Logo

dev-resources.site

for different kinds of informations.

On the Andy Pavlo's DB review

Published at
1/2/2025
Categories
discuss
database
postgres
ai
Author
pranav_aurora
Categories
4 categories in total
discuss
open
database
open
postgres
open
ai
open
Author
13 person written this
pranav_aurora
open
On the Andy Pavlo's DB review

The Andy Pavlo yearly review has a massive chokehold amongst the DB community. It's like the oscars of databases?

This year was a pretty special review, our project, pg_mooncake was mentioned.

Here are some thoughts from reading the review, and what we've learnt at Mooncake Labs in our first 121 days of existence.

1. Yes, we're guilty of 'Shoving Ducks everywhere'...

Our first project, pg_mooncakeadded a native columnstore table (Iceberg) to Postgres for 1000x faster analytics.

While, there are quite a few extensions on the market bringing DuckDB into Postgres; we focussed on making the columnar storage feel like a regular Postgres table. Things like transactional writes, triggers etc. See our architecture

To us, it feels like the final touch to complete the 'analytics in PG experience'. Almost a decade later from early projects like Citus, we're optimistic that analytics in Postgres will be a reality.

2. 2024 felt like year of the Data Lake.

Snowflake vs Databricks. elastic's 'search lake' (lol). s3 tables.

What I mean by the 'lake': serverless workloads on data in object storage.

In 2024, analytic (DatabricksSQL, Snowflake Iceberg) & Vector Search (Turbo Puffer, Lance) moved to the lake.

In 2025, I reckon there will be more workloads (lookups, full-text) running in this manner.

3. As for vector search...

Agents are everywhere; and yet vector search wasn't a topic at all... Couple thoughts.

  1. Just use Postgres
  2. If you have big 'data', LanceDB / Turbopuffer
  3. Vector search workloads moving toward full-text workloads. Something we've noticed a lot. Hybrid Search results are often ~95%+ full-text results.

4. As for AI / Agents

A lot of the AI companies we spend time with are each building a'systems of record' for each customer... And they're all storing structured/unstructured data in a 'Lake'. See Rox's architecture

Another trend we've seen: LLMs being used for data processing and ML tasks (feature extraction, classifiers).

It kind of makes sense too… on small data. Product engineers can use LLMs out of the box, instead of picking/training/deploying ML models for each task.

I am super super curious how Snowflake, Databricks and Redshift AI functions will play out this year.

2025 will be exciting.

Pranav

postgres Article's
30 articles in total
Favicon
Developing a project using Java Spring Framework, JSON, JPA and PostgreSQL
Favicon
PostgreSQL plan_cache_mode
Favicon
Deploy laravel application using vercel : Amazing
Favicon
Diesel vs SQLx in Raw and ORM Modes
Favicon
TimescaleDB in 2024: Making Postgres Faster
Favicon
Primeiros Passos no PostgreSQL: Um Guia Completo para Iniciantes
Favicon
What are the benefits of using bounded quantifiers in regex
Favicon
Tutorial: How We Built a Reverse Video Search System Directly in PostgreSQL
Favicon
Building a Semantic Search Engine with OpenAI, Go, and PostgreSQL (pgvector)
Favicon
TOAST: PostgreSQL's Secret Sauce for Handling Big Data 🍞
Favicon
Simplifying PostgreSQL User Management: Two Methods
Favicon
Change data capture com AWS RDS e PostgreSQL
Favicon
Creating a Multi-Tenant Application with Laravel and Neon
Favicon
Node.js Meets PostgreSQL and MongoDB in Docker: Docker Diaries
Favicon
Understanding JWT: The Backbone of Modern Web Authentication and Authorization
Favicon
Index Filtering in PostgreSQL and YugabyteDB (Index Scan instead of Index Only Scan)
Favicon
Choosing the Right Time Series Database for Your Use Case
Favicon
On the Andy Pavlo's DB review
Favicon
Steps to Add a New Column with Sequelize Migrations
Favicon
πŸš€ Building a RESTful API in Go: A Practical Guide
Favicon
What is pgAdmin?
Favicon
Master Test Data Generation With dbForge Studio for PostgreSQL
Favicon
Salvaging a Corrupted Table from PostgreSQL
Favicon
Monitoring AWS RDS Postgres Parameter Changes
Favicon
PostgreSQL table partitioning
Favicon
[Boost]
Favicon
Managing MQTT User Information with PostgreSQL in a Separate Container
Favicon
Made FOSS for simplifying NextJS dev with OAuth And Postgres
Favicon
Big Discovery - lowCalAlt_update 7
Favicon
CΓ³mo instalar PostgreSQL en una instancia EC2 con Amazon Linux 2023

Featured ones: