Pinecone pg_pinecone: Open Source Search That Lives Inside Postgres

The hardest part of building an AI feature is often not the model. It is finding the right records, fast, in the database you already run. Pinecone pg_pinecone is an open source Postgres extension that brings Pinecone's vector search inside Postgres itself. It works on the pgvector columns you already store, it keeps your tables and your SQL, and on its default engine your data never leaves Postgres (pinecone-io/pg_pinecone, GitHub, retrieved 2026-10-27). Pinecone announced pg_pinecone on October 27, 2026 and released it under the Apache License 2.0. The pitch is simple: search that used to need a second system can now run where your data already lives.

Key Takeaways

• pg_pinecone is an open source Postgres extension, released by Pinecone on October 27, 2026 under the Apache License 2.0, that adds vector search to the pgvector columns you already have.

• Each table uses one of three engines: native, pgvector, or remote. You switch engines with one pinecone.set_engine() call, and your queries do not change.

• The default native engine stores compact RaBitQ codes in clusters instead of a graph, so its index stays small and it applies filters during the search, not after it.

• In Pinecone's own benchmarks on 1 million 768-dimension vectors, the native index measured 136 MB, compared with 2.0 GB for HNSW on halfvec and 4.1 GB for HNSW on full-size vectors.

• The native engine needs no Pinecone account, API key, or network calls. Optional Pinecone features, including the remote engine, embed(), BM25 full-text search, and rerank(), need an API key.

To see why this matters, start with the usual setup. A support assistant, a product recommendations engine, or an agent that remembers past work all need to find the most relevant records quickly. For many teams, those records live in Postgres. Moving them to a separate vector database means a second system to run, a copy of the data to keep in sync, and a new failure mode to watch. pg_pinecone removes that copy for the common case: the table stays in Postgres, Postgres stays the system of record, and search runs in the same database.

What Is pg_pinecone?

pg_pinecone is a Postgres extension for search on the data you already store. That is the shortest accurate description, and it is worth unpacking word by word. It is an extension, not a separate database, so you install it into Postgres and enable it with CREATE EXTENSION pg_pinecone. It is for search, specifically vector search and, optionally, keyword search. And it works on data you already store, because it indexes existing pgvector vector columns rather than asking you to move your data into a new store.

Vector search is search by meaning rather than by exact words. A text passage is turned into an embedding, a long list of numbers that captures what the passage is about, and the database finds the stored embeddings closest to the query embedding. pgvector is the established Postgres extension that adds a vector column type and distance operators to Postgres. pg_pinecone builds on that column type, so an application that already sends SQL such as ORDER BY embedding <=> $1 LIMIT 10 can keep sending the same SQL.

The repository went public on GitHub on October 26, 2026, one day before the announcement, and at the time of writing it shows 44 commits, 23 stars, and 2 releases, under the Apache License 2.0 (pinecone-io/pg_pinecone, GitHub, retrieved 2026-10-27). Pinecone describes this as the project's first release and says it maintains pg_pinecone and reviews every pull request. That is an honest framing: this is day one software from a company with a large managed vector database business, not a mature community project with years of production history.

Three Engines Behind One Index

The design decision that makes pg_pinecone interesting is not a single search algorithm. It is the choice to put three engines behind one index and let a table switch between them. Each table uses one engine at a time, and switching takes one call to pinecone.set_engine(). Your queries do not change, and every switch can be reversed, according to the project documentation (pinecone-io/pg_pinecone, GitHub, retrieved 2026-10-27).

The native engine is the default and the main story of this release. It builds and searches its index entirely inside your Postgres database, with no Pinecone account, API key, or network calls. The pgvector engine searches an existing pgvector index instead, which is useful if you already run one and do not need the native engine's filtering. The remote engine sends search to an index in Pinecone's managed service, for tables that have outgrown a single Postgres server. With the remote engine, your embeddings sync to Pinecone in the background, searches keep running in Postgres until the copy finishes, and Postgres stays the source of truth.

That ladder matters because it changes the cost of a wrong choice. A team can start on the native engine, inside Postgres, and move a table to the managed service later without rewriting queries. It can also move back. The extension is therefore less of a bet on one deployment shape and more of a way to defer that decision until the data forces it.

How the Native Engine Searches

Like pgvector's HNSW, the native engine uses an approximate nearest neighbor algorithm. Approximate nearest neighbor (ANN) is a search method that finds very close matches quickly without guaranteeing the mathematically closest match every time. The difference is what the index stores and how it is walked. HNSW stores a graph: a full copy of every vector plus links to its nearest neighbors, and a search walks that graph, comparing full vectors at each hop, so the whole graph needs to stay in memory to be fast.

The native engine replaces the graph with an inverted file index, usually called IVF, that groups similar vectors into clusters. Its index stores a compact summary of each vector as a RaBitQ code, about one bit per dimension plus a few correction values, using a method credited to Gao and Long in a SIGMOD 2024 paper. The codes for each cluster are packed together, so a search reads a few contiguous pages instead of hopping across a graph. A search then runs in three steps: probe the nearest clusters by comparing the query vector with each cluster's centroid, compute an estimated distance for each row from its RaBitQ code and keep the best candidates, then rerank those candidates using their full vectors from your table.

That is why the index is so much smaller. A summary is cheaper to store than a full vector plus graph links, and a clustered layout is cheaper to read than a graph walk. The trade is that the estimate step must be good enough that the true best matches survive into the shortlist for reranking. Pinecone's benchmark numbers, discussed below, are its evidence that they do, at least on the test data it chose.

The Filter Problem, and Why It Matters More Than Speed

Most real searches are filtered. A support assistant searches one customer's tickets, a store searches one tenant's products, an agent searches one user's notes. The filter is usually a WHERE clause on a column such as tenant_id. The question is when that clause is applied, and the answer decides whether the results are complete.

pgvector's HNSW post-filters: it finds the nearest neighbors first, then applies the WHERE clause. With a narrow filter, like one tenant out of a hundred, most candidates are discarded, and a query can return fewer rows than its LIMIT. pgvector's iterative scans fill the list by searching further, at a cost in time, and in accuracy if relaxed_order is used, according to Pinecone's announcement (Pinecone's announcement, retrieved 2026-10-27).

The native engine filters during the search instead, and picks its strategy based on selectivity. It stores the value of one filter column next to each row's RaBitQ code and, at build time, counts how many rows carry each value. For a narrow filter, 3 percent of rows or fewer by default, it pre-filters: it scans every matching row across all clusters and scores each one with its full vector, so results are exact. For a wider filter, more than 3 percent of rows, it uses single-stage filtering: it searches more of the nearest clusters than an unfiltered search, up to half, and checks the filter inside each cluster before scoring.

That distinction is the strongest argument for the native engine, stronger than raw speed. Speed is easy to buy with more memory. Completeness under a narrow filter is a correctness property, and a search that silently returns fewer rows than requested is a bug that looks like a missing feature to the user. If your workload is mostly unfiltered, this advantage matters less, and the pgvector engine may be enough. If your workload is per tenant or per user, it matters a lot.

Keeping the index current is simpler than the search itself. When you insert new rows, each vector joins the nearest cluster and is searchable immediately. Centroids stay where the build placed them, so after large changes, like a bulk load that doubles the table or new data unlike what the index was built on, Pinecone advises running REINDEX or enabling automatic maintenance to rebalance the clusters. That is a normal operational cost for a clustered index, and it should be planned into any bulk load, not discovered after one.

Benchmarks: What Pinecone Reports, and What to Watch

All benchmark figures in this section are Pinecone's own, reported in its announcement and in its benchmark harness, and they have not yet been independently reproduced in published coverage. That caveat belongs at the top, because the numbers are striking and because the company that sells the managed service also wrote the test. The setup, as Pinecone describes it, is 1 million wiki_dpr vectors with 768 dimensions, on an AWS r6id.4xlarge instance with 16 vCPUs and 128 GB of RAM. Unfiltered latency, build time, and index size were measured in one session on PostgreSQL 17, filtered search in another on PostgreSQL 18, and a memory-capped configuration in a third. Latency is median p50 at k equal to 10 with one client, unless noted.

Read the table in two parts. The index size and build time gaps are large and structural: a clustered summary index should be smaller and faster to build than a graph of full vectors, and 136 MB against 2.0 GB and 4.1 GB is the kind of gap that changes whether an index fits in memory at all (Pinecone's announcement, retrieved 2026-10-27). The latency gaps are smaller, and they come with a trade: at a 10 percent filter, the native engine reaches 1.000 recall in 12.3 ms while HNSW with iterative scans reaches 0.920 recall at best in 5.9 ms. The native engine is more accurate there, but slower in absolute time, because it searches more clusters to get the filter right. That is a reasonable trade for a filtered workload, and a bad trade if you only care about raw speed on unfiltered queries.

Independent coverage so far restates Pinecone's figures rather than re-running them. Unite.AI's launch coverage reports the same index sizes, latencies, and filter behavior, and notes the same three engines and the same default of no account or network calls on the native engine. Treat the benchmark table above as a claim to verify on your own data, not as a settled result. The benchmark harness is in the repository, so a team can rerun the tests before committing to the extension, and that is the right next step for any production decision.

How to Try pg_pinecone

The requirements are narrow, so check them first. pg_pinecone requires PostgreSQL 17 or 18, superuser access, and pgvector installed on the server (pinecone-io/pg_pinecone, GitHub, retrieved 2026-10-27). It installs from a prebuilt release on GitHub or as the pineconeteam/pg_pinecone image on Docker Hub, which the README describes as the pgvector image with pg_pinecone added, with one tag per Postgres major version. It can also be built from source with a Rust toolchain and cargo-pgrx 0.19.2.

The quick start, from the announcement, is short. Create a table with a vector column, enable the extension, build an index, and query it the way you already do: CREATE EXTENSION pg_pinecone, then SELECT pinecone.create_index('docs', 'embedding', filter_column => 'tenant_id'), then SELECT id, body FROM docs WHERE tenant_id = $1 ORDER BY embedding <=> $2 LIMIT 10. The filter_column argument is worth setting from the start if your workload filters by tenant or user, because it is what makes the native engine's during-search filtering work.

Two operational notes deserve emphasis. First, running the remote engine also requires loading the extension through shared_preload_libraries, which is a server configuration change, not just a SQL statement, so plan a restart window. Second, the extension's documentation on docs.pinecone.io covers installation, the storage engines, engine switching, and full-text search, while the repository adds a SQL reference, an operator runbook, and a disaster recovery guide. Read the runbook before a production install, not after the first incident.

Optional Pinecone Features, and What They Cost

The native engine is free in the sense that matters most: no account, no key, no network, no bill from Pinecone. Everything else in the announcement is optional and tied to a Pinecone API key. That split is clean, but it is also a funnel, and it should be named as one.

With an API key, the same extension gives you four more tools, each optional and added one at a time. The remote engine moves search to Pinecone's managed service with one SQL command, SELECT pinecone.set_engine('docs', 'remote'). The embed() function calls Pinecone's inference service to turn text into embeddings from SQL, so a table with text but no embeddings can be filled without hosting a model or managing another provider's key, and Pinecone says its free tier covers a first build. BM25 full-text search runs alongside vector search on the same table, for exact terms like product names or error codes as well as search by meaning. And rerank() reorders top results with Pinecone's reranking models, which helps when only the top few results matter, as in retrieval augmented generation, or when keyword and vector results are merged into one list.

None of those features changes where search runs, except the remote engine itself. Only the embed call needs Pinecone, in the embed case, and only the rerank call needs Pinecone, in the rerank case. That separation is good design: a team can use embed() to bootstrap a table, then run all search locally on the native engine, and pay nothing beyond the free tier for that first build. The cost to watch is not the extension, which is Apache 2.0 software, but the managed calls, which are priced by Pinecone's service terms, and the data movement, if the remote engine syncs embeddings out of Postgres.

Who Should Use pg_pinecone, and Who Should Not

The best fit is a team that already runs Postgres 17 or 18, already stores vectors in pgvector, and mostly searches within a tenant, a user, or a category. For that team, pg_pinecone removes a second system, keeps the data in one place, and fixes the narrow filter case that post-filtering handles badly. It is also a good fit for a team that wants to start small and keep the option of moving to a managed service later, without rewriting queries.

Skip it, for now, if you run an older Postgres version, if you cannot get superuser access, or if your workload is unfiltered and already fast on HNSW. Skip the remote engine if your data cannot leave Postgres, because that engine's whole point is syncing embeddings to Pinecone's service. And treat the first release label seriously: Pinecone says it will build the public roadmap from feedback shared through GitHub issues, its Discord community, and monthly office hours, which means the feature set and the operational rough edges are still being discovered by early users.

A related question is how pg_pinecone sits next to other database stories on this site. If the idea of a database that changes shape appeals to you, our DuckDB 2.0 article covers a database that learned to be a server while staying in process. pg_pinecone is a different answer to a similar pressure: keep the database you have, and bring the new workload to it, instead of bringing your data to a new database.

Frequently Asked Questions

Is pg_pinecone free? The software is free and open source under the Apache License 2.0. The native engine needs no Pinecone account, API key, or network calls, so running it costs only your own Postgres infrastructure. Optional features that call Pinecone's service, including the remote engine, embed(), BM25 search through Pinecone, and rerank(), need an API key and are subject to Pinecone's service terms.

Does pg_pinecone replace pgvector? No. pg_pinecone works with pgvector columns you already have, and one of its three engines is the pgvector engine itself, which searches an existing pgvector index. It adds a native engine and a remote engine behind the same columns and the same SQL, rather than replacing the column type or the queries you already send.

Does my data leave Postgres on the default engine? No. On the default native engine, the index is built and searched entirely inside your Postgres database, with no Pinecone account, API key, or network calls. Data leaves Postgres only if you choose the remote engine, which syncs embeddings to a Pinecone index, or if you call optional Pinecone services such as embed() or rerank().

What Postgres versions does pg_pinecone support? PostgreSQL 17 or 18, with superuser access and pgvector installed on the server, according to the project README. It installs from a prebuilt release on GitHub or as the pineconeteam/pg_pinecone Docker image, with one tag per Postgres major version, or it can be built from source with a Rust toolchain and cargo-pgrx 0.19.2.

Why does filtering during search matter? Because post-filtering can return fewer rows than requested. If a search finds the nearest neighbors first and then applies a WHERE clause for one tenant out of a hundred, most candidates are discarded and the query can return fewer rows than its LIMIT. The native engine applies the filter during the search, so a narrow scope still returns a full set of results, and for narrow filters of 3 percent of rows or fewer it pre-filters for exact results.

The Bottom Line

pg_pinecone is a small extension with a clear idea: the database you already run should be able to search the data you already store, without a second system and without silently dropping filtered results. The native engine's clustered, compressed index is genuinely smaller than a graph index, the during-search filtering fixes a real correctness problem for multi-tenant workloads, and the three-engine ladder lets a team start inside Postgres and move out only if the data forces it. The benchmarks are Pinecone's own and should be rerun on your data before a production commitment, and this is a first release, so expect rough edges. But the default is the right default: no account, no key, no network, and no copy of your data. Start with the GitHub repository and the official announcement, or read the Indonesian version of this article.