Skip to main content

When to use it

pinecone-sparse-english-v0 is the right choice for workflows that need a learned sparse-vector representation — for example, when your application already produces sparse vectors upstream of Pinecone, or when you’re pairing it with a dense encoder in a single-index hybrid workflow on the Vectors API.For general-purpose text retrieval, we recommend full-text search: it runs over your raw text fields with BM25 ranking and Lucene query syntax.Built on the innovations of the DeepImpact architecture, the model directly estimates the lexical importance of tokens by using their context, unlike traditional retrieval models like BM25, which rely solely on term frequency. The model outperforms BM25 by up to 44% (average 23%) NDCG@10 on Text Retrieval Conference (TREC) Deep Learning Tracks and up to 24% (8% on average) on BEIR. For more information, see the cascading retrieval blog post.

Inputs, limits, and pricing

Monthly and per-minute limits depend on your plan. See Rate limits.

Quickstart

Installation

Integrated embedding on document indexes requires Python SDK 10.1.0 or later, or Node.js SDK 9.1.0 or later.

Create index

This creates a document index whose chunk_text field embeds with pinecone-sparse-english-v0. The write_parameters and read_parameters raise the max input tokens from 512 to 2,048 for upserts and queries.

Embed & upsert

Upsert documents with plain text on the embedded field. Pinecone embeds each value on write, up to 96 documents per request.

Query

Search with a sparse_embed clause. Pinecone embeds the query text with the field’s model and returns the documents that share the most important terms with it.

Parameters

When you call Generate vectors, set these in the request’s parameters. On a sparse_embed field, set them in write_parameters and read_parameters. Pinecone sets input_type to passage for writes and query for reads by default.

Good to know

You can call the Generate vectors operation for standalone embedding to turn text into vectors without writing to an index. That differs from integrated embedding on a document index, where Pinecone embeds a field’s text when you upsert or search. To see how many embedding tokens a request consumes, see Embedding tokens.