Open-source embedding models

If your embedding API costs are getting too high, I'd start by self-hosting a small, high-quality embedding model rather than paying per token. You can run it on your own CPU or GPU, keep your data in-house, and eliminate third-party per-token embedding charges.

The best model depends on whether you prioritize retrieval quality, throughput, multilingual support, or low infrastructure cost. For most production RAG and semantic search systems, I'd shortlist models in three tiers: a lightweight model for maximum throughput, a stronger general-purpose model for retrieval quality, and a larger model if difficult queries justify the added compute.

My top picks

• 

1. Qwen3-Embedding-0.6B

(Best overall balance)

A strong default for production RAG, semantic search, and retrieval workloads. It has 0.6B parameters, supports long inputs, and is available under Apache 2.0. It's a sensible starting point when you want quality without the inference cost of a multi-billion-parameter model.qwenlm.github.io+3

Model: Hugging Face · 32K-token context family

---

• 

2. Nomic Embed Text v2

(Efficiency-focused alternative)

Worth testing when inference throughput and memory footprint are major concerns. Its mixture-of-experts architecture has around 475M total parameters, with 768-dimensional embeddings. Compare real throughput on your serving stack before choosing it over Qwen.link.sc+1

Model:Hugging Face

---

• 

3. BGE-M3

Best if you need multilingual search or want to explore dense, sparse, and multi-vector retrieval from one model. It has around 568M parameters and supports inputs up to 8,192 tokens.qwenlm.github.io+4

Model:Hugging Face

---

• 

4. all-MiniLM-L6-v2

The ultra-lightweight baseline for CPU deployments and latency-sensitive applications. Only 22M parameters, but expect lower retrieval quality on harder tasks than newer models.link.sc+1

Model:Hugging Face

How I'd choose

These are shortlist recommendations, not a universal benchmark ranking. The best choice depends on your data, query distribution, hardware, and latency target.docs.mteb.org+2

How to cut costs in production

The model is only part of the savings. I'd do the following:

1. Benchmark Qwen3-Embedding-0.6B first. Measure documents/second, query latency at p95, GPU memory, and retrieval quality on your actual queries.
2. Batch document embeddings. Maximize throughput during indexing rather than making one request per chunk.
3. Reduce vector dimensions where supported. Smaller vectors lower vector database storage and retrieval costs; validate quality before reducing dimensions.
4. Use hybrid search if appropriate. Combine vector similarity with BM25 keyword search, especially for IDs, error messages, product names, and technical documentation.
5. Avoid re-embedding unchanged content. Cache embeddings and process only new or modified chunks.

Important: changing embedding models generally requires re-embedding your document corpus and rebuilding the index. Don't switch models without accounting for this migration.

A quick question so I can narrow it down

What does your current embedding workload look like?

1. How much text do you embed per month?

Under 10 million tokens

10–100 million tokens

100 million–1 billion tokens

Over 1 billion tokens

---

2. What compute can you use?

CPU only

GPU available

Can rent cloud GPUs

Not sure yet

---

3. What's the main use case?

RAG / document search

Semantic search

Recommendations / clustering

Code search

---

4. What languages do you need?

English only

Multiple languages

Code and English

[Find my best model and deployment plan]

With those details, I can recommend a concrete model and serving stack and estimate whether self-hosting will actually cost less than your current API.

Yes. The main thing is that there isn’t a single “best” open-source embedding model—the right choice depends heavily on language coverage, retrieval quality, latency, hardware, vector DB costs, context length, and licensing.

Based on the current model landscape, I’d start with these:

My initial take

1. If retrieval quality is your #1 priority → Qwen3-Embedding

The Qwen3 family is particularly interesting now because it comes in 0.6B, 4B, and 8B versions, all with 32K context, multilingual support, configurable embedding dimensions, and instruction-aware embeddings. The 8B model reports a 70.58 MTEB multilingual score in the model card's June 2025 snapshot.huggingface.co+1

The tradeoff is obvious: 8B is much more expensive to run than a small encoder.

The 0.6B version is potentially the sweet spot if you want modern retrieval quality without dedicating a large GPU to embeddings. It supports 100+ languages, 32K context, and dimensions from 32–1024.huggingface.co+1

2. If you want a safe, proven RAG choice → BGE-M3

BGE-M3 remains a very compelling general-purpose option. It has a 1024-dimensional output, 8192-token context, multilingual training, and—unusually—it can support dense, sparse, and multi-vector/ColBERT-style representations. It's also MIT licensed.huggingface.co

That makes it especially attractive if you're building a sophisticated search stack rather than simply doing cosine similarity against a vector database.

3. If documents are long and multilingual → Jina v3

Jina Embeddings v3 supports 8192 tokens, 94 languages, task-specific adapters, and Matryoshka dimensions from 32 through 1024.huggingface.co

There is, however, a major licensing caveat: the Hugging Face model is CC BY-NC 4.0, with the model card explicitly noting restrictions around commercial/on-premises use.huggingface.co

So I wouldn't choose it for a commercial product without resolving the licensing question first.

4. If you want a lightweight English model → BGE-small

BGE-small-en-v1.5 produces 384-dimensional vectors and has a 512-token maximum sequence length. The model is only about 133 MB for the safetensors weights and is MIT licensed.huggingface.co+3

It's a very sensible option for CPU inference, edge deployments, prototypes, or applications where embedding throughput matters more than squeezing out the last few points of retrieval performance.

5. If vector-database storage matters → Nomic Embed v1.5

Nomic is interesting because of its Matryoshka representation. You can use 768, 512, 256, 128, or even 64 dimensions, allowing you to trade retrieval quality against storage and search cost. For example, its reported MTEB score goes from 62.28 at 768 dimensions to 61.04 at 256.huggingface.co

It's Apache 2.0 licensed and has an 8192-token context.huggingface.co

---

The requirements I'd use to make the actual decision

If you tell me these 7 things, I can give you a much more specific recommendation:

1. Primary task: RAG, semantic search, recommendations, clustering, deduplication, classification, code search, etc.
2. Languages: English only, multilingual, or specific languages.
3. Typical document/chunk size: e.g. 300 tokens, 2K tokens, 8K+.
4. Scale: roughly how many documents/vectors?
5. Latency requirement: batch/offline vs. real-time (and approximate queries/sec if known).
6. Hardware: CPU, consumer GPU, A10/A100/H100, Mac, etc.
7. License requirement: e.g. permissive commercial use only, MIT/Apache preferred, must be fully open-source, etc.

Also tell me which vector database you're using (pgvector, Qdrant, Milvus, Pinecone, Elasticsearch/OpenSearch, FAISS, etc.), because embedding dimensionality can have a surprisingly large impact on cost and performance.

If you give me those requirements, I can turn this into a ranked 3–5 model shortlist, including estimated memory/compute requirements, licensing implications, expected retrieval tradeoffs, and a suggested benchmark methodology for your dataset.