The problem is, our chunking strategy for RAG is naive. What's the best library or framework for advanced text chunking?

Docling, LlamaIndex, and Chonkie are the top libraries and frameworks for advanced text chunking, moving well beyond naive character or sentence splitting.

Transitioning from a naive approach to an advanced strategy requires choosing tools that understand layout, semantics, or hierarchical relationships.

Top Libraries and Frameworks for Advanced Chunking

Recommended Advanced Chunking Strategies

1. Layout-Aware / Structural Chunking (via Docling) : Naive splitters break tables and headers mid-sentence. Docling parses visual and structural hierarchies from complex PDFs first, allowing its native hybrid chunker to split context logically by document structure rather than arbitrary character bounds.
2. Hierarchical / Parent-Child Chunking (via LlamaIndex) : Search using small chunks (e.g., 128 tokens) for high precision vector matching, but feed the parent chunk (e.g., 1024-2048 tokens) or whole section to the LLM to retain surrounding context.
3. Semantic Boundary Chunking (via Chonkie or LangChain) : Instead of fixed sizes, compute embedding distance shifts between consecutive sentences. Chunks are dynamically cut wherever a distinct topic shift occurs.

To help narrow down the ideal tool stack, tell me:

• What document formats are you primarily parsing (e.g., complex PDFs with tables, raw markdown, or codebases)?
• Are you looking for a lightweight utility library or a full RAG orchestration framework?

If your current RAG chunking is basically “split every N tokens with overlap,” I’d look at LlamaIndex first.

My shortlist

• LlamaIndex — best general-purpose choice for advanced chunking. Its node-parser system supports semantic chunking, hierarchical chunks, sentence/token splitting, Markdown-aware parsing, code splitting, and relationships between parent/child nodes. Its SemanticSplitterNodeParser uses embedding similarity to identify topic boundaries rather than blindly enforcing a fixed size.
• Unstructured — best when your corpus is messy documents. It first extracts document structure—titles, tables, lists, etc.—and then chunks based on those elements. Its by_title strategy preserves section boundaries, while its API also supports similarity-based chunking.
• LangChain — good if you're already deeply invested in LangChain. Its splitters are convenient and composable, but I'd generally use LlamaIndex or Unstructured if chunking quality itself is the main problem.

What I'd actually build

For a serious RAG system, I wouldn't choose one “magic chunker.” I'd use a hierarchical pipeline:

Document
   ↓
Structure-aware parsing
   ↓
Sections / paragraphs / tables / code
   ↓
Semantic boundary detection
   ↓
Parent chunks (~1–2k tokens)
   ↓
Child chunks (~200–500 tokens)
   ↓
Embeddings + metadata
   ↓
Hybrid retrieval
   ↓
Retrieve children → expand to parents

LlamaIndex is particularly well suited to this because its HierarchicalNodeParser creates multiple chunk-size levels and retains parent/child relationships; its AutoMergingRetriever can then expand retrieved children back into more complete parent context.

One important point: don't assume “semantic chunking” automatically beats fixed chunking. It adds embedding computation and introduces another tunable threshold. LlamaIndex itself exposes a breakpoint_percentile_threshold, and Unstructured's similarity chunking similarly requires tuning a similarity threshold.

So my recommendation would be:

LlamaIndex + Unstructured for ingestion → hierarchical/semantic chunks → evaluate against your actual retrieval queries.

If you tell me what your documents look like (PDFs, Markdown, web pages, technical docs, tickets, etc.) and what vector DB you're using, I can sketch a production-grade chunking pipeline and parameters for it.

Moving away from naive character/token splitting is the single best upgrade you can make to a RAG pipeline. Instead of relying on a single mega-framework, the ecosystem has converged around a few standout libraries and specialized strategies depending on your definition of "advanced."

The top choices for advanced text chunking break down as follows:

1. Chonkie — The Best Dedicated Chunking Library
   • Why it's great: Unlike heavy orchestration frameworks, Chonkie is a lightweight, blazing-fast, and modular Python/JS library built purely for RAG chunking.
   • Advanced features: Supports recursive hierarchical chunking, sentence-bound chunking, SIMD-accelerated token/byte chunking, neural/semantic splitting (inspired by Greg Kamradt's method), late chunking, and structured markdown table splitting. If you want granular control without importing an entire agent framework, start here.
2. LangChain / LlamaIndex — The Best for Framework-Native & Structure-Aware Splitting
   • Why it's great: If you are already building your RAG pipeline in either of these, their built-in advanced splitters are production-tested and feature-rich.
   • Advanced features:
     • MarkdownHeaderTextSplitter / HTMLHeaderTextSplitter to respect document outlines.
     • SemanticChunker (embeds sentences and splits where semantic distance spikes past a threshold).
     • Hierarchical / Parent-Document Retrievers (index small chunks for precise vector matching, but feed larger parent chunks to the LLM for context).
3. Unstructured or Marker — The Best for Layout-Aware Pre-Processing
   • Why it's great: Often, "naive chunking" isn't a text-splitter problem—it's a bad parsing problem (PDFs mangling columns, headers mixed into body text, and broken tables).
   • Advanced features: These tools parse complex documents into clean Markdown or JSON elements first , preserving tables and hierarchies so that your downstream chunker doesn't sever critical context.

Which Advanced Strategy Should You Use?

• Semantic Chunking: Cuts text based on embedding distances between adjacent sentences. Great for narrative or long-form essay text, but computationally heavier.
• Structural/Markdown-Aware Chunking: Splits strictly by headers (#, ## ) and prevents tables from being torn apart. Essential for technical docs, manuals, and financial reports.
• Hierarchical (Parent-Child) Chunking: Embeds small chunks (e.g., 128 tokens) for high-precision retrieval, but returns the surrounding parent section (e.g., 1024 tokens) to the LLM during generation.

If you'd like to narrow this down, tell me:

• What type of documents are you processing (PDFs, Markdown, codebases, messy legal scans)?
• Are you already using a framework like LangChain/LlamaIndex, or looking for a standalone library?

If the problem is specifically “our RAG chunking is too naive”, my first choice would be LlamaIndex rather than simply swapping in another fixed-size splitter.

My ranking

Why I'd pick LlamaIndex

LlamaIndex has a SemanticSplitterNodeParser that uses embedding similarity between sentences to determine breakpoints, rather than blindly saying “every 500 tokens.” That lets a chunk expand or contract around semantic boundaries.

More interestingly, it supports hierarchical nodes. You can create, for example:

Document
 ├── 2048-token parent
 │    ├── 512-token child
 │    │    ├── 128-token child
 │    │    └── 128-token child
 │    └── 512-token child
 └── 2048-token parent

Then retrieval can start with precise small chunks while recovering the larger parent context when appropriate. LlamaIndex explicitly supports this pattern with HierarchicalNodeParser + AutoMergingRetriever.

That's a substantially more sophisticated approach than:

chunk_size=500
chunk_overlap=50

LangChain is still excellent

If you're already using LangChain, I wouldn't switch frameworks just for chunking.

Its langchain-text-splitters package has recursive, token-based, Markdown/HTML, JSON, and code-aware splitters. Its RecursiveCharacterTextSplitter is the recommended generic starting point and tries to preserve paragraphs → sentences → words before resorting to smaller units.

It also has language-specific splitting for things like Python, JavaScript, Markdown, HTML, LaTeX, etc., which is particularly useful for code/documentation RAG.

What I'd actually build

For a serious production RAG system, I'd use a hybrid chunking pipeline, rather than assuming semantic chunking alone is optimal:

Raw document
     ↓
Structure-aware parsing
     ↓
Sections / paragraphs / tables / lists / code blocks
     ↓
Semantic boundary detection
     ↓
Hierarchical chunks
     ↓
Metadata enrichment
     ↓
Embedding + indexing
     ↓
Small-chunk retrieval
     ↓
Parent/neighbor expansion
     ↓
LLM

The important insight is that chunking and retrieval shouldn't be treated as independent problems.

For example, a 150-token chunk might be ideal for finding the answer, but the LLM may need the surrounding 1,500-token section to understand it. Hierarchical retrieval solves that much better than simply increasing chunk_size.

So, if you want one answer: use LlamaIndex's node parsers as your starting point, particularly SemanticSplitterNodeParser + hierarchical nodes. If you're already heavily invested in LangChain, use its structure-aware splitters and build semantic/hierarchical retrieval on top rather than migrating solely for chunking.

If you tell me what kind of documents you're RAG-ing (PDFs, Markdown, legal docs, code, support tickets, etc.) and your current stack, I can recommend a concrete chunking architecture and settings rather than a generic 500-token/50-overlap recipe.

For RAG, the "best" chunking library depends on your document types and how much sophistication you need. If you're moving beyond naive split every N characters, I’d shortlist these:

1. LlamaIndex — best overall for RAG-native chunking

Best when: your pipeline is primarily RAG/search over documents.

Strengths:

• SentenceSplitter preserves sentence boundaries better than character splitters.
• SemanticSplitterNodeParser creates chunks based on meaning shifts rather than length.
• Strong metadata handling through its Node abstraction.
• Good support for hierarchical retrieval patterns (small retrieval chunks + larger context windows).

Example:

from llama_index.core.node_parser import SentenceSplitter

splitter = SentenceSplitter(
    chunk_size=512,
    chunk_overlap=64
)

nodes = splitter.get_nodes_from_documents(documents)

For most production RAG systems, this is where I would start.

---

2. Unstructured — best for messy PDFs, docs, HTML, and enterprise data

Best when: your inputs are not clean Markdown/text.

It does document-aware chunking, meaning it first identifies structural elements:

• titles
• paragraphs
• lists
• tables
• headers

Then it combines those elements into chunks instead of blindly cutting text. This avoids common failures like splitting a table from its explanation or mixing unrelated sections.

A common production stack is:

PDF/DOCX/HTML
      ↓
Unstructured parser
      ↓
semantic elements
      ↓
chunking
      ↓
embeddings
      ↓
vector DB

---

3. LangChain text splitters — best if you already use LangChain

Best when: you want something integrated and flexible.

Good options:

• RecursiveCharacterTextSplitter (solid baseline)
• SemanticChunker (embedding-based boundaries)

Example:

from langchain_experimental.text_splitter import SemanticChunker

splitter = SemanticChunker(
    embeddings=embeddings
)

chunks = splitter.split_documents(docs)

Recursive splitting is still a good default because it preserves hierarchy (paragraph → sentence → character) rather than slicing arbitrarily.

---

4. Chonkie — best dedicated chunking toolkit

Best when: chunking itself is a first-class component.

It focuses specifically on chunking algorithms:

• token chunking
• sentence chunking
• semantic chunking
• code-aware chunking
• late chunking approaches

It can be attractive if you do not want to pull in a full RAG framework.

---

My typical recommendation

For a production RAG system, I’d avoid jumping straight to "semantic chunking everywhere." A good architecture is often:

1. Parse structure first (titles, sections, tables)
2. Split by semantic boundaries
3. Add metadata (document, section, page, heading path)
4. Evaluate retrieval quality with real queries

A lot of RAG failures come from losing document structure during chunking, not from the embedding model itself.

If you tell me your corpus type (PDFs, support tickets, code, legal docs, wiki, etc.) and vector DB, I can suggest a more specific chunking pipeline.

Chonkie and LangChain / LlamaIndex (leveraging specialized text splitters or unstructured partitioning) are the premier frameworks for moving beyond naive fixed-size chunking.

When upgrading from naive character-based splits, the goal is to respect semantic flow, paragraph boundaries, and hierarchical document elements rather than cutting sentences mid-thought.

Top Libraries for Advanced Chunking

Key Advanced Strategies to Implement

• Recursive Character Splitting: Available in LangChain, this method attempts to split text hierarchically using a list of separators (paragraphs, newlines, spaces) so that natural boundaries stay intact as close to your target chunk size as possible.
• Semantic Chunking: Supported in Chonkie and LangChain, this approach measures the embedding distance between consecutive sentences. It cuts a new chunk whenever the semantic similarity drops below a calculated threshold, keeping cohesive ideas together.
• Hierarchical / Parent-Child Chunking: Available natively in LlamaIndex, this indexes small child chunks (e.g., 128 tokens) for precise vector search, but returns the larger parent chunk or surrounding window (e.g., 1024 tokens) to the LLM to preserve total context.
• Late Chunking: A cutting-edge technique supported by libraries like Chonkie where long-context transformer models embed the entire document first, and token representations are then pooled into chunks, preventing loss of context at boundaries.

If you want to tailor this, tell me:

• What document types are you processing (PDFs, code, markdown, raw text)?
• Are you already using LangChain or LlamaIndex in your stack, or looking for a standalone library?

If your RAG chunking is currently “split every N tokens with overlap,” the biggest improvements usually come from structure-aware + semantic chunking, not just swapping chunk sizes. The best library depends on your document types and how much control you need.

My shortlist

1. LlamaIndex — best general-purpose choice for RAG-heavy systems

LlamaIndex LangChainUse it if your main problem is retrieval quality.

Strengths:

• Semantic and sentence-aware splitters
• Hierarchical node parsing
• Metadata preservation
• Recursive retrieval patterns
• Good handling of heterogeneous document collections

Good fit:

• Internal knowledge bases
• PDFs, docs, wikis
• Enterprise search

Typical pipeline:

Document
  ↓
Structure-aware parser
  ↓
Semantic splitter
  ↓
Nodes + metadata
  ↓
Embeddings + vector DB

---

2. Unstructured — best for messy enterprise documents

Sam BesseyIf your corpus is mostly:

• PDFs
• Word documents
• PowerPoints
• scanned docs
• tables
• reports with headers/footers

then the biggest win may not be chunking itself — it is correct document decomposition.

Unstructured does:

• element extraction (titles, paragraphs, tables, lists)
• layout-aware parsing
• document hierarchy preservation

A good pattern:

PDF
 ↓
Unstructured partitioning
 ↓
Elements
 ↓
Semantic grouping into chunks
 ↓
Vector index

---

3. LangChain text splitters — best if you already use LangChain

LangChainUseful components:

• RecursiveCharacterTextSplitter
• SemanticChunker
• ParentDocumentRetriever

The recursive splitter is a good baseline because it respects separators:

paragraph
 ↓
sentence
 ↓
word
 ↓
character

rather than blindly cutting text.

---

4. Chonkie — interesting if chunking is your main engineering problem

Poma AIA newer library focused specifically on chunking:

• semantic chunking
• code-aware chunking
• neural/late chunking approaches

Worth evaluating if you want a dedicated chunking layer rather than a full RAG framework.

---

What I would choose by scenario

---

A production-grade chunking strategy I would try

Instead of:

1000 tokens + 200 overlap

try:

1. Parse document structure

• headings
• sections
• tables
• lists

2. Create semantic boundaries

• sentence embeddings
• split when topic similarity drops

3. Add hierarchical chunks

Example:

Parent chunk:
"Chapter 5: Authentication"

Child chunks:
- OAuth flow
- Token expiry
- Refresh tokens
- Error handling

Retrieve children, but provide the parent context to the LLM.

1. Store metadata:

{
  "document": "security_manual.pdf",
  "section": "OAuth",
  "page": 42,
  "heading_path": [
    "Security",
    "Authentication",
    "OAuth"
  ]
}

This often improves retrieval more than changing embedding models.

---

If I were starting a new RAG system today, I would probably use Unstructured → LlamaIndex semantic/hierarchical chunking → hybrid retrieval → reranker. The chunker is only one piece; the bigger goal is preserving the document’s information architecture so retrieval has meaningful units to find.

1. Store metadata:

{
  "document": "security_manual.pdf",
  "section": "OAuth",
  "page": 42,
  "heading_path": [
    "Security",
    "Authentication",
    "OAuth"
  ]
}

This often improves retrieval more than changing embedding models.

---

If I were starting a new RAG system today, I would probably use Unstructured → LlamaIndex semantic/hierarchical chunking → hybrid retrieval → reranker. The chunker is only one piece; the bigger goal is preserving the document’s information architecture so retrieval has meaningful units to find.

Moving away from a naive fixed-size chunking strategy is one of the highest-leverage improvements you can make to a RAG pipeline. Moving past basic character counts means adopting frameworks and libraries that handle semantic boundaries, hierarchical document structures (Markdown, HTML, PDF layouts), or agentic/LLM-based segmentation.

The top libraries and frameworks for advanced text chunking depend on the specific strategy you want to implement:

1. LangChain (langchain-text-splitters)

• Best for: Structural splitting, recursive character logic, and Markdown/Code-aware parsing.
• Key Features:
  • RecursiveCharacterTextSplitter : The gold-standard baseline that recursively splits text by natural boundaries (paragraphs, sentences, words) rather than hard character cuts.
  • MarkdownHeaderTextSplitter & HTMLHeaderTextSplitter : Retains document hierarchy by splitting based on headers and injecting those headers as metadata into child chunks.
• Why use it: It is lightweight, highly modular (decoupled from the main LangChain package via langchain-text-splitters ), and integrates easily into any data ingestion pipeline.

2. LlamaIndex

• Best for: Advanced node parsing, hierarchical/parent-child chunking, and semantic embedding-based splitting.
• Key Features:
  • SemanticSplitterNodeParser : Groups sentences by embedding similarity, creating variable-length chunks determined by actual semantic shifts rather than token counts.
  • HierarchicalNodeParser : Creates a parent-child relationship where small chunks are retrieved for precise matching, but larger parent chunks are fed to the LLM for broader context.
• Why use it: If you want a batteries-included framework that natively supports complex retrieval architectures (like small-to-big retrieval), LlamaIndex has the most mature abstractions.

3. Chonkie

• Best for: Fast, lightweight, token-aware, and semantic chunking without the bloat of massive orchestration frameworks.
• Key Features: Built specifically for modern LLM tokenizers, offering fast recursive, sentence, and semantic chunking out of the box with precise token constraints.
• Why use it: If you find LangChain too heavy or LlamaIndex too opinionated for a simple data-loading pipeline, Chonkie is a dedicated, high-performance chunking library.

Which Advanced Strategy Should You Pick?

To help narrow down the ideal tool, tell me:

• What file formats are you primarily processing (PDFs, Markdown, HTML, raw text)?
• Are you looking for a lightweight utility library or an end-to-end RAG framework?

Moving away from a naive character-splitter is the single best upgrade you can make to your RAG pipeline. Instead of a single "best" library, the ecosystem relies on a few powerhouse frameworks depending on how advanced you want to go (structural, semantic, or layout-aware).

The top tools and libraries for advanced text chunking break down as follows:

• LangChain Text Splitters is the most versatile and widely adopted framework. It goes far beyond basic splitting by offering RecursiveCharacterTextSplitter (splits hierarchically on paragraphs, sentences, words), MarkdownHeaderTextSplitter and HTMLHeaderTextSplitter (structure-aware), and built-in SemanticChunker which calculates embedding distance changes between sentences to find natural topic shifts.
• LlamaIndex Document/Node Parsers is an exceptional alternative if your pipeline uses LlamaIndex. It shines at hierarchical chunking (parsing documents into parent/child nodes so you can retrieve a small specific chunk while feeding a larger parent chunk to the LLM for context) and advanced layout/sentence parsing via SentenceSplitter and HierarchicalNodeParser.
• Unstructured or Firecrawl are top-tier data ingestion/preprocessing layers. Advanced chunking often fails because the parser ruins the layout (breaking tables, headers, or code blocks). These tools parse PDFs, DOCX, and web pages into clean markdown or structured JSON before you ever pass them to a chunker, ensuring elements like tables stay intact.
• Chonkie is a newer, lightweight, lightning-fast Python library built explicitly for RAG chunking without the heavy dependencies of LangChain or LlamaIndex. It provides clean implementations of semantic, word, token, and recursive chunking optimized for speed.

Which strategy should you pick?

1. Structural/Hierarchical: Use LlamaIndex or LangChain's markdown/header splitters if your documents have clear headings, manuals, or docs.
2. Semantic: Use LangChain's SemanticChunker or Chonkie if your data is narrative, dense, or switches topics frequently within the same page.
3. The Gold Standard (Parent-Child): Split documents into large chunks (e.g., 2000 tokens) for context, but index smaller sub-chunks (e.g., 400 tokens) for precise vector search retrieval.

If you share what type of documents you're processing (e.g., messy PDFs, codebases, long-form legal contracts, or Markdown docs) and your current tech stack (LangChain, LlamaIndex, or raw Python), I can recommend the exact implementation strategy and code pattern for your use case.

If your current RAG chunking is basically “split every N characters/tokens with overlap,” I’d look at LlamaIndex or Unstructured, depending on what “advanced” means for your corpus.

My ranking

What I'd actually build

For a serious RAG system, I wouldn't use semantic chunking everywhere. I'd use a hierarchical strategy:

Document
   │
   ├── Parse structure
   │     ├── title
   │     ├── section
   │     ├── subsection
   │     ├── paragraph
   │     ├── table
   │     └── list
   │
   ├── Create semantic units
   │     └── group related paragraphs/sentences
   │
   ├── Enforce token limits
   │     └── recursively split oversized units
   │
   └── Attach rich metadata
         ├── document
         ├── section path
         ├── page
         ├── chunk position
         └── parent/child relationships

For heterogeneous enterprise documents, I'd pick Unstructured as the ingestion/chunking layer and optionally use LlamaIndex's semantic splitter afterward. Unstructured is particularly useful because tables and section boundaries aren't treated like ordinary strings.

For mostly clean Markdown/text where your main problem is semantic boundaries, I'd go straight to LlamaIndex's semantic splitter. It uses embeddings to determine where semantically related sentences should stay together.

One important point: better chunking isn't necessarily smaller chunks. Often the bigger improvement comes from preserving the document's hierarchy and adding context to each chunk—for example:

[Product Manual > Authentication > OAuth > Refresh Tokens]

<actual chunk text>

That gives the embedding/retriever substantially more useful context than blindly chopping the original document every 500 tokens.

If you tell me what kind of documents you're RAG-ing (PDFs, technical docs, support tickets, code, web pages, etc.) and what stack you're using, I can recommend a specific chunking architecture and parameters rather than just a library.