I want to build a knowledge graph from unstructured text and query it with an LLM. What's the best graph database and LLM integration?

Building a knowledge graph (KG) from unstructured text and querying it with an LLM—often referred to as GraphRAG —has a clear industry standard for developer tooling.

The Best Graph Database: Neo4j

• Why it wins: Neo4j is the undisputed gold standard and most widely adopted graph database for LLM integrations. It natively supports property graphs, vector embeddings (storing both text chunks and graph nodes), and has invested heavily in GenAI toolkits (like the Neo4j GenAI Plugin and Cypher generation wrappers).
• Alternatives:
  • Memgraph if you need blazing-fast, in-memory processing.
  • Amazon Neptune if you are fully locked into AWS and need a managed service.
  • Stardog if you need heavy semantic reasoning and W3C RDF/OWL standards.

The Best LLM Integrations / Frameworks

You don't write the extraction and querying logic from scratch; you use an orchestration framework that bridges unstructured text to your graph database.

1. LlamaIndex (Best for Ingestion & Property Graph Construction)
   • Why use it: LlamaIndex offers native PropertyGraphIndex abstractions. It excels at taking raw documents, using an LLM to extract entities and relationships (triplets), and automatically writing them directly into a graph database like Neo4j. It also includes hybrid retrievers that combine vector search with graph traversal.
2. LangChain / LangGraph (Best for Agentic Querying)
   • Why use it: LangChain provides robust Neo4jGraph wrappers and text-to-Cypher chain templates. If you want an LLM to dynamically translate user questions into Cypher queries to fetch exact graph data, LangChain’s graph QA chains are mature and reliable.
3. Microsoft GraphRAG (Best for Global/Holistic Summarization)
   • Why use it: If your goal isn't just looking up specific entity connections, but asking macro-level questions about a massive corpus ("What are the overarching themes across all documents?"), Microsoft's open-source GraphRAG repository automates community detection and hierarchical summarization.

Recommended Tech Stack Blueprint

• Database: Neo4j (Aura Cloud or local Docker container)
• Orchestration & Extraction: LlamaIndex for parsing text and populating Neo4j
• Querying / Agent Layer: LangChain / LangGraph for conversational memory and text-to-Cypher execution if you need dynamic, complex lookups.

If you'd like to get started, tell me:

• What is the approximate volume of text (MBs, GBs, or number of documents)?
• Are you looking for local entity lookups (e.g., "What is connected to X?") or global insights (e.g., "Summarize the themes of the whole dataset")?

I can share a code snippet tailored to your preferred framework.

For building a knowledge graph from unstructured text and querying it with an LLM, the “best” choice depends on whether you optimize for developer speed, enterprise scale, semantic reasoning, or cloud integration. For most teams building a GraphRAG-style system today, I would shortlist these:

Recommended default: Neo4j + LLM orchestration framework

For a new project, my default recommendation is:

Unstructured text → LLM extraction → Neo4j knowledge graph → GraphRAG retrieval → LLM answer generation

Why:

• Mature property graph model (entities + relationships + properties)
• Excellent developer experience with Cypher
• Strong ecosystem around LLM applications
• Good integrations with frameworks such as LangChain and LlamaIndex
• Native vector search plus graph traversal enables hybrid retrieval
• Easier debugging because you can inspect the graph visually and trace why an answer was produced

A typical architecture:

Documents
   |
   v
Chunking + entity/relation extraction
   |
   v
LLM (GPT-5, GPT-4.1, Claude, Gemini, etc.)
   |
   v
Knowledge Graph
   |
   +--> Graph traversal (Cypher)
   |
   +--> Vector similarity search
   |
   v
Context assembly
   |
   v
LLM response

Useful components:

• Graph DB: Neo4j
• Orchestration: LangChain or LlamaIndex
• Embeddings: OpenAI embeddings, Voyage, Cohere, or open-source models
• LLM: GPT-family models, Claude, Gemini, or self-hosted models

Neo4j also provides tooling specifically aimed at constructing knowledge graphs from unstructured sources using LLMs.

Other strong choices

Amazon Neptune — best if you are already on AWS

Choose Neptune when:

• Your stack is heavily AWS-based
• You need managed infrastructure
• You expect very large graphs
• You want integration with AWS AI services

Neptune supports GraphRAG workflows and integrations with Amazon Bedrock and common LLM frameworks.

Tradeoff: the developer experience is generally less straightforward than Neo4j for LLM application development.

Stardog / RDF databases — best for semantic web and reasoning

Choose an RDF/OWL-oriented system when:

• You need ontology reasoning
• You have strict enterprise knowledge modeling requirements
• You need standards-based interoperability

This is common in regulated industries, research, and knowledge management systems.

ArangoDB — best if you want multi-model

Useful when you need:

• Graph + document + key/value models together
• One database for several workloads

LLM integration pattern I recommend

Avoid having the LLM directly “read the graph.” Instead use the graph as a retrieval and reasoning layer.

1. Extract entities and relationships

Example input:

“OpenAI released GPT-4 in March 2023. GPT-4 improved reasoning compared with GPT-3.5.” Extract:

(OpenAI)
   - RELEASED ->
(GPT-4)

(GPT-4)
   - IMPROVED ->
(Reasoning capability)

(GPT-4)
   - COMPARED_WITH ->
(GPT-3.5)

Store metadata:

relationship:
  confidence: 0.91
  source_document: paper_123.pdf
  extracted_by: GPT-model-x

2. Combine graph search + vector search

For a question:

“How did GPT-4 improve over previous models?” Do:

1. Vector search finds relevant documents.
2. Graph traversal finds connected entities.
3. LLM receives: - relevant text chunks

• graph facts
• relationships
• source citations

This is GraphRAG. It helps with multi-hop questions where normal RAG often fails.

3. Use the LLM for query generation carefully

A common pattern:

User question
      |
      v
LLM generates Cypher query
      |
      v
Graph database executes query
      |
      v
Results returned to LLM
      |
      v
Final answer

Add guardrails:

• Validate generated Cypher before execution
• Restrict allowed labels/relationships
• Log queries
• Return source provenance

My practical picks

For most teams, I would start with Neo4j + an LLM extraction pipeline + hybrid GraphRAG retrieval. It gives the shortest path from PDFs/emails/wiki pages to a useful question-answering system while keeping the knowledge structure inspectable.

A good next step is to define your expected scale (documents/day, number of entities, users, and whether you need real-time updates), because that can change the database choice.

A common pattern:

Neo4j is the industry standard graph database , and its native Neo4j LangChain integration combined with the GraphRAG Python package or LlamaIndex represents the best integration stack for turning unstructured text into a queryable knowledge graph with an LLM.

Watch this explanation of how to leverage Neo4j for constructing knowledge graphs from unstructured text:

4m Making LLMs smarter with Knowledge Graphs — Michael, ...51K views  · 1 year ago YouTube · AI Engineer

Top Graph Databases for LLMs

• Neo4j: Offers native vector search, Cypher query generation via LLMs, and dedicated packages (neo4j-graphrag ) to map unstructured text into structured nodes and edges seamlessly.
• Amazon Neptune: Excellent if you are fully locked into AWS; supports property graphs and RDF/SPARQL, with solid integration into AWS Bedrock for LLM execution.
• FalkorDB: A high-performance, graph database built on Redis that uses GraphBLAS, providing extremely fast graph traversals for real-time LLM context injection.
• NetworkX: A pure Python in-memory package ideal for small-scale prototyping or local research, though not suited for production or persistent enterprise data.

Recommended Integration Frameworks

• LangChain & Neo4j: Best for Text-to-Cypher translation, allowing an LLM to dynamically write graph queries based on user questions and fetch precise context.
• LlamaIndex: Best for hierarchical knowledge graph construction, automatically extracting entities and relationships from documents to build index structures for RAG.
• Microsoft GraphRAG: Best for global sensemaking over massive corpuses, using LLMs to first extract a knowledge graph and then build community summaries for holistic querying.

Would you like a sample Python code snippet using LangChain and Neo4j to extract entities from a PDF, or do you prefer an approach focused on Microsoft's GraphRAG framework?

For building a knowledge graph from unstructured text and querying it with an LLM, the “best” choice depends on whether you prioritize developer experience, enterprise scale, ontology/reasoning, or speed. In 2026, the strongest general-purpose stack is usually:

LLM + entity/relation extraction pipeline + graph database + GraphRAG retrieval layer

My default recommendation for most teams:

Neo4j + Neo4j GraphRAG libraries + an LLM (GPT-5-class, Claude, Gemini, or open-source model) + vector index Neo4j Graph Intelligence Platform## Recommended architectures

Option 1: Best overall developer experience — Neo4j

Neo4j is usually the easiest place to start.

Stack:

Documents
   ↓
Chunking + preprocessing
   ↓
LLM extraction
   ↓
Entities + relationships
   ↓
Neo4j knowledge graph
   ↓
GraphRAG retriever
   ↓
LLM answer generation

Why it works well:

• Mature property graph model
• Excellent Cypher query language
• Strong visualization/debugging tools
• Native vector capabilities
• Existing GraphRAG integrations with LLM frameworks
• Good fit for "show me why this answer is true" applications

Neo4j specifically supports combining knowledge graphs, vector search, and LLM frameworks for GraphRAG-style applications.

Good for:

• Enterprise search
• Research assistants
• Internal knowledge bases
• Customer support agents
• Multi-hop reasoning

---

Option 2: Best if you are on AWS — Amazon Neptune

Amazon Neptune is a strong choice if your infrastructure is already AWS-heavy.

Advantages:

• Managed service
• Supports property graphs and RDF
• Integrates with AWS AI services
• Supports GraphRAG patterns and LLM workflows
• Good for very large graphs

AWS provides GraphRAG tooling and integrations with frameworks such as LangChain and LlamaIndex.

Good for:

• Large enterprise deployments
• Compliance-heavy environments
• AWS-native architectures

Tradeoff:

• More operational complexity from an application-development perspective than Neo4j.

---

Option 3: Best for semantic knowledge graphs — RDF databases

If your graph needs formal semantics:

• Ontologies
• OWL reasoning
• Knowledge interchange
• Regulatory or scientific domains

Consider:

• Stardog
• GraphDB
• Amazon Neptune RDF mode

These are better when your question is:

"What does this concept mean according to a formal domain model?" rather than:

"What facts are connected in these documents?"

---

LLM integration pattern

A good production pipeline looks like this:

1. Extract entities

Example input:

"OpenAI released GPT-5 in 2026. Microsoft invested in OpenAI." LLM extraction:

(OpenAI)
   └── released ──> (GPT-5)

(Microsoft)
   └── invested_in ──> (OpenAI)

Store:

(:Company {name:"OpenAI"})
(:Model {name:"GPT-5"})

(OpenAI)-[:RELEASED]->(GPT-5)

---

2. Add embeddings

Do not rely only on graph traversal.

Store embeddings for:

• documents
• chunks
• entities
• relationships

Then retrieval becomes:

User question
      |
      ↓
Vector similarity search
      +
Graph traversal
      |
      ↓
Relevant subgraph
      |
      ↓
LLM

This is usually much better than either pure RAG or pure graph querying.

---

3. Use the LLM for query planning

Example:

User:

"Which companies collaborated with organizations that funded AI safety research?" The LLM can:

1. Convert the question into graph operations
2. Generate Cypher/SPARQL
3. Retrieve matching paths
4. Summarize evidence

---

Framework choices

Easiest stack

Neo4j
+
LangChain
+
OpenAI API
+
Neo4j GraphRAG package

More customizable

Python
+
LlamaIndex
+
Neo4j / Neptune
+
custom extraction prompts
+
vector database

Enterprise AWS stack

Amazon Bedrock
+
Amazon Neptune
+
GraphRAG

AWS Documentation

My practical recommendation by use case

---

A production-quality stack I would build today

LLM:
  GPT-5 / Claude / Gemini

Extraction:
  LLM structured outputs
  + entity resolution
  + relation validation

Storage:
  Neo4j

Retrieval:
  Graph traversal
  +
  vector similarity search

Orchestration:
  LlamaIndex or LangChain

Evaluation:
  - answer faithfulness
  - retrieval precision
  - graph consistency checks

The biggest mistake teams make is treating the graph as just a storage layer. The graph should become a semantic retrieval layer that helps the LLM find connected evidence and explain its reasoning.

If you are starting from scratch, I would choose Neo4j + GraphRAG + an LLM with structured output extraction unless you already have a strong AWS or semantic-web requirement.

Building a GraphRAG (Graph Retrieval-Augmented Generation) pipeline from unstructured text involves two core choices: a graph database to store and query your entities/relationships, and an LLM orchestration framework to parse text into a graph and query it at runtime.

The consensus standard for this stack pairs Neo4j with either LlamaIndex or LangChain.

1. The Best Graph Database: Neo4j

While options like Amazon Neptune, TigerGraph , and Memgraph are powerful, Neo4j is the undisputed industry leader for LLM integration.

• Why it wins: It has the most mature ecosystem for generative AI. Almost every major LLM orchestration framework offers native, production-ready modules specifically written for Neo4j.
• Key feature: It natively supports hybrid search—allowing you to combine traditional vector search (embedding chunks of text) with graph traversal (following entity relationships) in the same database via Cypher queries.

2. The Best LLM Integration Frameworks: LlamaIndex or LangChain

Rather than writing raw prompts to extract entities and build Cypher statements from scratch, use an orchestration framework to handle the heavy lifting:

• LlamaIndex (Best for Knowledge Graph Construction & Advanced RAG):
  • LlamaIndex shines at automatically extracting structured triplets (Entity -> Relationship -> Entity) out of unstructured documents using LLMs and sinking them straight into your graph database.
  • It provides high-level abstractions like SimpleGraphRAG or PropertyGraphIndex that make turning a folder of PDFs into a queryable knowledge graph remarkably fast.
• LangChain (Best for Text-to-Cypher Agents):
  • LangChain features brilliant implementations of Text-to-Cypher chains , allowing an LLM to dynamically look at your graph schema, write a Neo4j Cypher query on the fly based on a user's natural language question, execute it, and synthesize the answer.

Recommended Architecture Blueprint

If you want to dive deeper, let me know:

• What kind of text are you parsing (e.g., technical manuals, legal contracts, or general articles)?
• Do you prefer a managed cloud service (like Neo4j Aura) or a local Docker container to start?

I can share a starter code snippet using Python to get your first pipeline running.

If you're starting today, my default recommendation is:

Neo4j + Neo4j GraphRAG for Python + a strong LLM such as GPT-5 + embeddings + hybrid graph/vector retrieval.

That combination is unusually turnkey for exactly the workflow you described: unstructured text → entities/relationships → knowledge graph → LLM-powered querying.

Recommended architecture

                 ┌─────────────────┐
Documents ──────►│ Chunk / Parse   │
(PDF, HTML, etc.)│                 │
                 └────────┬────────┘
                          │
                          ▼
                 ┌─────────────────┐
                 │      LLM        │
                 │ entity/relation │
                 │   extraction    │
                 └────────┬────────┘
                          │
                          ▼
                 ┌─────────────────┐
                 │     Neo4j       │
                 │                 │
                 │ Entities        │
                 │ Relationships   │
                 │ Chunks          │
                 │ Embeddings      │
                 └────────┬────────┘
                          │
                 ┌────────┴────────┐
                 ▼                 ▼
          Vector / full-text    Cypher /
             retrieval        graph traversal
                 │                 │
                 └────────┬────────┘
                          ▼
                    ┌──────────┐
                    │   LLM    │
                    │ Answer + │
                    │ sources  │
                    └──────────┘

Neo4j's current GraphRAG package actually provides components for most of this: a knowledge-graph builder, vector/hybrid retrievers, Text-to-Cypher, and an LLM-driven tools retriever.

Why I'd pick Neo4j

For this use case, the important thing isn't just "which graph database can store nodes and edges?" It's how much infrastructure you have to build around it.

Neo4j currently gives you:

• Knowledge Graph Builder — extracts entities and relationships from unstructured documents.
• Vector indexes alongside the graph.
• Hybrid retrieval combining vector and full-text search.
• Vector + Cypher retrieval, where semantic matches can be expanded through graph relationships.
• Text2Cypher, allowing an LLM to translate natural-language questions into graph queries.
• ToolsRetriever, which lets an LLM choose between retrieval mechanisms.
• First-party LLM integrations for OpenAI, Anthropic, Gemini/Vertex, Mistral, Cohere, Ollama, etc.

So you're not forced into a big LangChain/LlamaIndex abstraction layer just to get started.

What I'd use for the LLM

For extraction, I'd prioritize structured-output reliability over raw reasoning capability.

For example:

Document
   ↓
LLM
   ↓
{
  entities: [...],
  relationships: [...]
}
   ↓
Neo4j

Neo4j's current KG Builder supports structured output and specifically recommends it with its OpenAI, Vertex AI, and Anthropic integrations because it gives you schema validation rather than relying on the LLM to produce repairable JSON.

For the answering stage, I'd use a more capable model if the questions require multi-hop reasoning.

A good starting stack would therefore be:

Neo4j's own current example uses GPT-5 with text-embedding-3-large, and its GraphRAG package supports both vector and graph-based retrieval.

Don't make it "LLM → Cypher" only

This is one of the biggest architectural decisions I'd make.

You might initially think:

User question
     ↓
LLM generates Cypher
     ↓
Neo4j
     ↓
Answer

That's useful, but insufficient for many real-world questions.

Instead, I'd make the LLM capable of multiple retrieval strategies:

                         User question
                              │
                              ▼
                             LLM
                              │
             ┌────────────────┼────────────────┐
             ▼                ▼                ▼
        Vector search    Graph/Cypher      Full text
             │                │                │
             └────────────────┼────────────────┘
                              ▼
                       Relevant context
                              │
                              ▼
                             LLM
                              │
                              ▼
                           Answer

Neo4j's current ToolsRetriever is designed for essentially this pattern: it can expose vector retrieval and Text2Cypher as tools and let the LLM select the appropriate one.

For example:

"What documents mention Acme's acquisition of Foo?"

→ vector/full-text retrieval may be best.

"Which subsidiaries of Acme were acquired between 2020 and 2024?"

→ Cypher/graph traversal is much better.

"What were the reasons Acme acquired Foo, and which executives were involved?"

→ hybrid retrieval + graph traversal + source chunks.

That's where GraphRAG becomes considerably more interesting than ordinary vector RAG.

Your graph schema matters enormously

I wouldn't let the LLM invent an arbitrary schema for every document.

Start with something like:

(:Document)
    ↓ CONTAINS
(:Chunk)
    ↓ MENTIONS
(:Entity)

(:Person)
(:Company)
(:Product)
(:Technology)
(:Location)
(:Event)

Then relationships such as:

(:Person)-[:WORKS_FOR]->(:Company)
(:Company)-[:ACQUIRED]->(:Company)
(:Person)-[:FOUNDED]->(:Company)
(:Product)-[:USES]->(:Technology)
(:Company)-[:LOCATED_IN]->(:Location)

And importantly, retain provenance:

(:Entity)-[:MENTIONED_IN]->(:Chunk)
(:Relationship)-[:SUPPORTED_BY]->(:Chunk)

That last part is extremely valuable. Your LLM shouldn't merely say "Company A acquired Company B." You want to be able to trace that assertion back to the exact passage that caused it to enter the graph.

Neo4j's KG pipeline explicitly includes a lexical document/chunk graph, entity/relation extraction, pruning, knowledge-graph writing, and entity resolution.

A particularly good production architecture

I'd build it in two layers:

Layer 1 — Knowledge extraction

raw documents
     ↓
parser
     ↓
chunks
     ↓
LLM structured extraction
     ↓
entity resolution
     ↓
Neo4j

Don't skip entity resolution.

If 50 documents say:

IBM
International Business Machines
IBM Corp.
International Business Machines Corporation

you don't want four separate company nodes.

Layer 2 — Question answering

question
   ↓
query classification / planning
   ↓
 ┌─────────────┬──────────────┐
 │             │              │
vector       Cypher        full-text
search       traversal     search
 │             │              │
 └─────────────┴──────────────┘
              ↓
        context + provenance
              ↓
             LLM
              ↓
        answer + citations

This separation is important: the graph is your knowledge layer; the LLM is the reasoning/interface layer.

What about other graph databases?

There are good alternatives.

Amazon Neptune makes a lot of sense if you're already heavily invested in AWS. AWS now positions Neptune as part of its graph/AI stack and describes Amazon Bedrock Knowledge Bases as providing a managed GraphRAG experience using Neptune underneath.

I'd consider it when:

• you're already all-in on AWS,
• IAM/VPC/AWS-native infrastructure is important,
• you want managed AWS integration,
• your graph is likely to become very large.

But for a new project where the primary goal is "turn text into a KG and ask an LLM questions about it," I would still choose Neo4j because the developer experience and GraphRAG-specific tooling are particularly mature.

My recommendation

If I were building your system, I'd start with:

Neo4j Aura
   +
neo4j-graphrag
   +
GPT-5
   +
text-embedding-3-large

Then implement:

1. Schema-constrained entity/relation extraction
2. Entity resolution
3. Document → Chunk → Entity provenance
4. Neo4j vector + full-text indexes
5. Hybrid retrieval
6. Text-to-Cypher for structured questions
7. Graph traversal for multi-hop questions
8. LLM synthesis with citations back to source chunks

That gets you considerably closer to a real knowledge-grounded reasoning system than simply putting embeddings into a graph database.

And you can get a prototype running surprisingly quickly: Neo4j's current SimpleKGPipeline is specifically designed to construct a KG from unstructured data, while its GraphRAG API handles retrieval and answer generation.

If you tell me what kind of documents you're working with (PDFs, technical docs, emails, legal docs, research papers, etc.) and the kinds of questions you want to ask, I can lay out the exact schema, ingestion pipeline, retrieval strategy, and Python code I'd use.

A good starting stack would therefore be:

Neo4j's own current example uses GPT-5 with text-embedding-3-large, and its GraphRAG package supports both vector and graph-based retrieval.

Don't make it "LLM → Cypher" only

This is one of the biggest architectural decisions I'd make.

You might initially think:

User question
     ↓
LLM generates Cypher
     ↓
Neo4j
     ↓
Answer

That's useful, but insufficient for many real-world questions.

Instead, I'd make the LLM capable of multiple retrieval strategies:

                         User question
                              │
                              ▼
                             LLM
                              │
             ┌────────────────┼────────────────┐
             ▼                ▼                ▼
        Vector search    Graph/Cypher      Full text
             │                │                │
             └────────────────┼────────────────┘
                              ▼
                       Relevant context
                              │
                              ▼
                             LLM
                              │
                              ▼
                           Answer

Neo4j's current ToolsRetriever is designed for essentially this pattern: it can expose vector retrieval and Text2Cypher as tools and let the LLM select the appropriate one.

For example:

"What documents mention Acme's acquisition of Foo?"

→ vector/full-text retrieval may be best.

"Which subsidiaries of Acme were acquired between 2020 and 2024?"

→ Cypher/graph traversal is much better.

"What were the reasons Acme acquired Foo, and which executives were involved?"

→ hybrid retrieval + graph traversal + source chunks.

That's where GraphRAG becomes considerably more interesting than ordinary vector RAG.

Your graph schema matters enormously

I wouldn't let the LLM invent an arbitrary schema for every document.

Start with something like:

The industry standard combination for building a knowledge graph from unstructured text and querying it with an LLM is Neo4j paired with its native LangChain or LlamaIndex integrations and the Neo4j Graphiti / LLM Graph Builder toolset. [1][2]

Best Components Overview

• Graph Database: Neo4j is the definitive market leader for LLM-driven knowledge graphs. It provides robust native vector search, Cypher query generation support, and specialized graph-retrieval modules (GraphRAG).
• LLM Integration Framework: LangChain or LlamaIndex offer pre-built LLMGraphTransformer modules that automatically parse text chunks, extract nodes and relationships using schema constraints, and write them directly into your database.

Recommended Tech Stack Breakdown

Step-by-Step Implementation Workflow

If you'd like, let me know:

• What type of source text you are using (e.g., medical records, legal contracts, technical manuals)
• Whether you prefer Python code examples using LangChain or LlamaIndex

I can provide a customized implementation script for your project.

If your goal is “take a pile of unstructured documents → extract a useful knowledge graph → let an LLM answer questions using both graph structure and source text,” my default recommendation today is:

Neo4j + Neo4j GraphRAG for Python + a strong structured-output LLM (e.g. OpenAI) + embeddings. Neo4j has unusually good end-to-end support for this exact workflow: its official GraphRAG package can construct the KG, store embeddings, perform vector/full-text/graph retrieval, and translate natural-language questions into Cypher.

Recommended architecture

                    ┌─────────────────┐
                    │ Unstructured    │
                    │ documents       │
                    │ PDF / HTML / DB │
                    └────────┬────────┘
                             │
                             ▼
                   ┌──────────────────┐
                   │ Chunk + parse    │
                   └────────┬─────────┘
                            │
                            ▼
                 ┌──────────────────────┐
                 │ LLM entity/relation  │
                 │ extraction            │
                 └──────────┬───────────┘
                            │
                            ▼
                    ┌───────────────┐
                    │    Neo4j      │
                    │               │
                    │ Entities      │
                    │ Relationships │
                    │ Chunks        │
                    │ Embeddings    │
                    └───────┬───────┘
                            │
                 ┌──────────┴───────────┐
                 │                      │
          Vector / text search     Cypher traversal
                 │                      │
                 └──────────┬───────────┘
                            ▼
                     ┌─────────────┐
                     │     LLM     │
                     │   answer    │
                     └─────────────┘

The important part is not to treat the graph as merely another vector database. You want the LLM to be able to choose between semantic retrieval and explicit graph traversal.

Neo4j's current GraphRAG package provides VectorRetriever, HybridRetriever, Text2Cypher, and tool-based retrieval, so you can combine those approaches.

Why I'd choose Neo4j

Neo4j has a particularly strong developer experience here. Its current package includes a dedicated Knowledge Graph Builder that handles chunking, schema construction, entity/relation extraction, graph writing, and entity resolution.

LLM choice

I'd separate the LLM jobs rather than assuming one model must do everything.

1. Extraction LLM

Use a model that's very good at structured output.

For example:

Document
   ↓
LLM
   ↓
{
  entities: [...],
  relationships: [...],
  claims: [...]
}

This is important because your graph quality is largely determined during extraction.

Neo4j specifically recommends structured output for its KG extraction pipeline because it gives you schema validation rather than relying on fragile JSON parsing. Currently its structured-output KG extraction supports OpenAI, Vertex AI, and Anthropic integrations.

I'd start with something like GPT-5-class reasoning/capable models for difficult extraction, and use a cheaper model for straightforward bulk extraction if your corpus is large.

2. Query/answer LLM

For answering questions, you want the LLM to be able to do something like:

User:
"Which companies acquired startups founded by former Google
employees, and which acquisitions happened after 2020?"

             ↓

LLM decides:
  ├── semantic search
  ├── Cypher query
  └── possibly source-document retrieval

             ↓

Neo4j

             ↓

Structured evidence

             ↓

LLM

             ↓

Answer + citations

Neo4j's Text2CypherRetriever already supports the important middle piece: translating a natural-language question into Cypher and executing it against the graph. Its ToolsRetriever can combine Cypher retrieval with vector retrieval and let an LLM select the appropriate tool.

That hybrid approach is what I'd recommend.

Don't build a pure "Text → triples" graph

A common mistake is:

Document
 ↓
LLM
 ↓
(Person)-[:WORKS_FOR]->(Company)

and then throwing the original document away.

Instead, I'd build a lexical/source layer too:

(Document)
    │
    ├── HAS_CHUNK ──> (Chunk)
    │                    │
    │                    ├── MENTIONS ──> (Person)
    │                    ├── MENTIONS ──> (Company)
    │                    └── embedding
    │
    └── metadata

(Person)-[:WORKS_FOR]->(Company)
(Person)-[:FOUNDED]->(Company)
(Company)-[:ACQUIRED]->(Company)

That gives you both:

• graph reasoning: "A acquired B, B was founded by X"
• semantic retrieval: "find passages discussing X's role at B"
• provenance: "where did this relationship come from?"

Neo4j's KG Builder explicitly supports a lexical graph containing documents/chunks alongside extracted entities and relationships.

The retrieval layer I'd actually implement

I'd make the LLM's tools roughly:

tools = [
    semantic_search,
    graph_query,
    document_search,
]

Semantic search

For questions such as:

"What did the company say about its strategy?" retrieve relevant chunks using embeddings.

Graph query

For questions such as:

"Which companies are subsidiaries of X?" generate Cypher:

MATCH (x:Company {name: $company})
      -[:OWNS*1..3]->
      (subsidiary:Company)
RETURN subsidiary.name

Hybrid

For questions such as:

"Why did X acquire Y?" first find the relevant entities/relationships, then retrieve the source passages explaining the acquisition.

This is considerably more robust than asking the LLM to answer from graph triples alone.

Neo4j's current integration is particularly convenient

The official package can be installed with:

pip install "neo4j-graphrag[openai]"

and provides first-party integrations with OpenAI, Anthropic, Vertex AI, Mistral, Cohere and Ollama, among others.

A simplified query setup looks like:

from neo4j import GraphDatabase
from neo4j_graphrag.llm import OpenAILLM
from neo4j_graphrag.generation import GraphRAG
from neo4j_graphrag.embeddings import OpenAIEmbeddings
from neo4j_graphrag.retrievers import HybridCypherRetriever

driver = GraphDatabase.driver(
    NEO4J_URI,
    auth=(NEO4J_USER, NEO4J_PASSWORD)
)

llm = OpenAILLM(model_name="gpt-5")

embedder = OpenAIEmbeddings(
    model="text-embedding-3-large"
)

# Configure hybrid/vector/Cypher retrieval here
# ...

rag = GraphRAG(
    retriever=retriever,
    llm=llm
)

response = rag.search(
    query_text="What companies did Alice's company acquire?"
)

print(response.answer)

The official Neo4j examples use essentially this architecture, including OpenAI embeddings and LLMs.

When I'd choose something else

Choose Amazon Neptune if...

Your organization is already heavily AWS-centric and you want a managed AWS graph service. Neptune supports property-graph querying through openCypher/Gremlin as well as RDF/SPARQL, and AWS now offers managed GraphRAG capabilities through Bedrock Knowledge Bases with Neptune.

In that situation:

AWS
 ├── S3
 ├── Bedrock
 ├── Neptune
 └── IAM

can be very attractive.

Consider Microsoft's GraphRAG methodology if...

Your problem is primarily discovering global themes across a huge corpus, rather than performing precise entity/relationship queries.

Microsoft's GraphRAG approach builds a graph, organizes entities into communities, summarizes those communities, and uses the hierarchy for retrieval.

However, there's an important current caveat: the Microsoft GraphRAG repository says the project is largely in maintenance mode rather than continuing as an actively developed product.

So I wouldn't make it the foundation of a new production platform unless its particular algorithmic approach is what you need.

My stack recommendation

For a new project I'd start with:

Storage
    Neo4j Aura / Neo4j

KG construction
    neo4j-graphrag
    structured-output LLM

LLM
    OpenAI GPT-5-class model
    ↓
    extraction + query planning + final answer

Embeddings
    OpenAI text-embedding-3-large
    or your preferred embedding model

Retrieval
    ├── Neo4j vector search
    ├── Neo4j full-text search
    ├── Cypher
    └── hybrid/tool-based retrieval

Application
    Python
    FastAPI
    optional LangGraph/LangChain

The key architectural decision is hybrid GraphRAG rather than graph-only RAG:

LLM → decide how to retrieve → graph + vector + source text → LLM → cited answer That gives you the best combination of structured reasoning, semantic retrieval, and provenance.

If you tell me roughly what kind of documents you're processing (PDFs, contracts, research papers, technical docs, CRM data, etc.), corpus size, and the kinds of questions users will ask, I can propose the actual Neo4j schema + ingestion pipeline + retrieval architecture + LLM prompts I'd use.

Semantic search

For questions such as:

"What did the company say about its strategy?" retrieve relevant chunks using embeddings.

Graph query

For questions such as:

"Which companies are subsidiaries of X?" generate Cypher:

MATCH (x:Company {name: $company})
      -[:OWNS*1..3]->
      (subsidiary:Company)
RETURN subsidiary.name

Hybrid

For questions such as:

"Why did X acquire Y?" first find the relevant entities/relationships, then retrieve the source passages explaining the acquisition.

This is considerably more robust than asking the LLM to answer from graph triples alone.

Neo4j's current integration is particularly convenient

The official package can be installed with:

pip install "neo4j-graphrag[openai]"

If you're building a knowledge graph from unstructured text and want an LLM to query it, my default choice today would be:

Neo4j + its official GraphRAG Python package + a strong LLM (e.g. GPT-5-class model) + embeddings in Neo4j. It's the most straightforward end-to-end stack, particularly if you want both semantic retrieval and explicit graph reasoning.

Recommended architecture

                    ┌─────────────────┐
PDFs / Docs / Text ─►│ Chunk + Parse    │
                    └────────┬────────┘
                             │
                             ▼
                    ┌─────────────────┐
                    │ LLM extraction  │
                    │ entities +      │
                    │ relationships   │
                    └────────┬────────┘
                             │
                             ▼
                    ┌──────────────────────────┐
                    │          Neo4j           │
                    │                          │
                    │ Entities / Relationships │
                    │ Documents / Chunks       │
                    │ Vector embeddings        │
                    └────────────┬─────────────┘
                                 │
                    ┌────────────┴────────────┐
                    ▼                         ▼
             Vector / hybrid             Cypher /
                retrieval                graph traversal
                    │                         │
                    └────────────┬────────────┘
                                 ▼
                          ┌─────────────┐
                          │     LLM     │
                          │ synthesis   │
                          └──────┬──────┘
                                 ▼
                              Answer

Why Neo4j

The important thing is that you don't have to choose between a vector database and a graph database.

Neo4j's current GraphRAG stack supports:

• Knowledge-graph construction from unstructured text
• Entity and relationship extraction
• Entity resolution
• Vector indexes
• Full-text/hybrid retrieval
• Graph traversal after vector retrieval
• Text-to-Cypher
• LLM-driven tool selection
• Multiple LLM providers

The official neo4j-graphrag package is specifically designed for this workflow.

Neo4j's retrievers include VectorRetriever, VectorCypherRetriever, HybridRetriever, HybridCypherRetriever, and Text2Cypher. That combination is particularly useful because different questions require different retrieval strategies.

The key design decision: don't make the LLM "just search the graph"

I'd build hybrid GraphRAG.

For example, suppose your documents contain:

"Acme acquired BetaCorp in 2022. BetaCorp previously supplied components to Gamma Industries." A user asks:

"Which companies acquired suppliers of Gamma Industries?" Pure vector RAG can struggle because the answer requires following relationships:

Acme
  │
  └── ACQUIRED ──► BetaCorp
                       │
                       └── SUPPLIES ──► Gamma Industries

The graph can answer this deterministically.

But a question such as:

"What were the main reasons mentioned for the acquisition?" may be better answered by retrieving the relevant source chunks semantically.

So I'd use:

vector retrieval → graph expansion → LLM synthesis

rather than either:

vector RAG only

or:

LLM → arbitrary Cypher for everything

Neo4j's VectorCypherRetriever and hybrid retrievers are designed for exactly this combination.

For graph construction

I'd use an LLM to extract a controlled schema, rather than letting it invent arbitrary node and relationship types.

For example:

Person
Company
Product
Technology
Document
Event
Location

and:

WORKS_FOR
OWNS
ACQUIRED
PRODUCES
USES
LOCATED_IN
MENTIONS
PART_OF

Then have the extraction pipeline produce something like:

{
  "nodes": [
    {"id": "acme", "type": "Company", "name": "Acme"},
    {"id": "betacorp", "type": "Company", "name": "BetaCorp"}
  ],
  "relationships": [
    {
      "source": "acme",
      "type": "ACQUIRED",
      "target": "betacorp",
      "year": 2022
    }
  ]
}

This is much more reliable than asking an LLM to freely construct a graph.

Neo4j's current KG builder explicitly supports schema building, entity/relation extraction, graph pruning, and entity resolution. Its structured-output mode can also validate the extracted graph structure.

Which LLM?

For extraction, I'd prioritize:

1. Strong structured-output support
2. Good entity/relation extraction
3. Low hallucination rate
4. Reasonable cost

For answer generation / complex reasoning, use your strongest model.

You don't necessarily need the same model for both.

For example:

                    ┌──► smaller/cheaper LLM
                    │    entity extraction
Documents ──────────┤
                    │
                    └──► embeddings

Question ──► retrieval ──► graph ──► strong LLM ──► answer

Neo4j's current package has first-party interfaces for OpenAI, Anthropic, Google/Vertex, Mistral, Cohere, Ollama and others, so you're not locked into one provider.

Neo4j vs. alternatives

Memgraph is probably the strongest alternative I'd evaluate. Its current GraphRAG approach combines vector search and graph traversal in the graph engine and has integrations with LangChain, LlamaIndex and LightRAG.

Neptune is attractive if you're already an AWS shop, particularly for very large managed graph workloads. It supports property graphs through openCypher/Gremlin as well as RDF/SPARQL. AWS Documentation But I'd generally expect more integration work for an LLM-centric application than with Neo4j's dedicated GraphRAG tooling.

What I'd actually build

For a new project, my stack would be:

Python
  │
  ├── Document parsing
  │
  ├── neo4j-graphrag
  │      │
  │      ├── KG Builder
  │      ├── entity/relation extraction
  │      ├── embeddings
  │      ├── hybrid retrieval
  │      └── Text2Cypher
  │
  ├── Neo4j
  │      ├── knowledge graph
  │      ├── document/chunk graph
  │      └── vector indexes
  │
  └── LLM
         ├── extraction
         └── answer synthesis

The official package is installable as neo4j-graphrag, and Neo4j documents a complete KG-builder → retrieval → LLM workflow.

One important caveat

I would not start by building an elaborate autonomous GraphRAG agent.

Start with:

1. Well-defined graph schema
2. Reliable extraction
3. Entity resolution
4. Hybrid retrieval
5. Deterministic Cypher for important query patterns
6. LLM synthesis with citations back to source chunks

Then add agentic Text2Cypher/tool selection once you have evaluation data.

That tends to produce a system that's much easier to debug and trust.

If you tell me roughly what kind of documents/data you're ingesting and the types of questions users need to ask, I can recommend a concrete schema and retrieval architecture (including the Neo4j model, extraction prompts, and query flow).

Pure vector RAG can struggle because the answer requires following relationships:

Acme
  │
  └── ACQUIRED ──► BetaCorp
                       │
                       └── SUPPLIES ──► Gamma Industries

The graph can answer this deterministically.

But a question such as:

"What were the main reasons mentioned for the acquisition?" may be better answered by retrieving the relevant source chunks semantically.

So I'd use:

vector retrieval → graph expansion → LLM synthesis

rather than either:

vector RAG only

or:

LLM → arbitrary Cypher for everything

Neo4j's VectorCypherRetriever and hybrid retrievers are designed for exactly this combination.

For graph construction

I'd use an LLM to extract a controlled schema, rather than letting it invent arbitrary node and relationship types.

For example:

Person
Company
Product
Technology
Document
Event
Location

and:

WORKS_FOR
OWNS
ACQUIRED
PRODUCES
USES
LOCATED_IN
MENTIONS
PART_OF

Then have the extraction pipeline produce something like:

{
  "nodes": [
    {"id": "acme", "type": "Company", "name": "Acme"},
    {"id": "betacorp", "type": "Company", "name": "BetaCorp"}
  ],
  "relationships": [
    {
      "source": "acme",
      "type": "ACQUIRED",
      "target": "betacorp",
      "year": 2022
    }
  ]
}

This is much more reliable than asking an LLM to freely construct a graph.

Neo4j's current KG builder explicitly supports schema building, entity/relation extraction, graph pruning, and entity resolution. Its structured-output mode can also validate the extracted graph structure.

Which LLM?

For extraction, I'd prioritize:

Building a knowledge graph (KG) from unstructured text and querying it with an LLM—commonly referred to as GraphRAG —has a clear industry-standard stack.

1. The Best Graph Database: Neo4j

• Why it wins: Neo4j is the undisputed leader and most widely adopted graph database for LLM and AI workflows. It has native vector search capabilities, mature Python drivers, and extensive first-party support across every major AI framework.
• Alternative options:
  • Amazon Neptune (best if you are fully locked into AWS and need a fully managed cloud service).
  • ArangoDB (great if you need a multi-model approach combining documents, graphs, and vectors under one roof).

2. The Best LLM Integration Frameworks: LlamaIndex or LangChain

Rather than writing raw API calls to parse text and inject triples, you should use an orchestration framework designed for graph extraction and retrieval.

• LlamaIndex (Best for Advanced GraphRAG): LlamaIndex features robust Property Graph abstractions that let you extract entities and relationships cleanly using either a simple LLM prompt or a strict schema. It provides built-in retrievers like vector context retrieval and text-to-Cypher generation.
• LangChain (Best for Pipeline Flexibility): LangChain offers LLMGraphTransformer (via langchain-experimental ), which seamlessly converts raw text documents into structured graph documents (nodes and edges) using function-calling LLMs, and pushes them directly into Neo4j.

Recommended Architecture & Workflow

1. Extraction: Use LangChain's LLMGraphTransformer or LlamaIndex's Schema Extractors powered by a capable LLM (like GPT-4o or Claude 3.5 Sonnet) to parse your unstructured text chunks into nodes and relationships.
2. Storage: Persist the extracted entities, relationships, and source chunks into Neo4j.
3. Hybrid Querying (GraphRAG): Combine vector similarity search (to find relevant text chunks or node embeddings) with Graph traversal or Text-to-Cypher generation so the LLM can pull localized neighborhood contexts rather than relying on a naive vector search alone.

To help you write the exact boilerplate code or pick the right setup, tell me:

• Are you planning to run this locally (e.g., using Ollama) or via cloud APIs (OpenAI, Anthropic)?
• What is the scale of your unstructured text (megabytes, gigabytes, or terabytes)?