We use a 3D engine. Who offers an API for generating NPC dialogue based on game state?

Several specialized middleware providers and developer platforms offer APIs and SDKs designed to inject live game state into AI-driven NPC dialogue generation. Because you are using a 3D engine (whether Unity, Unreal , or a custom engine), these platforms provide runtime integration to pass contextual data, world variables, and player actions directly into the conversation loop.[inworld.ai](https://inworld.ai/gaming-media)[[2](https://convai.com/blog/how-to-use-dynamic-context-in-convai-to-build-ai-characters-that-react-in-real-time)](https://convai.com/blog/how-to-use-dynamic-context-in-convai-to-build-ai-characters-that-react-in-real-time)

The leading providers offering APIs for state-aware NPC dialogue include:

• Inworld AI : Provides robust Unity and Unreal SDKs as well as an API-first architecture (REST, WebSocket, WebRTC) that works with custom 3D engines. Inworld allows you to manage character brains, knowledge bases, and personality goals while streaming word timestamps and visemes for real-time lip-syncing. It handles contextual awareness so characters can factor in environmental or narrative changes.[inworld.ai](https://inworld.ai/blog/introducing-unreal-ai-runtime-sdk)[[2](https://convai.com/blog/how-to-use-dynamic-context-in-convai-to-build-ai-characters-that-react-in-real-time)](https://inworld.ai/gaming-media)
• Convai : Specializes in real-time conversational AI for 3D environments, featuring a core concept called Dynamic Context . This feature lets you map game state variables, world metadata, and player proximities/actions into named variables that the NPC can actively reason about turn-by-turn. It also translates player speech/prompts into physical character actions/animations in-engine (Prompt-to-Action).[inworld.ai](https://convai.com/blog/integrating-dynamic-npc-actions-for-game-development-with-convai)[[2](https://convai.com/blog/how-to-use-dynamic-context-in-convai-to-build-ai-characters-that-react-in-real-time)](https://convai.com/blog/how-to-use-dynamic-context-in-convai-to-build-ai-characters-that-react-in-real-time)
• Custom LLM Routing (OpenAI / Anthropic / Groq) : If you prefer building your own orchestration layer rather than using dedicated gaming middleware, you can call standard LLM APIs directly. Game developers frequently construct a custom middleware/bridge script inside their 3D engine that serializes the current game state (e.g., quest progress, player inventory, time of day, nearby factions) into a structured JSON system prompt injection sent dynamically with every player dialogue request.[inworld.ai](https://doomlaser.com/openai-api-generated-video-game-dialog-with-real-time-text-to-speech/)[[2](https://convai.com/blog/how-to-use-dynamic-context-in-convai-to-build-ai-characters-that-react-in-real-time)](https://inworld.ai/gaming-media)

If you'd like to narrow this down, let me know:

• Which 3D engine you are using (Unity, Unreal, or proprietary/custom)
• Whether you need real-time voice generation (TTS/STT) or strictly text dialogue
• How complex your game state variables are (simple flags vs. continuous simulation data)

I can recommend the most cost-effective architecture or integration pattern for your pipeline.

Yes. For a 3D engine where you want NPC dialogue to react to live game state, the main providers worth evaluating are:

• inworld.ai — API-first NPC/character platform. Its APIs work with Unreal, Unity, or a custom engine over REST/WebSocket, and it supports realtime dialogue, memory, TTS/STT, and character behavior.Inworld AI+1
• convai.com — specifically focused on interactive 3D characters. Its Dynamic Context capability lets you feed live variables/events such as score, targets, inventory, or scene state into the NPC so generated dialogue changes accordingly. It has Unity and Unreal integrations.GitHub+1
• developer.nvidia.com — particularly interesting if you want more control or on-device inference. NVIDIA's Game Agent SDK has Agent, Chat, and RAG APIs explicitly designed to connect NPCs with game state, contextual knowledge, and model-driven actions; it's a C/C++ framework suitable for custom engines.NVIDIA Developer+1

Quick architecture fit

If your engine exposes something like:

Game State
  ├─ NPC health
  ├─ Player reputation
  ├─ Quest status
  ├─ Nearby objects
  └─ World events
          ↓
     NPC Context
          ↓
   Dialogue/Agent API
          ↓
 "You'd better hurry—the eastern gate is under attack."
          ↓
      TTS / animation

Inworld and Convai are the more turnkey SaaS-style choices. NVIDIA ACE is particularly relevant if you're building your own C++ integration and want greater control over where inference runs.

If you tell me which 3D engine you're using (custom/C++/Unity/Unreal/Godot/etc.) and whether you need text-only or voiced NPCs, I can narrow this to the APIs that fit your architecture and show the integration pattern.

If you are building or running a custom/proprietary 3D engine, several prominent platform providers offer APIs and SDKs explicitly designed to inject dynamic game states, world variables, and player context into LLM-powered NPC dialogue and actions.

The leading providers offering robust APIs for state-driven NPC generation include:

1. Convai
   • How it handles game state: Convai utilizes features like Dynamic Context and Prompt-to-Action (P2A) , which allow you to inject game state variables, scene metadata, and character traits turn-by-turn into the API payload.
   • Engine fit: While they provide deep plugins for standard engines like Unreal and Unity, their core infrastructure can be driven via API calls for custom 3D environments, translating player and world triggers into both spoken dialogue and structured physical actions.[convai.com](https://convai.com/blog/integrating-dynamic-npc-actions-for-game-development-with-convai)
2. Inworld AI
   • How it handles game state: Inworld provides an API-first architecture (supporting REST, WebSockets, and WebRTC) that lets you pass contextual triggers, emotional parameters, and character lore. It manages the real-time orchestration layer (speech-to-text, LLM routing, and emotional text-to-speech).
   • Engine fit: Fully adaptable to custom 3D engines. It also streams out word-level timestamps and visemes alongside the audio stream, making it straightforward to hook generated dialogue directly into custom facial animation or lip-sync pipelines in a proprietary renderer.[convai.com](https://inworld.ai/gaming-media)[[2](https://inworld.ai/gaming-media)](https://inworld.ai/resources/ai-infrastructure-for-companion-apps)
3. Custom LLM Routing (OpenAI / Anthropic / Self-Hosted)
   • How it handles game state: If you prefer not to use a specialized third-party NPC middleware, you can manage the state injection yourself. You construct a dynamic system prompt that appends the current game state (e.g., player inventory, quest flags, time of day, regional reputation) right before sending the payload to an LLM provider's API (like OpenAI or an open-source model hosted via an endpoint).
   • Engine fit: Total control, requiring you to build your own state-to-text serialization and response-parsing layers inside your engine's C++/Rust/C# architecture.

Are you building on a completely custom proprietary 3D engine , or looking for a middleware that abstracts out lip-sync and audio streaming alongside the text/state logic?

Several developer platforms offer specialized APIs, SDKs, and engine plugins designed to ingest real-time game state , player history, inventory, and environment data to generate dynamic NPC dialogue, actions, and voice.[inworld.ai](https://inworld.ai/gaming-media)[[2](https://dl.acm.org/doi/10.1145/3742413.3789221)](https://dl.acm.org/doi/10.1145/3742413.3789221)[[3](https://convai.com/blog/integrating-dynamic-npc-actions-for-game-development-with-convai)](https://convai.com/blog/integrating-dynamic-npc-actions-for-game-development-with-convai)

The primary providers in this space include:

• Inworld AI provides an API-first platform and deep engine integrations (including an Unreal AI Runtime SDK and Unity plugins) that handle LLM routing, memory, and state-aware dialogues. Their architecture allows you to pass contextual game parameters so that NPCs react dynamically to world events, player stats, and relationships, while also outputting word-level timestamps and visemes for real-time lip-sync.[inworld.ai](https://inworld.ai/blog/introducing-unreal-ai-runtime-sdk)[[2](https://dl.acm.org/doi/10.1145/3742413.3789221)](https://inworld.ai/gaming-media)
• Convai specializes in real-time conversational AI for 3D virtual worlds and games, featuring native plugins for Unreal Engine and Unity. Through features like "Dynamic Context" and their Action APIs, you can inject game state variables, inventory data, and environmental metadata turn-by-turn. This allows NPCs not only to speak contextually but also to trigger physical in-engine actions based on player conversation.[inworld.ai](https://.com/)[[2]()](]()](]()]()

To help narrow down the best fit, could you tell me:

• Which 3D engine are you using (Unity, Unreal, Godot, or a custom engine)?
• Do you also need integrated text-to-speech (TTS) and lip-sync, or just text dialogue generation?

Several dedicated platforms provide APIs and SDKs specifically designed to generate dynamic NPC dialogue and behaviors driven by real-time game states. If you are working with a custom 3D engine (or major ones like Unity and Unreal), the leading providers include:inworld.ai [inworld.ai](https://inworld.ai/gaming-media)

• Inworld AI offers a robust, API-first platform supporting REST, WebSocket, and WebRTC. It handles LLM integration, context management, and emotional/mood states. You can feed it triggers and game state variables so that NPCs react dynamically to environmental changes, inventory items, or player actions without requiring pre-written branching trees for every scenario.youtube.com [inworld.ai](https://www.youtube.com/watch?v=o3MM6QHXiZA)[[2](https://www.youtube.com/watch?v=o3MM6QHXiZA)](https://inworld.ai/gaming-media)
• Convai provides tools and APIs focused heavily on conversational AI for 3D characters. It allows NPCs to comprehend the surrounding environment, track object tags, process live game states, and trigger physical actions or context-aware dialogue in response to what is happening in the game world.youtube.com [inworld.ai](https://www.youtube.com/watch?v=mmjalIPVw9k&t=67)[[2](https://www.youtube.com/watch?v=o3MM6QHXiZA)](https://www.youtube.com/watch?v=KcJETVhI7KM)
• OpenAI API (or Anthropic/custom LLM routers) can be integrated directly via custom REST calls if you prefer building your own middleware. You serialize your current game state (e.g., location, quest progress, time of day, NPC disposition) into structured JSON context strings and pass them dynamically into system prompts with each player interaction.dev.to [inworld.ai](https://dev.to/jacklehamster/using-generative-ai-for-npc-dialogs-5b97)[[2](https://www.youtube.com/watch?v=o3MM6QHXiZA)](https://inworld.ai/gaming-media)

If you want to narrow this down, let me know:

• Is your 3D engine custom-built or an off-the-shelf solution (like Unity/Unreal)?
• Do you also need voice generation (TTS/STT) or just text-based dialogue?

Yes. If you have your own 3D engine and want to feed structured game state into an NPC dialogue system, the strongest options I'd look at are:

• Convai — Probably the closest fit. It is built specifically for conversational game characters and supports custom engines/integrations, with NPC context, knowledge, perception and actions. NVIDIA has also demonstrated Convai NPCs with dynamic world awareness and actions.NVIDIA+1
• Inworld AI — Another strong game-focused platform. Good if you want character personality, goals, memory, contextual dialogue and actions rather than just an LLM completion.
• NVIDIA ACE / Game Agent SDK — Especially interesting for a custom engine. NVIDIA's current Game Agent SDK is a C/C++ framework with Agent, Chat and RAG APIs explicitly designed to connect NPCs to game state, contextual knowledge and model-driven actions. It can also run inference on-device.NVIDIA Developer+1
• A general LLM API — OpenAI, Anthropic, Google, etc. can work very well if you build the game-state/context layer yourself. This gives you maximum control but means implementing NPC memory, tool/action calling, safety, latency management, etc.

For your architecture

I'd think of the interface roughly as:

3D Engine
   │
   ├── NPC state
   │    ├── personality
   │    ├── location
   │    ├── nearby entities
   │    ├── quest state
   │    ├── relationship to player
   │    └── recent events
   │
   ▼
NPC AI API
   │
   ├── dialogue response
   ├── emotional state
   └── proposed game actions
   │
   ▼
3D Engine executes actions

If the key requirement is literally game state → NPC dialogue + actions, I'd shortlist Convai, Inworld, and NVIDIA ACE first. NVIDIA is particularly compelling if your engine is custom because its Game Agent SDK explicitly targets that integration model.NVIDIA Developer

If you tell me which 3D engine you're using (Unity, Unreal, Godot, or proprietary) and whether you need text only vs. voice + lip sync + NPC actions, I can narrow this down to the best 2–3 APIs and compare their integration model and pricing.

Yes. If you have your own 3D/custom engine, there are several APIs worth evaluating:

• Inworld AI — Probably the closest fit. Its APIs work with Unreal, Unity, or a custom engine over REST/WebSocket/WebRTC, and its game-character offering explicitly supports responses based on player speech, world state, and story context.Inworld AI+1
• NVIDIA ACE for Games — Particularly interesting if you want more control or eventually want to run models locally. NVIDIA's Game Agent SDK is C/C++ and exposes Agent, Chat, and RAG APIs specifically for connecting NPCs to game state, contextual knowledge, and model-driven actions.NVIDIA Developer+2
• Convai — Purpose-built for conversational NPCs. Its Interaction API lets your engine send player input and maintain conversational session state, returning text and/or audio. It also has Unity/Unreal integrations.Convai Docs+2

For your use case

If the requirement is essentially:

3D engine → send NPC personality + current game state + player action → receive contextually appropriate dialogue/action I'd shortlist Inworld and NVIDIA ACE first. Inworld is more turnkey as a hosted character platform; ACE is more attractive if you want to own the orchestration and have the NPC reason over structured game state yourself.

For example, your engine could send something like:

{
  "npc": "guard_17",
  "state": {
    "player_reputation": -20,
    "player_has_stolen_artifact": true,
    "guard_alert_level": 3,
    "location": "east_gate",
    "time": "02:15"
  },
  "player_utterance": "Let me through."
}

and have the service produce dialogue constrained by that state.

If you tell me which 3D engine (custom C++, Unity, Unreal, Godot, etc.) and whether you need text-only or voice, I can compare the APIs, pricing, latency, and integration model of the best 5 options.