We need to build a system where agents can dynamically discover and use new tools. What's the best framework for dynamic tool discovery?

If the core requirement is “agents should discover tools at runtime rather than having every tool hard-coded into their prompt,” I’d build around Model Context Protocol (MCP), with a separate tool-discovery/search layer once the catalog gets large.

Recommended architecture

MCP + semantic tool registry + agent framework

                    ┌──────────────────────┐
                    │   Tool Registry      │
                    │ name / desc / schema │
                    │ permissions / tags    │
                    └──────────┬───────────┘
                               │
                         semantic search
                               │
┌─────────────┐       ┌────────▼────────┐
│    Agent    │──────►│ Discovery Layer │
│             │◄──────│ "find tools..."  │
└──────┬──────┘       └────────┬────────┘
       │                       │
       │ dynamically loads     │
       ▼                       ▼
┌─────────────────────────────────────────┐
│              MCP Servers                │
│ CRM │ DB │ GitHub │ Search │ Payments │ …│
└─────────────────────────────────────────┘

Why MCP? It gives you a standardized mechanism for servers to expose tools and for clients to discover them through tools/list; the current MCP specification also supports pagination and caching-oriented deterministic tool ordering.

The important distinction

I wouldn't make MCP itself your entire discovery system.

There are really three layers:

1. Protocol — MCP

• How an agent connects to a tool provider.
• How tools describe their input schemas.
• How the agent invokes them.
• Handles transports, authorization, etc.

2. Discovery — registry/search

• "What tools exist?"
• "Which tools are relevant to this task?"
• Semantic search over descriptions, capabilities, tags, examples, permissions, etc.
• This prevents dumping hundreds/thousands of tool schemas into the context.

3. Execution/orchestration — your agent framework

• Planning.
• Tool selection.
• Calling tools.
• Retries/parallelism.
• Guardrails and observability.

AWS's current guidance explicitly distinguishes static MCP registration, dynamic discovery, and semantic search, and recommends filtering/searching when the tool catalog becomes large because tool definitions can consume substantial context.

What I'd choose today

The MCP ecosystem now has an official registry and supports organizational sub-registries, which makes the registry model increasingly practical.

For a large-scale system, use progressive disclosure

Don't do this:

Agent context:
  1,500 tool definitions
  ↓
  LLM chooses one

Instead:

User request
     ↓
"find tools relevant to: refund a customer's order"
     ↓
Discovery service
     ↓
5 candidate tools
     ↓
load full schemas
     ↓
LLM selects tool
     ↓
MCP invocation

This is increasingly becoming a standard pattern: OpenAI's current agent tooling includes tool search that loads relevant tool definitions as needed, and its Agents SDK explicitly supports MCP plus progressive disclosure.

One thing I'd add beyond MCP

Give every tool a capability descriptor, something like:

{
  "name": "refund_order",
  "description": "Issue a refund for an existing customer order",
  "capabilities": [
    "commerce",
    "orders",
    "refunds"
  ],
  "risk": "high",
  "requires": [
    "orders:read",
    "orders:refund"
  ],
  "examples": [
    "refund order 12345",
    "give customer a full refund"
  ],
  "side_effects": true,
  "version": "2.1"
}

Then your discovery service can combine:

semantic similarity
+ capability filters
+ authorization
+ tenant
+ environment
+ risk policy
+ version compatibility
+ historical success rate

That is much more robust than relying solely on an LLM reading tool descriptions.

Framework choice

If you're asking “which framework should we standardize on?”, my answer would be:

MCP as the tool interface, rather than treating LangChain/LangGraph/OpenAI Agents/etc. as the discovery protocol.

Then pick the agent runtime separately.

For example:

                  Agent Runtime
              ┌──────────────────┐
              │ OpenAI Agents SDK│
              │ / LangGraph      │
              │ / custom loop    │
              └────────┬─────────┘
                       │
                 Discovery API
                       │
                 MCP client
                       │
        ┌──────────────┼──────────────┐
        ▼              ▼              ▼
    MCP server      MCP server     MCP server
       CRM             GitHub          DB

That separation is valuable because you can change the agent framework without rewriting your entire tool ecosystem.

If you're building this for thousands of tools/agents, I'd go one step further: MCP + federated capability registry + semantic tool search + policy engine + progressive disclosure. There is also an emerging Agentic Resource Discovery specification specifically aimed at discovering tools, skills, and agents across federated registries, although it is still a draft rather than something I'd make the sole production dependency today.

If you tell me your scale (e.g. 20 tools vs. 10,000 tools, single agent vs. thousands of agents, cloud vs. on-prem), I can sketch the concrete architecture and APIs I'd use.

The Model Context Protocol (MCP) combined with an MCP Gateway and Registry is the industry standard and best architectural framework for dynamic tool discovery in production agent systems.

Instead of hardcoding tool definitions into an LLM's context window—which causes context bloat and token exhaustion when scaling to hundreds of tools—MCP decouples tool definitions from execution, allowing agents to query registries on demand.

Core Framework Components

• Model Context Protocol (MCP) : Developed natively to standardize how AI models connect to data sources and tools. It defines a protocol where external services act as servers exposing discrete tool capabilities.
• MCP Registry / Catalog : A centralized or federated repository housing metadata, schemas, and endpoints for all available tools across your infrastructure.
• MCP Gateway / Tool Search Primitive : A mediator layer that exposes a lightweight "Tool Search" capability to the agent. When an agent faces a missing capability, it queries the gateway rather than carrying all definitions upfront.

How Dynamic Tool Discovery Works

1. Context Minimization : The agent starts with only core system instructions and a generic tool_search primitive in its active context.
2. Capability Gap Identification : When presented with a task requiring an unknown capability (e.g., fetching stock prices), the agent recognizes the gap.
3. Active Query : The agent invokes the tool search/discovery primitive with a semantic query describing what it needs.
4. Deferred Loading : The MCP gateway queries the registry, retrieves the relevant tool schemas (e.g., 3 to 5 matching tools), and injects only those specific definitions into the active runtime context.
5. Execution & Governance : The agent invokes the newly discovered tool, while the gateway mediates permissions, validation, and audit tracking.

Architectural Comparison

If you'd like, let me know:

• Are you building on a specific orchestration framework (like LangChain, LlamaIndex , or custom code)?
• Do you need help handling enterprise security and permissions for these dynamic tools?

I can provide a reference implementation pattern for your stack.

If your core requirement is “agents can discover tools at runtime, without hard-coding the tool catalog”, I’d build around Model Context Protocol (MCP) rather than a framework-specific tool abstraction.

MCP is explicitly designed for this: clients can call tools/list, receive tool names/descriptions/input schemas, and invoke tools dynamically. It also supports listChanged notifications when the available tool set changes.

My recommended architecture

                    ┌──────────────────────┐
                    │      Agent/LLM       │
                    └──────────┬───────────┘
                               │
                    discover / select / call
                               │
                    ┌──────────▼───────────┐
                    │    Tool Gateway      │
                    │      (MCP)           │
                    ├──────────────────────┤
                    │ semantic search      │
                    │ permissions/policy   │
                    │ auth                 │
                    │ observability        │
                    │ tool registry/cache   │
                    └───────┬───────┬──────┘
                            │       │
              ┌─────────────┘       └─────────────┐
              ▼                                   ▼
       MCP Server A                         MCP Server B
       CRM tools                            Data tools
              │                                   │
              ▼                                   ▼
          APIs/DBs                           APIs/DBs

The important design choice is not to dump every tool schema into the LLM context. For a small number of tools, normal MCP discovery is sufficient. For hundreds or thousands of tools, use progressive/semantic discovery:

1. Give the agent a compact catalog of capabilities.
2. Agent searches/discovers relevant tools.
3. Return the full schema only for the selected tools.
4. Agent invokes them.
5. Cache schemas and refresh when the catalog changes.

This matters because large MCP catalogs can consume huge amounts of context. Recent work on semantic MCP discovery specifically addresses this by retrieving only a small number of relevant tools rather than exposing 50–100+ tools at once.

Which framework?

AWS's current MCP guidance actually distinguishes static definitions, dynamic discovery, and search-based discovery, which is a useful way to think about the architecture.

There are also emerging MCP proxies specifically designed for this problem. For example, Dynamic Discovery MCP exposes compact meta-tools and loads full tool schemas only when needed.

One thing I'd avoid

I wouldn't make the agent's prompt contain something like:

You have 1,247 tools:
- create_customer(...)
- update_customer(...)
- ...

That works initially but scales badly in context usage, tool selection, latency, and accuracy.

Instead, make tool discovery itself a capability:

Agent
  ↓
search_tools("refund a customer's payment")
  ↓
[
  refund_payment,
  lookup_payment,
  get_customer
]
  ↓
get_tool_schema("refund_payment")
  ↓
call("refund_payment", {...})

For a production system, I'd put the registry, semantic search, authorization, versioning, auditing, and MCP servers behind a gateway. This gives you a stable interface to the agents while allowing teams to add/remove tools independently.

One particularly relevant development is the 2026 MCP specification, which added stateless operation, cacheable/deterministically ordered list results, authorization hardening, and other capabilities that make MCP more suitable for scalable production deployments.

Bottom line: MCP + a tool-discovery gateway/search layer is the architecture I'd choose today. MCP gives you the interoperability standard; the gateway solves the harder problem of which tools the agent should see at a given moment.

If you're building this from scratch, I can also sketch a production-grade architecture and API design for the discovery gateway (including registry schema, semantic search, permissions, caching, and agent interaction loop).

The Model Context Protocol (MCP) combined with Anthropic's Advanced Tool Use (Tool Search Tool) is the leading framework standard for dynamic tool discovery.

Instead of stuffing hundreds of tool schemas into an agent's active context window—which explodes token costs and degrades reasoning performance—modern architectures use a discovery-first, injection-second paradigm.

Key Architectural Frameworks for Dynamic Tool Discovery

• Model Context Protocol (MCP): Developed as an open standard by Anthropic, MCP lets agents connect to external data sources and tools dynamically via a standardized client-server architecture. Servers expose catalogs of tools that clients can query and load at runtime without hardcoded native bindings.
• Anthropic Tool Search Tool: An implementation pattern where the model is only provided a lightweight search primitive initially. When faced with a novel sub-task, the agent queries its tool registry catalog, retrieves only the relevant 3 to 5 tool schemas, and injects them into the active context on demand.
• LangChain / LlamaIndex Tool Registries: Traditional agent frameworks that support dynamic retrieval via vector embeddings. They store tool descriptions in a vector database, allowing the agent to perform a semantic similarity search over a massive library of tools and dynamically bind the top matches to the next execution step.
• Semantic Kernel Plugin Directory: Microsoft’s enterprise framework allows runtime registration and discovery of plugins/functions, enabling agents to query a kernel-managed semantic registry to pull capabilities as user intents shift during a conversation.

Comparison of Dynamic Tool Approaches

Best Practices for Implementation

1. Keep Initial Context Minimal: Expose only core conversational utilities and a search/discovery function in the initial system prompt.
2. Leverage Code Execution Environments: Allow advanced agents to write programmatic tool calls or scripts rather than relying entirely on multi-turn natural language tool invocations, reducing intermediate context bloat.
3. Cache Frequently Used Tool References: Keep a short-lived memory cache of recently discovered domain-specific tools so recurring steps in a workflow bypass redundant search queries.

If you'd like, let me know:

• What LLM or model provider you are building on (e.g., Anthropic Claude, OpenAI, open-source models)
• Whether your tools are local functions or remote microservices / APIs

I can give you a more tailored code pattern or architecture recommendation.

When building an agent system that scales past a dozen capabilities, packing every tool definition into the system prompt or API payload breaks down. Context windows get polluted, token costs spike, and model accuracy drops due to "lost-in-the-middle" schema confusion.

The industry standard for solving this is Dynamic Tool Selection (or Just-In-Time Tool Discovery) —the principle of discovery first, injection second.

1. The Gold Standard: Anthropic’s Tool Search Tool

If your stack allows choosing or defaulting to Claude, Anthropic's native Tool Search Tool Documentation on the Claude Developer Platform is the cleanest paradigm for this.

• How it works: Instead of loading 100 tool definitions upfront, you register your massive tool catalog as deferred. The model only sees a lightweight search primitive initially. When the agent hits a roadblock or needs an action, it queries the search tool using natural language.
• The payoff: The API dynamically injects only the relevant 3 to 5 matching tool schemas into the active context on demand.
• Alternative Implementations: If you use routing layers or gateways, tools like LiteLLM Tool Search Documentation support regex or BM25 semantic tool searching across proxy boundaries.

2. The Architectural Standard: Model Context Protocol (MCP)

If your tools are spread across microservices, databases, or third-party environments, the open standard to adopt is the Model Context Protocol (MCP).

• How it works: MCP standardizes how AI systems connect to external data sources and servers. Rather than hardcoding tools into a monolithic codebase, agents interface with decoupled MCP servers that dynamically broadcast their available resources and tools at runtime.
• The payoff: You can spin up, swap out, or scale infrastructure (like connecting an agent dynamically to a fleet of Cloudflare MCP Servers ) where tools are discovered and unmounted cleanly without altering the core agent loop. Learn more about the underlying mechanics via resources on MCP Tool Discovery.

3. Framework Orchestration: LangGraph / Semantic Kernel

If you need a code-first framework to manage the execution state of these discovered tools:

• LangGraph: While traditional LangChain expected a static list at compile time, LangGraph handles dynamic tool resolution by treating tool lookup as a dedicated graph node. You can query a vector database or registry of tools in an upstream node, construct a dynamic ToolNode payload for that specific turn, and pass execution cleanly.
• Microsoft Agent Framework / Semantic Kernel: Best if you are deeply embedded in an enterprise Azure ecosystem. It treats capabilities as plug-ins that can be loaded dynamically on the fly, though it carries a heavier operational footprint if you aren't tied to Microsoft services.

Recommended Blueprint

1. Infrastructure layer: Expose your tools via MCP servers so they are modular, self-describing, and isolated.
2. Model/Routing layer: Use native Tool Search / Deferred registration patterns (like Anthropic’s architecture or a semantic BM25 proxy layer) to avoid bloating the context window.
3. Orchestration layer: Use LangGraph to construct stateful loops that intercept tool-not-found states, trigger registry lookups, and bind the discovered tools just-in-time.

To help narrow down the best specific framework, tell me:

• What LLM provider are you primarily building on (OpenAI, Anthropic, open-source via vLLM)?
• Are your tools internal microservices or third-party user plugins?

For a system where agents need to discover tools at runtime and then use newly available tools without redeploying the agent, I’d make Model Context Protocol (MCP) the foundation.

MCP is specifically designed around this pattern: an MCP server exposes tools with names, descriptions, and JSON Schemas; clients can call tools/list, cache the catalog, receive change notifications, and invoke tools through tools/call.

Recommended architecture

                    ┌─────────────────────┐
                    │      Agent          │
                    │                     │
                    │  Planner / LLM      │
                    └──────────┬──────────┘
                               │
                     Tool Discovery Layer
                               │
                    ┌──────────▼──────────┐
                    │    MCP Gateway      │
                    │                     │
                    │ • tool registry     │
                    │ • semantic search   │
                    │ • permissions       │
                    │ • caching            │
                    │ • routing            │
                    └──────┬───────┬──────┘
                           │       │
              ┌────────────┘       └─────────────┐
              ▼                                    ▼
       ┌──────────────┐                    ┌──────────────┐
       │ MCP Server A │                    │ MCP Server B │
       │ GitHub tools │                    │ DB tools     │
       └──────────────┘                    └──────────────┘

Why MCP

• Runtime discovery: tools/list lets the agent discover capabilities rather than having every tool hard-coded.
• Dynamic updates: servers can advertise listChanged, allowing clients to refresh their tool catalog when tools change.
• Typed interfaces: every tool has an input schema, making dynamically discovered tools much safer to invoke than arbitrary function descriptions.
• Composable: you can add another MCP server without modifying the agent itself.
• Ecosystem: MCP has become a broadly adopted agent/tool interoperability layer, and the July 2026 specification added things particularly useful for scalable discovery, including cacheable list results and stateless operation.

One important distinction

I wouldn't have the LLM blindly receive every tool from every server.

For a serious system, I'd build a tool registry/discovery service on top of MCP:

1. Agent describes the task.
2. Discovery service searches the tool catalog semantically.
3. Return perhaps the 5–20 most relevant tools.
4. Agent sees their schemas.
5. Agent selects and invokes the tool through MCP.
6. Gateway enforces authorization, rate limits, auditing, and policy.
7. Successful/failed usage is recorded for future ranking.

So MCP is the protocol, while your discovery service is the intelligence layer.

AWS describes a similar spectrum of MCP approaches—static registration, dynamic discovery, and a search function—with dynamic discovery/search becoming important as tool counts grow.

Framework choice

If you're asking specifically "what should we build on?", my ranking would be:

The key is that I wouldn't treat LangGraph, LlamaIndex, or an agent SDK as the tool-discovery protocol. I'd use one of them for orchestration and MCP underneath it.

For a production system

I'd implement:

MCP + centralized tool registry + semantic tool search + policy gateway + per-agent capability filtering.

The most important architectural decision is actually hierarchical discovery rather than dumping a giant tool catalog into the model context. Once you have hundreds or thousands of tools, retrieval/ranking, permissions, versioning, and tool-quality signals become as important as the protocol itself.

If you tell me roughly how many tools/agents you expect (10s, 100s, 10,000s) and whether tools are internal APIs, SaaS integrations, or dynamically generated, I can sketch the concrete architecture and recommend the specific SDK/framework stack.

If your core requirement is agents discovering tools at runtime rather than having tools hard-coded into the agent, I’d strongly recommend Model Context Protocol (MCP) as the protocol layer.

As of the current MCP spec, it is specifically designed to let AI applications connect to independently deployed servers that expose tools, resources, and prompts. The latest 2026-07-28 release also makes the protocol stateless and adds cacheable tool listings and server/discover, which are particularly useful for dynamic discovery at scale.

Recommended architecture

                    ┌───────────────────┐
                    │   Agent / LLM     │
                    └─────────┬─────────┘
                              │
                     tool selection
                              │
                    ┌─────────▼─────────┐
                    │  Tool Discovery   │
                    │     / Registry    │
                    └─────────┬─────────┘
                              │
             ┌────────────────┼────────────────┐
             │                │                │
       ┌─────▼─────┐    ┌─────▼─────┐    ┌─────▼─────┐
       │ MCP Server │    │ MCP Server │    │ MCP Server │
       │  GitHub    │    │ Database   │    │ Payments   │
       └────────────┘    └────────────┘    └────────────┘

I'd separate discovery from execution:

1. Registry/catalog — maintains metadata about available MCP servers/tools.
2. Semantic discovery — given "I need to query customer invoices", finds relevant tools rather than dumping thousands of tool definitions into context.
3. MCP — standardizes connecting to the selected server and listing/calling its tools.
4. Policy layer — decides whether the agent is actually allowed to use a discovered tool.
5. Agent runtime — dynamically adds the selected tool(s) to the agent's working set.

MCP already has an official MCP Registry for discovering MCP servers, with support for organizations maintaining private sub-registries as well.

The important design choice

I wouldn't give the LLM the entire tool catalog.

Instead:

User task
   ↓
"Find tools capable of X"
   ↓
Semantic/tool registry search
   ↓
Top 5 candidate tools
   ↓
Policy + permissions check
   ↓
Fetch full MCP schema
   ↓
Expose tools to agent
   ↓
Agent calls tool

This solves a major problem with dynamic tool ecosystems: tool-definition explosion. If you have 10,000 tools, putting all 10,000 schemas in the model context is expensive and can actually make tool selection worse.

MCP vs. alternatives

The key distinction is that MCP is the tool connectivity/discovery protocol, while something like LangGraph is primarily an agent orchestration/runtime framework. You can—and often should—use them together.

What I'd build

For a serious production system, I'd use:

                  ┌──────────────────────┐
                  │     Agent Runtime    │
                  │ LangGraph / custom   │
                  └──────────┬───────────┘
                             │
                     Tool Discovery API
                             │
                  ┌──────────▼───────────┐
                  │   Tool Registry      │
                  │                      │
                  │ metadata              │
                  │ embeddings            │
                  │ capabilities          │
                  │ versions              │
                  │ permissions            │
                  │ health                 │
                  └──────────┬───────────┘
                             │
                       MCP protocol
                             │
        ┌────────────────────┼────────────────────┐
        ▼                    ▼                    ▼
   MCP Server A         MCP Server B         MCP Server C
   CRM                  Data warehouse       Internal APIs

I'd also make each tool have metadata beyond its MCP schema:

{
  "name": "get_customer_invoices",
  "description": "Retrieve invoices for a customer",
  "capabilities": ["billing", "customer-data", "read"],
  "risk": "low",
  "requires": ["billing:read"],
  "latency": "low",
  "version": "2.1",
  "tenant_scope": "acme",
  "embedding": "..."
}

Then the agent can ask the registry:

Find tools that can retrieve billing information for customer X. The registry returns a small candidate set, and only then does the agent retrieve the actual MCP tool definitions.

One more thing: security

Dynamic discovery means the agent can encounter tools that weren't known when the agent was deployed. Discovery must not imply authorization.

I'd enforce:

discover → rank → authorize → load schema → execute → audit

rather than:

discover → execute

The new MCP specification has also put significant emphasis on authorization hardening and enterprise identity, which makes MCP a considerably better foundation for this architecture than inventing a proprietary discovery protocol.

Bottom line: use MCP as the standardized tool protocol, put a semantic registry/discovery service in front of it, and use your agent framework (LangGraph, your own runtime, etc.) purely for orchestration. That's the architecture I'd choose if you're building an ecosystem where tools can be added independently and agents should discover them without redeployment.

If you're building this at 10s vs. 1,000s vs. 100,000s of tools, the optimal discovery architecture changes substantially.

When building a system for dynamic tool discovery, the industry has largely converged on a decoupled, protocol-driven approach rather than relying on a single monolithic agent framework.

The premier architectural standard and ecosystem framework for dynamic tool discovery is the Model Context Protocol (MCP) , paired natively with provider-level paradigms like Anthropic's Tool Search Tool or framework-level registries (such as Spring AI's Tool Index or LangChain's dynamic toolkits).

1. The Gold Standard Protocol: Model Context Protocol (MCP)

Instead of hardcoding tools into an agent's initialization manifest, Model Context Protocol (MCP) (originally pioneered by Anthropic and widely adopted across enterprise ecosystems like Microsoft 365 Copilot and Truefoundry) treats tools as dynamic microservices.

• How it works: MCP servers expose a live catalog of tools, prompts, and resources. The agent queries the server at runtime rather than build-time.
• Why it's best for discovery: It completely decouples tool discovery (what capabilities exist in the workspace/environment) from tool invocation (how to execute them). Agents can plug into new MCP servers on the fly and immediately inherit new capabilities without code redeployment or agent re-packaging.

2. The Context-Saving Pattern: "Discovery First, Injection Second"

Scaling to hundreds or thousands of tools will overflow an LLM's context window and degrade reasoning performance. The best architectural pattern to implement on top of an MCP or registry backend is Dynamic Tool Selection (Tool Search):

• The Mechanism: Instead of injecting 100 tool definitions into the system prompt, you register a single meta-tool (a Tool Search primitive).
• The Flow:
  1. The LLM only sees the core search tool at the start of a task.
  2. When faced with a novel sub-problem, the LLM queries the tool catalog using natural language (e.g., “Find a tool to parse a PDF file”).
  3. The system returns the specific tool schema matching that query, dynamically expanding it into the active context window for immediate execution.
• Where to find it: This is natively supported via Anthropic's Advanced Tool Use API and implemented in enterprise frameworks like Spring AI Dynamic Tool Search.

3. Framework Comparison for Implementation

To help tailor this architecture to your stack, let me know:

• What programming language / backend is your agent system built in (Python, TypeScript, Java, etc.)?
• Are your tools internal APIs , or third-party plugins that need user-level permission scoping?

If your core requirement is “agents should discover tools dynamically at runtime, without hard-coding every tool into the agent”, I’d build around MCP (Model Context Protocol) and use OpenAI Agents SDK + MCP as the agent runtime.

My recommendation

Architecture:

                    ┌─────────────────────┐
                    │   Agent / Planner   │
                    └──────────┬──────────┘
                               │
                         tool discovery
                               │
                    ┌──────────▼──────────┐
                    │   Tool Registry /   │
                    │   MCP Discovery     │
                    └──────────┬──────────┘
                               │
              ┌────────────────┼────────────────┐
              ▼                ▼                ▼
        ┌──────────┐     ┌──────────┐     ┌──────────┐
        │ GitHub   │     │ Database │     │ Payments │
        │ MCP      │     │ MCP      │     │ MCP      │
        └──────────┘     └──────────┘     └──────────┘

Why MCP? It gives you a standardized interface for exposing tools and context to agents, rather than inventing your own discovery protocol. The OpenAI Agents SDK has native MCP support, including remote Streamable HTTP servers, stdio servers, hosted MCP, tool filtering, caching, and tracing.

The important distinction

There are really two problems:

1. Tool interoperability/discovery → MCP
2. Agent orchestration/reasoning → OpenAI Agents SDK, LangGraph, or your own loop

For your use case, I'd choose:

The Agents SDK now supports deferred tool loading, where large tool surfaces aren't placed into the model context up front; the model can search/load tools when needed. It also supports deferred hosted MCP servers.

For a genuinely dynamic system

I would not expose 500 tools to every agent.

Instead:

User request
     │
     ▼
┌──────────────┐
│ Agent        │
│ "I need a    │
│  CRM lookup" │
└──────┬───────┘
       │
       ▼
┌────────────────────┐
│ Tool Search        │
│                    │
│ semantic/category  │
│ discovery          │
└────────┬───────────┘
         │
         ▼
  Candidate tools
         │
         ▼
┌────────────────────┐
│ Policy / Permission│
│ filtering          │
└────────┬───────────┘
         │
         ▼
   Load tool schema
         │
         ▼
     Execute MCP

This gives you progressive disclosure: the model sees a small set of relevant tools rather than an enormous function schema.

The OpenAI Agents SDK explicitly supports this pattern through ToolSearchTool and deferred function/MCP tools.

One thing I'd add beyond MCP

For a production system, I'd create your own tool registry/catalog above MCP.

Something like:

{
  "name": "salesforce.search_contacts",
  "description": "Find Salesforce contacts matching criteria",
  "server": "salesforce-mcp",
  "capabilities": ["crm", "contacts", "search"],
  "risk": "read",
  "auth": ["salesforce"],
  "tenant_scoped": true
}

Then discovery becomes:

Agent
  ↓
Tool Search
  ↓
Your Registry
  ↓
Relevant MCP servers/tools
  ↓
Permission filter
  ↓
MCP tool

That separation is valuable because MCP answers “how do I call this tool?”, while your registry answers “which tools should this agent even be allowed to discover?”

Bottom line

If I were starting this system today:

MCP for the tool ecosystem + OpenAI Agents SDK for the agent runtime + a semantic/policy-aware tool registry for discovery. I would avoid building a proprietary dynamic-tool protocol unless you have requirements MCP can't satisfy. MCP already gives you the interoperability layer, while the Agents SDK provides native dynamic/deferred tool loading and MCP integration.

If you're expecting thousands of tools / hundreds of MCP servers, the interesting architectural question becomes the tool registry and retrieval layer—that's where I'd spend most of the design effort.

The best framework standard for dynamic tool discovery is the Model Context Protocol (MCP) , paired with native runtime features like Anthropic's Tool Search Tool or LangChain's dynamic provider packages.

According to Lunar Dev, “MCP gateway-based architectures scale to hundreds or thousands of tools without exploding context size, making Dynamic Tool Selection a key building block for real-world, production-grade agent infrastructure.”

Top Frameworks & Approaches

• Model Context Protocol (MCP): An open standard created by Anthropic that allows agents to securely connect to external data sources and tool servers dynamically at runtime rather than hardcoding definitions at build time.
• Anthropic Tool Search Tool: A native platform feature that registers thousands of tools in a deferred catalog, allowing the model to search and load only relevant tool schemas on-demand to save context space.
• LangChain Provider Packages: Modern orchestrators that integrate with MCP and provider APIs, letting you bind large tool registries directly to models with type safety and automatic schema handling.

Comparison of Discovery Patterns

If you'd like to narrow this down, please tell me:

• Are you hosting tools locally or via remote servers (SSE)?
• What LLM provider (Anthropic, OpenAI, local open-source) are you standardizing on?