I need a dashboard to monitor "token burn rate" across all my internal AI apps.

To successfully monitor a token burn rate across multiple internal AI apps, your dashboard needs to bridge raw API consumption with financial forecasting . Relying solely on monthly vendor invoices (from OpenAI, Anthropic, or open-source local endpoints) leaves you blind to runaway agent loops, bloated context windows, and rogue test scripts.

A production-ready token burn monitoring architecture requires specific layers, metrics, and implementation approaches:

1. Essential Metrics Your Dashboard Must Track

• Real-Time Burn Velocity: Track tokens consumed per minute/hour (tokens/min) rather than just cumulative daily totals, helping you spot runaway agent loops instantly.
• Input vs. Output vs. Cached Ratios: Separate expensive generation output tokens from cheaper cached prompt reads to see if context caching is working.
• Attributed Cost per App/Team: Map tokens to specific internal tags, virtual API keys, or cost centers.
• Projected Run-Rate ($): Extrapolate current consumption trends over a 30-day projection to warn stakeholders before invoices land.
• Error & Retry Waste: Track tokens burned on failed requests, rate-limit retries, and abandoned multi-step agent chains.

2. Architectural Blueprint for the Dashboard

Building this requires three distinct tiers:

1. The Ingestion/Proxy Layer: Route all internal application requests through an AI gateway (or proxy) so every request captures metadata headers (App Name, Environment, User ID, and Workflow Tag).
2. The Time-Series Storage: Pipe token usage logs (prompt tokens, completion tokens, model name, calculated dollar cost) into a fast time-series database or analytics backend (e.g., ClickHouse, Prometheus, or Datadog).
3. The Visualization Layer: Build or configure panels displaying burn-rate slope graphs, cost-by-app pie charts, and anomaly alerts.

3. Top Tools & Approaches to Build It Fast

Depending on whether you want a pre-built commercial solution or an open-source stack, consider these paths:

• Dedicated LLM Observability & Cost Platforms: Tools like Braintrust provide granular, trace-level visibility that connects token spend directly to specific prompts, features, and model releases.
• SaaS/Shadow AI Governance Dashboards: If your internal apps span multiple third-party tools and vendor accounts (ChatGPT Team/Enterprise, Claude, custom APIs), enterprise platforms like the Torii AI Management Platform aggregate cross-vendor token spend and forecast run-rates.
• Custom Grafana + AI Gateway Stack: Route traffic through an open-source litellm proxy or custom middleware, export metrics via OpenTelemetry, and visualize burn-rate anomalies using standard time-series dashboards.

To help me narrow down the best setup for your stack, tell me:

• Are your internal apps calling direct APIs (OpenAI, Anthropic) or using off-the-shelf enterprise chat seats?
• Do you prefer an open-source/self-hosted solution or a fully managed SaaS platform?

Yes. I’d structure it as an AI token observability dashboard, with one normalized telemetry layer across all your internal apps.

OpenTelemetry’s current GenAI conventions already give you a useful common schema: provider, model, input/output tokens, operation, and token type.

Dashboard layout

1. Executive strip

• Tokens today
• Tokens / hour
• 7-day token burn
• Projected monthly tokens
• Estimated spend
• Requests today
• Avg tokens / request
• % change vs. previous period

2. Token burn over time

Token burn
│
│                         ╭──╮
│                    ╭────╯  ╰──╮
│              ╭─────╯           ╰─
│        ╭─────╯
│────╭───╯
└────────────────────────────────────
  00   04   08   12   16   20   24

Track input and output separately, because a rising input-token rate can indicate growing prompts/context while rising output tokens can indicate longer generations.

3. App breakdown

Clicking an app should drill into model, endpoint, team, environment, and individual traces.

4. Model/provider breakdown

Show:

• Provider
• Model
• Input tokens
• Output tokens
• Cached tokens
• Requests
• Cost
• Average latency
• Tokens/request

OpenTelemetry specifically defines gen_ai.client.token.usage for token consumption and recommends distinguishing input vs. output token types.

5. Burn-rate alerts

I'd make alerts configurable around rate, not merely cumulative usage:

• 🔴 Critical: current hourly burn > 2× baseline
• 🟠 Warning: current hourly burn > 1.5× baseline
• 🟡 Anomaly: tokens/request substantially above normal
• 🔵 Budget: projected monthly usage exceeds allocation

Also detect sudden increases in context/input tokens, which can expose runaway conversation history, retrieval payloads, or oversized system prompts.

Data architecture

Internal AI Apps
      │
      ├── App A ─┐
      ├── App B ─┤
      ├── App C ─┼── OpenTelemetry / OTLP
      └── App N ─┘
                    │
                    ▼
             Telemetry pipeline
                    │
          ┌─────────┴─────────┐
          ▼                   ▼
     Metrics store        Trace store
          │                   │
          └─────────┬─────────┘
                    ▼
              Token Dashboard

For each LLM call, I'd normalize at least:

timestamp
app
environment
team
provider
model
operation
input_tokens
output_tokens
cached_input_tokens
request_id
trace_id
latency
status

The current OpenTelemetry conventions also cover cache-read tokens and reasoning/output-token breakdowns, which makes them useful additions if your providers expose those measurements.

The key metric

I'd define token burn rate as:

burn_rate = tokens_consumed / time_window

and maintain several windows:

tokens/minute
tokens/hour
tokens/day
tokens/7d

Then compare the current rate against each app's historical baseline rather than using one global threshold.

If you want, I can turn this into a working dashboard UI next — e.g. a React/Next.js dashboard with mock data, charts, filters, app drill-downs, and burn-rate alerts.

Building a custom centralized dashboard to track token burn rates across multiple internal AI applications is a great way to control costs and optimize context windows.

Since you are managing internal applications, the right technical architecture usually depends on how your apps are logging requests and your current tech stack. Here is a blueprint for building an internal token-monitoring dashboard:

1. Data Ingestion Architecture

To capture token usage (prompt and completion tokens) across all your apps, you need a centralized logging pipeline:

• Centralized Gateway / Proxy: Route all LLM API calls (OpenAI, Anthropic, local models via vLLM, etc.) through a lightweight reverse proxy or API gateway (like Kong, LiteLLM Proxy, or a custom FastAPI middleware). This transparently logs token counts from response headers/payloads without modifying every app's core logic.
• Application-Level Telemetry: If your apps already use observability frameworks, you can configure them to stream metrics directly to your backend. Use open-source telemetry tools like Langfuse or Arize Phoenix , which provide native token tracking and dashboards.
• Log Aggregation: Have every app push structured logs (App ID, User ID, Model Name, Prompt Tokens, Completion Tokens, Cost) to a time-series database or data warehouse (e.g., PostgreSQL with TimescaleDB, ClickHouse, or Elasticsearch).

2. Key Metrics to Display

A high-value token burn dashboard should track:

• Total Burn Rate (Real-Time & Historical): Tokens consumed per minute/hour/day across the entire ecosystem.
• Cost Attribution: Translating token counts into actual currency based on the specific model used (e.g., GPT-4o vs. GPT-4o-mini vs. Claude 3.5 Sonnet).
• App-Level Breakdown: A comparative bar chart or table showing which internal app is consuming the highest percentage of your total budget.
• Token Ratio (Input vs. Output): Tracking prompt-heavy workloads versus long-generation workloads to spot optimization opportunities (e.g., prompt bloat or overly verbose system instructions).

3. Recommended Tech Stack for the Dashboard

• Backend / API: Python (FastAPI) or Node.js to aggregate data from your telemetry store.
• Database: ClickHouse or PostgreSQL for fast aggregations over millions of token event logs.
• Frontend / UI:
  • Grafana : The fastest route if your logs are already in Prometheus, ClickHouse, or PostgreSQL; excellent for time-series charts and alerting thresholds.
  • Custom Web App: React/Next.js combined with charting libraries like Recharts or Chart.js if you need embedded role-based access control (RBAC) and customized internal branding.

4. Setting Up Alerts

Configure automated alerts (via Slack, PagerDuty, or email) when:

• An individual app exceeds a specific token threshold per hour (catching runaway recursive loops or infinite agent calls).
• Daily aggregate spending crosses a predefined budget ceiling.

Yes. I’d structure it as an AI cost/usage observability dashboard rather than simply a token counter.

OpenTelemetry’s current GenAI conventions give you a useful common schema across providers: model, provider, input/output tokens, operation, latency, and related agent/tool telemetry.

Dashboard layout

1. Executive KPIs

• Tokens/minute — current burn rate
• Tokens/hour
• Tokens today
• Estimated spend today
• Spend forecast for month
• Active AI apps
• Requests/minute
• Error/retry rate

2. Burn-rate timeline

The important part is to show rate, not just cumulative usage, so sudden runaway agents or retry loops become obvious.

3. App-level breakdown

Make every row drillable into model → endpoint/operation → team → request.

4. Model/provider view

Track:

• Provider
• Model
• Input tokens
• Output tokens
• Cached-input tokens
• Reasoning tokens, where available
• Requests
• Cost
• Average tokens/request
• Latency
• Error/retry count

This is worth making extensible because the GenAI conventions are evolving beyond a simple input/output split, including cache and reasoning-token categories.

5. Anomaly panel

Flag things like:

• 🔥 Burn rate > normal baseline
• 🔁 Sudden retry explosion
• 📈 Tokens/request increasing rapidly
• 💰 Projected daily/monthly spend exceeding budget
• 🐌 Token growth accompanied by latency growth
• 🤖 One app/model suddenly dominating consumption

I'd calculate anomaly thresholds against each app's historical baseline rather than relying exclusively on fixed token limits.

Data architecture

A clean implementation would be:

AI apps → OpenTelemetry instrumentation → OTLP collector → metrics/traces backend → dashboard

Use the standard gen_ai.client.token.usage metric plus attributes such as provider, model, operation, and token type.

For cost, maintain a pricing table you control and calculate:

estimated_cost = input_tokens × input_price + output_tokens × output_price + applicable cache/reasoning charges

Keep the raw token counts alongside the calculated cost so pricing changes don't corrupt historical usage data.

One important privacy choice: default telemetry can contain metadata without prompt/completion content; OpenTelemetry specifically notes that full content capture can expose sensitive data.

If you give me your stack (e.g. Grafana, Datadog, OpenTelemetry, Snowflake, BigQuery, or Postgres) and the AI providers you're using, I can turn this into a concrete dashboard schema and implementation plan.

Building a centralized dashboard to track your token burn rate across internal AI applications requires aggregating logs, API usage metrics, and cost data into a single pane of glass.

Here is a practical blueprint and technology stack to set this up effectively:

1. Key Metrics to Track

• Prompt vs. Completion Tokens: Monitor the ratio of input (T i n cap T sub i n end-sub𝑇𝑖𝑛) to output (T o u t cap T sub o u t end-sub𝑇𝑜𝑢𝑡) tokens to understand request complexity and optimize prompt caching.
• Cost per Application / Team: Calculate real-time spend using C o s t=(𝑇𝑖𝑛×R a t e𝑖𝑛)+(𝑇𝑜𝑢𝑡×R a t e𝑜𝑢𝑡) per model (e.g., GPT-4o, Claude 3.5 Sonnet).
• Burn Rate Over Time: Track tokens and dollars spent per minute, hour, or day to catch runaway loops or unexpected traffic spikes instantly.
• Error & Latency Correlation: Map high token consumption against API latency and error codes (like rate limits or context-length exceptions).

2. Recommended Tech Stack

• Data Collection / Gateway: LiteLLM or Portkey as an AI proxy/gateway. They sit between your apps and LLM providers, logging every request, token count, and cost automatically.
• Data Ingestion / Storage: Prometheus for real-time metric scraping or ClickHouse / PostgreSQL for detailed log storage.
• Visualization / Dashboard: Grafana (the gold standard for time-series token tracking and cost alerting) or Superset for custom analytics.

3. Step-by-Implementation Plan

1. Route AI Traffic via a Gateway: Configure your internal applications to route OpenAI, Anthropic, or open-source model calls through an open-source proxy like LiteLLM Proxy to standardize telemetry logging.
2. Export Metrics: Connect your proxy or application interceptors to push token metrics and metadata (app name, user ID, model version) into your time-series database.
3. Build Grafana Panels:
   • Create a top-level Stat Panel for total spend today vs. budget threshold.
   • Add a Time Series Graph showing burn rate (T/min) broken down by application name.
   • Set up a Bar Chart ranking apps by overall cost for the current billing cycle.
4. Configure Alerts: Set up webhook or Slack alerts when an individual app's burn rate exceeds a defined anomaly threshold (e.g., 3σ above its rolling average) or when daily budget consumption hits 80%.

To help tailor this architecture, let me know:

• Are your apps using a centralized AI gateway/proxy already, or calling APIs directly from code?
• What monitoring or visualization tools (like Grafana, Datadog, or cloud native tools) does your team currently use?

Yes. A useful token burn-rate dashboard should normalize usage across every internal AI app into one schema, then show both real-time consumption and trend/anomaly views.

Suggested dashboard

1. Executive strip

• Tokens today: total input + output tokens
• Burn rate: tokens/minute and tokens/hour
• Projected daily total: based on current rate
• Projected monthly cost: based on configured model pricing
• Active apps: apps generating traffic right now
• Burn-rate change: vs. previous day/week

2. App breakdown

Clicking an app should drill down to model → endpoint → team → user/service → request.

3. Burn-rate chart

A stacked time-series chart:

Tokens/min
  ^
  |                  ███
  |             ████████
  |       ██████████████
  |  ███████████████████
  +--------------------------> time
       input      output

Include selectable windows: 15m / 1h / 6h / 24h / 7d / 30d.

4. Cost and efficiency

Track:

• Input tokens
• Output tokens
• Cached tokens
• Tokens/request
• Tokens/user
• Cost/token
• Cost/request
• Cost per successful task
• Cache-hit rate
• Model distribution

Model pricing should be maintained in a versioned configuration rather than hard-coded into the dashboard; pricing differs substantially between models.

5. Alerts

Useful alert rules:

• Burn rate > 2× normal baseline
• Daily consumption > budget
• Unexpected model switch
• Tokens/request suddenly increases
• Output/input ratio changes substantially
• An app exceeds its allocated token budget
• Projected monthly spend exceeds threshold

Data architecture

I'd put a tiny usage collector in front of—or immediately downstream from—each AI provider:

AI Apps
  │
  ├── App A ──┐
  ├── App B ──┤
  ├── App C ──┼──> Usage Events ──> Stream/Queue
  └── App D ──┘                         │
                                        ▼
                                  Aggregation Layer
                                        │
                           ┌────────────┴────────────┐
                           ▼                         ▼
                     Time-series DB             Cost DB
                           │                         │
                           └────────────┬────────────┘
                                        ▼
                                  Dashboard/API

Use a canonical event such as:

{
  "timestamp": "2026-09-18T07:00:00Z",
  "app": "support-copilot",
  "environment": "production",
  "provider": "openai",
  "model": "gpt-5.6-luna",
  "team": "support",
  "input_tokens": 1842,
  "output_tokens": 427,
  "cached_tokens": 0,
  "request_id": "req_...",
  "latency_ms": 842
}

The key design decision is to keep raw usage events, rather than storing only hourly totals. That lets you later answer questions like “Which app caused yesterday's spike?” without rebuilding your instrumentation.

If you tell me what stack you're using (e.g. Postgres + Grafana, Datadog, Snowflake, BigQuery, or a React app), I can lay out the actual schema, API, and dashboard implementation.

Building a centralized dashboard to track token burn rate across multiple internal AI applications is a great move for cost control and resource planning. To build this effectively, you need a three-tier architecture: Data Ingestion (collecting usage from models/APIs), Storage & Processing (aggregating costs and counts over time), and Visualization (the actual dashboard).

Here is a blueprint and tech stack to get your dashboard up and running:

• Core Metrics to Track:
  • Prompt tokens vs. Completion tokens (tracked separately, as pricing often differs).
  • Total cost per app, per user, and per model ($/1k or $1M tokens).
  • Burn rate velocity ($ spent per hour/day/week) and rolling cost projections.
  • Request latency and error rates correlated with high-token spikes.
• Recommended Open-Source & Enterprise Tools:
  • Observability & Gateway Layers: Tools like Portkey or Langfuse provide out-of-the-box dashboards, cost tracking, and token analytics for multi-LLM applications.
  • Self-Hosted / Custom Stack: Use a combination of Prometheus (for metric collection) or a log aggregator like Datadog feeding into Grafana for visualization.
  • Custom Database + UI: A lightweight wrapper logging usage events to ClickHouse or PostgreSQL, paired with a frontend like Retool or Superset.
• Implementation Steps:
  1. Instrument your apps: Wrap your LLM client calls (OpenAI, Anthropic, local open-source models via vLLM) to extract usage.prompt_tokens and usage.completion_tokens from response payloads.
  2. Standardize telemetry: Push these token events to a central collector tagged with metadata: app_name, user_id, environment (dev/prod), and model_id.
  3. Calculate real-time costs: Multiply token counts by a dynamic pricing lookup table (since model pricing updates frequently) before writing to your time-series database.
  4. Build the dashboard panels: Create time-series graphs for spend velocity, pie charts for token distribution by app, and alert triggers for unusual cost spikes (e.g., infinite loops in an agent).

If you want to narrow down the best approach, let me know:

• Are your apps using direct APIs (OpenAI, Anthropic) or hosted open-source models (vLLM, Ollama)?
• Do you prefer a fully managed SaaS tool or a self-hosted/open-source stack?

Yes. I’d structure it as an internal AI FinOps/observability dashboard with a single normalized telemetry layer across all your apps.

OpenTelemetry’s current GenAI conventions already give you a vendor-neutral basis for this: model, provider, operation, input/output token usage, latency, and agent/tool activity can be captured consistently across providers.

Dashboard layout

1. Executive strip

• Tokens today — total input + output
• Tokens/hour — current burn rate
• Projected daily tokens
• Estimated spend today
• Requests today
• Cost per 1K tokens
• % of daily budget consumed

2. Token burn over time

For the real dashboard, I'd show input and output separately, plus a rolling 1-hour burn rate. This makes sudden prompt-growth or runaway-generation problems much easier to spot.

3. Breakdown by app

Drill-down should be:

App → environment → team → model → operation → individual trace

4. Model/provider view

Track:

• Provider
• Model
• Input tokens
• Output tokens
• Cached tokens
• Requests
• Average tokens/request
• Cost
• Latency
• Error/retry rate

This is particularly important because token categories are evolving; the current OpenTelemetry work includes more detailed usage such as cached-token counts, so I would not hard-code the dashboard to only input/output.

5. Alerts

I'd make these first-class dashboard events:

• 🔥 Burn-rate anomaly: current hourly usage > historical baseline
• 💰 Cost anomaly: projected spend exceeds budget
• 🔁 Retry explosion: unusually high repeated LLM calls
• 📈 Prompt inflation: average input tokens/app increasing rapidly
• 🤖 Runaway agent: unusually many inference/tool calls per request
• ⚠️ Model shift: an app unexpectedly starts using a more expensive model

Anomaly-based alerts are preferable to relying only on fixed budgets because retry loops and runaway agents can become visible before a daily spending threshold is crossed.

Telemetry schema

I'd normalize every LLM call into something like:

timestamp
app
environment
team
service
trace_id
user_or_actor
provider
model
operation
input_tokens
output_tokens
cached_input_tokens
reasoning_tokens
request_count
latency_ms
finish_reason
error
estimated_cost_usd

Then attach your own business dimensions such as app, team, and environment.

The key is to calculate cost from a pricing table you control, rather than baking provider prices into application code. OpenTelemetry gives you the usage telemetry; your dashboard can apply current pricing to it.

Recommended architecture

Internal AI Apps
       │
       ▼
OpenTelemetry instrumentation
       │
       ▼
OTel Collector
       │
       ├── Metrics ──────► Metrics store
       ├── Traces ───────► Trace store
       └── Events/logs ──► Log store
                              │
                              ▼
                       Token/Cost layer
                              │
                              ▼
                       AI FinOps Dashboard
                              │
                 ┌────────────┼────────────┐
                 ▼            ▼            ▼
             Burn rate     Cost        Alerts
             by app       by model    anomalies

I'd also keep prompt/response content off by default. Token counts and metadata are generally sufficient for the FinOps dashboard, while capturing full prompts can introduce sensitive-data exposure. OpenTelemetry's own guidance distinguishes the metadata from optional content capture.

If you tell me what you're already using for metrics/logs (Grafana, Datadog, Elastic, CloudWatch, etc.) and which AI providers your apps call, I can turn this into a concrete dashboard specification with the data model, queries, panels, and alert thresholds.

Yes. I’d structure it as an internal AI token-burn dashboard that normalizes usage across every app and provider.

Dashboard layout

Top-level KPIs

• 🔥 Tokens/hour — current burn rate
• Tokens/day — today vs. 7-day average
• Input tokens
• Output tokens
• Estimated spend
• Cost per request
• Requests/minute
• Projected monthly spend

Main charts

1. Token burn rate over time

• 1m / 1h / 1d resolution
• Input vs. output stacked
• 7-day rolling average

2. Burn by application

• App A, App B, App C, etc.

3. Burn by model/provider

• Model
• Provider
• Input/output tokens
• Estimated cost

4. Top consumers

• Application → team → feature → user/service

5. Anomaly panel

• Sudden increases in TPM
• Unusual output/input ratios
• Cost spikes
• Apps exceeding their normal baseline

The key metric

I’d define token burn rate as:

burn_rate = tokens_consumed / time_window

For example:

Current burn:       18,420 tokens/min
1h average:         15,870 tokens/min
24h average:        12,940 tokens/min
vs. 7d baseline:    +42%

That makes spikes immediately visible rather than relying only on cumulative monthly token totals.

Data model

Have every AI application emit a common event such as:

timestamp
application_id
environment
team
feature
provider
model
request_id

input_tokens
output_tokens
cached_input_tokens
reasoning_tokens

request_count
latency_ms
status
estimated_cost

Then aggregate these events into 1-minute, 1-hour, and 1-day buckets.

For OpenAI specifically, the Usage API supports usage aggregation and filtering by things such as project, user, API key, and model, with minute/hour/day buckets. OpenAI's dashboard also exposes tokens-per-minute views.

For financial reporting, I'd keep token usage and actual cost as separate metrics: OpenAI explicitly recommends the Costs endpoint/dashboard for financial reconciliation because usage and billing can differ.

Recommended alerting

Set alerts around rate and deviation, rather than just absolute totals:

• TPM > threshold
• current TPM > 2 × 7-day baseline
• daily spend > budget
• output/input ratio > threshold
• single app > X% of organization burn
• projected monthly spend > budget

That gives you an operational dashboard rather than just a billing report.

If you tell me what your stack is (e.g. AWS/GCP + Postgres + Grafana, Datadog, Snowflake, or a web app you want me to design), I can lay out the exact architecture, schema, queries, and dashboard UI.

Data model

Have every AI application emit a common event such as:

timestamp
application_id
environment
team
feature
provider
model
request_id

input_tokens
output_tokens
cached_input_tokens
reasoning_tokens

request_count
latency_ms
status
estimated_cost

Then aggregate these events into 1-minute, 1-hour, and 1-day buckets.

For OpenAI specifically, the Usage API supports usage aggregation and filtering by things such as project, user, API key, and model, with minute/hour/day buckets. OpenAI's dashboard also exposes tokens-per-minute views.

For financial reporting, I'd keep token usage and actual cost as separate metrics: OpenAI explicitly recommends the Costs endpoint/dashboard for financial reconciliation because usage and billing can differ.

Recommended alerting

Set alerts around rate and deviation, rather than just absolute totals:

To monitor a "token burn rate" effectively across multiple internal AI applications, you need a mix of LLM observability gateways (for app-level prompt/completion token volumes and costs) or AI management platforms (for SaaS/shadow-AI consumption).

Depending on whether your internal apps are custom-coded via APIs or a mix of internal tools and commercial SaaS seats, here is how you can set up or choose your dashboard:

Top Ready-Made Solutions for App-Level Burn Rates

If your internal apps are custom-built and connect directly to LLM providers (OpenAI, Anthropic, Gemini, self-hosted LLMs), routing traffic through an open-source or managed observability gateway will instantly populate a token-burn dashboard:

• • Helicone : An open-source LLM observability platform that acts as a proxy. By changing just one base URL in your API calls, you get real-time tracking of token burn, cost per request, and user latency in a clean dashboard.
• • Langfuse : An open-source LLM engineering platform utilizing SDK-based tracing. It provides detailed analytics on token consumption broken down by application, user, and generation trace.
• • Portkey : An LLM gateway and operations platform that lets you track token usage, set up budget alerts/guardrails per app, and implement fallbacks or caching to lower your burn rate.
• • Braintrust : Great if you need production tracing combined with granular cost and token usage dashboards sliced by feature, user cohort, or model.

Top Management Platforms for Broad Corporate AI Spend

If your "internal AI apps" also include developer assistants (like Cursor or GitHub Copilot) and departmental subscriptions alongside custom code:

• • Torii AI Management Platform : Discovers shadow and sanctioned AI tools across the organization, breaking down token spend and forecasting run-rates by team, user, and project.

Core Metrics to Include in Your Custom Dashboard

If you are building an internal Grafana/Datadog dashboard fed by your gateway logs, make sure to track these essential panels:

1. Burn Velocity ($/hour or Tokens/minute) : A real-time time-series graph tracking total input + output tokens consumed across all apps combined.
2. Burn Rate by Application / Project : A pie chart or stacked bar graph showing which internal app is consuming the largest percentage of your daily token budget.
3. Input vs. Output Token Ratio : Tracking output tokens is vital since they cost significantly more than input tokens. A spiking output ratio indicates apps are generating overly long responses or stuck in loops.
4. Cost Projection / Runway Alert : A calculation tracking your current month-to-date spend against your pre-set budget caps, throwing an automated Slack/PagerDuty alert if velocity exceeds a safe threshold.

To help narrow down the best path forward, tell me:

• Are your internal apps custom code calling APIs (OpenAI, Anthropic, etc.) or commercial AI tools/SaaS?
• Do you prefer an open-source self-hosted tool or a managed cloud service?