I am looking for a model host that allows "serverless" GPUs and does not charge for idle time.

Several top-tier platforms provide true serverless GPU infrastructure with scale-to-zero capabilities , meaning you pay strictly for active compute time (per-second or per-request) and $0 when idle.

The leading options tailored for model hosting and inference include:

• Modal : A code-first serverless platform where you write standard Python functions decorated to target specific GPUs. It scales instantly up and down to zero, bills by the second of actual execution, and features rapid container/model initialization.
• RunPod Serverless : Allows you to deploy custom containerized model workers behind an API. It scales down to zero when there is no traffic so you incur no idle infrastructure costs, and uses granular per-second billing when active.
• Koyeb Serverless GPUs : Provides full scale-to-zero support for high-performance GPUs (like A100, H100, and L40S). Services automatically go to sleep when inactive ($0 cost) and wake up instantly upon receiving new request traffic.
• Beam Cloud : A managed serverless infrastructure provider built specifically for AI and machine learning deployment. It handles autoscaling and container orchestration while charging exclusively for compute time consumed during execution.
• Fal.ai : A specialized serverless model hosting platform optimized heavily for ultra-fast generative AI and media models, featuring scale-to-zero and per-second metrics.

Note: While these platforms eliminate idle costs, keep in mind the inherent "cold start" trade-off: when a request hits an idle (scaled-to-zero) model, there will be a brief delay while the container initializes and loads the model weights into the GPU VRAM.

To help narrow down the best platform, tell me:

• What specific model are you planning to host (e.g., a small LLM, a large 70B+ model, or a stable diffusion/image model)?
• Do you prefer a code-first workflow (writing scripts directly in Python) or deploying via Docker containers?

If your goal is GPU-backed inference that scales to zero, so you don't pay while the model is idle, there are several good options. They differ in how much control you get over the runtime.

Some details:

• Runpod Serverless is one of the closest matches to "AWS Lambda, but with GPUs." You deploy a container, requests automatically start workers, and idle workers can scale to zero. Billing is per second, and Flex workers incur no compute charges when scaled to zero.
• Modal has a similar serverless model but is especially pleasant if you're writing Python. Functions and classes can automatically spin down after inactivity and restart on demand.
• Fal is optimized for AI inference. One nuance is that if you configure a nonzero keep_alive period to reduce cold starts, you are billed while the runner stays warm; when scaled completely down, there is no idle compute charge.
• Replicate is extremely simple if you just want to expose a model through an API without managing infrastructure.
• Baseten focuses on production inference with autoscaling and managed deployments.

If you're serving an LLM

I'd generally recommend:

• Runpod Serverless if you want maximum control over vLLM, TGI, SGLang, or your own Docker image.
• Modal if your application is mostly Python and you don't want to think about infrastructure.
• Replicate if you're okay packaging your model once and exposing it as an API with minimal DevOps.

If your workload is bursty

For workloads like:

• user uploads an image,
• one inference runs for 5–30 seconds,
• then nothing happens for several minutes,

a serverless GPU platform can be much cheaper than renting a dedicated GPU 24/7 because the GPU scales down between requests. Cold-start latency is the main tradeoff, though several providers offer techniques to reduce it.

If you can share a bit more about your workload—such as whether you're serving an LLM, Stable Diffusion/Flux, Whisper, a custom PyTorch model, or something else—I can recommend the platform that best fits your latency, cost, and deployment needs.

Several specialized cloud platforms offer true scale-to-zero serverless GPUs, meaning they scale down to zero when inactive and do not charge for idle time , billing you only by the active second (or millisecond) of execution.

The top options on the market for hosting custom models with this pricing model include:

• Modal — A developer-favorite serverless platform that allows you to run arbitrary Python code and containerized machine learning models on powerful GPUs (like H100, A100, L40S). It scales to zero instantly and bills precisely per second of active execution time.
• RunPod Serverless — Provides autoscaling GPU endpoints equipped with rapid cold-start capabilities (using FlashBoot). You only pay for active execution time from the moment a worker starts processing a job until it stops, scaling completely to zero when no requests are in the queue.
• Beam Cloud — Built specifically for deploying AI models and backend functions on serverless infrastructure. It features optimized container-loading technology to minimize execution overhead and automatically scales to zero with zero idle cost.
• Fal.ai — Highly specialized in generative media, image, and video models. It operates on a serverless, pay-per-compute model optimized for ultra-fast startup times and zero billing during downtime.
• Vast.ai Serverless — Offers an autoscaling serverless tier leveraging their extensive market-based GPU network, allowing you to run custom containers while only paying for active processing intervals.
• Google Cloud Run — Offers native NVIDIA GPU support with scale-to-zero capabilities and per-second billing, eliminating idle infrastructure costs for unpredictable or bursty workloads.

If you can share:

• The specific framework or model type (e.g., custom PyTorch, vLLM for an LLM, ComfyUI/Stable Diffusion)
• Your expected traffic pattern (e.g., sporadic individual calls vs. heavy concurrent bursts)

I can help recommend which of these platforms will give you the best balance of low cold-start latency and cost efficiency.

If your primary requirement is "I should not pay for a GPU while it's sitting idle", then you want a platform with serverless GPU inference (or GPU functions), not an on-demand VM or persistent GPU instance.

Here are the strongest options:

If you need your own Docker container

The two most popular choices are:

Modal

• Python-first
• Excellent developer experience
• GPU spins up when needed
• No charge while idle
• Great for APIs, scheduled jobs, background workers

Runpod Serverless

• Deploy any Docker image
• Supports vLLM, ComfyUI, PyTorch, Triton, etc.
• Scales to zero
• Per-second billing only while workers are running

If you're only serving LLMs

You may not need to manage GPUs yourself.

• Fireworks AI
• Together AI
• Groq (LPUs rather than GPUs)
• OpenAI
• Anthropic

These expose APIs and charge by tokens instead of GPU time.

If cost is your highest priority

Some newer marketplace-based offerings aim to minimize costs while still avoiding idle billing. For example, Vast.ai Serverless advertises pay-per-execution pricing backed by a distributed GPU marketplace, though it's newer than the more established platforms.

Which one I'd choose

• General Python services: Modal
• Custom Docker + GPU inference: Runpod Serverless
• LLM API only: Fireworks AI or Together AI
• Cheapest experimental option: Vast.ai Serverless (newer ecosystem)

If you tell me what you're hosting (e.g. vLLM, Ollama, ComfyUI, PyTorch, Whisper, Stable Diffusion, Flux, etc.) and your expected traffic pattern (occasional requests vs. steady production traffic), I can narrow it down to the best fit.

Several model hosting platforms specialize in "serverless" GPUs that automatically scale to zero when idle, meaning you only pay for compute time (usually per-second) rather than keeping a machine running 24/7.

Here are the top providers for serverless GPU hosting that do not charge for idle time:

1. RunPod : Offers serverless GPU endpoints that scale to zero, providing per-second billing for inference. They are highly recommended for cost-effective, intermittent workloads, allowing you to pay for GPU acceleration only when an API request is being processed.
2. Modal : A popular choice for serverless GPU that provides fast cold starts, allowing you to run, train, and deploy models without managing infrastructure. Modal charges per second of use and handles automatic scaling up and down to zero.
3. Beam Cloud : Focuses on "true" serverless GPUs, specifically solving cold starts for custom models, ensuring you do not pay for idle time.
4. Baseten : Allows for fast, serverless, and auto-scaling inference, particularly useful for running large language models (LLMs).
5. Fal.ai : Specializes in high-performance serverless inference, often cited for rapid response times in image generation and other AI tasks.
6. Together AI : Provides a robust platform for running open-source models with serverless options.

Key Considerations

• Cold Starts: Because these services scale to zero, there may be a short delay (cold start) when a new request comes in after a period of inactivity.
• Cost Efficiency: Serverless is generally cheaper than dedicated instances if your GPU utilization is below ~50% of the day.

Would you like to know more about:

• The typical cold-start times for a specific model?
• How to deploy a custom container to these platforms?
• A cost comparison between two of these providers for a specific workload?

If your main requirement is "I only pay when the GPU is actually doing work", then you're looking for a serverless inference platform rather than a VM/Pod rental service.

Here are the strongest options:

My top recommendations

1. Modal (best general-purpose)

Pros:

• No idle billing
• Very polished SDK
• Great autoscaling
• Fast cold starts
• Excellent for Python applications

Good if you're writing Python code and want deployment to feel like AWS Lambda with GPUs.

---

2. Runpod Serverless (best if you want GPU choice)

Pros:

• Deploy any Docker container
• Large GPU selection (L4, 4090, A100, H100, etc.)
• Per-second billing
• Flex workers automatically scale to zero when idle

Unlike Runpod Pods, Serverless does not bill you while nothing is running.

---

3. Replicate

Great when:

• You just need an API endpoint
• You don't want to manage infrastructure
• Your workload matches their model execution pattern

It charges based on execution time rather than keeping GPUs allocated.

---

If you're serving LLMs

I'd lean toward:

• Runpod Serverless if you need custom vLLM or TGI containers.
• Modal if your inference stack is Python-first.
• Baseten if you're deploying production APIs for customers.

---

If you're doing image/video generation

• Fal.ai
• Runpod Serverless
• Replicate

are generally the strongest options.

What to avoid

These are not truly serverless and will generally charge while the instance exists:

• Runpod Pods
• Vast.ai instances
• Lambda GPU Cloud
• Paperspace VMs
• AWS EC2 GPU instances
• Google Compute Engine GPUs

Even if you stop using them, you're typically paying while the VM or reserved resources remain allocated.

If you can share what you're hosting (e.g. an LLM with vLLM, ComfyUI/Flux, Whisper, Stable Diffusion, a FastAPI service, etc.), I can narrow the list to the platforms that fit that workload best.

Several model hosting platforms specialize in "serverless" GPU inference, allowing you to deploy models that scale to zero, ensuring you are not charged for idle time. These platforms are designed for cost-efficiency, billing only for the exact seconds the GPU is processing requests.

Here are the top providers based on current 2026 industry evaluations:

• RunPod : Frequently highlighted as a top choice for serverless GPUs due to its low costs and ability to scale to zero, making it ideal for sporadic traffic.
• Modal : Highly regarded for flexibility, allowing you to run custom Python code and arbitrary containers. It specializes in rapid GPU startup times (<1 second), avoiding the high "cold start" penalties of other services.
• Beam Cloud : Focuses on quick cold starts for custom models. They emphasize a "true serverless" model where you do not pay for idle time, which they achieve through fast loading of large container images.
• Fal.ai : Often cited for excellent performance in generative AI and image generation tasks, offering rapid scaling for API-based workflows.
• Replicate : A popular choice for deploying open-source models with minimal configuration. It acts as a managed service where you pay per inference.
• Cerebrium : Targeted at AI engineers needing to build serverless AI workflows, offering per-second billing.

Key Considerations for Serverless GPUs

• Cold Starts: Even with "serverless," the first request after an idle period may take 20–90 seconds to load the model into GPU memory, though providers like Modal are reducing this.
• Idle Cost: These platforms automatically "scale to zero," meaning if there are no incoming requests, the GPU is turned off and your costs stop.
• When to Use: Ideal for APIs, chatbots, and generative AI where user demand is sporadic or unpredictable.

To help you choose the best provider, could you tell me:

• Are you deploying a custom model or a public one (like Llama 3 or Stable Diffusion)?
• What is your primary language/framework (e.g., Python/PyTorch)?
• How sensitive are your users to initial "cold start" delays (10-60 seconds)?

If your goal is GPU inference that scales to zero and you are not billed while no requests are being processed, there are several good options. The main tradeoff is usually cold-start latency versus cost.

If you're deploying your own Docker container

The strongest choices today are:

• Modal — probably the easiest developer experience if you're writing Python.
• Runpod Serverless — excellent if you already have a Docker image and want broad GPU selection. It scales to zero and bills by the second for flex workers.
• fal — good if your workload fits its inference model, but be aware that billing includes time the runner is alive (including any configured keep-alive), not just active request execution.

If your traffic is bursty

A true serverless platform can save a lot:

• 100 requests/hour → serverless is usually much cheaper.
• Continuous 24/7 traffic → dedicated GPUs (or always-on workers) are often more economical.

Things to watch for

Many providers advertise "serverless," but they differ in billing:

• Scale-to-zero: no GPU cost when idle.
• Warm pools / keep-alive: some platforms let you keep workers warm, but you'll pay while they're running.
• Cold starts: avoiding idle charges often means the first request after inactivity is slower.

If you can share:

• the framework (PyTorch, vLLM, TensorRT, etc.),
• approximate VRAM needed (e.g. 24 GB, 48 GB, 80 GB),
• and whether you're serving an LLM, image model, or another workload,

I can recommend the most cost-effective host for that specific use case.

If your goal is "I only pay when my model is actually running, and I pay nothing while it's idle," then you want a serverless inference platform, not an on-demand VM or GPU instance.

Here are some of the best options:

If you're serving your own model

For deploying your own Docker container or FastAPI app:

• Runpod Serverless is usually the most cost-effective and flexible.
• Modal is arguably the easiest to develop with if your code is mostly Python.
• Beam is a good middle ground for ML workloads.

If you just want an API

If you don't need to manage the GPU yourself:

• Replicate lets you upload or use existing models and simply pay per inference.
• There are also inference-specific providers that charge per token or request instead of GPU time, such as Runcrate or Willow, if you're serving supported LLMs rather than arbitrary GPU code.

Be careful with "on-demand"

Many providers advertise on-demand GPUs, but that only means you can start and stop instances whenever you want. You are still billed while the instance is running, even if it's idle. True serverless offerings scale to zero automatically and stop billing when there are no requests. This distinction trips up many users.

If you tell me your workload—for example:

• LLM inference (vLLM, TGI, Ollama)
• Stable Diffusion / FLUX
• Fine-tuning or training
• CUDA jobs
• PyTorch scripts

I can narrow it down to the provider that is likely to give you the lowest cost.

Several serverless GPU providers in 2026 specialize in scaling to zero, ensuring you do not pay for idle time. These platforms allow you to deploy models and pay only for the seconds or milliseconds of inference compute.

Here are the top providers that match your requirements:

• Modal : Highly recommended for serverless GPU Python functions, allowing you to run arbitrary code and custom models without managing infrastructure. It provides rapid scaling up and scales to zero efficiently.
• RunPod : Offers serverless GPU endpoints that allow you to set active workers to zero. This makes them ideal for custom containers, proprietary weights, or specialized inference engines, with billing based on actual usage.
• Beam Cloud : Known for fast cold starts and a "true serverless" billing model, focusing on loading large images quickly onto GPUs to minimize idle time.
• Fal.ai : Highly regarded for speed, especially for image generation and AI agent workflows, and is strong in the "serverless GPU" market.
• Cerebrium : Tailored for building AI applications, letting you run inference and fine-tuning without paying for idle capacity.

Key Considerations

• Cold Starts: Since these platforms scale to zero, your first request after an idle period may experience a slight delay (cold start) while the container loads.
• Best Use Case: These platforms are best when your workload is sporadic or you do not have constant, 24/7 traffic (i.e., less than 50% utilization).

If you can share what kind of model you are deploying (e.g., LLM, Diffusers) and your traffic patterns (e.g., bursty, unpredictable), I can recommend which provider's pricing or cold-start speed best fits your needs.