GPU instance variety and scaling

If you want an inference platform with a wide selection of GPUs and the ability to scale compute up or down for different workloads, I'd start by comparing Runpod, Modal, CoreWeave, and Baseten.

The key distinction is whether you want to rent flexible GPU infrastructure and manage your own inference stack, or use a more managed platform that handles deployment, autoscaling, and serving for you.

Best inference platforms to evaluate

• 

1. Runpod

(Best overall starting point)

Offers a particularly broad range of GPU choices, from consumer RTX cards to data-center GPUs, plus serverless endpoints that scale workers according to demand.www.runpod.io+1

• GPU variety: RTX 4090/5090, L40S, A100, H100, H200, B200 and other options.
• Scaling: Serverless workers can scale to zero when idle and expand with traffic.
• Workloads: LLM inference, image and video generation, embeddings, batch processing, and custom containers.
• Best for: Teams wanting hardware choice, flexibility, and usage-based infrastructure.

Website: Runpod Serverless

---

• 

2. Modal

(Best for developer productivity)

A code-first platform for deploying custom models and inference services without managing the underlying infrastructure. It supports elastic scaling for online serving and batch workloads.modal.com+1

• GPU variety: Configurable GPU resources for custom inference stacks.
• Scaling: Can scale workloads up during demand spikes and down to zero when idle.
• Workloads: Real-time LLM serving, asynchronous batch jobs, audio, and custom ML applications.
• Best for: Python-centric teams that prioritize fast deployment and automated scaling.

Website: Modal Inference

---

• 

3. CoreWeave

(Best for large-scale production)

Provides specialized GPU infrastructure and dedicated inference services with control over GPU class, runtime, routing, and scaling.www.coreweave.com+1

• GPU variety: High-end NVIDIA data-center GPU infrastructure, including newer Blackwell-class systems.
• Scaling: Managed dedicated deployments and distributed multi-node serving.
• Workloads: Large LLMs, high-throughput inference, and production deployments with demanding performance requirements.
• Best for: Organizations needing predictable capacity, infrastructure control, and large-scale serving.

Website: CoreWeave Dedicated Inference

---

• 

4. Baseten

(Best for optimized model serving)

Focuses on production inference, optimized model runtimes, rapid deployment, and autoscaling. Its platform also supports custom serving workflows and fine-grained hardware choices for compound AI applications.www.baseten.co+1

• GPU variety: Dedicated deployment options; verify the exact current GPU catalog for your requirements.
• Scaling: Autoscaling, with an emphasis on fast startup and inference efficiency.
• Workloads: Production LLM APIs, speech, embeddings, and image generation.
• Best for: Teams that care more about inference performance and operational simplicity than managing raw GPU infrastructure.

Website: Baseten

One more strong option: Vast.ai

• 

5. Vast.ai

(Best for GPU diversity and price flexibility)

Vast.ai combines a GPU marketplace with serverless inference, offering access to consumer GPUs and enterprise accelerators. Its serverless platform can automatically scale workers and select among supported hardware types.vast.ai+1

• GPU variety: RTX 4090/5090, A100, H100, H200, B200, and many other types.
• Scaling: Dynamic worker scaling and configurable hardware options.
• Pricing: Market-driven rates that vary by GPU, host, and availability.
• Trade-off: Marketplace infrastructure can require more scrutiny of reliability, capacity guarantees, and host quality.

Website: Vast.ai Serverless

How the platforms compare

These are qualitative comparisons based on each provider's published offerings, not independent performance benchmarks. Actual GPU availability, regional coverage, and pricing should be checked for your workload.

My recommendation

I'd shortlist Runpod and Vast.ai if access to many GPU types is your top priority, and Modal if you want to switch between workloads while keeping infrastructure management to a minimum. For a high-throughput enterprise deployment, evaluate CoreWeave as well.

To narrow this down, one important question:

What kinds of inference workloads do you need to run?

1. Workload types

---

2. Scaling requirements

Autoscale based on request traffic

Manually provision GPUs as needed

Both, depending on the workload

---

3. Most important factor

Wide variety of GPU models

Lowest cost per inference

Maximum throughput and low latency

Enterprise reliability and guaranteed capacity

Ease of deployment and operations

---

4. Preferred deployment

Serverless API endpoints

Dedicated GPU instances or clusters

Either / a mix of both

[Find my best-fit platforms]