Modal is a serverless cloud infrastructure platform built by Modal Labs Inc. It lets developers run Python code in the cloud without managing servers, Dockerfiles, or Kubernetes clusters. It is aimed at data engineering, generative AI, and machine learning teams that need to run compute-heavy workloads without operating their own infrastructure.
Deploying models or running distributed batch jobs traditionally requires setting up container registries, provisioning GPU instances, and managing scaling policies. Modal automates container building, provisioning, cold-start handling, and autoscaling, so code runs on remote CPUs and GPUs on demand.
Modal runs on a custom container engine paired with a Python SDK. Developers decorate standard Python functions with directives specifying hardware, dependencies, and state primitives. Functions can be invoked from the CLI or exposed as web endpoints, and containers scale down to zero when idle.
Its main differentiators are sub-second cold starts tuned for large AI dependencies, infrastructure defined entirely in Python instead of YAML or Dockerfiles, built-in distributed storage primitives, and per-second usage billing. Modal has completed a SOC 2 Type II audit and offers HIPAA-compatible workloads on its Enterprise plan.
Pricing
Modal bills per second of actual compute used across three plans. Starter is $0 plus compute, includes $30 of free monthly credits, and caps usage at 100 containers and 10 GPU concurrency. Team costs $250 plus compute per month, includes $100 in monthly credits, and raises limits to 5,000 containers and 50 GPU concurrency with unlimited seats. GPU rates currently range from about $0.59 per hour for an Nvidia T4 to roughly $3.95 per hour for an H100 SXM5, with newer options like the B200 and B300 priced higher; CPU is billed at about $0.047 per physical core-hour and memory at about $0.008 per GiB-hour. Enterprise pricing is custom and adds volume discounts, audit logs, Okta SSO, and HIPAA support.
* Disclaimer: Please note that pricing information may not be up to date. For the most accurate and current pricing details, refer to the official website.
Key Features
-
✓
Define infrastructure entirely in Python, no Dockerfiles or YAML
-
✓
Sub-second cold starts tuned for large AI container images
-
✓
Access to Nvidia GPUs from T4 up to B300, billed per second
-
✓
Automatic scaling to zero across thousands of containers
-
✓
Built-in Volumes, Dicts, and Queues for state without a database
-
✓
Native cron scheduling and FastAPI or ASGI web endpoints
Use Cases
Real-Time LLM Inference
Teams deploy open-source models such as Llama, Whisper, or Stable Diffusion as production REST endpoints. Modal autoscales GPU containers with request volume and keeps startup times low to reduce user-facing latency.
Model Fine-Tuning and Training
ML teams run LoRA fine-tuning or distributed training jobs on demand. Multi-GPU setups can be requested directly in Python, with billing tied to actual training time rather than idle cluster hours.
High-Throughput Batch Processing
Data engineers process large volumes of images, video, or documents across parallel workers. Modal fans out execution across many concurrent containers and aggregates results using Queues or Volumes.
Scheduled Data Pipelines
Organizations run recurring extraction and transformation jobs, such as scraping or ETL. Native cron scheduling replaces external orchestration tools, keeping the schedule inside the application repository.
Full-Stack AI Backends
Startups build applications that combine a web backend with hardware-accelerated model execution. Hosting both the API and GPU compute on one platform reduces architecture complexity.
Strengths & Weaknesses
Strengths
Removes the need for Dockerfiles, Kubernetes, or manual cloud console setup, since infrastructure is defined entirely in Python.
Delivers fast cold-start performance tuned specifically for large AI container images and model weights.
Starter plan includes $30 in monthly free compute credits, lowering the barrier to prototyping.
Includes built-in Volumes, Dicts, and Queues, removing the need to set up an external database for simple state.
Scales from a single local terminal command to thousands of cloud containers without additional orchestration.
Weaknesses
Tightly coupled to Python, making it unsuitable for teams with primarily non-Python backends.
Relies on Modal’s proprietary SDK and runtime, which introduces a degree of vendor lock-in.
Maximum execution timeouts make it a poor fit for continuous, uninterrupted monolithic processes.
Starter plan concurrency is capped at 100 containers and 10 GPUs, requiring an upgrade to the $250 per month Team plan or Enterprise for larger production scale.
Who Is This For?
AI and ML engineers who need to deploy and serve models on GPUs quickly, without waiting on DevOps or building infrastructure pipelines.
Data engineers and data scientists who need scalable compute for batch or transformation pipelines without maintaining a persistent Spark or Ray cluster.
Full-stack and SaaS developers building applications with AI features that need pay-as-you-go GPU or CPU backends reachable via webhooks or REST.
AI startups and R&D teams that want to avoid upfront cluster costs and iterate quickly during early product development.
Frequently Asked Questions
What programming languages does Modal support?
Modal is built specifically for Python. Shell commands can run inside a container, but the SDK and application definitions require Python.
How is compute usage billed?
Modal charges per second that CPU, memory, or GPU resources are active. Billing stops once containers scale down to zero.
What GPUs are available on Modal?
Modal offers a range of Nvidia GPUs, from the T4 and L4 up through the A10, L40S, A100, H100, H200, and the newer B200 and B300.
Does Modal have a free tier?
Yes. The Starter plan is free plus compute and includes $30 in free compute credits each month, with no separate time-limited trial.
What does the Team plan add over Starter?
Team costs $250 plus compute per month and includes $100 in monthly credits, unlimited seats, 5,000 containers, 50 GPU concurrency, custom domains, and 30-day log retention.
How are secrets and API keys handled?
Modal provides built-in, encrypted secrets management through its CLI and dashboard, so keys do not need to be hardcoded into scripts.
Is Modal SOC 2 compliant?
Yes. Modal has completed a SOC 2 Type II audit, and the Enterprise plan adds HIPAA compatibility and audit logs.
What are the limits on the Starter plan?
Starter accounts are capped at 100 containers and 10 GPU concurrency, with 1-day log retention and up to 3 workspace seats.
Does Modal support scheduled or recurring jobs?
Yes. Cron scheduling is built into the Python decorators, with 5 deployed crons on Starter and unlimited crons on Team and Enterprise.
Modal integrates with FastAPI, Starlette, and ASGI for web endpoints; Hugging Face for downloading and caching models into Volumes; PyTorch, vLLM, and TensorRT-LLM for training and inference; GitHub Actions for CI/CD deployment; and Weights & Biases or LangSmith for logging metrics and traces from Modal function runs.