SERVERLESS ENDPOINTS

Production inference.
Zero overhead.

Run models with automatic scaling, optimized routing, and token-based pricing.

What are Serverless Endpoints?

Fast, scalable inference endpoints without managing infrastructure.

Run top open-source models, auto-scale with traffic, and pay only for what you use - tokens in, tokens out.

HOW IT WORKS

Blazing fast inference
Serve open-source models fast with minimized cold starts and real-time responsiveness.
Effortless auto-scaling
Scales automatically to meet peak demand—no setup, no ops, no interruptions.
Only pay for tokens
Pay only for input and output tokens—never for idle time or unused capacity.
Fully managed inference
Serve models instantly with a single API call—no infra, setup, or scaling required.

Optimized to deliver open source model inference – at scale

SCALE
1000+
GPUs to scale to
SPEED
60s
or less to scale

Run scalable inference of leading open-source models on

fast serverless endpoints

LLM
Llama 3.1 70B
META - NVIDIA H100 SXM
A multilingual LLM, pre-trained and instruction-tuned model, with top performance on key benchmarks.
LLM
Llama 3.1 8B
META - NVIDIA L40S
A small multilingual instruction-tuned LLM with top performance for it size.
LLM
Mistral 7B-Instruct
MISTRAL - NVIDIA L40S
Instruct fine-tuned version of the Mistral-7B-v0.3.
LLM
Qwen 2.5 1.5B
QWEN - NVIDIA L4
The small language model from the Qwen 2.5 family with up to 128k context length.
LLM
Llama 3.2 3B
META - NVIDIA L40S
A multimodal LLM fine-tuned for image reasoning, visual recognition and answering image-based queries.
LLM
TinyLlama 1.1B
TINYLLAMA - NVIDIA L4
A small language model for use with constrained memory footprint.