// category · 10 tools · 3 open source
Inference & Hosting
Serving engines, local runtimes, gateways and cloud platforms for running models fast and cheaply in production or on your laptop.
Ollamaollama.comThe simplest way to download and run open models locally, with a CLI, desktop app and local API.Inference & HostingOpen source★ 182.3k
Basetenbaseten.coProduction inference platform for deploying open and custom models with dedicated, autoscaling GPUs.Inference & HostingPaid★ 1.2k
OpenRouteropenrouter.aiUnified API to hundreds of models from many providers with automatic fallbacks and price routing.Inference & HostingFreemiumHN 34
Modalmodal.comServerless platform for running Python and GPU workloads, from inference endpoints to sandboxes, defined in code.Inference & HostingFreemiumHN 18
vLLMdocs.vllm.aiHigh-throughput LLM serving engine built around PagedAttention, with an OpenAI-compatible server.Inference & HostingOpen source★ 93.2k
LiteLLMlitellm.aiPython SDK and proxy gateway that calls many LLM providers through one OpenAI-style interface with budgets and logging.Inference & HostingOpen source★ 60.2k
Replicatereplicate.comRun and deploy open models through a simple API, packaging custom models with Cog.Inference & HostingPaid★ 9.5k
Fireworks AIfireworks.aiFast inference platform for open models with fine-tuning and compound AI tooling.Inference & HostingPaidHN 5
Together AItogether.aiCloud for running, fine-tuning and training open models, plus on-demand GPU clusters.Inference & HostingPaidHN 4
Groqgroq.comInference API running on custom LPU chips, known for very fast token generation.Inference & HostingFreemiumHN 1