Results for “llm inference”

106 tools in this category · Updated 2 Nov 2026

“llm inference”CodingRuns on Self-hostedShowing 106 tools

Manifest LLM Gatewayby ManifestAn open-source gateway for routing LLM requests, handling provider failures, and tracking inference use.Freemium
vLLMby vLLMLLM inference and serving library.Free
Inference.netby inference.netInference infrastructure for serving, monitoring, and training LLMsFreemium
Red Hat AI Inferenceby IBMSelf-managed vLLM and llm-d inference stack for running open models on your own acceleratorsContact sales
Superlinked Inference Engineby SuperlinkedManaged Superlinked Inference Engine API for embeddings, reranking, extraction and chat with small modelsPay as you go
Cloudera AI Inferenceby ClouderaModel serving for deploying and scaling private AI applications, agents, and assistants.Pay as you go
Makora Inferenceby MakoraAI inference service with agent-driven optimization across the serving stack$20+/mo
LLM Compressorby IBMOpen-source Python library for quantizing and pruning LLMs to run faster in vLLMFree
any-llmby Mozilla.aiSwitch LLM providers through a common Python interfaceFree
Deepchecks LLM Evaluationby DeepchecksProduction-grade LLM evaluation, testing, and monitoring platformPaid
RubyLLMby RubyLLMBuild model-powered applications and agents in RubyFree
LMCacheby LMCacheOpen-source KV-cache infrastructure for storing and reusing LLM inference cache.Free
Inferactby InferactProduction LLM inference platform from the creators of vLLMContact sales
Roboflow Inferenceby RoboflowYC S20Self-hosted computer vision inference server with WorkflowsFree
LiteLLMby LiteLLMYC W23Open-source AI gateway that puts many LLM providers behind one API with spend caps and routingFreemium
Nutanix Enterprise AIby NutanixEnterprise AI platform for governing agents, models, and inference across hybrid infrastructureContact sales
.txt APIby .txtStructured outputs API that enforces JSON Schema, regex, and grammar constraints on LLM responses.Pay as you go
Basetenby BasetenHosted model APIs and dedicated GPU inference for production AI appsPay as you go
NVIDIA Dynamoby NVIDIAOpen-source framework for serving LLMs across multi-GPU, multi-node clustersFree
NVIDIA NIMby NVIDIAPrebuilt inference containers for running AI models on NVIDIA GPUsFreemium
OpenVINOby IntelOpen source AI inference toolkit for deploying models across Intel hardwareFree
Magnitudeby MagnitudeYC S25Local open-model inference engine with hardware-tuned kernels, desktop app, and CLI.Free
Fabric Gatewayby PostmanGoverned gateway for LLMs, MCP servers, and internal APIsFreemium
Agora Conversational AI Engineby AgoraVoice AI agent engine that connects any LLM to real-time speech.Pay as you go