Results for “distributed inference”

55 tools in this category · Updated 2 Nov 2026

“distributed inference”CodingRuns on Self-hostedShowing 55 tools

Red Hat AI Inferenceby IBMSelf-managed vLLM and llm-d inference stack for running open models on your own acceleratorsContact sales
Superlinked Inference Engineby SuperlinkedManaged Superlinked Inference Engine API for embeddings, reranking, extraction and chat with small modelsPay as you go
Cloudera AI Inferenceby ClouderaModel serving for deploying and scaling private AI applications, agents, and assistants.Pay as you go
Inference.netby inference.netInference infrastructure for serving, monitoring, and training LLMsFreemium
Makora Inferenceby MakoraAI inference service with agent-driven optimization across the serving stack$20+/mo
Roboflow Inferenceby RoboflowYC S20Self-hosted computer vision inference server with WorkflowsFree
vLLMby vLLMLLM inference and serving library.Free
Nutanix Enterprise AIby NutanixEnterprise AI platform for governing agents, models, and inference across hybrid infrastructureContact sales
Manifest LLM Gatewayby ManifestAn open-source gateway for routing LLM requests, handling provider failures, and tracking inference use.Freemium
Argmax SDKby ArgmaxA developer SDK for running speech and other foundation-model inference on supported devices.Freemium
LMCacheby LMCacheOpen-source KV-cache infrastructure for storing and reusing LLM inference cache.Free
Inferactby InferactProduction LLM inference platform from the creators of vLLMContact sales
Gimlet Cloudby Gimlet LabsManaged inference API for agentic AI workloads across heterogeneous hardwareUnknown
Basetenby BasetenHosted model APIs and dedicated GPU inference for production AI appsPay as you go
NVIDIA NIMby NVIDIAPrebuilt inference containers for running AI models on NVIDIA GPUsFreemium
OpenVINOby IntelOpen source AI inference toolkit for deploying models across Intel hardwareFree
Flowerby FlowerYC W23Build federated AI systems across distributed datasetsFreemium
Magnitudeby MagnitudeYC S25Local open-model inference engine with hardware-tuned kernels, desktop app, and CLI.Free
NVIDIA Run:aiby NVIDIAOpen source GPU scheduling and workload governance for shared Kubernetes AI clustersFree
PyTorchby PyTorch FoundationOpen source machine learning framework for building and training deep learning modelsFree
LLM Compressorby IBMOpen-source Python library for quantizing and pruning LLMs to run faster in vLLMFree
NVIDIA Dynamoby NVIDIAOpen-source framework for serving LLMs across multi-GPU, multi-node clustersFree
Otariby Mozilla.aiControl model routing, budgets and access through one AI gatewayUnknown
llamafileby Mozilla.aiRun local language models from portable executable filesFree