Results for “llm inference”

38 tools in the directory · Updated 2 Nov 2026

“llm inference”Ways to use: Command lineUpdated: last 7 daysShowing 38 tools

Manifest LLM Gatewayby ManifestAn open-source gateway for routing LLM requests, handling provider failures, and tracking inference use.Freemium
vLLMby vLLMLLM inference and serving library.Free
Tinfoil Inferenceby TinfoilYC X25Call AI models through an attested confidential-inference APIPaid
LMCacheby LMCacheOpen-source KV-cache infrastructure for storing and reusing LLM inference cache.Free
Doublewordby DoublewordHosted inference API for open-weight models, with real-time, async, batch and dedicated-capacity options.Pay as you go
Magnitudeby MagnitudeYC S25Local open-model inference engine with hardware-tuned kernels, desktop app, and CLI.Free
Vast.aiby Vast.aiProvision GPU compute for AI training and inferencePay as you go
Orq.aiby Orq.aiAI gateway and platform for routing, governing and observing LLM and agent traffic across 500+ models.Freemium
ChainForgeby ChainForgeOpen-source visual environment for prompt engineering and LLM evaluation.Free
Otariby Mozilla.aiControl model routing, budgets and access through one AI gatewayUnknown
Oumiby OumiA platform for evaluating, training, and deploying custom AI models.Freemium
Guardrails AIby Guardrails AIValidate model inputs and outputs with reusable Python guardsContact sales
Instructorby InstructorA developer library for extracting structured, validated data from language models.Unknown
llamafileby Mozilla.aiRun local language models from portable executable filesFree
encoderfileby Mozilla.aiPackage embedding and classification models as portable executablesFree
Flower Agentby FlowerYC W23Build and run agent applications with managed context and toolsUnknown
MiroThinkerby MiroMindMiroThinker is a deep-research agent and open-source model family for research and prediction tasks.Freemium
Overmindby OvermindA platform for tracing AI applications, preparing data, evaluating models, and training and serving specialized models.Freemium
Union.aiby Union.aiOperate durable AI workflows in your own cloudFreemium
Simplismartby SimplismartDeploy, serve, and optimize AI models on shared or dedicated infrastructure.Pay as you go
Mixedbreadby MixedbreadTurn files into searchable evidence for AI agentsFreemium
mlpackby mlpackOpen-source C++ machine-learning library with Python, Julia, R, Go and command-line interfaces.Free
Repomixby RepomixPackage repository context for an AI assistantFree
MLflowby MLflowOpen-source platform for tracing, evaluating, and managing AI applications and machine-learning models.Free