Results for “async inference”

14 tools in the directory · Updated 2 Nov 2026

“async inference”Download or installRuns on Command lineShowing 14 tools

vLLMby vLLMLLM inference and serving library.Free
LMCacheby LMCacheOpen-source KV-cache infrastructure for storing and reusing LLM inference cache.Free
OpenVINOby IntelOpen source AI inference toolkit for deploying models across Intel hardwareFree
Transformersby Hugging FaceOpen-source Python library defining state-of-the-art ML models for inference and trainingFree
Magnitudeby MagnitudeYC S25Local open-model inference engine with hardware-tuned kernels, desktop app, and CLI.Free
Junieby JetBrainsJetBrains' coding agent that plans and edits code, with support for multiple AI models.Freemium
mlpackby mlpackOpen-source C++ machine-learning library with Python, Julia, R, Go and command-line interfaces.Free
Haystackby deepsetOpen-source Python framework for building RAG, agent, and multimodal LLM applicationsFreemium
Codeflashby CodeflashCodeflash optimizes code and benchmarks changes while checking whether behavior is preserved.Freemium
smolagentsby Hugging FaceOpen-source Python library for building agents that write their actions as codeFree
LLM Compressorby IBMOpen-source Python library for quantizing and pruning LLMs to run faster in vLLMFree
Rayby AnyscaleOpen-source Python framework for scaling AI and ML workloadsFree
Ollamaby OllamaYC W21Run and build with open AI models locally or in the cloudFreemium
LIBSVMby LibSVMLIBSVM is a software library and command-line toolkit for support vector classification, regression, and one-class estimation.Free