Results for “gpu serving”

• vLLMby vLLMLLM inference and serving library.FreeWatch
• NVIDIA AI Workbenchby NVIDIADesktop app for building and managing GPU-backed AI projects across local and remote machinesFreemiumWatch
• LMCacheby LMCacheOpen-source KV-cache infrastructure for storing and reusing LLM inference cache.FreeWatch
• LLM Compressorby IBMOpen-source Python library for quantizing and pruning LLMs to run faster in vLLMFreeWatch
• Rayby AnyscaleOpen-source Python framework for scaling AI and ML workloadsFreeWatch
• Lambda Stackby LambdaCurated Ubuntu software stack for machine learning and deep learningUnknownWatch
• LeRobotby Hugging FaceOpen-source Python library for training and running robot policies from demonstrationsFreeWatch
• Transformersby Hugging FaceOpen-source Python library defining state-of-the-art ML models for inference and trainingFreeWatch
• Magnitudeby MagnitudeYC S25Local open-model inference engine with hardware-tuned kernels, desktop app, and CLI.FreeWatch

That’s all 9 tools.