Results for “gpu serving”

• vLLMby vLLMLLM inference and serving library.FreeWatch
• LMCacheby LMCacheOpen-source KV-cache infrastructure for storing and reusing LLM inference cache.FreeWatch
• Rayby AnyscaleOpen-source Python framework for scaling AI and ML workloadsFreeWatch
• PyTorchby PyTorch FoundationOpen source machine learning framework for building and training deep learning modelsFreeWatch
• FastEmbedby QdrantLightweight Python library for generating text, sparse, image and reranking embeddingsFreeWatch
• LanceDBby LanceDBYC W22AI-native multimodal lakehouse for training data, features, and retrievalFreemiumWatch
• Magnitudeby MagnitudeYC S25Local open-model inference engine with hardware-tuned kernels, desktop app, and CLI.FreeWatch

That’s all 7 tools.