LMCache

What it does

LMCache is a modular KV-cache layer for teams operating large language model inference. It takes computed KV-cache data and can store it beyond GPU memory in CPU memory, local disk or configured external storage. When later requests contain reusable prompt content, LMCache can retrieve cached chunks to avoid repeating some prefill computation. It can also search for reusable chunks beyond exact prefix matches with CacheBlend, compress cache data and transfer it across workers or engines.

The workflow is infrastructure-oriented: install LMCache alongside a supported inference engine, configure cache storage and a connector, then run LMCache in-process or as a separate server. The multiprocess mode supports sharing cache across engine instances and provides management and observability endpoints. Teams can inspect cache behavior through logs, metrics and traces. It is not a hosted model or end-user chat application; the inference engine and model execution remain part of the serving setup.

LMCache is intended for developers and infrastructure teams building or operating LLM serving systems, particularly where prompts, conversation history or retrieved content recur. The official quickstart documents vLLM and SGLang setup; compatibility and deployment details depend on engine, versions, hardware and storage backend. SGLang currently requires multiprocess mode, and the documented TensorRT-LLM connector depends on components not yet in stable releases.

What it can do

• Offload KV cache to CPU memory, local disk or external storage
• Reuse cached KV data across requests
• Search reusable cache chunks beyond exact prefix matches with CacheBlend
• Compress KV cache data
• Transfer cache across workers and inference engines
• Observe cache behavior through metrics, logs and traces

Works with

• vLLM
• SGLang
• TensorRT-LLM (source installation required; integration components not yet in stable releases)
• Redis/Valkey
• Amazon S3
• Google Cloud Bigtable

Before you choose

• The SGLang integration supports multiprocess mode only; in-process mode is not supported.
• Engine versions and connector behavior matter: the quickstart documents a vLLM connector distinction at vLLM 0.20.0.
• The documented TensorRT-LLM integration requires source installs because its connector preset and matching adapter have not shipped in stable releases.

Listed under

• Coding
• Sandboxes and infrastructure
• Developers
• GPU memory
• Inference infrastructure
• Kv cache
• LLM inference
• ML engineers

Get started

The quickstart shows installation with vLLM and commands to configure the connector and run requests.

1. Create a Python environment and install LMCache with the inference engine required for your deployment.
2. Choose in-process or multiprocess deployment, then configure the connector, cache capacity and storage backend.
3. Start the inference engine and LMCache as applicable, send requests with reusable prompt content, and check logs or metrics for cache activity.

Apache-2.0 software; inference, hardware, storage and hosting remain separate operating costs.

Source: github.com/LMCache/LMCache, checked 1 Nov 2026. Prices can change; check before you buy.

Sources for this overview

Product pages, documentation, and pricing checked 1 Nov 2026. Each source supports part of the overview; availability and prices can change.

• LMCache – Building the foundation of AI memory tensor with KV Cache Infrastructure ↗lmcache.ai
• Quickstart | LMCache ↗docs.lmcache.ai
• GitHub - LMCache/LMCache: LMCache: Supercharge Your LLM with the Fastest KV Cache Layer · GitHub ↗github.com
• Contact Us – LMCache ↗lmcache.ai

Company website

lmcache.ai: product, pricing and policy pages

YC directory mirror

Batch and company name

Web search

Confirms the official site

Social accounts

Accounts the company links, and its posts on X

Human review

Checked by a person

knowledge_base

manual_review

Last checked 1 Nov 2026. Every field keeps its source and date; unknown means we couldn’t confirm it.