Red Hat AI Inference

What it does

Red Hat AI Inference is an integrated inference stack that takes in open source or custom language models and serves them to applications, agents and chatbots. It is powered by vLLM and llm-d. It produces model-serving endpoints on your own accelerators, plus telemetry such as Time-to-First Token, KV-cache hit rate and GPU utilization. A model optimization toolkit can compress models through techniques like quantization or sparsity.

Teams deploy it as a container on OpenShift Container Platform, in disconnected environments, or as distributed inference with llm-d on OpenShift. Early access to llm-d is offered on Azure, AWS or CoreWeave Kubernetes. The aim is a Model-as-a-Service setup that runs any model on any supported accelerator, with a collection of validated models. Documentation also covers speculative decoding, tool calling, Mistral 3 and geospatial models.

It suits platform and infrastructure teams at organizations that run their own GPUs or other accelerators across hybrid cloud and need vendor support. It is sold as a standalone product or within Red Hat AI Enterprise and OpenShift AI, and is priced per accelerator. It is not a hosted chat app: you need supported hardware and Linux or Kubernetes skills. A separate pay-per-token managed version runs on IBM Cloud.

What it can do

• Serve open source and custom models with a vLLM-based runtime
• Run distributed inference with llm-d on OpenShift, with early access on third-party Kubernetes
• Gen-AI telemetry: Time-to-First Token, KV-cache hit rate, GPU utilization
• Model optimization toolkit using sparsity or quantization
• Run sparse Mixture of Experts architectures
• Deploy in disconnected environments with a mirror image registry

Works with

• Red Hat OpenShift
• Red Hat Enterprise Linux
• Red Hat OpenShift AI
• Red Hat AI Enterprise
• Microsoft Azure Kubernetes
• CoreWeave Kubernetes Service
• AWS
• IBM Cloud

Example workflows

Serve a model on OpenShift

Inputs: an OpenShift cluster with supported accelerators and a validated model. Steps: follow the getting-started guide, deploy the standalone container, serve the model. Output: a model endpoint with performance telemetry.

Compress a model before serving

Inputs: a custom or foundation model. Steps: apply quantization or sparsity with the model optimization toolkit, then serve it. Output: a smaller model that needs less hardware.

Before you choose

• Priced per accelerator; amounts are not published.
• llm-d on third-party Kubernetes is early access; third-party support follows Red Hat's policy.
• You supply and operate the hardware and cluster.

Listed under

• Coding
• Sandboxes and infrastructure
• Distributed inference
• Documentation
• Enterprise
• Enterprise AI
• Inference
• Kubernetes

Get started

Standalone or bundled purchase

1. Check the supported product and hardware configurations in the documentation.
2. Start a trial or contact Red Hat sales.
3. Deploy the container or llm-d following the getting-started guide.

Priced per accelerator; no amounts, trial length or free tier were confirmed. A separate pay-per-token managed service exists on IBM Cloud.

We couldn’t find published prices for Red Hat AI Inference. See their pricing page ↗

Sources for this overview

Product pages, documentation, and pricing checked 29 Oct 2026. Each source supports part of the overview; availability and prices can change.

• Red Hat AI Inference ↗www.redhat.com
• Red Hat AI Inference | 3.4 | Red Hat Documentation ↗docs.redhat.com

Company website

redhat.com: product, pricing and policy pages

YC directory mirror

Batch and company name

Web search

Confirms the official site

Social accounts

Accounts the company links, and its posts on X

Human review

Checked by a person

knowledge_base

Last checked 29 Oct 2026. Every field keeps its source and date; unknown means we couldn’t confirm it.