ChainForge

What it does

ChainForge is an open-source visual programming environment for developers and researchers evaluating prompts and text-generation models. Users create prompt variations, query multiple LLMs or one model at different settings, then compare and inspect responses. The visual workflow supports templated conversations, code- or LLM-based scoring, and plots across prompts, parameters, models, and settings. Experiments can be cached and exported to Excel; the online version also lets users share experiments by link.

Use it to test prompt robustness, including against prompt-injection attacks; check consistency of output formats; compare prompt and model responses; evaluate conversations turn by turn; and visualize scorer results. ChainForge runs in a browser, either through its online version or locally after installing the Python package and serving the app. The local setup supports additional features, including environment-variable API keys and querying locally run Alpaca/Llama models hosted via Dalai. Users need API keys for the LLMs they query. The online version is limited, and keys must be set again each time it is loaded. The documentation describes ChainForge as an open beta test. Budget for any configured model-provider usage and local operating resources.

What it can do

• Query multiple LLMs or one LLM at different settings
• Compare responses across prompt variations, models, and settings
• Create and inspect templated multi-turn conversations
• Score responses using code or LLM-based metrics
• Plot evaluation results across prompts, parameters, models, and settings
• Cache and export prompt results to Excel

Works with

• LLM APIs requiring user-provided API keys
• Locally run Alpaca/Llama models hosted via Dalai

Before you choose

• The online version has a more limited feature set than the local version.
• LLM API keys are required. The documentation says keys are not stored and must be set again on each load; local users can set keys as environment variables.
• The official documentation describes the product as an open beta test; model-provider costs are not described.

Listed under

• Coding
• MCP and agent tools
• Developers
• LLM evaluation
• Model comparison
• No-code
• Open source
• Prompt engineering

Get started

Local installation uses pip install chainforge followed by chainforge serve; the online version is at chainforge.ai/play.

1. For local use, install with `pip install chainforge`.
2. Run `chainforge serve` and open `localhost:8000` in Chrome, Firefox, Edge, or Brave.
3. Provide API keys for the LLMs you want to query; the online version requires keys to be set again each time it is loaded.

The project is open source. External model usage and infrastructure can carry separate charges; no hosted-service commercial plan is verified.

We couldn’t find published prices for ChainForge.

Sources for this overview

Product pages, documentation, and pricing checked 1 Nov 2026. Each source supports part of the overview; availability and prices can change.

• ChainForge Documentation ↗chainforge.ai
• ChainForge: A visual programming environment for prompt engineering ↗chainforge.ai
• For Developers - ChainForge Documentation ↗chainforge.ai

Company website

chainforge.ai: product, pricing and policy pages

YC directory mirror

Batch and company name

Human review

Checked by a person

knowledge_base

manual_review

Last checked 1 Nov 2026. Every field keeps its source and date; unknown means we couldn’t confirm it.