Bloom

What it does

Bloom generates evaluations for a specific behavior you want to measure in an AI model. You describe that behavior and configure a seed; the framework develops scenarios, runs interactions with the target model, and uses model-based judges to score what happened. It is intended for researchers who need many targeted tests rather than a single fixed benchmark.

The pipeline separates understanding the behavior, designing scenarios, running conversations, and judging results. Researchers can inspect transcripts, compare models, adjust scenario variations, and run larger sweeps. Examples include measuring sycophancy, self-preservation, or other behavior relevant to model evaluation. The seed and configuration are part of interpreting a score, and model-based judgments still need validation.

Anthropic originally released Bloom, but the linked repository now states that development and maintenance have moved to Meridian Labs. The old standalone repository is frozen, and new projects are directed to the maintained Petri/Bloom project. The software is open source; running evaluations uses your own environment and model APIs, whose costs depend on the models and scale of the experiment.

What it can do

• Generate scenarios for a chosen model behavior
• Run evaluations across target models
• Score and inspect conversation transcripts
• Configure scenario variations and judge models
• Export results for analysis and experiment tracking

Works with

• Model APIs through LiteLLM
• Weights & Biases
• Inspect-compatible transcripts

Example workflows

Measure a specific behavior

Define a behavior, inspect a small generated suite, refine the seed, then compare target models using the same configuration.

Investigate a model change

Repeat a documented evaluation configuration and review transcript-level differences before interpreting aggregate scores.

Before you choose

• The original Anthropic repository is frozen; current ownership and listing placement need reconciliation.
• Open-source software does not make model API calls or compute free.

Listed under

• Coding
• Code review and testing
• AI safety
• Alignment research
• Benchmarking
• Developer tools
• Evaluations
• Open source

Get started

Set up Python and model API credentials

1. Follow the maintained Petri/Bloom project linked from the original repository.
2. Prepare a Python environment and credentials for the models used in the evaluation.
3. Start with a small seed configuration, inspect outputs and costs, then scale the run.

No software purchase is shown. Evaluation costs depend on target and judge models, scenario count, and your compute.

Source: github.com/safety-research/bloom, checked 28 Oct 2026. Prices can change; check before you buy.

Sources for this overview

Product pages, documentation, and pricing checked 28 Oct 2026. Each source supports part of the overview; availability and prices can change.

• Introducing Bloom: Automated behavioral evals \ Anthropic ↗www.anthropic.com
• GitHub - safety-research/bloom: bloom - evaluate any behavior immediately  🌸🌱 · GitHub ↗github.com

Company website

anthropic.com: product, pricing and policy pages

YC directory mirror

Batch and company name

Web search

Confirms the official site

Social accounts

Accounts the company links, and its posts on X

Human review

Checked by a person

knowledge_base

Last checked 28 Oct 2026. Every field keeps its source and date; unknown means we couldn’t confirm it.