Run a short experiment

This example asks OpenEvolve to improve the optimizer settings of a small regression model. Each evaluation trains on 4,096 examples for 64 steps, then measures loss on 1,024 held-out examples. Training and loss calculation run on a C3 GPU. You do not need a GPU, CUDA, or OpenEvolve on your laptop.

Use Python 3.11 or later, the supplied C3 pilot checkout, and a staging API key with account credits. Run the commands from the repository root.

1. Install the SDK

python -m venv .venv
source .venv/bin/activate
python -m pip install ./modules/python-sdk
export C3_API_ENDPOINT=https://test.api.cthree.cloud

Set C3_API_KEY in your shell or secret manager to your staging key. If you already use the C3 CLI, c3 apikey create autoresearch creates a key after login; use the key for the same staging account. Keep the value out of source files and shared notebooks. The SDK also accepts C3_AUTH_TOKEN.

2. Submit the experiment

Save this as first_research.py in the repository root:

from pathlib import Path
from c3 import C3

example = Path("modules/research-runner/examples/batched-regression")

with C3() as client:
    run = client.research.run(
        initial_program=example / "candidate.py",
        evaluator=example / "evaluate.py",
        name="My first autoresearch experiment",
        engine="openevolve",
        engine_version="0.2.26",
        engine_config={
            "max_iterations": 2,
            "llm": {
                "models": [{"name": "openai/gpt-4.1-mini", "weight": 1.0}],
                "max_tokens": 1024,
            },
            "prompt": {
                "system_message": (example / "objective.txt").read_text(),
            },
        },
        c3={"hardware_profile": "l40", "provider": "nextgen"},
        idempotency_key="autoresearch-quickstart-002",
    )
    Path("research-run-id.txt").write_text(run.id)
    print(run.id)
    print(run.url)

python first_research.py

The script prints a run ID and link, then exits. The remote run continues. engine_config.max_iterations=2 requests one baseline plus two generated candidates, or up to three evaluations in total. This example uses the normal six-hour campaign and one-hour evaluation limits; it stops when those three evaluations finish. The limits are ceilings, not the expected duration. Each supplied evaluation is small, but cold GPU provisioning can add several minutes, and provider recovery can take longer.

Rerunning identical inputs with the same idempotency key returns the same run. Use a new key when you intentionally want another experiment. A changed request with an existing key is rejected.

3. Watch the run

Open the printed link and sign in with the C3 account that submitted it. Python API-key authentication does not sign your browser in. The dashboard shows:

• OpenEvolve, the selected model, NextGen compute and L40 hardware.
• An overview of the baseline, candidate-generation and evaluation loop.
• Individual scores, elapsed time, spending and private compute logs and artifacts.

This experiment's score is 1 / (1 + validation_mse), so a higher score is better. Compare the baseline with the best valid candidate. The model may propose an invalid candidate or fail to improve the score; the measured result is what matters. A candidate error can be fed back into the next iteration.

4. Read the saved result

Run this in a new Python process, even after closing the original terminal:

from pathlib import Path
from c3 import C3

with C3() as client:
    run = client.research.get(Path("research-run-id.txt").read_text().strip())
    print(run.status, run.url)
    result = run.results().get("result")
    if result and result.get("code"):
        Path("best_candidate.py").write_text(result["code"])
        print(result["metrics"])
    else:
        print("A final best program is not available yet; inspect the run page.")

Successful runs retain the selected program, metrics and checkpoint references. You can also download the result and each evaluation's artifacts from the run page. A run that stops early may retain completed evaluations without a final best-program result.

Adapt the candidate and evaluator using Configuration. See Runs and results for cancellation and failure behavior.