Algorithm Discovery Loop

Run one automated-research evaluation batch across 3 C3 jobs.

The included evaluator is intentionally small and CPU-only. Replace search.py with your benchmark, verifier, simulation, GPU kernel, RL rollout, or training script.

Files

c3-examples/
  algorithm-discovery-loop/
    .c3
    run.sh
    launch_shards.py
    search.py
    merge.py

Prerequisites

c3 login

Your account needs research-alpha access. The deploy output prints the job IDs used by c3 pull.

Local Smoke Test

git clone https://github.com/c3-research/c3-examples.git
cd c3-examples/algorithm-discovery-loop
C3_ARTIFACTS_DIR=results bash run.sh
cat results/leaderboard.md

This checks the Python code and artifact format before submitting C3 jobs.

Submit 3 Shards

python3 launch_shards.py --shards 3
cat shard-jobs.txt
c3 squeue

The launcher submits one C3 job per shard. Each job requests one L40, runs one candidate shard, uploads artifacts, and exits.

Free trial users can run a 3-chip burst. Team accounts can burst to 50 chips, subject to live capacity (one machine is one chip, GPU or CPU):

python3 launch_shards.py --shards 50 --candidates 200

If c3 squeue shows the shard jobs as PENDING, C3 is waiting for capacity or provisioning from the cold pool. You can leave them to start automatically or cancel the batch:

while read job; do c3 cancel "$job"; done < shard-jobs.txt

Pull And Merge

while read job; do c3 pull "$job"; done < shard-jobs.txt
python3 merge.py job_*/artifacts
cat merged-results/leaderboard.md

Use --shards to change the burst size.

Generated C3 Config

The local smoke path uses .c3:

project: algorithm-discovery-loop
script: run.sh
gpu: l40
time: "00:05:00"
job_name: algorithm-discovery-loop

The multi-GPU launcher writes temporary SBATCH files under .c3-shards/, one per shard:

#SBATCH --job-name=algo-loop-s0
#SBATCH --gres=gpu:l40:1
#SBATCH --time=00:05:00
#C3 GPU l40

python3 search.py --candidates 32 --seeds 4 --rounds 2 --workers 8 --shard 0 --shards 3

Run Script Path

#!/bin/bash
set -euo pipefail

python3 --version
nvidia-smi --query-gpu=name,memory.total --format=csv,noheader || true

python3 search.py --candidates 32 --seeds 4 --rounds 2 --workers 8

Artifacts

Each shard writes:

artifacts/
  candidates.json
  evaluations.jsonl
  leaderboard.json
  leaderboard.md

Each shard job writes a compact leaderboard:

# Algorithm Discovery Leaderboard

Rounds: 2
Evaluations: 264
Elapsed seconds: 0.223

| Rank | Candidate | Shard | Round | Parent | Mean reward | Median best value | Evaluations |
| ---: | --- | ---: | ---: | --- | ---: | ---: | ---: |
| 1 | r0-policy-24 | 0 | 0 | - | 1.658096 | 0.007018627 | 12 |

The merge step writes one cross-shard leaderboard:

# Merged Algorithm Discovery Leaderboard

Shard artifacts: 3
Evaluations: 768

Replace The Evaluator

Edit search.py to run your real evaluation. Keep the same boundary:

• launch_shards.py splits the batch and submits C3 jobs.
• search.py runs one shard and writes artifacts.
• merge.py merges pulled shard artifacts into one leaderboard.

For the broader agent-compute rationale, read Agents need compute too.

Execution Diagram