Open-source proof-of-concept infrastructure for distributed reinforcement-learning post-training using Apple-silicon MLX workers, CUDA GPUs, and Cloudflare R2.
stoa separates reinforcement-learning post-training into rollout and training planes. Apple-silicon Macs generate rollouts with MLX, while CUDA GPUs train through Megatron/slime; both sides communicate through Cloudflare R2. Workers can join, leave, or crash during a run, while the trainer continues processing available rollouts.