MASS improved a 27B model by 1.2 to 1.6×. An agent team wrote its own textbook

Sakana AI and UC Berkeley had Qwen3.6-27B design multi-agent workflows, judge their outputs and learn from the best runs. After two cycles, performance per output token reached 1.2 to 1.6 times the baseline on four research benchmarks, while two software benchmarks exposed weaker transfer.

One model rotates through solver, judge and trainer roles

Multi-Agent Self-Supervision, or MASS, uses the same model in three roles. Qwen3.6-27B solves the task, compares results and revises the workflow that divides work among several agents. An inner loop searches for better coordination, then an outer loop selects the best runs and fine-tunes the model on them with LoRA.

The experiment ran for two cycles. The researchers created 12 synthetic tasks in finance, robotics and pharmacy. Nine were designated for training and three were held out; eight training tasks ultimately supplied data because workflow search failed to improve one of them. Judgments from GPT-5.5 and Claude Opus 4.8 served as an external measure and were withheld from MASS during training.

On the three held-out synthetic tasks, the win rate against the base model rose from 53.9% after the first cycle to 69.9% after the second. On MLR-Bench, DSBench, ScienceAgentBench and AstaBench, the model reached 1.2 to 1.6 times the baseline score per output token.

The strongest gain comes from organizing work before changing weights

MASS first changes a textual workflow: agent roles, instructions, output contracts and call order. Only then does the winning team produce training data. The improvement combines process search with post-training rather than an unconstrained rewrite of the model architecture.

That is a practical signal for teams building agents. Quality can rise through precise contracts between subagents and better sequencing. The authors report that a multi-agent student achieved a 68.3% win rate against the base after processing 28.8 million supervised training tokens, while a single-agent variant reached 64.0% after 42 million tokens.

The gains stay close to the research tasks used for training

Transfer remains uneven. The model achieved 0.96 times the baseline score per output token on Terminal-Bench 2.0 and 0.94 times on SWE-bench Verified. The four benchmarks showing gains resemble the research training tasks more closely, so the result currently supports effective specialization rather than a general increase across every capability.

The second risk sits inside the loop. One model generates work, judges it and selects data for the next generation. A shared blind spot can pass through all three roles and be reinforced by the next fine-tune. External models helped measure the outcome, but they did not provide human verification of every result.

Replication must test transfer and resistance to shared errors

The next step is independent replication across different models, tasks and judges. The important questions are whether gains survive in domains far from training and whether additional cycles begin optimizing against weaknesses in the model's own evaluator.

Practical use also requires the full compute bill. Score per output token excludes the cost of workflow search, 18 runs of the champion workflow per task, pairwise ranking and fine-tuning. A useful method has to improve both selected outputs and the economics of the entire loop.

Lilith's verdict

When the same student writes the paper, grades it and rewrites the textbook from that grade, rapid progress is possible. MASS shows two successful rounds; an independent pair of eyes still has to catch any mistake being printed into the next edition.