Lightnews — Scholar-powered news

sakanaai.bsky.social @sakanaai.bsky.social · 13d

By making ShinkaEvolve open-source, our goal is to democratize access to advanced discovery tools. We envision it as a companion to help scientists and engineers, building efficient, nature-inspired systems to unlock the future of AI research.

GitHub Project: github.com/SakanaAI/Shi...

GitHub - SakanaAI/ShinkaEvolve

Contribute to SakanaAI/ShinkaEvolve development by creating an account on GitHub.

github.com

6

sakanaai.bsky.social @sakanaai.bsky.social · 13d

ShinkaEvolve's efficiency comes from three key innovations:

1) Adaptive parent sampling to balance exploration and exploitation.

2) Novelty-based rejection filtering to avoid redundant work.

3) A bandit-based LLM ensemble that dynamically picks the best model for the job.

1 3

sakanaai.bsky.social @sakanaai.bsky.social · 13d

3/ LLM Training: It discovered a novel load balancing loss for MoE models, improving model performance and perplexity.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · 13d

2/ Competitive Programming: On ALE-Bench, it improved an existing agent's solution, turning a 5th place result into a 2nd place leaderboard rank for one task.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · 13d

We applied ShinkaEvolve to a diverse set of hard problems:

1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering a Pareto frontier of solutions trading performance for efficiency.

1 3

sakanaai.bsky.social @sakanaai.bsky.social · 13d

On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a massive leap in efficiency compared to previous methods that required thousands of evaluations.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · 13d

Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · 13d

We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency. It leverages LLMs to find state-of-the-art solutions, orders of magnitude faster!

Blog: sakana.ai/shinka-evolve/
Paper: arxiv.org/abs/2509.19349

1 2 13

sakanaai.bsky.social @sakanaai.bsky.social · Aug 30

How Sakana AI’s new evolutionary algorithm builds powerful AI models without expensive retraining
venturebeat.com/ai/how-sakan...

How Sakana AI’s new evolutionary algorithm builds powerful AI models without expensive retraining

M2N2 is a model merging technique that creates powerful multi-skilled agents without the high cost and data needs of retraining.

venturebeat.com

2 8

sakanaai.bsky.social @sakanaai.bsky.social · Aug 29

We are honored that Sakana AI’s CEO David Ha (@hardmaru.bsky.social) has been named to the TIME 100 AI 2025 list. Full List: time.com/time100ai

We’re truly grateful for the recognition and will continue our mission to build a frontier AI company in Japan.

Thank you for your support!

1 12

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

We hope this work sparks more interest in these under-explored ideas!

Please read our paper: arxiv.org/abs/2508.16204

Code: github.com/SakanaAI/nat...

Published in ACM GECCO’25: Proceedings of the Genetic and Evolutionary Computation Conference. DOI: doi.org/10.1145/3712...

Competition and Attraction Improve Model Fusion

Model merging is a powerful technique for integrating the specialized knowledge of multiple machine learning models into a single model. However, existing methods require manually partitioning model p...

arxiv.org

7

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

This approach is central to our mission. Rather than scaling monolithic models, we envision a future where ecosystems of specialized models co-evolve and combine, leading to more adaptive, robust, and creative AI. 🐙

1 4

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

Does it work on multimodal models?

We merged several text-to-image models by adapting them only for Japanese prompts. The resulting model not only improved on Japanese but also retained its strong English capabilities—a key advantage over fine-tuning, which can suffer from catastrophic forgetting.

1 3

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

Does it scale?

We used M2N2 to merge a math specialist LLM with an agentic specialist LLM. The resulting model excelled at both math and web shopping tasks, significantly outperforming other methods. The flexible split-point was crucial.

1 3

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

Does it work?

This is the first time model merging has been used to evolve models entirely from scratch, outperforming other evolutionary algorithms. In one experiment, M2N2 evolved an MNIST classifier from random networks that achieved performance comparable to CMA-ES, but more efficiently.

1 1

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

3/ Attraction and Mate Selection 💏: Merging is computationally expensive. M2N2 introduces an “attraction” heuristic that intelligently pairs models based on their complementary strengths, making the evolutionary search much more efficient.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

2/ Diversity through Competition 🐠: To ensure a rich pool of models to merge, M2N2 makes them compete for limited resources (i.e., data points). This forces models to specialize and find their own “niche,” creating a population of diverse, high-performing specialists.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

1/ Evolving Merging Boundaries 🌿: Instead of merging models using pre-defined, static boundaries, M2N2 dynamically evolves the “split-points” for merging. This allows a more flexible exploration of parameter combinations, like swapping variable-length segments of DNA.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

Our new paper proposes M2N2 (Model Merging of Natural Niches), a more fluid method which overcomes this with 3 key, nature-inspired ideas:

1/ Evolving Merging Boundaries 🌿

2/ Diversity through Competition 🐠

3/ Attraction and Mate Selection 💏

1 3

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

A key limitation in earlier work remained: model merging required manually defining how models should be partitioned (e.g., by fixed layers) before they could be combined. What if we could let evolution figure that out too?

1 2

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

Our research follows an evolutionary path. We started with using evolution to find merge "recipes" (Nature Machine Intelligence), then explored maintaining diversity for new skills (ICLR 2025). Now, we're combining these ideas into a full evolutionary system.

1 2

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

Summary of Paper 🧵

At Sakana AI, we draw inspiration from nature’s evolutionary processes. Instead of one giant monolithic AI, we envision an ecosystem of specialized models that collaborate and combine their skills, like a school of fish 🐟 where collective intelligence emerges from the group.

2 5

sakanaai.bsky.social @sakanaai.bsky.social · Aug 25

What if we could evolve AI models like organisms, letting them compete, mate, and combine their strengths to produce ever-fitter offspring?

Excited to share our new paper, “Competition and Attraction Improve Model Fusion” presented at GECCO 2025 (runner-up for best paper)!

arxiv.org/abs/2508.16204

Competition and Attraction Improve Model Fusion

Model merging is a powerful technique for integrating the specialized knowledge of multiple machine learning models into a single model. However, existing methods require manually partitioning model parameters into fixed groups for merging, which restricts the exploration of potential combinations and limits performance. To overcome these limitations, we propose M2N2, an evolutionary algorithm with three key features: 1/ dynamic adjustment of merging boundaries to progressively explore a broader range of parameter combinations; 2/ a diversity preservation mechanism inspired by the competition for resources in nature, to maintain a population of diverse, high-performing models that are particularly well-suited for merging; and 3/ a heuristic-based attraction metric to identify the most promising pairs of models for fusion. Our experimental results demonstrate, for the first time, that model merging can be used to evolve models entirely from scratch. Specifically, we apply M2N2 to evolve MNIST classifiers from scratch and achieve performance comparable to CMA-ES, while being computationally more efficient. Furthermore, M2N2 scales to merge specialized language and image generation models, achieving state-of-the-art performance. Notably, it preserves crucial model capabilities beyond those explicitly optimized by the fitness function, highlighting its robustness and versatility.

2 10 36

sakanaai.bsky.social @sakanaai.bsky.social · Aug 22

Software Engineerを含む、Sakana AIのApplied Teamの仕事の概要、働き方については以下の記事もご参照ください。

sakana.ai/open-house-2...

Sakana AI

【イベントレポート】Applied Engineer Open House 2025：金融・防衛の難関課題に挑む、AI社会実装の最前線

sakana.ai

2

sakanaai.bsky.social @sakanaai.bsky.social · Aug 22

Sakana AI が募集しているSoftware Engineerの募集要項（Job Description）をアップデートしました。

sakana.ai/careers/#sof...

Sakana AIにおけるSoftware Engineerは、Applied Teamの一員としてビジネスのインパクトにつながるプロダクト開発を行っています。Frontend、Backend、Infrastructure構築の全体にわたって、AI技術を組み込んだアプリケーションの設計・開発に挑戦いただける方のご応募をお待ちしております！

1 1 1