Lightnews — Scholar-powered news

Nous Research @nousresearch.com · Jun 3

Controlling text generation and structure remains a difficult problem to solve.

Our newest blog post and release from Researcher in Residence @yaboilyrical (on Twitter) explores how this problem becomes solvable using Sequential Monte Carlo approximation.

nousresearch.com/steering-the...

Steering the Shoggoth: Taming LLMs with Sequential Monte Carlo - NOUS RESEARCH

In this blog post, we present our findings from an exciting direction in controlling text generation with large language models. We can programmatically define constraints on the output of a model, en...

nousresearch.com

9

Nous Research @nousresearch.com · May 15

Come join Nous and the Solana Foundation in NYC on Thursday, May 22nd, to discuss decentralized AI and Nous's efforts to democratize intelligence, including Psyche.

Limited capacity. Apply below👇
lu.ma/39b7e9pu?v=1

Nous Research x Solana Foundation · Luma

Join Nous Research and the Solana Foundation for a private gathering in the Meatpacking District to discuss decentralized AI and Nous's efforts to democratize…

lu.ma

5

Nous Research @nousresearch.com · May 14

As always, we couldn't have gotten here without your help. Special thanks to our team, our community, and the open source movement.

1

Nous Research @nousresearch.com · May 14

Psyche’s initial training infrastructure is just the beginning of our journey. We plan to integrate full post training stages - supervised finetuning and reinforcement learning workloads, inference, and other parallelizable workloads in the creation and serving of AI going forward.

1 1

Nous Research @nousresearch.com · May 14

Looking ahead, we will draw model ideas from the community via our forum and Discord. By enabling highly parallel and scalable experimentation, we’re betting that the next innovation in model creation and design will come from the open source community

1

Nous Research @nousresearch.com · May 14

The resulting model will be small enough to train on a single H/DGX and run on a 3090, but will be powerful enough to serve as the basis for strong reasoning models and creative pursuits. The model will be trained continually without a final annealing step, resulting in a true unaltered base model.

1

Nous Research @nousresearch.com · May 14

We are launching testnet with the pre-training of a 40B parameter LLM:

- MLA Architecture
- Dataset consisting of FineWeb (14T) + FineWeb-2 minus some less common languages (4T), and The Stack v2 (1T)

40b parameters. 20t tokens. MLA architecture.

1

Nous Research @nousresearch.com · May 14

If you have 64+ H100 GPUs, Contact [email protected] to apply to provide hardware to the network’s training pool.

1

Nous Research @nousresearch.com · May 14

While compute on the network needs to be trusted and approved at this time, we plan to support trustless, community-owned compute resources. For now, open source enthusiasts can contribute via our mining pool, and we will be onboarding more nodes over the next weeks.

1

Nous Research @nousresearch.com · May 14

Psyche uses the Solana blockchain to decentralize parts of the core infrastructure for coordination and stores attestations for the nodes operating within the network. This design takes meaningful steps towards decentralization while ensuring training does not become too costly or redundant.

1 1

Nous Research @nousresearch.com · May 14

Training used to have a bandwidth constraint that kept the process centralized. In 2024, Nous's DisTrO optimizers broke through that constraint. With Psyche, we have created a custom peer-to-peer networking stack to coordinate globally distributed GPUs running DisTrO.

1 1

Nous Research @nousresearch.com · May 14

All the links!

Blog: nousresearch.com/nous-psyche/
Training dashboard: psyche.network
Code: github.com/PsycheFounda...
Docs: docs.psyche.network
Forum: forum.psyche.network
HuggingFace: huggingface.co/PsycheFounda...
Discord: discord.com/invite/jqVph...

Democratizing AI: The Psyche Network Architecture - NOUS RESEARCH

Psyche is an open infrastructure that democratizes AI development by decentralizing training across underutilized hardware. Building on DisTrO and its predecessor DeMo, Psyche reduces data transfer by...

nousresearch.com

1 1

Nous Research @nousresearch.com · May 14

This run represents the largest pre-training run conducted over the internet to date, surpassing previous iterations that trained smaller models on much fewer data tokens.

1 1

Nous Research @nousresearch.com · May 14

We are launching our testnet today with the pre-training of a 40B parameter LLM, a model powerful enough to serve as a foundation for future pursuits in open science.

1 2

Nous Research @nousresearch.com · May 14

Psyche is a decentralized training network that makes it possible to bring the world’s compute together to train powerful AI, giving individuals and small communities access to the resources required to create new, interesting, and unique large scale models.

1 2

Nous Research @nousresearch.com · May 14

Announcing the launch of Psyche

nousresearch.com/nous-psyche/

Nous Research is democratizing the development of Artificial Intelligence. Today, we’re embarking on our greatest effort to date to make that mission a reality: The Psyche Network

A diagram of the Psyche network. It shows the relationship between the Solana Coordinator, an individual training client, the DisTrO optimizer inside a client, forward/backward passes to create gradients, the transmission and dissemination of the created DisTrO results, and the ingestion of data from a data provider.

1 4 26

Nous Research @nousresearch.com · Mar 13

To ensure a smooth rollout, we made a waitlist: portal.nousresearch.com
- Access will be granted on a first-come, first-served basis
- Once granted access, you can create API keys and purchase credits
- OpenAI-compatible API
- Right now all accounts start off with $5.00 in free credits.

Nous Portal

Nous Research is a leader in the development of human-centric language models and simulators. Manage your account and API keys here.

portal.nousresearch.com

5

Nous Research @nousresearch.com · Mar 13

Today we’re releasing our Inference API that serves Nous Research models. We heard your feedback, and built a simple system to make our language models more accessible to developers and researchers everywhere.

The initial release features two models - Hermes 3 Llama 70B and DeepHermes 3 8B Preview

Nous Portal

Nous Research is a leader in the development of human-centric language models and simulators. Manage your account and API keys here.

portal.nousresearch.com

1 7

Nous Research @nousresearch.com · Jan 27

Apply to attend Nous Research x Solana Foundation this Thursday in NYC: lu.ma/nousxsolana

How to Play with Shoggoths | Nous x Solana | NYC · Luma

Nous Research and the Solana Foundation present HOW TO PLAY WITH SHOGGOTHS, an evening of lively debate and discussion 𒀭 Shoggoths are shapeshifting…

lu.ma

3

Nous Research @nousresearch.com · Jan 27

You can now experiment with Psyche’s DisTrO-enabled training code on our GitHub, and the larger open-sourced distributed training stack will be released alongside testnet. github.com/PsycheFounda...

GitHub - PsycheFoundation/psyche: An open infrastructure to democratize and decentralize the development of superintelligence for humanity.

An open infrastructure to democratize and decentralize the development of superintelligence for humanity. - PsycheFoundation/psyche

github.com

1 4

Nous Research @nousresearch.com · Jan 27

Read more in our blog post: nousresearch.com/nous-psyche/

1 4

Nous Research @nousresearch.com · Jan 27

Recent AI breakthroughs challenge the status quo narrative that only closed, mega labs have the ability to push the frontier of superintelligence.

Today we announce Nous Psyche built on
@solana.com

www.youtube.com/watch?v=XMWI...

The Story of Psyche

YouTube video by Nous Research

www.youtube.com

1 5 18

Nous Research @nousresearch.com · Dec 12

Run Hermes on phones, laptops, and CPUs without sacrificing speed, and may also be a great pairing with 70B for speculative decoding!

Learn more about Hermes, see our technical report, and chat with it now: nousresearch.com/hermes

Hermes 3 - NOUS RESEARCH

Hermes 3 contains advanced long-term context retention and multi-turn conversation capability, complex roleplaying and internal monologue abilities, and enhanced agentic function-calling. Our training...

nousresearch.com

10

Nous Research @nousresearch.com · Dec 12

Introducing a smol Hermes 3 LLM!

Hermes 3 3B is now available on huggingface alongside quantized GGUF versions to make it even smaller.

More info and download links here: huggingface.co/NousResearch...

Hermes 3 3B was built by Teknium, Roger Jin, Jeffrey Quesnelle and "nullvaluetensor".

3 4 31

Nous Research @nousresearch.com · Dec 10

Thursday December 12th
Doors @ 6pm, Talks @ 7pm
DCTRL, 436 W Pender St, Vancouver
Open Entry. Food + Drink + Merch.

DisTrO Demystified - Jeffrey Quesnelle , Bowen Peng

Why Decentralization Matters - Mark Murdock

Mapping Uncertainty at Inference Time - _xjdr

1 1 16