Lightnews — Scholar-powered news

Reposted by fairseq2

juanpino2000.bsky.social @juanpino2000.bsky.social · Jun 27

In line with FAIR mission, we open source Seamless Interaction Dataset to the community in order to accelerate the field. Download on github github.com/facebookrese... and @hf.co huggingface.co/datasets/fac...

GitHub - facebookresearch/seamless_interaction: Foundation Models and Data for Human-Human and Human-AI interactions.

Foundation Models and Data for Human-Human and Human-AI interactions. - facebookresearch/seamless_interaction

github.com

1 1 2

fairseq2 @fairseq2.bsky.social · Jun 17

[4/4] Collaborate efficiently with reproducible experiment setups using FAIRSeq2. Identify root causes swiftly and share lessons learned with the community. Create your benchmarks and contribute!

fairseq2 @fairseq2.bsky.social · Jun 17

[3/4] Beyond TensorBoard and WanDB, FAIRSeq2 supports torch profilers (set trainer.profile and common.profilers.torch.enabled=True) to inspect potential infra issues. Dive deep into your training processes with various profilers and metric recorders.

1

fairseq2 @fairseq2.bsky.social · Jun 17

[2/4] e.g., the tokens/s metric allows easy computation of Model FLOP Utilization – a great metric to check resource utilization efficiency. We achieve up to 48% MFU on 8 GPUs and maintain 37.6% across 4 nodes (32 GPUs). Experience effective and efficient distributed training!

1

fairseq2 @fairseq2.bsky.social · Jun 17

[1/4] 🛠️ FAIRSeq2 – your go-to tool for reliable benchmarking and diagnosing infra issues! With native logging of metrics, monitor training performance in real-time and ensure great visibility. #AI #MachineLearning #fairseq2

2 1

fairseq2 @fairseq2.bsky.social · Mar 10

🚀 Transform your LLM post-training with fairseq2! We make complex post-training into a breeze, so that you can make fairseq2 your paper machine!

Feel free to check our tutorials out:
- SFT: facebookresearch.github.io/fairseq2/sta...
- DPO: facebookresearch.github.io/fairseq2/sta...

1

Reposted by fairseq2

Matthew Finlayson @mattf1n.bsky.social · Feb 25

This project was made feasible by the excellent open-source LLM training library @fairseq2.bsky.social; I highly recommend giving it a look! It made both SFT and DPO a piece of cake 🍰

Matthew Finlayson @mattf1n.bsky.social · Feb 25

🧵 Adapting your LLM for new tasks is dangerous! A bad training set degrades models by encouraging hallucinations and other misbehavior. Our paper remedies this for RAG training by replacing gold responses with self-generated demonstrations. Check it out here: https://arxiv.org/abs/2502.10

3 10

fairseq2 @fairseq2.bsky.social · Feb 19

Nothing explains better than a vivid example:

fairseq2 @fairseq2.bsky.social · Feb 19

🚀 Big news for LLM researchers! #fairseq2 now has native support in #vLLM. Deploy your fine-tuned language models with vLLM in just one command for lightning-fast performance. Ready to accelerate your research like in FAIR? Check this out: facebookresearch.github.io/fairseq2/sta...

End-to-End Fine-Tuning - fairseq2 DocumentationContentsMenuExpandLight modeDark modeAuto light/dark, in light modeAuto light/dark, in dark mode

facebookresearch.github.io

1 1

fairseq2 @fairseq2.bsky.social · Feb 14

Large Concept Models extends fairseq2 and demonstrates what we can imagine beyond traditional sequence models -- check it out here: ai.meta.com/research/pub...

Large Concept Models: Language Modeling in a Sentence Representation Space | Research - AI at Meta

LLMs have revolutionized the field of artificial intelligence and have emerged as the de-facto tool for many tasks. The current established technology of...

ai.meta.com

fairseq2 @fairseq2.bsky.social · Feb 14

Seamless Communication as the best innovation recognized by TIMES and Nature -- check it out here: ai.meta.com/research/sea...

Seamless Communication - AI at Meta

A significant step towards removing language barriers through expressive, fast and high-quality AI translation

ai.meta.com

1

fairseq2 @fairseq2.bsky.social · Feb 14

🖼️ A gallery of open-source projects and papers powered by #fairseq2! 🚀

Seamless Communication and Large Concept Models are 2 vivid examples that showcase the potential of what we are building.

More exciting FAIR research built on fairseq2 is on the way!

1 1

fairseq2 @fairseq2.bsky.social · Feb 12

Check out our doc for more details: facebookresearch.github.io/fairseq2/stabl…

https://facebookresearch.github.io/fairseq2/stabl…

fairseq2 @fairseq2.bsky.social · Feb 12

👋 Hello world! We’re thrilled to announce the v0.4 release of fairseq2 — an open-source library from FAIR powering many projects at Meta. pip install fairseq2 and explore our trainer API, instruction & preference finetuning (up to 70B), and native vLLM integration.

1 2 4