Lightnews — Scholar-powered news

Alexander Doria

@dorialexander.bsky.social

6.8K followers 640 following 1.5K posts

LLM for the commons.

Posts Media Videos Starter Packs

Pinned

Alexander Doria @dorialexander.bsky.social · 11d

And new paper out: Pleias 1.0: the First Family of Language Models Trained on Fully Open Data

How we train an open everything model on a new pretraining environment with releasable data (Common Corpus) with an open source framework (Nanotron from HuggingFace).

www.sciencedirect.com/science/arti...

8 51 170

Alexander Doria @dorialexander.bsky.social · 1h

Not surprised :D

Alexander Doria @dorialexander.bsky.social · 6h

Proof it’s not ai generated.

(To be updated soon anyway, current r&d is very different)

Alexander Doria @dorialexander.bsky.social · 14h

Welcome to the ongoing Pleias research program.

1 3

Alexander Doria @dorialexander.bsky.social · 16h

They publish the full dump on open access (I even used that in the past). If people were behaving normally rather than brute forcing agents on full web pages shouldn’t even happen.

1 6

Alexander Doria @dorialexander.bsky.social · 16h

Ok while I do think many issues with AI are overblown, web agents hitting hard on infra sustainability and knowledge access. Was just browsing the reference database for Disney comics: now login only indefinitely.

2 5 17

Alexander Doria @dorialexander.bsky.social · 17h

Too many hyper/super/adaptive intelligence. Not enough people caring about small agi.

2 8

Alexander Doria @dorialexander.bsky.social · 18h

I think it goes both ways. Some AI people have been relentlessly targeted on here even though very mild pro-AI and much more progressive than the usual tech bro. Extremes simply ignore each others.

1 3

Alexander Doria @dorialexander.bsky.social · 19h

I would admit, core idea is only edgy. Real cringe is in the expression.

Alexander Doria @dorialexander.bsky.social · 19h

In fact just noticed this post but full endorsement. Working daily with models means I’m just perpetually unsatisfied. bsky.app/profile/theo...

rev. howard arson @theophite.bsky.social · 21h

"the AI bubble will collapse because the technology has plateaued and cannot improve"

will_smith_spaghetti.mpeg

"we are on the verge of superintelligence"

seahorse_emoji.txt

Alexander Doria @dorialexander.bsky.social · 20h

Have to admit there are pro-ai takes on the other site that are as cringe as anti-ai folks on here.

3 1 36

Alexander Doria @dorialexander.bsky.social · 22h

Going to set it as my default answer on Linked*n.

2 17

Alexander Doria @dorialexander.bsky.social · 23h

Two different ones.

Alexander Doria @dorialexander.bsky.social · 1d

No was just shitpost on my side.

Completely agree on your last pint and actually spending quite a bit of time on personality tuning right now.

1 1

Alexander Doria @dorialexander.bsky.social · 1d

I guess that's some brand recognition

1 13

Alexander Doria @dorialexander.bsky.social · 1d

So losing interns to Mistral, now ex-OpenAI. At least I know where to find quality people just can’t keep them on standard EU salaries.

1 11

Alexander Doria @dorialexander.bsky.social · 1d

In my experience, layer design (especially of the deeper kind) is the one non-data thing (with tokenizer) with most significant impact. Really determine whether internal representations are siloed or yield deeper interconnected patterns.

Alexander Doria @dorialexander.bsky.social · 1d

Well after 18 months trying to, still can't publish anything data related. But at least we'll get a gazillion Qwen/RL experiments (is it Qwenology?)

Alexander Doria @dorialexander.bsky.social · 1d

Also refusal to consider it is also primarily training data behavior (big ML conferences still hate that).

Eryk Salvaggio @eryk.bsky.social · 1d

The "you don't understand how (AI/LLMs/Diffusion Models/NNs) work" posture in any debate is often just "we don't agree on how to interpret what (AI/LLMs/Diffusion Models/NNs) are doing"

1 1 13

Alexander Doria @dorialexander.bsky.social · 1d

Actually hear me out: anti-AI social bots to keep that one loud Anti-AI crowd in its own corner.

3 4

Alexander Doria @dorialexander.bsky.social · 1d

All you need now is to better engineer heavenbanning for the users that really ask for it.

1 5

Alexander Doria @dorialexander.bsky.social · 1d

Clearer sign that bluesky actually made it.

John David Pressman @jdp.extropian.net · 1d

Already seen one dude loudly announce they're leaving and then keep posting lol.

2 3 54

Alexander Doria @dorialexander.bsky.social · 1d

Here it's all synthetic generation on Wikipedia, but that also include (fictive) literary works. Idea is to always circle on the same knowledge pool so that the model gets some anchor despite the constrained search space.

Alexander Doria @dorialexander.bsky.social · 1d

I think at this point Macron is just trolling everyone. Not even sure I can blame him.

Alexander Doria @dorialexander.bsky.social · 2d

Yeah now it’s all French politics. At least some takes are funny.

Alexander Doria @dorialexander.bsky.social · 4d

A minor issue on here is that I have zero interest in US politics at the moment. I’m in the EU, can’t do a thing, sorry if it doesn’t help (!)

1 5

Alexander Doria @dorialexander.bsky.social · 1d

That's actually the trick: very small vocab size (8k tokenizer, english-only support), relatively deep configuration for the size (32 layers) and synthetic reasoning dataset centered on core Wikipedia knowledge (essentially cutting the slack).

1 3

Alexander Doria @dorialexander.bsky.social · 1d

50m

1 3