And in general my impression of LLM writing is that the author hasn't really made any choices about what they're really trying to say.

I certainly don't think it's impossible that an llm could do this, but doing so would in some meaningful way mean that it is not adhering to the prompt.

When a human writes a story about cows, if the story is any good, the person reading it probably does not think "this is a story about cows" as their main impression. They probably think something else entirely about it. LLMs in their current form give you something tight to "A story about cows."

I think they were all a byproduct of the utopian spirit of the times. If people had approached the internet with different ideas and with less idealism I don't think that sort of thing would have happened.

This is not the shape I expected the graph to take. Encouraging.

Well, to the contrary, I think that era's spirit that the internet was a collective project that we all contributed a little bit to, and gave away/consumed for free is mostly gone now, and so is the accompanying activity.

I think it is very difficult to bootstrap such a thing without a near-religious ideology pushing people to put in the work. The tech is the trivial part, but it only exists as actual text because the spirit of the times convinced a lot of people that it's what they should do.

Related. bsky.app/profile/moul...

Yes, I made exactly that point the other day on twitter.

They are best when the nonverbal visual component is as important as the narrative voiceover. They are very information dense for that type of information, but you may not be able to verbalize what you learned.

Even now I think they are primarily guaranteed to win due to first mover advantage. Nothing else is within striking distance of doing economically valuable work, and you probably have to scale up any technique you use to that size before it starts to make money.

LLMs feel very historically contingent to me. If Tim Berners-Lee and the rest of the early internet pioneers hadn't had such utopian ideals about everyone publishing everything online, maybe internet content is two orders of magnitude smaller and all behind paywalls, and LLMs don't exist.

"What do people mean by X?" no longer depends on X being a coherent enough concept for the model to rediscover it on its own.

I turned off that setting because pages reloading when I revisited them was making me feel like I had some serious mental disorder whenever I tried to get work done. There were these big lags at every single step that made me feel like an invalid.

Some of the tabs that I need for work use a completely unreasonable amount of memory when they are open, so I end up just rage killing tabs quite a bit, and that helps with the profusion.

Not the main point, but "low powered" is equivalent to sparsity, which is generally considered to indicate that a model is learning the correct model. This is a bit like "writing a long essay because I didn't have time to write a short one."