Lightnews — Scholar-powered news

Shubhendu Trivedi @shubhendu.bsky.social · 5h

PS: Since I way overpost on this website (a bit unusual for me), and because such websites have a lot of literalists -- just joking. I’m poking at the “gradient descent in the brain” line of work, and also because there are a lot of (serious) neuro papers on bloom filter-like neural structures.

2

Shubhendu Trivedi @shubhendu.bsky.social · 5h

"Does the Brain Implement BM25? Evidence for Lexical Ranking from Intracranial Recordings." Science 799 (7813), 167-171.

1

Shubhendu Trivedi @shubhendu.bsky.social · 5h

Still haven’t been able to develop the right mental hashing and retrieval frameworks for scanning cs CL efficiently. Perhaps they’re forming up there, waiting to compile and become automatic in another eight months.

Shubhendu Trivedi @shubhendu.bsky.social · Jun 5

Somehow, I don’t like relying on social media for this. I prefer looking through the lists myself, having developed (I think) reasonably good thin-slice judgments of papers that help me sift through them quickly. I can still manage the firehose of cs LG pretty easily otherwise, but not LLM papers.

1

Shubhendu Trivedi @shubhendu.bsky.social · 13h

i.e. after a positive reflexive loop. The magical, alchemical, and completely unknown solution is to just prepare enough, and not let the loop happen. :P

2

Shubhendu Trivedi @shubhendu.bsky.social · 13h

I love talking, and hence tend to get into this reflexive loop (since we are talking Soros and the theory of reflexivity) of taking things for granted, overdoing it (e.g. adding in too many minute details, or too many slides, or too many papers), and then go through a negative reflexive loop.

1

Shubhendu Trivedi @shubhendu.bsky.social · 14h

Just gave a 1.5 hour talk that I was very happy with, after a string of overly compressed (200 slides in one hour, for example), or hyper, or needlessly detailed, talks. Turns out you need to make sure to just prepare adequately. Who would have thought. :(

1 6

Shubhendu Trivedi @shubhendu.bsky.social · 16h

For AI, something similar will happen eventually. But it will look more violent than it happens to be (because of the capital concentration), and because for a while the narrative will undergo a negative reflexive cycle. But the tech will mature, play out like in the dot com case (but bigger).

5

Shubhendu Trivedi @shubhendu.bsky.social · 16h

But eventually, it is a whole weird thing where it simultaneously (numerically) larger as a bubble than the dot com era, but also more robust than it long term. The dot com era didn't "finish the internet," the bubble eliminated the excesses for some time and the technology and industry matured.

1 2

Shubhendu Trivedi @shubhendu.bsky.social · 16h

If the spoils of the ZIRP era were more diversified (say into energy, manufacturing, semiconductors), random companies, 5 days old, without even a product plan wouldn't be valued at 20 B. The stock bubble does seem to have some tulip mania aspect to it-based on hysteria and narrative contagion.

1 2

Shubhendu Trivedi @shubhendu.bsky.social · 16h

I think it's not a bubble in the sense of the south sea bubble, or the Dutch tulip mania. Those were actual bubbles--where "actual" means that they were almost fully driven by fraud. In contrast, the bubbly aspect of AI comes from the fact that a huge volume of risk capital is all in one space.

1 14

Shubhendu Trivedi @shubhendu.bsky.social · 23h

What is a good reference/survey for the post-Bayes stuff? I have been meaning to learn what it is about.

1 1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

BTW, while too old now, I'd have loved to hear how Soros would have thought of and interpreted this reflexive cycle's evolution. It basically looks like the exact thing he loved talking about, he would probably frame this as the “alchemy” phase, potentially veering towards instability.

2

Shubhendu Trivedi @shubhendu.bsky.social · 1d

It's notable that just a couple of years ago there was a whole anti-google mania (and how they had lost out at their own game), but they are amongst the few who look sober (and still arguably undervalued long term despite the recent rise) in this frothy cycle. Others include energy companies.

1 1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

Everyone knows it is fake, but they all play along, not wanting to miss the action. Or that the circular AI stock bubble powers on.. until, eventually, some completely random (unexpected) trigger ends the party of this rapidly circulating (but identical) capital. And when it does, it will be ugly.

1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

While I appreciate AMD, their thing was not any less funny. They decide to give OpenAI 10% stock (~ $35B pre-market), which gives OA rights to buy 6GW capacity worth of chips over the next few years. The stock went up 27% pre-market. In the end, OA gets resources to buy chips without raising cash.

1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

So far these circular announcements function as "infinite money glitch" generators. The funniest was Oracle. It went up 45% in a single day based on some random capex guide using revenue that doesn't exist from OpenAI. Then it flipped into a sureshot short and retraced most of it in two weeks.

Carl Quintanilla @carlquintanilla.bsky.social · 1d

NVIDIA and OpenAi:

Concerns that their “increasingly complex and interconnected web of business transactions is artificially propping up the trillion-dollar AI boom.“

@bloomberg.com $NVDA 👀
www.bloomberg.com/news/feature...

1 7

Shubhendu Trivedi @shubhendu.bsky.social · 1d

To be clear, for the first book, this is understandable. But I feel sad at not being able to recollect any of the specific imagery. The other two are short stories, the one on the right had a cruel hue, another is somewhat like Bruno Schulz (Streets of Crocodiles), but in a surrealistic register.

Shubhendu Trivedi @shubhendu.bsky.social · 1d

Same for these two, although were read years later. Older photo and hence overcrowded.

1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

Hmm, the post reminded me of this. I tried for a bit to distill what I could about this book (other than the subject). All I recall is some sort of residue--it being quite uneven, and wandering, but still quite distinct.

1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

benefitting from that leverage, including scaling, and orchestration. The latter category, of course, already makes sense even without needing what you said in its strong form.

Shubhendu Trivedi @shubhendu.bsky.social · 1d

I mean, putting money in consolidated vehicles that invest in starting companies aiming either to monetize the automation of scientific discovery itself (Lila could be an example of such an ambition; although not a fan for idiosyncratic, not rational, reasons), or to build setups oriented toward

1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

I think good avenues that explicitly adopt the thesis that human technical labour will become commoditized in that way don’t really exist yet, at least not directly (though some do so indirectly). But some approaches could include “going meta”: for instance, investing in capital-as-research.

1

Shubhendu Trivedi @shubhendu.bsky.social · 1d

Should not share without permission, but this (which arrived in the morning) just proves my point perfectly.

2

Shubhendu Trivedi @shubhendu.bsky.social · 1d

They show that a ML + UQ approach helped them understand when the physical experiment wouldn't be a lucky shot.

Shubhendu Trivedi @shubhendu.bsky.social · 1d

Ignition here refers to producing more energy via fusion than the amount of laser energy delivered to the target. The process creates extreme plasma conditions that are not trivial to simulate, and require extensive manual tuning.

1