They aren't really non-deterministic. If you keep a consistent RNG seed, start with a clear context window and set the temperature to 0, you will always get the same answer. The logits in the next token generator are deterministic for a fixed set of inputs.

Did you say that these neo-cloud companies are using the AI chips as collateral? Who accepts an asset that depreciates by 50% every two years as collateral? How long are their loan terms?

I like the volume.

It isn't just the building that is expensive. Each 1GW of data center will cost about $9B/yr to operate between chip depreciation and energy costs. 17GW is significantly more power than NYC uses under peak demand. Even if they could afford to build it, operation would cost $150B/yr.

Thank you! Either Youtube or Audible are my platforms of choice. Did he talk about industry suggestions in Better Offline? If so, you could just give me an episode number or title.

Thanks again!

I'm not subscribed to the newsletter. I enjoy listening to his podcasts. I happen to enjoy using AI locally with Ollama, but I'm somewhat disappointed by the limitations. Ed makes more sense than the pro-AI podcasts, and I would like to hear his suggestions if you would recommend an episode.

Which pod episodes? I would love to listen to them. I've listened to a lot of Ed's stuff, it is all very well researched. It is usually heavy on identifying the problems and light on the solutions. Not a criticism, it is good regardless, but I must have missed the content you are referring to.

Prediction is hard, but everyone does it, and everyone gets things wrong most of the time. That doesn't mean I'm not curious about his opinion. I'm not looking for some sort of gotcha. He does good research and is insightful, and he's changed my opinion on a few things.

Why did it upset you when someone asked for suggestions? I would like to hear some if you have them. Do you believe that the technology is so flawed that there is nothing that is worth saving? If not, how you would fix/change it? I'm honestly curious, you make well researched arguments.

Ed's completely right about inference. It doesn't matter if you cut per token prices in half if you have to more than double the number of tokens you produce to get an answer. Deep Seek's "innovation" was to hide costs in "reasoning" tokens. Every episode I get pulled closer to Ed's point of view.