AI experts in the past: 1) Defined an optimization problem as a proxy to a business problem, 2) Optimized on a business-specific dataset, 3) Proved the solution is optimal, 4) Applied it to the business problem.

The growth in LLMs from pretraining on larger datasets and parameter count increase has reached its peak. It took 2 years, which was fun to watch.

Those of us who learned to code before LLMs will know how to instruct them in generating good quality code.

Those who started building software by talking to LLMs will always be limited in what they can explain verbally.

Learning to code is crucial to being able to effectively not code.

In doing research for my book, I discovered that byte-pair encoding (BPE), the algorithm used to tokenize data for modern language models, one of the most important algorithms of our times, is described incorrectly in almost all online resources.

Hello, world!