A favorite book of mine is Buciarelli’s “Designing Engineers”, which follows three engineering teams through their design process and does an ethnographic analysis of each. I’m still waiting to see an equivalent in software.

Oh absolutely.

One of the challenges I see is that these tools can be very helpful for experts who can tell *quickly* if the result is bad or not, and iterate. I’ve seen senior folks get huge value out of them.

But if you don’t have that experience you can go off the rails real fast

But I’ll happily use an LLM to help write a one-off script to restructure a JSON file into CSV, because fundamentally what I want is some set of data to be in a spreadsheet I can mess with. 🤷‍♂️

So e.g. I don’t use an LLM to write a design doc, because the core purpose of that doc is to help a human audience understand an intended design.

I don’t use an LLM to write my D&D campaign, because the point is to entertain my players and make them feel a certain way.

* “Primarily” is arguable, readability is important especially in long-running code bases.

But eg the core purpose of a video transcoder is not to teach about transcoding, it’s to convert a video file into the desired format for you to watch it

I periodically remind folks that even if everyone stopped training LLMs for whatever reason… any given Apple-silicon machine can now write code and do homework at a level unthinkable five years ago.

I don’t think that’s a bad thing! But it’s a thing we all need to manage now, either way.

I periodically remind folks that even if everyone stopped training LLMs for whatever reason… any given Apple-silicon machine can now write code and do homework at a level unthinkable five years ago.

I don’t think that’s a bad thing! But it’s a thing we all need to manage now, either way.

It’s easy for software folks to overestimate LLMs, because code kinda lives their sweet spot:

- Highly structured text-based work
- (Mostly) easy to test results quickly and mechanistically
- A *huge* online corpus of decent, un-paywalled examples for training data

Most fields are… not that.

I feel like I’m telling people at work to consider their audience on a weekly basis at minimum. It is apparently challenging! But you’d expect a politician to be good at this.

Yep, I made a ton of it for Thanksgiving breakfast so we’re still working through it 😁

Also I listened to this while making some delightful cream biscuits from Smitten Kitchen, pictured here with some homemade lemon curd 😋

oh nice, are you following a particular recipe? I used the one from smitten kitchen a few years ago

I finished my baking perhaps 10m after you. Congratulations!