And the blog post...

willmcgugan.github.io

But, but... it won't scale to petabytes of data!!

Not my problem. Never will be.

I'm bullish on DuckLake for the same reason I'm bullish on DuckDB itself - it will scale to all data sizes I care about, and for those data sizes it's both faster and simpler.

Publishing a package on S3 means: building, uploading and indexing the package. To make it even easier to rebuild software, we added a mechanism to automatically increment the build number (based on what's already there).

 We also have new fancy docs: rattler-build.prefix.dev/latest/publ...

This is a problem because of the old adage:
"Debugging is twice as hard as writing the code in the first place."

Complex solutions have a much higher maintenance cost which will limit your ability to support the code base. AI just supercharges the ability to generate tech-debt if... you allow it.

There isn't an AI development session I have where Claude doesn't say "Yes, you're absolutely right! It will be much simpler and more elegant to do it that way!"

Given tests, they can almost always come up with *a* solution to a problem but, like any junior developer, they solve problems by writing more and more code - adding more and more (unnecessary) complexity.

I would phrase that as "can produce excellent code with expert guidance".

My experience is that they're still terrible at designing complex systems.