Author | Lightnews

Surviving the AI code Deluge: Data quality in the Spotlight

3

Adrian Brudaru @datateam.bsky.social · 12d

The real AI win isn't superhuman agents, it's scaled mediocrity.
Doing less with less at massive scale unlocks tasks that were once uneconomical.
The magic is in aggregate value, not perfect outputs. Empower teams with practical AI tools.
🔗 https://dlthub.com/blog/the-real-ai-win-scaled-mediocrity

1 1 3

Adrian Brudaru @datateam.bsky.social · 15d

This isn't about AI taking jobs. It's about AI exposing what the real job should have been all along.
Building systems of trust. Asking "do I trust this output?" instead of "how do I write this script?"
The intern is here. It's waiting for a manager.

This is, we’re told, the great democratization of data engineering. The tedious work is gone. The barrier to entry is gone. Everyone can now be a data engineer.

1 1

Adrian Brudaru @datateam.bsky.social · 15d

Then the LLMs arrived.
Suddenly, there's an AI intern who can write that heroic query in seconds. An intern who can build a thousand handcrafted pipelines a day.
It's here to automate the firefighter's job into oblivion.

Adrian Brudaru @datateam.bsky.social · 15d

For years, the most celebrated person on the data team was the one who could write the heroic, last-minute query.
We celebrated firefighters. Craftsmen.
We built a culture around reacting, not architecting. A leverage trap of our own making.

Introduction to data load tool (dlt): A Python Library for Simple Data Ingestion

Adrian Brudaru @datateam.bsky.social · 19d

5/5 Read Erfan's full post: https://pipeline2insights.substack.com/p/introduction-to-data-load-tool-dlt

#DataEngineering #Python #dlt #OpenSource #FridayReads

Discover the basics of dlt and its role in modern data engineering workflows

pipeline2insights.substack.com

Adrian Brudaru @datateam.bsky.social · 19d

4/5 Whether you're 😤 tired of rebuilding ingestion pipelines, 🔧 looking to integrate Python tools into your data stack or just 💡 curious about modern ELT approaches, this guide has something for you.

Adrian Brudaru @datateam.bsky.social · 19d

3/5 The best part? Anyone who knows Python can build senior-level pipelines.

No new frameworks. No specialized expertise. Just Python doing what it does best.

Adrian Brudaru @datateam.bsky.social · 19d

2/5 What Erfan's guide covers:

⚡ ETL vs ELT & how to avoid data swamps
🐍 Why Python devs need simpler data tools
⚙️ dlt automates schema evolution, incremental loading & data contracts
💻 A practical MySQL-to-DuckDB example you can try today
☁️ Plus real deployment options: Lambda, Airflow, Kubernetes

Adrian Brudaru @datateam.bsky.social · 19d

1/5 Perfect Friday reading: Erfan Hesami's guide to dlt makes data pipelines actually simple.

Our co-founder spent a decade rebuilding the same pipelines. One question changed everything: "What if there was a way to reuse code?"

That became dlt.

🧵👇

1 3

Adrian Brudaru @datateam.bsky.social · 22d

The Zero Theorem (2013)

A lonely data worker tries to prove that all his computation adds up to zero. Ten years later, we call it AI productivity.

Workshop: Productizing Data Platforms - Infrastructure for Orchestrating dlt Pipelines by The Scalable Way

Adrian Brudaru @datateam.bsky.social · Sep 9

Pipelines working...but platform missing?

This hands-on workshop covers lightweight infrastructure, CI/CD, and flow automation for real-world data workflows.

Location: Online

Date: September 24th, 2025

Time: 16:00 (CET | Berlin)
community.dlthub.com/productizing...

Learn to productize your data platform and orchestrate dlt pipelines. This hands-on workshop covers lightweight infrastructure, CI/CD, and flow automation, giving you practical steps to build a scalab...

community.dlthub.com

Scalable Lakehouse Architecture with Iceberg & Polaris: A Battle-tested Playbook

Adrian Brudaru @datateam.bsky.social · Jun 12

Scalable Lakehouse Architectures with Iceberg and Polaris!

Simon from Tactile shared insights on tackling bottlenecks in data loading using Apache Iceberg and our open-source dlt library.

youtu.be/gb5fwIO4pX0?...

#databs #iceberg

YouTube video by Apache Iceberg

youtu.be

1 5

Adrian Brudaru @datateam.bsky.social · May 15

"If this song could stop aging like fine wine, that would be swell"

www.youtube.com/watch?v=HLUX...

Flobots - Handlebars (Official Video)

YouTube video by FlobotsVEVO

Vibe code dltHub Jaffle Shop API: using Curl to get responses for pagination

2

Adrian Brudaru @datateam.bsky.social · May 13

Trump and Putin together are supporting a fascist candidate in the Romanian presidential elections.

Enough said.

Adrian Brudaru @datateam.bsky.social · May 4

I’ve been quiet on this too long. I’m not speaking as a founder here. Just as a person who believes that racism, hate, have no place in a just society.

Trumpism isn’t “just politics.” It’s an ideology that dehumanizes.

I won’t make space for it in my life, my conversations, or my circle.

7

Adrian Brudaru @datateam.bsky.social · Apr 19

Vibe code dltHub Jaffle Shop API: using Curl to get responses for pagination.

With this workflow you allow your agent to grab information that isn't available in docs such as response structure

#dataengineering #databs

www.youtube.com/watch?v=fpNZ...

YouTube video by dltHub

Data Engineering with Python and AI/LLMs – Data Loading Tutorial

4

Adrian Brudaru @datateam.bsky.social · Apr 19

We just dropped a 4h knowledge bomb in collaboration with
@freecodecamp.bsky.social and #DataTalksClub

It's designed for people already in the data field who want to upskill to senior DE knowledge in data loading best practices.

www.youtube.com/watch?v=T23B...

#dataengineering #databs

YouTube video by freeCodeCamp.org

Firebringer | I don't really want to do the work today

1 4

Adrian Brudaru @datateam.bsky.social · Apr 18

vibe coding in a nutshell: www.youtube.com/watch?v=OhYW...

#databs

YouTube video by Sebastian Andras

#apacheiceberg #moderndatastack #datapipelines #dlthub | Adrian Brudaru

Adrian Brudaru @datateam.bsky.social · Apr 11

4

Adrian Brudaru @datateam.bsky.social · Apr 3

Loved the energy around Apache Iceberg at the Amsterdam meetup.

At dltHub, we’re focused on making Iceberg actually usable—modernizing legacy stacks with auto-ingestion, schema handling, and pipeline orchestration.

Modernization doesn’t have to hurt.

www.linkedin.com/posts/data-t...

Violetta’s talk with Lakekeeper showed exactly what we’re focused on at dltHub: making Iceberg actually usable, by modernizing your data stack and automating ingestion, schema evolution, and pipeline ...

www.linkedin.com

3

Adrian Brudaru @datateam.bsky.social · Apr 1

2/2

In this blog post, I describe an approach to benchmarking EL pipeline creation by separating the generation problem into 3 parts - Extraction, intuitive memory, and code creation.

dlthub.com/blog/towards...

Towards a Benchmark for AI-Generated Data Pipelines

Building pipelines with AI isn’t one task, it's many. In this post, we explore how to split and test them individually, so failures are easier to diagnose and fix.

Erase data loading tech debt with dlt + Cursor + LLMs

Adrian Brudaru @datateam.bsky.social · Apr 1

1/2 It’s not that LLMs can’t code pipelines. It’s that they read your API docs and said: ‘nah.’

Just dropped a benchmark test on AI-generated EL pipelines using Pipedrive.

TL;DR – bad docs reliably break even the best models.

Got 3 solid examples of where things went wrong.

1 2

Adrian Brudaru @datateam.bsky.social · Mar 27

One more experiment that went great - it looks feasible to convert any pipeline to better code or another framework.

dlthub.com/blog/moderni...

#databs
#dataengineering

Modernisation at its finest, from trash to cutting edge in seconds. It works amazing, just give it a try, stop paying for tech debt

From Airbyte YAML to Python with dlt, Cursor and LLMs

7

Adrian Brudaru @datateam.bsky.social · Mar 25

Vibe coding EL pipelines - an exploration and first success. Inside:

- problem definition
- a first experiment ended surprisingly successful, some learnings enclosed
- I will keep pushing and see how far it's possible to go with more complex cases

dlthub.com/blog/convert...

#databs

In this microblog + video we explore generating python pipelines (dlt REST API) from Airbyte low code yaml spec. Tl;dr: it works well.