Julien Le Dem
@julien.ledem.net
Principal Engineer, Founder, Angel, Advisor, OSS.
LFAI&data: OpenLineage, Marquez, ASF: Parquet, Arrow, Iceberg, 🐖
he/him.
Me: https://julien.ledem.net/
Blog: https://sympathetic.ink
LFAI&data: OpenLineage, Marquez, ASF: Parquet, Arrow, Iceberg, 🐖
he/him.
Me: https://julien.ledem.net/
Blog: https://sympathetic.ink
In the past few years, we’ve seen a cambrian explosion of new columnar formats, challenging the hegemony of Parquet. Presumably, the design of yore is not going to cut it moving forward. I spent some time to understand a bit better how things actually changed.
sympathetic.ink/2025/12/11/C...
sympathetic.ink/2025/12/11/C...
Column Storage for the AI Era
In the past few years, we’ve seen a cambrian explosion of new columnar formats, challenging the hegemony of Parquet: Lance, Fastlanes, Nimble, Vortex, AnyBlox, F3 (File Format for the Future). The thi...
sympathetic.ink
December 11, 2025 at 7:29 PM
In the past few years, we’ve seen a cambrian explosion of new columnar formats, challenging the hegemony of Parquet. Presumably, the design of yore is not going to cut it moving forward. I spent some time to understand a bit better how things actually changed.
sympathetic.ink/2025/12/11/C...
sympathetic.ink/2025/12/11/C...
If you missed my talk "The advent of the open data lake" at AI By the Bay, the recording is now available.
ai.bythebay.io/talks/the-ad...
www.youtube.com/watch?v=xHGV...
ai.bythebay.io/talks/the-ad...
www.youtube.com/watch?v=xHGV...
The advent of the open data lake | Julien Le Dem, AI By the Bay 2025
YouTube video by FunctionalTV
www.youtube.com
December 9, 2025 at 9:03 PM
If you missed my talk "The advent of the open data lake" at AI By the Bay, the recording is now available.
ai.bythebay.io/talks/the-ad...
www.youtube.com/watch?v=xHGV...
ai.bythebay.io/talks/the-ad...
www.youtube.com/watch?v=xHGV...
It turns out that Friday’s NYT’s mini crosswords was written by my 13-year-old.
December 6, 2025 at 5:02 PM
It turns out that Friday’s NYT’s mini crosswords was written by my 13-year-old.
Parquet praise in the wild :) Nice chatting with you at Datacouncil Hannes!
www.youtube.com/watch?v=zeon...
www.youtube.com/watch?v=zeon...
Introducing DuckLake
YouTube video by DuckDB
www.youtube.com
December 1, 2025 at 10:47 PM
Parquet praise in the wild :) Nice chatting with you at Datacouncil Hannes!
www.youtube.com/watch?v=zeon...
www.youtube.com/watch?v=zeon...
Come say hi, Wednesday at 10am. I'll be speaking at the AI By the Bay Conference about "The Advent of The Open Data Lake".
ai.bythebay.io/talks/the-ad...
ai.bythebay.io/talks/the-ad...
The advent of the open data lake | AI By the Bay
ai.bythebay.io
November 18, 2025 at 12:38 AM
Come say hi, Wednesday at 10am. I'll be speaking at the AI By the Bay Conference about "The Advent of The Open Data Lake".
ai.bythebay.io/talks/the-ad...
ai.bythebay.io/talks/the-ad...
If you missed me giving it live, the slides of my talk, "Column Storage for the AI era" are available here:
docs.google.com/presentation...
docs.google.com/presentation...
Column Storage for the AI era
(illustration hand generated in 1958) “Column Storage for the AI era” © 2025 by Julien Le Dem is licensed under CC BY-NC-SA 4.0. To view a copy of this license, visit https://creativecommons.org/licen...
docs.google.com
November 14, 2025 at 11:20 PM
If you missed me giving it live, the slides of my talk, "Column Storage for the AI era" are available here:
docs.google.com/presentation...
docs.google.com/presentation...
Netflix: "We capture over a trillion events a day."
Datadog: www.datadoghq.com/blog/enginee....
Datadog: www.datadoghq.com/blog/enginee....
November 12, 2025 at 10:14 PM
Netflix: "We capture over a trillion events a day."
Datadog: www.datadoghq.com/blog/enginee....
Datadog: www.datadoghq.com/blog/enginee....
Good evening golden gate.
November 9, 2025 at 1:08 AM
Good evening golden gate.
Higher latency but higher throughput on improving the overall data ecosystem.
"if you want to go fast, go alone; If you want to go far, go together"
New Apache Parquet Community page is up: parquet.apache.org/community/
New Apache Parquet Community page is up: parquet.apache.org/community/
November 7, 2025 at 9:17 PM
Higher latency but higher throughput on improving the overall data ecosystem.
Reposted by Julien Le Dem
"if you want to go fast, go alone; If you want to go far, go together"
New Apache Parquet Community page is up: parquet.apache.org/community/
New Apache Parquet Community page is up: parquet.apache.org/community/
November 7, 2025 at 8:06 PM
"if you want to go fast, go alone; If you want to go far, go together"
New Apache Parquet Community page is up: parquet.apache.org/community/
New Apache Parquet Community page is up: parquet.apache.org/community/
Reposted by Julien Le Dem
The future of data connectivity is columnar. Today we launched
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
Announcing Columnar
Back to the future of data connectivity
columnar.tech
October 29, 2025 at 10:51 PM
The future of data connectivity is columnar. Today we launched
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
@columnar.tech to accelerate the shift from slow, row-oriented APIs like ODBC and JDBC to >10x faster alternatives powered by @arrow.apache.org. Learn more 👇
Experimenting with the laser cutter to make 3d objects.
November 3, 2025 at 12:21 AM
Experimenting with the laser cutter to make 3d objects.
This just went in the oven
October 31, 2025 at 12:40 AM
This just went in the oven
If you missed my talk: "Data Observability and OpenLineage" at the Datadog summit in SF, here is your chance to catch up on the recording.
www.youtube.com/watch?v=uhNo...
www.youtube.com/watch?v=uhNo...
Data Observability and OpenLineage
YouTube video by Datadog
www.youtube.com
October 28, 2025 at 10:30 PM
If you missed my talk: "Data Observability and OpenLineage" at the Datadog summit in SF, here is your chance to catch up on the recording.
www.youtube.com/watch?v=uhNo...
www.youtube.com/watch?v=uhNo...
Great time seeing Garbage at the Warfield tonight.
October 25, 2025 at 5:52 AM
Great time seeing Garbage at the Warfield tonight.
I'm speaking tomorrow at "Embracing AI in Open Source"
If you've been wondering why we see a flurry of new columnar formats, come see me present "Column Storage for the AI Era" I'll talk about what has changed, new advances in data encoding and how that's pushing Parquet to evolve.
luma.com/pxikwty3
If you've been wondering why we see a flurry of new columnar formats, come see me present "Column Storage for the AI Era" I'll talk about what has changed, new advances in data encoding and how that's pushing Parquet to evolve.
luma.com/pxikwty3
Next-Gen Data Engineering: Embracing AI in Open Source · Luma
Next-Gen Data Engineering: Embracing AI in Open Source
Join us October 23rd at the Silicon Valley AI Hub in Snowflake’s Menlo Park campus for an evening…
luma.com
October 22, 2025 at 9:47 PM
I'm speaking tomorrow at "Embracing AI in Open Source"
If you've been wondering why we see a flurry of new columnar formats, come see me present "Column Storage for the AI Era" I'll talk about what has changed, new advances in data encoding and how that's pushing Parquet to evolve.
luma.com/pxikwty3
If you've been wondering why we see a flurry of new columnar formats, come see me present "Column Storage for the AI Era" I'll talk about what has changed, new advances in data encoding and how that's pushing Parquet to evolve.
luma.com/pxikwty3
Who’s solving the Louvre robbery?
Right answers only
Right answers only
October 20, 2025 at 5:12 AM
Who’s solving the Louvre robbery?
Right answers only
Right answers only
First attempt at making prints with the laser cutter. I don’t know what I’m doing but happy to have made something.
October 19, 2025 at 8:28 PM
First attempt at making prints with the laser cutter. I don’t know what I’m doing but happy to have made something.
And then he said:
« Bring me to your leader! »
« Bring me to your leader! »
October 17, 2025 at 1:29 AM
And then he said:
« Bring me to your leader! »
« Bring me to your leader! »
Felt it
The epicenter of the quake, which was reported at 9:23 a.m., was in the center of campus, according to preliminary information from the U.S. Geological Survey.
Magnitude 3.1 earthquake shakes UC Berkeley campus an hour before planned quake drill
The epicenter of the quake, which was reported at 9:23 a.m., was in the center of campus, according to preliminary information from the U.S. Geological Survey.
www.berkeleyside.org
October 16, 2025 at 4:45 PM
Felt it
Reposted by Julien Le Dem
"It is not 100% clear to me how a new file format (or three) will drive additional ecosystem adoption :thinking:"
However, I absolutely think this adds to the pressure for Parquet to evolve.
Speaking of, anyone interested in helping add new encodings to parquet?
lists.apache.org/thread/djnbb...
However, I absolutely think this adds to the pressure for Parquet to evolve.
Speaking of, anyone interested in helping add new encodings to parquet?
lists.apache.org/thread/djnbb...
October 1, 2025 at 7:21 PM
"It is not 100% clear to me how a new file format (or three) will drive additional ecosystem adoption :thinking:"
However, I absolutely think this adds to the pressure for Parquet to evolve.
Speaking of, anyone interested in helping add new encodings to parquet?
lists.apache.org/thread/djnbb...
However, I absolutely think this adds to the pressure for Parquet to evolve.
Speaking of, anyone interested in helping add new encodings to parquet?
lists.apache.org/thread/djnbb...
"Why Datadog Chose Airflow 3: Multi-Tenancy, Observability, and the Future of Event-Driven Workflows"
Zach and I will be talking about how Datadog adopted Airflow 3 at the Airflow summit next week. Come say hi!
airflowsummit.org/sessions/202...
Zach and I will be talking about how Datadog adopted Airflow 3 at the Airflow summit next week. Come say hi!
airflowsummit.org/sessions/202...
October 1, 2025 at 3:39 PM
"Why Datadog Chose Airflow 3: Multi-Tenancy, Observability, and the Future of Event-Driven Workflows"
Zach and I will be talking about how Datadog adopted Airflow 3 at the Airflow summit next week. Come say hi!
airflowsummit.org/sessions/202...
Zach and I will be talking about how Datadog adopted Airflow 3 at the Airflow summit next week. Come say hi!
airflowsummit.org/sessions/202...