Martin Chesbrough
@mchesbro.bsky.social
Socio-technical data designer
I wrote something about *Real* Data Engineering… then added the AI bit
martinchesbrough.net/data-and-ai-...
martinchesbrough.net/data-and-ai-...
Data and AI Work … is it Engineering?
Inspired by Glenn Vanderburg’s talk on Real Software Engineering I ask what do we need to do to make Data and AI work Real Engineering?
martinchesbrough.net
May 31, 2025 at 5:53 AM
I wrote something about *Real* Data Engineering… then added the AI bit
martinchesbrough.net/data-and-ai-...
martinchesbrough.net/data-and-ai-...
I am signing up for Chad Sanderson's Shift Left Data Manifesto www.gable.ai/blog/shift-l...
It is more than data contracts but we all need to start somewhere ...
It is more than data contracts but we all need to start somewhere ...
Gable Blog - The Shift Left Data Manifesto
A core idea behind shifting Data Left is simple but often overlooked: data is code. Or more accurately—data is produced by code. It’s not just some downstream artifact that lives in tables and gets pi...
www.gable.ai
March 28, 2025 at 4:19 AM
I am signing up for Chad Sanderson's Shift Left Data Manifesto www.gable.ai/blog/shift-l...
It is more than data contracts but we all need to start somewhere ...
It is more than data contracts but we all need to start somewhere ...
Adding more layers to your data architecture is unlikely to make it easier to manage data or make it work better - consider alternatives martinchesbrough.net/against-laye...
#dataengineering #dataarchitecture #datamesh #softwarearchitecture #datamodeling
#dataengineering #dataarchitecture #datamesh #softwarearchitecture #datamodeling
Against Layers: an argument for pluralism in analytical data architecture
Analytical data architectures have tended to over-rotate towards layers. This essay is an attempt to argue for more diverse thinking.
martinchesbrough.net
March 22, 2025 at 5:04 AM
Adding more layers to your data architecture is unlikely to make it easier to manage data or make it work better - consider alternatives martinchesbrough.net/against-laye...
#dataengineering #dataarchitecture #datamesh #softwarearchitecture #datamodeling
#dataengineering #dataarchitecture #datamesh #softwarearchitecture #datamodeling
I've been meaning to write this for a while - it is titled "Against Layers" and is a plea for pluralism in data architecture.
March 15, 2025 at 4:02 AM
I've been meaning to write this for a while - it is titled "Against Layers" and is a plea for pluralism in data architecture.
Reposted by Martin Chesbrough
If you work with groups and teams, the Fun Retrospectives website is an excellent resource for ideas. I enjoy using my favorite techniques and experimenting with new ones to keep things exciting. #QiSky #Facilitation
Futurespective | FunRetrospectives
Have fun, learn from the past and prepare for the future!
buff.ly
December 18, 2024 at 4:20 PM
If you work with groups and teams, the Fun Retrospectives website is an excellent resource for ideas. I enjoy using my favorite techniques and experimenting with new ones to keep things exciting. #QiSky #Facilitation
Reposted by Martin Chesbrough
A ClickHouse query optimization guide. SQL-based observability in the wild. PIVOTing in ClickHouse, and more. It must be our final newsletter of 2024! 🎄
December 2024 newsletter
Welcome to the December ClickHouse newsletter, which will round up what’s happened in real-time data warehouses over the last month.
buff.ly
December 19, 2024 at 10:26 AM
A ClickHouse query optimization guide. SQL-based observability in the wild. PIVOTing in ClickHouse, and more. It must be our final newsletter of 2024! 🎄
I was talking to a friend who’s very into Gen AI at a recent Xmas party and I put it to home that my views is that the LLM model “wars” are now pretty well over …
December 15, 2024 at 12:37 PM
I was talking to a friend who’s very into Gen AI at a recent Xmas party and I put it to home that my views is that the LLM model “wars” are now pretty well over …
I've had a lot of fun today playing with Gemini 2.0 through aistudio.google.com
- I got it to read an academic paper on my screen (Claude does this as well but Gemini does it better)
- I got it to read an academic paper on my screen (Claude does this as well but Gemini does it better)
Google AI Studio
Google AI Studio is the fastest way to start building with Gemini, our next generation family of multimodal generative AI models.
aistudio.google.com
December 13, 2024 at 6:34 AM
I've had a lot of fun today playing with Gemini 2.0 through aistudio.google.com
- I got it to read an academic paper on my screen (Claude does this as well but Gemini does it better)
- I got it to read an academic paper on my screen (Claude does this as well but Gemini does it better)
Another day, another book recommendation. This time it is "Managing Data as a Product" by Andrea Gioia, published by Packt
www.packtpub.com/en-au/produc...
www.packtpub.com/en-au/produc...
December 10, 2024 at 11:02 PM
Another day, another book recommendation. This time it is "Managing Data as a Product" by Andrea Gioia, published by Packt
www.packtpub.com/en-au/produc...
www.packtpub.com/en-au/produc...
Reposted by Martin Chesbrough
Calling univ profs & lecturers! If you want to bring concepts from Doughnut Economics into your teaching, we've just launched a webpage bursting with resources for you: 7 slide decks, reading lists, videos & activities - all open access. Dive in & pls reshare! doughnuteconomics.org/university-c...
November 25, 2024 at 12:17 PM
Calling univ profs & lecturers! If you want to bring concepts from Doughnut Economics into your teaching, we've just launched a webpage bursting with resources for you: 7 slide decks, reading lists, videos & activities - all open access. Dive in & pls reshare! doughnuteconomics.org/university-c...
Because of a discussion on misuse of the term “data warehouse” I referenced a section from James Serra’s book “Deciphering Data Architectures”.
That led me to re-read the book. It’s a great book BTW and worth a read.
That led me to re-read the book. It’s a great book BTW and worth a read.
December 8, 2024 at 1:14 AM
Because of a discussion on misuse of the term “data warehouse” I referenced a section from James Serra’s book “Deciphering Data Architectures”.
That led me to re-read the book. It’s a great book BTW and worth a read.
That led me to re-read the book. It’s a great book BTW and worth a read.
I like using BlueSky as a short-form essay writer so here goes with my latest thoughts.
The essay might be titled:
"The Future of Software is AI ... NOT"
or
"Software eats the world ... this time with an AI sauce"
or
"The Future of Business is ... unreasonable AI agents"
or
something else
The essay might be titled:
"The Future of Software is AI ... NOT"
or
"Software eats the world ... this time with an AI sauce"
or
"The Future of Business is ... unreasonable AI agents"
or
something else
November 30, 2024 at 5:38 AM
I like using BlueSky as a short-form essay writer so here goes with my latest thoughts.
The essay might be titled:
"The Future of Software is AI ... NOT"
or
"Software eats the world ... this time with an AI sauce"
or
"The Future of Business is ... unreasonable AI agents"
or
something else
The essay might be titled:
"The Future of Software is AI ... NOT"
or
"Software eats the world ... this time with an AI sauce"
or
"The Future of Business is ... unreasonable AI agents"
or
something else
I’m having great fun with building AI agents but the various agent orchestration frameworks are still immature (Autogen 0.2.39, LangGraph 0.1.40, CrewAI v0.83.0, LlamaIndex v0.12.2). It feels like early Hadoop days.
I wonder how fast these frameworks will proceed. I’m excited for what happens next
I wonder how fast these frameworks will proceed. I’m excited for what happens next
It's a full stack team (including AI) building an AI product just like you would build any digital product. AI skills are just part of the team!
And the beauty of it was that it used a generic model - no specialised model building and training required.
Product development has changed ... forever!
And the beauty of it was that it used a generic model - no specialised model building and training required.
Product development has changed ... forever!
November 29, 2024 at 8:50 AM
I’m having great fun with building AI agents but the various agent orchestration frameworks are still immature (Autogen 0.2.39, LangGraph 0.1.40, CrewAI v0.83.0, LlamaIndex v0.12.2). It feels like early Hadoop days.
I wonder how fast these frameworks will proceed. I’m excited for what happens next
I wonder how fast these frameworks will proceed. I’m excited for what happens next
Just listened to this conference talk from @simon.bvssh.com - well the dude drops more names in a 30 minute talk than I can fit into a month - BUT it's a great talk!
youtu.be/ASE6Pt3fnxM?...
youtu.be/ASE6Pt3fnxM?...
Simon Rohrer — Modern Enterprise Architecture: architecting for outcomes "Modern Enterprise A..."
YouTube video by Agile meets Architecture
youtu.be
November 28, 2024 at 10:59 PM
Just listened to this conference talk from @simon.bvssh.com - well the dude drops more names in a 30 minute talk than I can fit into a month - BUT it's a great talk!
youtu.be/ASE6Pt3fnxM?...
youtu.be/ASE6Pt3fnxM?...
Reposted by Martin Chesbrough
Apply Data Mesh principles to solve case studies, an online workshop with Jacek Majchrzak.
New dates added. Take advantage of the early bird: https://buff.ly/3HWRNrX
New dates added. Take advantage of the early bird: https://buff.ly/3HWRNrX
November 28, 2024 at 9:00 AM
Apply Data Mesh principles to solve case studies, an online workshop with Jacek Majchrzak.
New dates added. Take advantage of the early bird: https://buff.ly/3HWRNrX
New dates added. Take advantage of the early bird: https://buff.ly/3HWRNrX
I’m keen to get some feedback …
I’ve been playing around with building Gen AI agents using Autogen, Langchain and Llamaindex.
And I’m coming to the conclusion that these are just small apps or applets.
I’ve been playing around with building Gen AI agents using Autogen, Langchain and Llamaindex.
And I’m coming to the conclusion that these are just small apps or applets.
November 26, 2024 at 7:55 AM
I’m keen to get some feedback …
I’ve been playing around with building Gen AI agents using Autogen, Langchain and Llamaindex.
And I’m coming to the conclusion that these are just small apps or applets.
I’ve been playing around with building Gen AI agents using Autogen, Langchain and Llamaindex.
And I’m coming to the conclusion that these are just small apps or applets.
The 2nd time that you read a book I think you get a lot more out of it than the first time.
This was worth a 2nd read
www.manning.com/books/archit...
This was worth a 2nd read
www.manning.com/books/archit...
Architecture Modernization
Proven techniques and principles for modernizing legacy systems into new architectures that deliver serious competitive advantage.</b>
For a business to thrive, it needs a modern software architectur...
www.manning.com
November 25, 2024 at 3:02 AM
The 2nd time that you read a book I think you get a lot more out of it than the first time.
This was worth a 2nd read
www.manning.com/books/archit...
This was worth a 2nd read
www.manning.com/books/archit...
Reposted by Martin Chesbrough
I made an infra engineer starter pack. Folks posting about databases, stream processing, durable execution, orchestrators, service meshes, and more.
go.bsky.app/SCZe42X
go.bsky.app/SCZe42X
October 25, 2024 at 1:16 AM
I made an infra engineer starter pack. Folks posting about databases, stream processing, durable execution, orchestrators, service meshes, and more.
go.bsky.app/SCZe42X
go.bsky.app/SCZe42X
How can you tell if a specific agile practice is relevant to your current situation?
About a week ago I started parallel conversations on the theoretical underpinnings of agile methods in different communities to try and answer this question. I received some very interesting responses.
About a week ago I started parallel conversations on the theoretical underpinnings of agile methods in different communities to try and answer this question. I received some very interesting responses.
Agile Time Machine community initiative
Agile Time Machine Overview of the pivotal moments and momentous ideas worth spreading again, and a general feeling of the original essence (Last updated on 2-Oct-2024) Read Me This document is the re...
bit.ly
November 25, 2024 at 12:14 AM
How can you tell if a specific agile practice is relevant to your current situation?
About a week ago I started parallel conversations on the theoretical underpinnings of agile methods in different communities to try and answer this question. I received some very interesting responses.
About a week ago I started parallel conversations on the theoretical underpinnings of agile methods in different communities to try and answer this question. I received some very interesting responses.
Reposted by Martin Chesbrough
How can #WardleyMapping, #DDD, and #TeamTopologies be combined in a structured way to optimize systems for flow? I developed an opinionated structured approach and visualized it in an Architecture for Flow Canvas.
You can find more information at susannekaiser.net/architecture-for-flow-canvas.
You can find more information at susannekaiser.net/architecture-for-flow-canvas.
November 21, 2024 at 9:01 PM
How can #WardleyMapping, #DDD, and #TeamTopologies be combined in a structured way to optimize systems for flow? I developed an opinionated structured approach and visualized it in an Architecture for Flow Canvas.
You can find more information at susannekaiser.net/architecture-for-flow-canvas.
You can find more information at susannekaiser.net/architecture-for-flow-canvas.
Reposted by Martin Chesbrough
ICYMI I recently interviewed the creator of @clickhouse.com, Alexey Milovidov.
We dove deep into what ClickHouse is so fast, why AI companies love it (hint: its had a vector data type since 2012), rewriting Zookeeper (yikes), and how math informs database design.
www.rilldata.com/blog/rill-cl...
We dove deep into what ClickHouse is so fast, why AI companies love it (hint: its had a vector data type since 2012), rewriting Zookeeper (yikes), and how math informs database design.
www.rilldata.com/blog/rill-cl...
Rill | Data Talks on the Rocks 4 - Alexey Milovidov, ClickHouse
A video interview with Alexey Milovidov, co-founder and CTO of ClickHouse, that dives deep into AI, ClickHouse’s architecture, different database categories, and pain points in the market that ClickHo...
www.rilldata.com
November 20, 2024 at 9:41 PM
ICYMI I recently interviewed the creator of @clickhouse.com, Alexey Milovidov.
We dove deep into what ClickHouse is so fast, why AI companies love it (hint: its had a vector data type since 2012), rewriting Zookeeper (yikes), and how math informs database design.
www.rilldata.com/blog/rill-cl...
We dove deep into what ClickHouse is so fast, why AI companies love it (hint: its had a vector data type since 2012), rewriting Zookeeper (yikes), and how math informs database design.
www.rilldata.com/blog/rill-cl...
Has Data Engineering forgotten about socio-technical architecture? I think so and I wrote up a blog post about it.
www.linkedin.com/posts/martin...
www.linkedin.com/posts/martin...
Martin C. on LinkedIn: Learning Socio-Technical Data Architecture
6 weeks ago I had the great pleasure of attending #dataengbytes in Melbourne and learning a heap from the great speakers and attendees at this community data…
www.linkedin.com
November 21, 2024 at 9:13 PM
Has Data Engineering forgotten about socio-technical architecture? I think so and I wrote up a blog post about it.
www.linkedin.com/posts/martin...
www.linkedin.com/posts/martin...
Just did this course. I like the parallel video-notebook screen so you can play with the code as the course runs.
It is very much a beginner style course (that's ok) and any dev would be well placed to take an AWS-GCP-Azure-Anthropic-other course to get a broader knowledge of prompting.
It is very much a beginner style course (that's ok) and any dev would be well placed to take an AWS-GCP-Azure-Anthropic-other course to get a broader knowledge of prompting.
This seems excellent, an online course for Prompt Engineering from OpenAI in combination with Deeplearning.ai (from Andrew NG) https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/
November 21, 2024 at 7:50 AM
Just did this course. I like the parallel video-notebook screen so you can play with the code as the course runs.
It is very much a beginner style course (that's ok) and any dev would be well placed to take an AWS-GCP-Azure-Anthropic-other course to get a broader knowledge of prompting.
It is very much a beginner style course (that's ok) and any dev would be well placed to take an AWS-GCP-Azure-Anthropic-other course to get a broader knowledge of prompting.
Reposted by Martin Chesbrough
"Blah blag blah decoupled blah blah blah."
Decoupled. Decouple. Decoupling. Decoupled.
“You Keep Using That Word. I Do Not Think It Means What You Think It Means.” -- Inigo Montoya
Semantics matter.
Thank me by subscribing and watching my following episodes.
youtu.be/ZffqJxLULx8?...
Decoupled. Decouple. Decoupling. Decoupled.
“You Keep Using That Word. I Do Not Think It Means What You Think It Means.” -- Inigo Montoya
Semantics matter.
Thank me by subscribing and watching my following episodes.
youtu.be/ZffqJxLULx8?...
November 20, 2024 at 11:32 PM
"Blah blag blah decoupled blah blah blah."
Decoupled. Decouple. Decoupling. Decoupled.
“You Keep Using That Word. I Do Not Think It Means What You Think It Means.” -- Inigo Montoya
Semantics matter.
Thank me by subscribing and watching my following episodes.
youtu.be/ZffqJxLULx8?...
Decoupled. Decouple. Decoupling. Decoupled.
“You Keep Using That Word. I Do Not Think It Means What You Think It Means.” -- Inigo Montoya
Semantics matter.
Thank me by subscribing and watching my following episodes.
youtu.be/ZffqJxLULx8?...
AI has evolved from a specialist nerd science into an everyday part of your product development team. Here's my recent experiences that support this argument.
But first some history (to help make the argument)
But first some history (to help make the argument)
November 20, 2024 at 10:35 PM
AI has evolved from a specialist nerd science into an everyday part of your product development team. Here's my recent experiences that support this argument.
But first some history (to help make the argument)
But first some history (to help make the argument)