Daniel Wallace
@developyr.bsky.social
I design and document reproducible, production‑grade data lakehouse stacks — from cloud‑native to open‑source, real‑time, and ML‑driven. My work blends deep technical execution with clear, practical explanations you can actually use.
I've built for everyone else. Now I'm building for me.
My career has been filled with amazing projects for incredible people and companies. It's a truly wild feeling to see your own idea finally start coming to life.
It's a different kind of stake, a different kind of excitement.
#FoundersJourney
My career has been filled with amazing projects for incredible people and companies. It's a truly wild feeling to see your own idea finally start coming to life.
It's a different kind of stake, a different kind of excitement.
#FoundersJourney
November 4, 2025 at 12:00 AM
I've built for everyone else. Now I'm building for me.
My career has been filled with amazing projects for incredible people and companies. It's a truly wild feeling to see your own idea finally start coming to life.
It's a different kind of stake, a different kind of excitement.
#FoundersJourney
My career has been filled with amazing projects for incredible people and companies. It's a truly wild feeling to see your own idea finally start coming to life.
It's a different kind of stake, a different kind of excitement.
#FoundersJourney
Stop building pollers for your docker agents. An isolated container behind a firewall is just an IoT device.
We used the IoT playbook to get "zero inbound port" command & control. It's secure for the customer, scalable, and the future of SaaS.
#data #docker
developyr.substack.com/publish/post...
We used the IoT playbook to get "zero inbound port" command & control. It's secure for the customer, scalable, and the future of SaaS.
#data #docker
developyr.substack.com/publish/post...
Zero Inbound Ports: How We Use AWS IoT to Command 10,000 Containers in a Customer's VPC
We stopped treating our on-prem agents like SaaS and started treating them like smart thermostats. This is the secure, scalable pattern for hybrid cloud.
developyr.substack.com
November 1, 2025 at 1:55 PM
Stop building pollers for your docker agents. An isolated container behind a firewall is just an IoT device.
We used the IoT playbook to get "zero inbound port" command & control. It's secure for the customer, scalable, and the future of SaaS.
#data #docker
developyr.substack.com/publish/post...
We used the IoT playbook to get "zero inbound port" command & control. It's secure for the customer, scalable, and the future of SaaS.
#data #docker
developyr.substack.com/publish/post...
I have been asked about how will AI be incorporated into the Lokryn suite?
It won’t.
AI will be ready one day, but for now, for compliance, we think that should be deterministic.
This may not be the popular current opinion, but it’s one we stand on. We have kept AI out, on purpose. #data
It won’t.
AI will be ready one day, but for now, for compliance, we think that should be deterministic.
This may not be the popular current opinion, but it’s one we stand on. We have kept AI out, on purpose. #data
October 26, 2025 at 8:10 PM
I have been asked about how will AI be incorporated into the Lokryn suite?
It won’t.
AI will be ready one day, but for now, for compliance, we think that should be deterministic.
This may not be the popular current opinion, but it’s one we stand on. We have kept AI out, on purpose. #data
It won’t.
AI will be ready one day, but for now, for compliance, we think that should be deterministic.
This may not be the popular current opinion, but it’s one we stand on. We have kept AI out, on purpose. #data
Been spending some evenings and Sundays working on this, I think we are getting close. Check out products and pricing.
www.lokryn.com
www.lokryn.com
Lokryn
www.lokryn.com
October 24, 2025 at 9:22 PM
Been spending some evenings and Sundays working on this, I think we are getting close. Check out products and pricing.
www.lokryn.com
www.lokryn.com
Pipe Audit: Rust-Fast Data Validation. Data quality tools are too heavy. Pipe Audit is a blazing-fast, language-agnostic validation engine. Configure simple .toml files and validate data anywhere.
Free CLI/lib. Heavy Duty with Dashboard from $99/mo.
#dataengineering #rust #devops #dataquality
Free CLI/lib. Heavy Duty with Dashboard from $99/mo.
#dataengineering #rust #devops #dataquality
October 24, 2025 at 7:51 PM
Pipe Audit: Rust-Fast Data Validation. Data quality tools are too heavy. Pipe Audit is a blazing-fast, language-agnostic validation engine. Configure simple .toml files and validate data anywhere.
Free CLI/lib. Heavy Duty with Dashboard from $99/mo.
#dataengineering #rust #devops #dataquality
Free CLI/lib. Heavy Duty with Dashboard from $99/mo.
#dataengineering #rust #devops #dataquality
Product Coming Soon--Flight Recorder: Flat-Rate Logging. Stop paying per-GB. Flight Recorder is your self-hosted logger with tamper-evident logs for audits, minus the enterprise price tag.
Free forever with unlimited volume. Heavy Duty plan with full UI & alerts from $99/mo.
#logging #observability
Free forever with unlimited volume. Heavy Duty plan with full UI & alerts from $99/mo.
#logging #observability
October 24, 2025 at 7:49 PM
Product Coming Soon--Flight Recorder: Flat-Rate Logging. Stop paying per-GB. Flight Recorder is your self-hosted logger with tamper-evident logs for audits, minus the enterprise price tag.
Free forever with unlimited volume. Heavy Duty plan with full UI & alerts from $99/mo.
#logging #observability
Free forever with unlimited volume. Heavy Duty plan with full UI & alerts from $99/mo.
#logging #observability
Product Coming Soon--Tracks: Feature Flags. Feature flags aren't free—they're tech debt. Tracks finds stale flags and integrates with Jira to help you clean them up.
Free for 10k MAU. Heavy Duty plan with Unlimited MAUs & Stale Flag Reports from $69/mo.
#featureflags #devops #cicd #techdebt
Free for 10k MAU. Heavy Duty plan with Unlimited MAUs & Stale Flag Reports from $69/mo.
#featureflags #devops #cicd #techdebt
October 24, 2025 at 7:48 PM
Product Coming Soon--Tracks: Feature Flags. Feature flags aren't free—they're tech debt. Tracks finds stale flags and integrates with Jira to help you clean them up.
Free for 10k MAU. Heavy Duty plan with Unlimited MAUs & Stale Flag Reports from $69/mo.
#featureflags #devops #cicd #techdebt
Free for 10k MAU. Heavy Duty plan with Unlimited MAUs & Stale Flag Reports from $69/mo.
#featureflags #devops #cicd #techdebt
Product Coming Soon--Foreman: The Language-Agnostic Orchestrator. Stop compromising. Forman is a workflow engine that runs in your environment. No language-lock, your code & data never leave your control.
Free dev sandbox. Heavy Duty plan with Visual UI from $69/mo.
#orchestration #selfhosted
Free dev sandbox. Heavy Duty plan with Visual UI from $69/mo.
#orchestration #selfhosted
October 24, 2025 at 7:47 PM
Product Coming Soon--Foreman: The Language-Agnostic Orchestrator. Stop compromising. Forman is a workflow engine that runs in your environment. No language-lock, your code & data never leave your control.
Free dev sandbox. Heavy Duty plan with Visual UI from $69/mo.
#orchestration #selfhosted
Free dev sandbox. Heavy Duty plan with Visual UI from $69/mo.
#orchestration #selfhosted
Product Coming Soon--re-Deliver: Guaranteed Webhook Reliability. Missed callbacks are a silent killer. re-Deliver secures every event, letting you manage and replay missed webhooks in order.
Free for 10k callbacks. Heavy Duty plan with 7-day ordered replay from $69/mo.
#webhooks #devops #api
Free for 10k callbacks. Heavy Duty plan with 7-day ordered replay from $69/mo.
#webhooks #devops #api
October 24, 2025 at 7:45 PM
Product Coming Soon--Gatehouse: Auth without the bloat. Stop choosing between risky DIY auth and bloated enterprise platforms. Get the expert features you actually need, like passwordless & MFA, without the enterprise overhead.
Free for 1000 MAU. Heavy Duty plan from $49/mo.
#auth #security #saas
Free for 1000 MAU. Heavy Duty plan from $49/mo.
#auth #security #saas
October 24, 2025 at 7:45 PM
Tired of dev tools for small teams being either "free toys" or "enterprise bloat"? 😤
Me too. So I'm building the alternative.
Announcing Lokryn: a new suite of pro-level, self-hosted tools that don't cost a junior dev's salary.
open.substack.com/pub/developy...
#dev #devtools #saas #startups
Me too. So I'm building the alternative.
Announcing Lokryn: a new suite of pro-level, self-hosted tools that don't cost a junior dev's salary.
open.substack.com/pub/developy...
#dev #devtools #saas #startups
Announcing Lokryn: Professional Tools, Not Enterprise Overhead
We're stuck choosing between "free toys" that can't scale and "enterprise suites" that break the bank. I'm building the alternative. Announcing Lokryn: a new suite of pro-value dev tools for small bus...
open.substack.com
October 23, 2025 at 9:52 PM
Tired of dev tools for small teams being either "free toys" or "enterprise bloat"? 😤
Me too. So I'm building the alternative.
Announcing Lokryn: a new suite of pro-level, self-hosted tools that don't cost a junior dev's salary.
open.substack.com/pub/developy...
#dev #devtools #saas #startups
Me too. So I'm building the alternative.
Announcing Lokryn: a new suite of pro-level, self-hosted tools that don't cost a junior dev's salary.
open.substack.com/pub/developy...
#dev #devtools #saas #startups
WIP Friday! 👀 Here's a first look at the PipeAudit dashboard we're building.
The goal is to make your data validation results and audit trails super easy to visualize.
No more hunting for answers. Just clear visibility. More to come!
#PipeAudit #DataEngineering #WIP #UIUX #RustLang
The goal is to make your data validation results and audit trails super easy to visualize.
No more hunting for answers. Just clear visibility. More to come!
#PipeAudit #DataEngineering #WIP #UIUX #RustLang
October 3, 2025 at 9:58 PM
WIP Friday! 👀 Here's a first look at the PipeAudit dashboard we're building.
The goal is to make your data validation results and audit trails super easy to visualize.
No more hunting for answers. Just clear visibility. More to come!
#PipeAudit #DataEngineering #WIP #UIUX #RustLang
The goal is to make your data validation results and audit trails super easy to visualize.
No more hunting for answers. Just clear visibility. More to come!
#PipeAudit #DataEngineering #WIP #UIUX #RustLang
Most data tools are built for the enterprise. They're expensive, monolithic, and overkill for startups.
Announcing Lokryn: a new suite of data tools built for SMBs. Think modular, developer-friendly, and compliance-first. And yes, it's all built on Rust 🦀.
www.lokryn.com
Announcing Lokryn: a new suite of data tools built for SMBs. Think modular, developer-friendly, and compliance-first. And yes, it's all built on Rust 🦀.
www.lokryn.com
Lokryn
Compliance‑first, Rust‑powered platform — fast, modular, and built for scale.
www.lokryn.com
September 30, 2025 at 2:26 PM
Most data tools are built for the enterprise. They're expensive, monolithic, and overkill for startups.
Announcing Lokryn: a new suite of data tools built for SMBs. Think modular, developer-friendly, and compliance-first. And yes, it's all built on Rust 🦀.
www.lokryn.com
Announcing Lokryn: a new suite of data tools built for SMBs. Think modular, developer-friendly, and compliance-first. And yes, it's all built on Rust 🦀.
www.lokryn.com
Real talk for data folks: a lot of our time feels less like engineering and more like archaeology, right? Digging through logs after something has already broken.
I wrote down some thoughts on how we can change that pattern.
developyr.substack.com/p/your-data-...
#DataFolk #DataEngineering
I wrote down some thoughts on how we can change that pattern.
developyr.substack.com/p/your-data-...
#DataFolk #DataEngineering
Your Data Quality is Broken. Here’s Why No One’s Talking About It.
“Most teams discover data quality issues in production. PipeAudit flips that: fast, boring, compliance‑ready validation you can trust.”
developyr.substack.com
September 30, 2025 at 2:09 PM
Real talk for data folks: a lot of our time feels less like engineering and more like archaeology, right? Digging through logs after something has already broken.
I wrote down some thoughts on how we can change that pattern.
developyr.substack.com/p/your-data-...
#DataFolk #DataEngineering
I wrote down some thoughts on how we can change that pattern.
developyr.substack.com/p/your-data-...
#DataFolk #DataEngineering
Building an agentic AI data pipeline the hard way. 🤖
I took a deep dive into the real-world struggles of getting an AI agent to work autonomously. No hype, just the honest journey.
Watch the full code walkthrough:
youtu.be/x4KCMFlOwxI
#AgenticAI #DataEngineering #LLM #Gemini #Llama3
I took a deep dive into the real-world struggles of getting an AI agent to work autonomously. No hype, just the honest journey.
Watch the full code walkthrough:
youtu.be/x4KCMFlOwxI
#AgenticAI #DataEngineering #LLM #Gemini #Llama3
Building an Agentic AI Pipeline: The Hard Way (Llama 3, Gemini 1.5 & Many Failures)
YouTube video by Developyr
youtu.be
September 21, 2025 at 8:06 PM
Building an agentic AI data pipeline the hard way. 🤖
I took a deep dive into the real-world struggles of getting an AI agent to work autonomously. No hype, just the honest journey.
Watch the full code walkthrough:
youtu.be/x4KCMFlOwxI
#AgenticAI #DataEngineering #LLM #Gemini #Llama3
I took a deep dive into the real-world struggles of getting an AI agent to work autonomously. No hype, just the honest journey.
Watch the full code walkthrough:
youtu.be/x4KCMFlOwxI
#AgenticAI #DataEngineering #LLM #Gemini #Llama3
I used Microsoft Copilot to help build an Agentic AI data pipeline, forcing it to remove the usual guardrails to test the hype vs. reality.
The results?
Read the full "Lab Notes" experiment here:
#AgenticAI #DataEngineering #LLM
developyr.substack.com/p/i-let-copi...
The results?
Read the full "Lab Notes" experiment here:
#AgenticAI #DataEngineering #LLM
developyr.substack.com/p/i-let-copi...
I Let Copilot Help Build My Agentic AI Data Pipeline (It Got Weird)
I let an AI build my agentic data pipeline to see what would happen. It got weird. 🤖Here's an honest, hands-on look at the gap between the promise of agentic AI and the current, often brittle, reality...
developyr.substack.com
September 18, 2025 at 6:36 PM
I used Microsoft Copilot to help build an Agentic AI data pipeline, forcing it to remove the usual guardrails to test the hype vs. reality.
The results?
Read the full "Lab Notes" experiment here:
#AgenticAI #DataEngineering #LLM
developyr.substack.com/p/i-let-copi...
The results?
Read the full "Lab Notes" experiment here:
#AgenticAI #DataEngineering #LLM
developyr.substack.com/p/i-let-copi...
How do you let devs & QA test with fresh production data without exposing PII?
My 'Lab Notes' article walks through using Trino, Nessie & Iceberg to create ephemeral, zero-copy environments with on-the-fly PII masking. 🛡️
developyr.substack.com/p/lab-notes-...
#DataEngineering #DataSecurity #Trino
My 'Lab Notes' article walks through using Trino, Nessie & Iceberg to create ephemeral, zero-copy environments with on-the-fly PII masking. 🛡️
developyr.substack.com/p/lab-notes-...
#DataEngineering #DataSecurity #Trino
Lab Notes: Ephemeral, PII-Masked Environments with Trino
How do you safely analyze fresh production data across the entire SDLC without creating costly copies? My new 'Lab Notes' article walks through a pattern using Trino, Nessie, and Iceberg to spin up ep...
developyr.substack.com
September 15, 2025 at 2:09 PM
How do you let devs & QA test with fresh production data without exposing PII?
My 'Lab Notes' article walks through using Trino, Nessie & Iceberg to create ephemeral, zero-copy environments with on-the-fly PII masking. 🛡️
developyr.substack.com/p/lab-notes-...
#DataEngineering #DataSecurity #Trino
My 'Lab Notes' article walks through using Trino, Nessie & Iceberg to create ephemeral, zero-copy environments with on-the-fly PII masking. 🛡️
developyr.substack.com/p/lab-notes-...
#DataEngineering #DataSecurity #Trino
Consulting isn’t just advice — it’s partnership. We design scalable, adaptable data systems so your team can move from firefighting to forward momentum. Let’s build clarity together. #DataConsulting #DataStrategy #ScalableSystems #Developyr #DataOps
September 14, 2025 at 6:23 PM
Consulting isn’t just advice — it’s partnership. We design scalable, adaptable data systems so your team can move from firefighting to forward momentum. Let’s build clarity together. #DataConsulting #DataStrategy #ScalableSystems #Developyr #DataOps
No data is perfect, some is actionable.
The sooner you stop chasing “perfect,” the sooner you start making progress.
Comfort in the Unknown isn’t about having all the answers — it’s about moving forward when you don’t.
#DataEngineering #ProblemSolving #ComfortInTheUnknown
The sooner you stop chasing “perfect,” the sooner you start making progress.
Comfort in the Unknown isn’t about having all the answers — it’s about moving forward when you don’t.
#DataEngineering #ProblemSolving #ComfortInTheUnknown
September 11, 2025 at 2:36 PM
No data is perfect, some is actionable.
The sooner you stop chasing “perfect,” the sooner you start making progress.
Comfort in the Unknown isn’t about having all the answers — it’s about moving forward when you don’t.
#DataEngineering #ProblemSolving #ComfortInTheUnknown
The sooner you stop chasing “perfect,” the sooner you start making progress.
Comfort in the Unknown isn’t about having all the answers — it’s about moving forward when you don’t.
#DataEngineering #ProblemSolving #ComfortInTheUnknown
Tackling a classic data engineering dilemma for my next 'Lab Notes' article:
How do you analyze FRESH production data without
a) creating slow, costly copies, and
b) exposing sensitive PII?
It feels like you have to compromise on one of these.
#DataEngineering #DataSecurity #PII #substack
How do you analyze FRESH production data without
a) creating slow, costly copies, and
b) exposing sensitive PII?
It feels like you have to compromise on one of these.
#DataEngineering #DataSecurity #PII #substack
September 10, 2025 at 1:32 PM
Tackling a classic data engineering dilemma for my next 'Lab Notes' article:
How do you analyze FRESH production data without
a) creating slow, costly copies, and
b) exposing sensitive PII?
It feels like you have to compromise on one of these.
#DataEngineering #DataSecurity #PII #substack
How do you analyze FRESH production data without
a) creating slow, costly copies, and
b) exposing sensitive PII?
It feels like you have to compromise on one of these.
#DataEngineering #DataSecurity #PII #substack
Evening thought: The most elegant data architecture is often the simplest one that gets the job done. It's easy to get lost in complex tools, but clarity and simplicity usually win in the long run.
#DataArchitecture #DataEngineering
#DataArchitecture #DataEngineering
September 8, 2025 at 11:39 PM
Evening thought: The most elegant data architecture is often the simplest one that gets the job done. It's easy to get lost in complex tools, but clarity and simplicity usually win in the long run.
#DataArchitecture #DataEngineering
#DataArchitecture #DataEngineering
Monday morning question for all the data folks ☕:
What's your go-to tool for quick, local data exploration on your laptop?
Are you team DuckDB, Polars, Pandas 2.0, or something else entirely? Curious to see what everyone is building with.
#DataEngineering #DataScience #DuckDB #Polars #Python
What's your go-to tool for quick, local data exploration on your laptop?
Are you team DuckDB, Polars, Pandas 2.0, or something else entirely? Curious to see what everyone is building with.
#DataEngineering #DataScience #DuckDB #Polars #Python
September 8, 2025 at 12:42 PM
Monday morning question for all the data folks ☕:
What's your go-to tool for quick, local data exploration on your laptop?
Are you team DuckDB, Polars, Pandas 2.0, or something else entirely? Curious to see what everyone is building with.
#DataEngineering #DataScience #DuckDB #Polars #Python
What's your go-to tool for quick, local data exploration on your laptop?
Are you team DuckDB, Polars, Pandas 2.0, or something else entirely? Curious to see what everyone is building with.
#DataEngineering #DataScience #DuckDB #Polars #Python
Just submitted a talk proposal for DataTune 2026 on "Decoding the Modern Data Lakehouse." 🤞
My goal is to bring a clear framework to the 6 core lakehouse patterns we see in the wild. Hope to see some local tech folks there!
#DataEngineering #NashvilleTech #DataArchitecture #DataTune2026
My goal is to bring a clear framework to the 6 core lakehouse patterns we see in the wild. Hope to see some local tech folks there!
#DataEngineering #NashvilleTech #DataArchitecture #DataTune2026
September 7, 2025 at 10:49 PM
Just submitted a talk proposal for DataTune 2026 on "Decoding the Modern Data Lakehouse." 🤞
My goal is to bring a clear framework to the 6 core lakehouse patterns we see in the wild. Hope to see some local tech folks there!
#DataEngineering #NashvilleTech #DataArchitecture #DataTune2026
My goal is to bring a clear framework to the 6 core lakehouse patterns we see in the wild. Hope to see some local tech folks there!
#DataEngineering #NashvilleTech #DataArchitecture #DataTune2026
Developyr’s Substack
developyr.substack.com?r=69mg6q&utm...
developyr.substack.com?r=69mg6q&utm...
Developyr’s Substack | Substack
Exploring modern data engineering, from local pipelines to cloud-scale patterns. Click to read Developyr’s Substack, a Substack publication. Launched 19 minutes ago.
developyr.substack.com
September 7, 2025 at 8:04 PM
Developyr’s Substack
developyr.substack.com?r=69mg6q&utm...
developyr.substack.com?r=69mg6q&utm...
What is a "data lakehouse," really? The term is everywhere, but it's not one-size-fits-all. 🗺️
It's actually a set of architectural patterns. I break it down into the 6 core archetypes.
developyr.substack.com/p/decoding-t...
#DataLakehouse #DataArchitecture #DataEngineering #CloudComputing
It's actually a set of architectural patterns. I break it down into the 6 core archetypes.
developyr.substack.com/p/decoding-t...
#DataLakehouse #DataArchitecture #DataEngineering #CloudComputing
Decoding the Modern Data Lakehouse: The 6 Core Archetypes You Need to Know
The term 'data lakehouse' is everywhere, but it's not one-size-fits-all. I've broken down the 6 core architectural patterns that define the industry today, from Cloud-Native to Open Source. Find out w...
developyr.substack.com
September 7, 2025 at 8:00 PM
What is a "data lakehouse," really? The term is everywhere, but it's not one-size-fits-all. 🗺️
It's actually a set of architectural patterns. I break it down into the 6 core archetypes.
developyr.substack.com/p/decoding-t...
#DataLakehouse #DataArchitecture #DataEngineering #CloudComputing
It's actually a set of architectural patterns. I break it down into the 6 core archetypes.
developyr.substack.com/p/decoding-t...
#DataLakehouse #DataArchitecture #DataEngineering #CloudComputing