Lightnews — Scholar-powered news

Gerd Zellweger @zgerd.bsky.social · Aug 25

Slack app to round robin pick a user from a channel? That'd be 5 USD/user/month pls 🤯

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Jun 18

New tech talk:

How to Incrementally Get Results from Your Graph Data with Feldera.

Interested in practical techniques for incremental computation or want to see how this fits into modern data engineering workflows?

Check it out: youtu.be/vpVAZbaZ2Hg

How to Incrementally Get Results from Your Graph Data with Feldera

YouTube video by Feldera, Inc.

youtu.be

1 2

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Jun 11

📢 Webinar - 6/18 at 9am PST!
Stop re-running complex recursive queries when your graph data changes. Feldera incrementally evaluates recursive graph computations. Learn to easily build these mechanisms with #SQL, without the hassle of constant recomputation.
tinyurl.com/rb5my7d8

4 7

Reposted by Gerd Zellweger

Lalith Suresh @lalithsuresh.bsky.social · May 10

🚀 Mihai Budiu will be giving a talk about DBSP, the IVM foundation powering @feldera.bsky.social at the CMU Database Group seminar on Monday, May 12, 2025, @ 12:00 pm EDT.

Zoom link below. 👇

db.cs.cmu.edu/events/dbsp-...

#databs #feldera #ivm

DBSP: Incremental Computation on Streams and Its Applications to Databases - Carnegie Mellon Database Group

We describe DBSP, a framework for incremental computation. Incremental computations repeatedly evaluate... Read More +

db.cs.cmu.edu

1 1 5

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · May 8

Learn how to transform your slow batch jobs into real-time, always-up-to-date insights with our new user guide for converting Spark jobs to incremental Feldera pipelines.
www.feldera.com/blog/warpspeed

1 3 2

Reposted by Gerd Zellweger

Lalith Suresh @lalithsuresh.bsky.social · Apr 15

Where we're at with the war on Rust compilation times.

Cutting Down Rust Compile Times With One Thousand Crates

We'll explore how we can leverage all cores on a machine with rustc using many crates to speed up compilation.

www.feldera.com

1 2 11

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Mar 28

Join us for an in-depth webinar on how Feldera transforms batch analytics with its incremental computation engine.
Register shorturl.at/zQTLu

Wednesday, April 9th at 9am PST, 12pm EST

3 3

Gerd Zellweger @zgerd.bsky.social · Mar 20

thanks for posting it :)

1

Gerd Zellweger @zgerd.bsky.social · Mar 20

with how good the rest of github is, actions could use some love www.feldera.com/blog/the-pai...

The Pain That is Github Actions

Developing a complex CI pipeline with github actions is full of gotchas.

www.feldera.com

1 1 6

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Mar 5

Last month, we announced S3 support for our pipelines.

Today, we're making it real: You can now try it out directly in the online sandbox.

No SQL changes are needed—tweak a bit of JSON in your pipeline settings, and you're set.

www.feldera.com/blog/leverag...

3 5

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Feb 26

@feldera.bsky.social v0.38 release - packed with improvements:

- A new redis output connector, easily and efficiently send data to the popular KV-store.

- Support for input connector orchestration so you can easily manage complex backfill scenarios

Learn more: www.feldera.com/blog/increme...

2 4

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Jan 28

Feldera v0.36 is here:

-📈 runtime & compilation performance for big programs & complex SQL schemas
-Optimized LEFT JOINs for ⬇️memory usage & ⬆️throughput
-Data compression capabilities for ⬆️storage $
-Multiway merging strategy for improved LSM-tree mgmt
www.feldera.com/blog/increme...

Incremental Update 15 at Feldera

Big records and JSON!

www.feldera.com

2 5

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Jan 22

Unlike traditional databases that slow down with data growth, our fully automatic incremental computation maintains consistent performance. Ideal for both batch and real-time data processing.
Learn more about steady performance as your #data grows!
www.feldera.com/blog/batch-p...

#datastreaming

Implementing Batch Processes with Feldera

www.feldera.com

4 3

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Jan 21

It's another incremental update for all you fine folks!
This latest release includes:
Fixed handling of large records in ad-hoc queries for web-console and fda.
Switched from YAML to JSON for internal configuration.
Stay tuned for upcoming features!
Details: www.feldera.com/blog/increme...

Incremental Update 15 at Feldera

Big records and JSON!

www.feldera.com

1 2

Gerd Zellweger @zgerd.bsky.social · Jan 20

I strongly prefer the swiss model (where many don't even know who's the president currently) over this ;)

Reposted by Gerd Zellweger

Ben Pfaff @benpfaff.bsky.social · Jan 15

Have you heard of fine-grained authorization (FGA)? It uses a lot of compute, which makes it expensive. But with Feldera, you can implement it more cheaply using incremental computation. See the full story in the @feldera.bsky.social blog at: www.feldera.com/blog/fine-gr...

Solving Fine-Grained Authorization by Turning the Problem on its Head

Build a high-performance policy engine with only a few lines of SQL.

www.feldera.com

3 6

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Jan 13

We've posted two exciting new use cases to our documentation

🔐 a policy engine for fine-grained authorization

🚀 building live and real-time web applications

We have also improved HTTP performance and sprinkled in some SQL compiler bug fixes.
www.feldera.com/blog/increme...

Incremental Update 14 at Feldera

Updated documentation, faster HTTP ingest and SQL fixes!

www.feldera.com

3 1

Reposted by Gerd Zellweger

Nico Ritschel @nicoritschel.com · Jan 9

What a great demo & presentation. Easily one of the most interesting tech demos I've seen in a half decade.

Feldera Webinar: Incremental Billion Cell Spreadsheets

YouTube video by Feldera, Inc.

www.youtube.com

1 4 13

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Dec 18

Tired of slow batch processing as your data grows? @felderainc 's incremental processing maintains lightning-fast performance no matter your data size. Perfect for batch, real-time, or hybrid processing needs! ⚡️
www.feldera.com/blog/batch-p...

4 7

Reposted by Gerd Zellweger

Lalith Suresh @lalithsuresh.bsky.social · Jan 6

We're doing a webinar this week on building real-time web apps! Join us on the 8th to see @zgerd.bsky.social build a billion-cell spreadsheet app using Feldera in 100 lines of SQL and some Rust (recursive SQL and Rust UDFs ftw!)

Feldera Webinar: Incremental Billion Cell Spreadsheets

Join our Feldera Webinar to learn how to build a real-time application that can scale effortlessly!

www.eventbrite.com

1 2 8

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Dec 12

📢New recursive query support in Feldera's SQL compiler.
Allows for mutually recursive views without needing common table expressions, making it easier to express complex queries.
Unlocking use cases for graph compute, security policy engines, & more
www.feldera.com/blog/recursi...

Recursive SQL queries in Feldera

www.feldera.com

2 2

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Dec 11

🚀 Feldera has released version 0.32!
This update features an optimized compilation cache that cuts down on unnecessary Rust compilations.
The Delta Lake connector now processes large datasets much more efficiently-spend less time sorting.
Read more here: www.feldera.com/blog/increme...

Incremental Update 12 at Feldera

Better compiler cache, better delta-lake adapter and many bugs fixed.

www.feldera.com

3 3

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Dec 2

Feldera v0.31 introduces a powerful new feature: recursive queries in SQL. Recursive queries let users tackle complex problems like hierarchical data traversal & graph algorithms with a straightforward syntax. Also included: UX/design improvements to the WebConsole.
www.feldera.com/blog/increme...

Incremental Update 11 at Feldera

Recursive, incremental SQL and a new UI design! A quick overview of what's new in Feldera v0.31.

www.feldera.com

1 5

Reposted by Gerd Zellweger

Feldera @feldera.bsky.social · Nov 26

A NEW preview of our fault tolerance feature is available now. We allow your pipelines to pick up from where they left off using remote storage, even when a machine crashes, shielding you from any data loss. It's just one more way we ensure operational continuity. www.feldera.com/blog/fault-t...

Fault tolerance now in preview!

Fault tolerance is now in preview for our enterprise offering! As a user, this means you can run Feldera pipelines on network-attached or remote storage, but then crash or completely take out the und...

www.feldera.com

3 3

Reposted by Gerd Zellweger

Lalith Suresh @lalithsuresh.bsky.social · Nov 19

Please welcome @benpfaff.bsky.social! Ben is a legendary engineer, and I've learnt so much from him over the years. He's the founder and long-time lead developer of Open vSwitch and OVN. He was also the founding engineer at Nicira.

bsky.app

1 4 6