Lightnews — Scholar-powered news

Reposted by Ryan Abernathey

Earthmover @earthmover.io · 29d

Climate Week NYC is coming fast, and there's a slew of events that CEO @rabernat.bsky.social willl be at: register now for events @brightbangtan.bsky.social and www.geospatialrisk.com are organizing. See m.earthmover.io/events for the list, including a hands-on workshop and the relaunch of GeoNYC!

www.geospatialrisk.com

2 1

Reposted by Ryan Abernathey

Earthmover @earthmover.io · 20d

#ClimateWeekNYC is almost upon us, as is the "What is the future of AI Weather Forecasting?" panel with @rabernat.bsky.social, @juliangsf.bsky.social, @mikepritchard.bsky.social, Google, Sorcerer and NVIDIA on Tuesday, September 23. Come join us! luma.com/svimvi2y?tk=...

What is the Future of AI Weather Forecasting? Climate Week NYC · Luma

A panel of AI Weather experts from Brightband, Earthmover, Google, NVIDIA, and Sorcerer will discuss the future of AI weather forecasting, and answer questions…

luma.com

2 4

Reposted by Ryan Abernathey

Earthmover @earthmover.io · May 20

Our latest fundamentals blog post provides an overview of @zarr.dev and its open-source ecosystem. Read more: earthmover.io/blog/what-is...

Fundamentals: What Is Zarr? A Cloud-Native Format for Tensor Data - Earthmover

What Zarr is, and how it enables fast, scalable access to multidimensional array data in the cloud.

earthmover.io

3 10

Reposted by Ryan Abernathey

Earthmover @earthmover.io · May 7

1/ Our team just got back from an inspiring week at the @cloudnativegeo.org conference in Snowbird, Utah, and one thing is clear: 𝐙𝐚𝐫𝐫 𝐚𝐝𝐨𝐩𝐭𝐢𝐨𝐧 𝐢𝐬 𝐬𝐮𝐫𝐠𝐢𝐧𝐠 𝐚𝐜𝐫𝐨𝐬𝐬 𝐠𝐞𝐨𝐬𝐩𝐚𝐭𝐢𝐚𝐥.

1 3 8

Reposted by Ryan Abernathey

Earthmover @earthmover.io · Apr 28

📣 Our team is heading to the
@cloudnativegeo.org
Conference in Snowbird, Utah. We’ll be sharing our latest open-source work on Xarray, Zarr, VirtualiZarr, Icechunk, and leading a hands-on workshop, 𝑍𝑎𝑟𝑟, 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘, & 𝑋𝑎𝑟𝑟𝑎𝑦 𝑓𝑜𝑟 𝐶𝑙𝑜𝑢𝑑-𝑛𝑎𝑡𝑖𝑣𝑒 𝐺𝑒𝑜𝑠𝑝𝑎𝑡𝑖𝑎𝑙 𝐷𝑎𝑡𝑎-𝑐𝑢𝑏𝑒 𝐴𝑛𝑎𝑙𝑦𝑠𝑖𝑠.

Read more: bit.ly/4lRWNAJ

1 7

Reposted by Ryan Abernathey

Earthmover @earthmover.io · Apr 30

👋 Stop by our booth at @cloudnativegeo.org to say hi to our team, see a demo of our platform, and pick up some stickers! #CNG2025

1 8

Reposted by Ryan Abernathey

Prof. Tatiana Ilyina @tatianailyina.bsky.social · Apr 29

The great @jacbeh.bsky.social interviewed me for the blog of the @egu.eu Ocean Sciences (OS) Division. It was fun to talk about my career path, research that has led to receiving the Fridtjof Nansen Medal, as well as challenges faced in academia
blogs.egu.eu/divisions/os...
🧪 🌊

Tatiana Ilyina Receives the Fridtjof Nansen Medal – About Her Career and Work

Tatiana Ilyina received the Fridtjof Nansen Medal from the European Geosciences Union (EGU) for her outstanding research in oceanography. We spoke with her about her career path and the work that led ...

blogs.egu.eu

2 10 40

Ryan Abernathey @rabernat.bsky.social · Apr 23

💡 Another banger technical blog post by @functionth.bsky.social! This one explains Icechunk's transaction system via the example of bank account transfers. Having true ACID transactions for the array data model is such a game changer!

Earthmover @earthmover.io · Apr 23

1/ 🚨 New Blog Post Alert: "𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝐴𝑏𝑜𝑢𝑡 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝐶𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑎 𝐶𝑙𝑖𝑐ℎ𝑒́𝑑 𝑏𝑢𝑡 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑣𝑒 𝐸𝑥𝑎𝑚𝑝𝑙𝑒" 🏦🔁

👉 Read it here: earthmover.io/blog/learnin...

Learning about Icechunk consistency with a clichéd but instructive example - Earthmover

In this post we’ll show what can happen when more than one process write to the same Icechunk repository concurrently, and how Icechunk uses transactions and conflict resolution to guarantee consisten...

earthmover.io

1

Reposted by Ryan Abernathey

Kyle Barron @kylebarron.dev · Apr 21

With today's Zarr Python release, switch to the new obstore-based backend for **3x faster** reading from S3, Google Cloud Storage, and Azure than the default fsspec backend.

zarr.readthedocs.io/en/stable/us...
developmentseed.org/obstore/late...

1 9 34

Reposted by Ryan Abernathey

Dr. Dawn Wright @deepseadawn.bsky.social · Apr 17

"Fundamentals: What is Cloud-Optimized Scientific Data?
Why naively lifting scientific data to the cloud falls flat."
earthmover.io/blog/fundame...
Thank you @tegnicholas.bsky.social

Zarr cartoon about how having to download all the data before looking at any of it is an antiquated paradigm.

1 6 18

Reposted by Ryan Abernathey

Earthmover @earthmover.io · Apr 17

1/ 💡 Our latest blog post in the fundamentals series, written by @tegnicholas.bsky.social, demystifies cloud-optimized scientific data formats!

Read more: earthmover.io/blog/fundame...

Fundamentals: What is Cloud-Optimized Scientific Data?

What cloud-optimized data really means, and how Zarr and Icechunk enable fast access to massive scientific datasets in cloud object storage.

earthmover.io

2 9 16

Ryan Abernathey @rabernat.bsky.social · Apr 16

Imagine if all you had to do to serve geopspatial data over OGC APIs was to ingest @zarr.dev datasets into cloud object storage...and the platform handled the rest. Scaling, logging, metrics, auth, etc. That's Flux! Join our webinar if you want to learn more.

Earthmover @earthmover.io · Apr 16

1/ Today we are launching Flux, a powerful new addition to the Earthmover platform. Flux is a high-performance gateway for exploring, querying, and visualizing geospatial data via standards-compliant APIs (EDR, WMS, and OPeNDAP).

2 7

Ryan Abernathey @rabernat.bsky.social · Apr 9

I have to admit that I too was confused about how S3 prefixes and scalability work before reading this post by @functionth.bsky.social.

Earthmover @earthmover.io · Apr 9

📣 Blog post alert! 𝐄𝐱𝐩𝐥𝐨𝐫𝐢𝐧𝐠 𝐈𝐜𝐞𝐜𝐡𝐮𝐧𝐤 𝐬𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲: 𝐮𝐧𝐭𝐚𝐧𝐠𝐥𝐢𝐧𝐠 𝐒𝟑'𝐬 𝐩𝐫𝐞𝐟𝐢𝐱 𝐬𝐭𝐨𝐫𝐲. This technical post by @functionth.bsky.social dives deep into the internals of how S3 shards data, showing that distributed Icechunk can easily perform 230,000 object reads/sec and beyond. earthmover.io/blog/explori...

Exploring Icechunk scalability: untangling S3's prefix story | Earthmover

We show Icechunk can scale to extremely high concurrency levels, and explain how it achieves this in modern object stores.

earthmover.io

2 4

Ryan Abernathey @rabernat.bsky.social · Apr 3

This one was a lot of fun to work on! Why @xarray.bsky.social and @zarr.dev are faster than DuckDB and Parquet for gridded data...from first principles.

Earthmover @earthmover.io · Apr 3

⭐ We just released the first post in our Fundamentals series. This one is called 𝐓𝐞𝐧𝐬𝐨𝐫𝐬 𝐯𝐬. 𝐓𝐚𝐛𝐥𝐞𝐬 - 𝐖𝐡𝐲 𝐭𝐚𝐛𝐮𝐥𝐚𝐫 𝐭𝐨𝐨𝐥𝐬 𝐭𝐫𝐢𝐩 𝐨𝐯𝐞𝐫 𝐠𝐫𝐢𝐝𝐝𝐞𝐝 𝐝𝐚𝐭𝐚. earthmover.io/blog/tensors...

Fundamentals: Tensors vs. Tables | Earthmover

Why tabular tools trip over gridded data.

earthmover.io

3 8

Ryan Abernathey @rabernat.bsky.social · Mar 28

I am so excited to share this new project with NASA and @developmentseed.org . By applying Icechunk and the "virtualization" approach, we can make Petabytes of archival data accessible to high performance analytics. Unlocking NASA's cloud archives will have immense downstream benefits for the world.

Earthmover @earthmover.io · Mar 28

1/ 🚀 Solving #NASA ’s cloud data dilemma: Icechunk unlocks 100x faster access to archival data formats

We're thrilled to publish results from our pilot project with NASA and @developmentseed.org to enable high-performance cloud-native access for NASA’s 100s of petabytes of Earth observation data.

1 12 34

Reposted by Ryan Abernathey

Earthmover @earthmover.io · Mar 18

1/ As a data platform company, we get a unique view of how other companies use data. In light of the recent cuts at @noaa.gov, our CEO, @rabernat.bsky.social, reached out to our customers to get their perspective on the value of NOAA’s data and services.

1 4 10

Reposted by Ryan Abernathey

kosmi.bsky.social @kosmi.bsky.social · Mar 16

An icy swirl at the coast of Antarctica.

Image taken by Sentinel-2 on March 15, 2025.

Lat, lng:

-65.86, 54.96

8 41

Reposted by Ryan Abernathey

Joe Hamman @jhamman.bsky.social · Feb 24

Here on Bluesky, I follow two topics: 1) software & data infrastructure and 2) cycling. I mostly post about software but today, I can share a bit about a fun cycling adventure of my own 🚵‍♂️.

Last week, I got to ride in the 24 Hours in the Old Pueblo - placing 4th in the Open Men Duo category:

2 1 14

Reposted by Ryan Abernathey

TEGNicholas.bsky.social @tegnicholas.bsky.social · Feb 27

We now have a public slack channel for VirtualiZarr development and questions, feel free to join!

earthmover-community.slack.com/archives/C08...

Slack

earthmover-community.slack.com

4 5

Reposted by Ryan Abernathey

Earthmover @earthmover.io · Feb 27

Denver folks — join us for a Geospatial Happy Hour on Wednesday, March 12th at 7 PM MST. Come enjoy drinks, tacos, discussing geospatial data, and meeting the Earthmover team. Register: lu.ma/o6r39p6j

Denver Geospatial Happy Hour · Luma

Come have a drink, grab a bite, and nerd out with the Earthmover team on geospatial data. Earthmover will provide food and drink tickets. Our event will be…

lu.ma

3 5

Ryan Abernathey @rabernat.bsky.social · Feb 26

One reason we started @earthmover.io was to create a home for amazing open source devs building towards a shared vision for a cloud-native future. So I couldn't be more excited to have @tegnicholas.bsky.social joining the team! 🚀 Tom's capabilities as an engineer are unparalleled!!!

Earthmover @earthmover.io · Feb 26

We’re thrilled to welcome @tegnicholas.bsky.social to the team! Tom is a prolific contributor to @xarray.bsky.social and @pangeo.io. He created Xarray Datatree and VirtualiZarr. Tom is working as a Forward Engineer, supporting our customers and the broader open-source community.

1 1 14

Ryan Abernathey @rabernat.bsky.social · Feb 20

Have you ever waited a long time for @xarray.bsky.social to open up a large @zarr.dev dataset in cloud storage? Upgrade to Zarr 3 for a massive performance improvement! 🚀

Earthmover @earthmover.io · Feb 20

1/ Check out our latest blog post earthmover.io/blog/xarray-... to learn about the dramatic improvement and performance of Xarray’s Zarr backend. We achieved improved the “time to first byte” metric, building on Zarr-Python’s new asyncio internals.

Accelerating Xarray with Zarr-Python 3 | Earthmover

We have recently dramatically improved the performance of Xarray’s Zarr backend. This post explores how we’ve improved the “time to first byte” metric, building on Zarr-Python’s new asyncio internals.

earthmover.io

6 9

Reposted by Ryan Abernathey

Earthmover @earthmover.io · Feb 17

Register for tomorrow’s webinar, Data Version Control for Arrays with Icechunk. bit.ly/3WVPSf2 @rabernat.bsky.social will discuss Icechunk’s application in AI/ML model development, how it adds safety + flexibility to data pipelines, & how it enables git-style version control to Zarr Arrays + Groups

1 2

Ryan Abernathey @rabernat.bsky.social · Feb 14

Next week I'll be giving a webinar on one of my favorite parts of Icechunk--the ability to apply git-style version control to Zarr arrays and Groups. This capability makes data workflows so more safe, robust, and reproducible. Come check it out! (Registration link below.)

Earthmover @earthmover.io · Feb 11

Join our webinar, Data Version Control for Arrays with #Icechunk. @rabernat.bsky.social will explain how Icechunk’s transactions, snapshots, tags, + branches can add safety & flexibility to data pipelines and workflows. Register here: bit.ly/3WVPSf2

3 6

Reposted by Ryan Abernathey

Earthmover @earthmover.io · Feb 11

Join our webinar, Data Version Control for Arrays with #Icechunk. @rabernat.bsky.social will explain how Icechunk’s transactions, snapshots, tags, + branches can add safety & flexibility to data pipelines and workflows. Register here: bit.ly/3WVPSf2

1 7