#icechunk
For climate predictors and weather analysts: You didn't actually miss the workshop "Replicating official NYC climate risk predictions from ERA5 using Arraylake and Icechunk"- we moved it to this Thursday, October 30! Join us: m.earthmover.io/replicatingo...
October 27, 2025 at 6:41 PM
For climate predictors and weather analysts with #climateweekNYC FOMO: We will reprise our hands-on workshop, “Replicating official NYC climate risk predictions from ERA5 using Arraylake and Icechunk," online this Thursday, October 23 at 9AM EST: join us! m.earthmover.io/replicatingo...
October 20, 2025 at 6:31 PM
Icechunk: Революция в хранении транзакционных тензоров для облачных решений

https://kripta.biz/posts/7AF01EA3-6110-4936-8D4C-31857A6FDE9F
October 20, 2025 at 10:55 AM
Icechunk:面向未来的事务型张量存储引擎解析

https://qian.cx/posts/23E59D45-55EC-4B4F-9970-58C525CAD88A
October 20, 2025 at 10:55 AM
It won't be long until Icechunk is used to virtualize Parquet stores, referencing compressed chunks by byte ranges, and stored in their mysterious Rust Zarrtabase
October 9, 2025 at 1:33 PM
Zarr is great for big multidimensional data—but not for collaboration. Our latest post by Lindsey Nield shows how Icechunk adds Git-like transactions + version control, so teams can safely share and scale Zarr workflows.
earthmover.io/blog/multi-p...
Multi-Player Mode: Why Teams That Use Zarr Need Icechunk - Earthmover
Icechunk brings Git-like version control to Zarr. Ensure atomic updates, consistent reads, and scalable collaboration for shared datasets.
earthmover.io
August 25, 2025 at 10:45 PM
Impressive overview of how VirtualiZarr and Icechunk work together when handling large numbers of files e.g. NetCDF youtu.be/rD3eeK56tJY?... @tegnicholas.bsky.social
Tom Nicholas - How to build a cloud-optimised datacube of archival files in 3 lines of xarray
YouTube video by SciPy
youtu.be
August 8, 2025 at 10:30 AM
August updates: How to access 50 years of ERA5 data with Earthmover, the Icechunk 1.0 release, and an invitation to our August webinar: How To Turn Weather Radar Archives into Usable Science with Xarray DataTree and Zarr
August Newsletter
🌏 50 Years of ERA5 Access in Earthmover: The Arraylake / Icechunk edition is an analysis-ready dataset that retains only the single-level surface fields—18 hourly variables such as 2 m temperature, 10...
www.linkedin.com
August 6, 2025 at 9:55 PM
Not sure , a bigger problem is the new icechunk variant that you can't use without the company's Rust lib
July 29, 2025 at 10:11 AM
Super excited to see #icechunk v1.0 ship today. Stable format, stable API, and ready for production. Take it for a spin and let us know how it goes. 🚀🚀🚀
Today at SciPy 2025 we released Icechunk 1.0, an open source package and specification that enables database-style transactions against petabyte-scale array datasets using only cloud object storage as infrastructure. Read about it on our blog earthmover.io/blog/icechun..., or visit earthmover.io
Icechunk 1.0: Production-Grade Cloud-Native Array Storage Is Here - Earthmover
A year ago, we made an important internal decision which set Earthmover on a new course—we decided to refactor and open source our core technology for storing array-based data in the cloud. This took ...
earthmover.io
July 10, 2025 at 7:15 PM
Today at SciPy 2025 we released Icechunk 1.0, an open source package and specification that enables database-style transactions against petabyte-scale array datasets using only cloud object storage as infrastructure. Read about it on our blog earthmover.io/blog/icechun..., or visit earthmover.io
Icechunk 1.0: Production-Grade Cloud-Native Array Storage Is Here - Earthmover
A year ago, we made an important internal decision which set Earthmover on a new course—we decided to refactor and open source our core technology for storing array-based data in the cloud. This took ...
earthmover.io
July 10, 2025 at 7:06 PM
Struggling with huge geospatial datasets in the cloud? @earthmover.io's Tom Nicolas #CNG2025 talk on VirtualiZarr + IceChunk will show you how to optimize data for cloud-native environments in just 3 lines of Python code. No more copying data
▶️ youtu.be/QBkZQ53vE6o
[CNG 2025] VirtualiZarr + Icechunk: Build Cloud-Optimised DataCube of Archival Files – Tom Nicholas
YouTube video by Cloud-Native Geospatial Forum
youtu.be
June 25, 2025 at 3:17 PM
Icechunk?
June 12, 2025 at 1:04 AM
Maybe icechunk zarr is for you.
June 12, 2025 at 12:37 AM
Daily json? Haha sigh
What about that next gen zarr icechunk thing?
June 11, 2025 at 11:51 PM
New #CNG2025 talk alert!
Lindsey Nield & @deepakcherian.bsky.social from @earthmover.io present on Zarr, Icechunk & Xarray for cloud-native #geospatial data cube analysis.
Watch now: youtu.be/DSxmCWfobqQ
[CNG 2025] Zarr, Icechunk & Xarray for Cloud-Native Geospatial Data-cube Analysis – Nield & Cherian
YouTube video by Cloud-Native Geospatial Forum
youtu.be
June 5, 2025 at 7:04 PM
I also wonder about Zarr in relation to TileDB, which seems to already have stuff like IceChunk built in and the user expectation is you write to a consistent open source API in any language instead of the Python-first Zarr ecosystem. We built silvimetric.com on TileDB for this reason.
SilviMetric — SilviMetric
silvimetric.com
June 3, 2025 at 12:04 PM
𝐻𝑜𝑤 𝑑𝑜𝑒𝑠 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝑎𝑣𝑜𝑖𝑑 𝑟𝑒𝑑𝑢𝑛𝑑𝑎𝑛𝑡 𝑠𝑡𝑜𝑟𝑎𝑔𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑑𝑎𝑡𝑎 𝑣𝑒𝑟𝑠𝑖𝑜𝑛𝑠?

Icechunk stores only new or changed chunks for each version —no redundant copies or rewrites. You get instant time travel, branching, and efficient updates, all with negligible storage overhead.

More: bit.ly/3F1XFST
Icechunk: Efficient storage of versioned array data - Earthmover
We recently got an interesting question in Icechunk’s community Slack channel (thank you Iury Simoes-Sousa for motivating this post): I’m new to Icechunk. How is the storage managed for redundant info...
earthmover.io
May 14, 2025 at 4:09 PM
Some Postdoc throwback today riding the NJtransit to Princeton.

On my way to the #WCRP km scale hackathon for the week.

www.wcrp-esmo.org/activities/w...

Excited to play around with #healpix #zarr #icechunk and some super high res data.
May 12, 2025 at 3:52 PM
I'll be at the CNG conference in Snowbird next week. I wrote a short blog post about what the Earthmover team will be up to.

tldr; we'll be talking about @zarr.dev, #icechunk, @xarray.bsky.social and cloud-native data cubes.

Details in the blog post 👇
📣 Our team is heading to the
@cloudnativegeo.org
Conference in Snowbird, Utah. We’ll be sharing our latest open-source work on Xarray, Zarr, VirtualiZarr, Icechunk, and leading a hands-on workshop, 𝑍𝑎𝑟𝑟, 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘, & 𝑋𝑎𝑟𝑟𝑎𝑦 𝑓𝑜𝑟 𝐶𝑙𝑜𝑢𝑑-𝑛𝑎𝑡𝑖𝑣𝑒 𝐺𝑒𝑜𝑠𝑝𝑎𝑡𝑖𝑎𝑙 𝐷𝑎𝑡𝑎-𝑐𝑢𝑏𝑒 𝐴𝑛𝑎𝑙𝑦𝑠𝑖𝑠.

Read more: bit.ly/4lRWNAJ
April 28, 2025 at 8:19 PM
📣 Our team is heading to the
@cloudnativegeo.org
Conference in Snowbird, Utah. We’ll be sharing our latest open-source work on Xarray, Zarr, VirtualiZarr, Icechunk, and leading a hands-on workshop, 𝑍𝑎𝑟𝑟, 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘, & 𝑋𝑎𝑟𝑟𝑎𝑦 𝑓𝑜𝑟 𝐶𝑙𝑜𝑢𝑑-𝑛𝑎𝑡𝑖𝑣𝑒 𝐺𝑒𝑜𝑠𝑝𝑎𝑡𝑖𝑎𝑙 𝐷𝑎𝑡𝑎-𝑐𝑢𝑏𝑒 𝐴𝑛𝑎𝑙𝑦𝑠𝑖𝑠.

Read more: bit.ly/4lRWNAJ
April 28, 2025 at 7:33 PM
Most people think of @zarr.dev as a "file format". With #Icechunk, we've turned Zarr into a database. @functionth.bsky.social's post shows how Icechunk can be used to solve a problem where transactional databases are often required.
1/ 🚨 New Blog Post Alert: "𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝐴𝑏𝑜𝑢𝑡 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝐶𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑎 𝐶𝑙𝑖𝑐ℎ𝑒́𝑑 𝑏𝑢𝑡 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑣𝑒 𝐸𝑥𝑎𝑚𝑝𝑙𝑒" 🏦🔁

👉 Read it here: earthmover.io/blog/learnin...
Learning about Icechunk consistency with a clichéd but instructive example - Earthmover
In this post we’ll show what can happen when more than one process write to the same Icechunk repository concurrently, and how Icechunk uses transactions and conflict resolution to guarantee consisten...
earthmover.io
April 24, 2025 at 6:40 PM
Had the idea of using Icechunk (an multi-dimensional array database) for something I would never use Icechunk for
1/ 🚨 New Blog Post Alert: "𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝐴𝑏𝑜𝑢𝑡 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝐶𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑎 𝐶𝑙𝑖𝑐ℎ𝑒́𝑑 𝑏𝑢𝑡 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑣𝑒 𝐸𝑥𝑎𝑚𝑝𝑙𝑒" 🏦🔁

👉 Read it here: earthmover.io/blog/learnin...
Learning about Icechunk consistency with a clichéd but instructive example - Earthmover
In this post we’ll show what can happen when more than one process write to the same Icechunk repository concurrently, and how Icechunk uses transactions and conflict resolution to guarantee consisten...
earthmover.io
April 23, 2025 at 3:37 PM
4/ You’ll learn:
• How concurrent processes can cause consistency issues
• Why Icechunk transactions are 𝐚𝐭𝐨𝐦𝐢𝐜 𝐚𝐧𝐝 𝐢𝐬𝐨𝐥𝐚𝐭𝐞𝐝 𝐛𝐲 𝐝𝐞𝐟𝐚𝐮𝐥𝐭
• How 𝐫𝐞𝐛𝐚𝐬𝐢𝐧𝐠 𝐬𝐞𝐬𝐬𝐢𝐨𝐧𝐬 unlock faster, conflict-aware concurrency
• The role of 𝐌𝐕𝐂𝐂 𝐚𝐧𝐝 𝐠𝐚𝐫𝐛𝐚𝐠𝐞 𝐜𝐨𝐥𝐥𝐞𝐜𝐭𝐢𝐨𝐧 in Icechunk’s design
April 23, 2025 at 3:30 PM
3/ We use bank account transfers to explore how 𝐈𝐜𝐞𝐜𝐡𝐮𝐧𝐤 ensures 𝐭𝐫𝐚𝐧𝐬𝐚𝐜𝐭𝐢𝐨𝐧𝐚𝐥 𝐜𝐨𝐧𝐬𝐢𝐬𝐭𝐞𝐧𝐜𝐲 in concurrent environments. This isn’t about banking; it’s about understanding how Icechunk handles conflicting writes, retries, rebases, and optimistic concurrency control for @zarr.dev data.
April 23, 2025 at 3:30 PM