Lightnews — Scholar-powered news

Reposted by Zarr

scverse @scverse.bsky.social · Aug 11

Anndata now supports @zarr.dev v3, great for both performance and interoperability. Check out our guide for more info, especially sharding: anndata.readthedocs.io/en/stable/tu.... Don’t worry though, we remain fully v2 backwards compatible - upgrade fearlessly!

1 2 6

Reposted by Zarr

ESIP - Earth Science Information Partners @esipfed.bsky.social · Jul 7

♻️ 7/8 - Webinar: Reusable Deep Learning with EOPF Zarr Samples - ESA Copernicus Earth Observation Processor Framework

Learn how to reuse a deep learning model originally trained with data from the openEO API of the Copernicus Dataspace Ecosystem (CDSE).

#datascience #deeplearning

Welcome! You are invited to join a webinar: Reusable Deep Learning with EOPF Zarr Samples. After registering, you will receive a confirmation email about joining the webinar.

The aim of this webinar is to demonstrate how to reuse a deep learning model originally trained with data from the openEO API of the Copernicus Dataspace Ecosystem (CDSE) with the data and tools provi...

us02web.zoom.us

1 1 2

Reposted by Zarr

Al Merose (he/him) @al.merose.com · Jun 27

I've been waiting a few years for this: Zarr v2 support was just launched in Google Earth Engine! Check out this demo that loads ARCO-ERA5 into GEE 🔥

code.earthengine.google.com/67ac1e48848d...
developers.google.com/earth-engine...

The Google Earth Engine code editor displaying an evaporation map loaded directly from ARCO-ERA5! The black and white map overlooks the continental US. It's easy to make out land masses, but there are also cloud-looking visuals on the map.

1 4 16

Reposted by Zarr

Lazaro Alonso @lazarusa.bsky.social · Jun 24

If you missed today's presentation at the LPS by @thejeran.bsky.social on Browzarr, a browser-based visualization toolkit for exploring and analyzing Zarr data stores, go and check out the version online, and open issues and do feature requests on GitHub. github.com/EarthyScienc...

1 3

Reposted by Zarr

Petra Korlević @petrathepostdoc.bsky.social · Jun 18

doodles of two Tree of Life department speakers @sangerinstitute.bsky.social this week, Evgeny Zakharov talking about insanely large scale barcode sequencing and Jerome Kelleher talking about large scale human genomic data analysis.

that's a lot of data 🧬📄

comic doodle of Evgeny Zakharov in a suit and stripey shirt, with a small plane to the left, while DNA strands and bases are above, a camera is taking a photo of a feather and a wasp, the coloring is grey shades on a red background.

comic doodle of Jerome Kelleher in a t-shirt and stripey shirt, the background is full of small peon people with DNA strands in them and 4 files with the word ZARR written on them, the coloring is grey shades on a green background.

1 4

Reposted by Zarr

Open Sustainable Technology @opensustaintech.bsky.social · Jun 9

We welcome a new project to OpenSustain.tech #climate #sustainability #opensource #opensustain #openscience github.com/zarr-develop...

GitHub - zarr-developers/zarr-python: An implementation of chunked, compressed, N-dimensional arrays for Python.

An implementation of chunked, compressed, N-dimensional arrays for Python. - zarr-developers/zarr-python

github.com

2 10

Zarr @zarr.dev · Jun 11

ICYMI: Check out this alternative storage approach for variation data using #Zarr. Rather than grouping all data for a given variant together, its groups all data for a given field, and store as chunked, compressed N-D arrays (tensors). 🤩

doi.org/10.1093/giga...

Analysis-ready VCF at Biobank scale using Zarr

AbstractBackground. Variant Call Format (VCF) is the standard file format for interchanging genetic variation data and associated quality control metrics.

doi.org

7

Reposted by Zarr

Rita Strack @ritastrack.bsky.social · May 13

OME-Zarr 4 life

1 3 9

Reposted by Zarr

Kit Macleod @kitmacleod.bsky.social · May 19

"The commitment from major entities like Copernicus, USGS, and Google Earth Engine signals a paradigm shift in how we will store, access, and analyze geospatial data in the cloud... #Zarr is poised to become a cornerstone of the cloud-native geospatial ecosystem, unlocking new frontiers..."

Earthmover @earthmover.io · May 7

4/ The Earthmover team is proud to play a leading role in the Zarr project as this community grows.

Read about our takeaways from CNG here: earthmover.io/blog/zarr-ta...

Zarr takes Cloud-Native Geospatial by storm - Earthmover

Our takeaways from the Cloud-Native Geospatial conference on Zarr’s surging adoption and its impact on the future of Earth Observation data. Our team just returned from an action-packed week at the Cl...

earthmover.io

2 3

Reposted by Zarr

Earthmover @earthmover.io · May 7

1/ Our team just got back from an inspiring week at the @cloudnativegeo.org conference in Snowbird, Utah, and one thing is clear: 𝐙𝐚𝐫𝐫 𝐚𝐝𝐨𝐩𝐭𝐢𝐨𝐧 𝐢𝐬 𝐬𝐮𝐫𝐠𝐢𝐧𝐠 𝐚𝐜𝐫𝐨𝐬𝐬 𝐠𝐞𝐨𝐬𝐩𝐚𝐭𝐢𝐚𝐥.

1 3 8

Reposted by Zarr

Copernicus ECMWF @copernicusecmwf.bsky.social · Mar 18

Curious about climate patterns? The new ERA Explorer app lets you compare 85 years of climate data worldwide. Analyse climatological averages at any map point or explore month-by-month data.Dive deeper with ERA Explorer! era-explorer.climate.copernicus.eu?lat=50.86&ln...
#C3S #beautifuldata

3 46 85

Zarr @zarr.dev · May 19

@zarr.dev even 🤩 Love it #beautifuldata

1

Zarr @zarr.dev · May 16

🧠 Interested in extending #Zarr?

Let us know what you think about ZEP10 "Generic extensions". Current plan is for voting on the proposal mid-June. 📅

github.com/zarr-develop...

ZEP10: Generic extensions (v3.2 spec changes) by joshmoore · Pull Request #344 · zarr-developers/zarr-specs

This is the implementation of ZEP10 which introduces a generic extensions object. ☝🏽 👉🏽 Rendered ZEP10 page: https://zeps--67.org.readthedocs.build/en/67/draft/ZEP0010.html 🗓️ Proposed...

github.com

2 4

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · May 7

Just back from #CNG2025, I wrote up some thoughts on the traction the @zarr.dev format is gaining in the broader geospatial industry (specifically Earth Observation). I also summarize what features and tools the user and developer communities are asking for to fully leverage Zarr. 👇👇👇

Earthmover @earthmover.io · May 7

1/ Our team just got back from an inspiring week at the @cloudnativegeo.org conference in Snowbird, Utah, and one thing is clear: 𝐙𝐚𝐫𝐫 𝐚𝐝𝐨𝐩𝐭𝐢𝐨𝐧 𝐢𝐬 𝐬𝐮𝐫𝐠𝐢𝐧𝐠 𝐚𝐜𝐫𝐨𝐬𝐬 𝐠𝐞𝐨𝐬𝐩𝐚𝐭𝐢𝐚𝐥.

2 3

Reposted by Zarr

Earthmover @earthmover.io · Apr 17

1/ 💡 Our latest blog post in the fundamentals series, written by @tegnicholas.bsky.social, demystifies cloud-optimized scientific data formats!

Read more: earthmover.io/blog/fundame...

Fundamentals: What is Cloud-Optimized Scientific Data?

What cloud-optimized data really means, and how Zarr and Icechunk enable fast access to massive scientific datasets in cloud object storage.

earthmover.io

2 9 16

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · Apr 16

🚀 Big launch today: introducing Flux — a game-changing way to serve geospatial data via standard APIs in seconds. Built for scale, speed, and simplicity.

Earthmover @earthmover.io · Apr 16

1/ Today we are launching Flux, a powerful new addition to the Earthmover platform. Flux is a high-performance gateway for exploring, querying, and visualizing geospatial data via standards-compliant APIs (EDR, WMS, and OPeNDAP).

1 3 5

Reposted by Zarr

Ryan Abernathey @rabernat.bsky.social · Apr 3

This one was a lot of fun to work on! Why @xarray.bsky.social and @zarr.dev are faster than DuckDB and Parquet for gridded data...from first principles.

Earthmover @earthmover.io · Apr 3

⭐ We just released the first post in our Fundamentals series. This one is called 𝐓𝐞𝐧𝐬𝐨𝐫𝐬 𝐯𝐬. 𝐓𝐚𝐛𝐥𝐞𝐬 - 𝐖𝐡𝐲 𝐭𝐚𝐛𝐮𝐥𝐚𝐫 𝐭𝐨𝐨𝐥𝐬 𝐭𝐫𝐢𝐩 𝐨𝐯𝐞𝐫 𝐠𝐫𝐢𝐝𝐝𝐞𝐝 𝐝𝐚𝐭𝐚. earthmover.io/blog/tensors...

Fundamentals: Tensors vs. Tables | Earthmover

Why tabular tools trip over gridded data.

earthmover.io

3 8

Reposted by Zarr

Copernicus ECMWF @copernicusecmwf.bsky.social · Mar 20

What's behind ERA Explorer's speed? Meet the ZARR archive! 🚀 ZARR revolutionises data processing, enabling fast, efficient access to over 80 years of climate data.
Test out its speed 👉 era-explorer.climate.copernicus.eu?lat=50.86&ln...

#C3S @zarr.dev #beautifuldata

1 9 19

Reposted by Zarr

Jackson Burns @fastprop.bsky.social · Mar 24

Just started using Zarr - literally unbelievably good.

Zarr @zarr.dev · Nov 27

Your friendly neighborhood #chunked, #compressed, & #cloud-ready #n-dimensional array format is now verified as @zarr.dev on @bsky.app!

Visit us at zarr.dev for more info!

Zarr

Have beautiful data in Zarr? Show us on Bluesky!

zarr.dev

2 2

Zarr @zarr.dev · Mar 29

🤩 " #NASA ... leveraged #Icechunk’s virtualization capabilities to present the entire collection of over a #million files as a single analysis-ready #Zarr data cube. "

Joe Hamman @jhamman.bsky.social · Mar 28

🚨 New blog post 🚨

In it, we show off our recent work deploying #icechunk on top of #NASA's existing archives of Earth observation data. The results: 100x speed up when extracting time series from existing datasets stored as netCDF.

Earthmover @earthmover.io · Mar 28

1/ 🚀 Solving #NASA ’s cloud data dilemma: Icechunk unlocks 100x faster access to archival data formats

We're thrilled to publish results from our pilot project with NASA and @developmentseed.org to enable high-performance cloud-native access for NASA’s 100s of petabytes of Earth observation data.

1 6

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · Mar 28

🚨 New blog post 🚨

In it, we show off our recent work deploying #icechunk on top of #NASA's existing archives of Earth observation data. The results: 100x speed up when extracting time series from existing datasets stored as netCDF.

Earthmover @earthmover.io · Mar 28

1/ 🚀 Solving #NASA ’s cloud data dilemma: Icechunk unlocks 100x faster access to archival data formats

We're thrilled to publish results from our pilot project with NASA and @developmentseed.org to enable high-performance cloud-native access for NASA’s 100s of petabytes of Earth observation data.

4 12

Reposted by Zarr

Josh Moore @joshmoore.bsky.social · Mar 18

ICYMI: A MATLAB implementation of @zarr.dev has dropped from @mathworks.bsky.social 🚀

github.com/mathworks/MA...

GitHub - mathworks/MATLAB-support-for-Zarr-files: This contribution provides an interface in MATLAB to read and write Zarr files. Zarr is a file format for scientific data (like HDF5 and netCDF) which...

This contribution provides an interface in MATLAB to read and write Zarr files. Zarr is a file format for scientific data (like HDF5 and netCDF) which is optimized for cloud storage and parallel co...

github.com

1 6 21

Reposted by Zarr

Al Merose (he/him) @al.merose.com · Mar 7

Really cool new neuroscience benchmark just dropped.

Data is in @zarr.dev and demos use tensorstore.

zapbench-release.storage.googleapis.com/landing.html

ZAPBench

ZAPBench evaluates how well different models can predict the activity of over 70,000 neurons in a novel larval zebrafish dataset.

zapbench-release.storage.googleapis.com

3 6

Reposted by Zarr

Michael Morehead @michael-syglass.bsky.social · Jan 14

Very happy to announce our winter release of syGlass: 2.2.0!
Now control syGlass with your voice. We've also upgraded the auto-segmentation, mask handling & proofreading, and the narration system. We've added basic support for @zarr.dev. Check out the changelog: www.syglass.io/blog/announc...

7 14

Reposted by Zarr

Chris Holmes @opencholmes.bsky.social · Feb 10

You, yes you, can help the Cloud Native Geospatial Movement!

Have you benefitted from Cloud-Optimized GeoTIFF’s? SpatioTemporal Asset Catalogs? Zarr, COPC or GeoParquet? And the ecosystem around them?

Learn how to give back in my post: cholmes.medium.com/you-yes-you-...

You, yes you, can help the Cloud Native Geospatial Movement!

Have you benefitted from COG’s, STAC, Zarr, COPC or GeoParquet? Join us in Utah for the first CNG Conference and help just by attending.

cholmes.medium.com

1 4 7