Lightnews — Scholar-powered news

TEGNicholas.bsky.social

@tegnicholas.bsky.social

330 followers 490 following 72 posts

Open-Source Software for science at Earthmover.io, built on Pangeo.io. One of many xarray.dev core devs. https://tom-nicholas.com/ Previously dabbled in oceanography at [C]Worthy and Columbia Uni., originally did fusion plasma physics.

tom-nicholas.com

Posts Media Videos Starter Packs

Pinned

TEGNicholas.bsky.social @tegnicholas.bsky.social · Jan 31

I've been thinking a lot recently about how one thing science needs generally is a **social network for sharing big data**, so I wrote this.

hackmd.io/wKKm4cIDR6a9...

🧵

Science needs a social network for sharing big data - HackMD

hackmd.io

4 10 38

Reposted by TEGNicholas.bsky.social

Ketan Joshi @ketanjoshi.co · Sep 3

"[Matt Yglesias] can write whole essays claiming that fracking is good and we need fossil fuel friendly energy policies, dismissing progressives as childish, while never engaging with the scientific literature on the consequences of climate change"

Matt Yglesias Is Confidently Wrong About Everything

The Biden administration’s favorite centrist pundit produces smug pseudo-analysis that cannot be considered serious thought. He ought to be permanently disregarded.

www.currentaffairs.org

8 69 360

TEGNicholas.bsky.social @tegnicholas.bsky.social · Aug 27

This reads like the writings of climate denialists - constant strawmanning, shifting of goalposts, selection bias in research quoted, and conspiracy theorizing.

1 2

Reposted by TEGNicholas.bsky.social

Kyle Barron @kylebarron.dev · Aug 4

New blog post on Obstore, fast, multi-provider cloud storage access for Python:

developmentseed.org/blog/2025-08...

We Needed Better Cloud Storage for Python so We Built Obstore — Development Seed

Obstore solves the friction we kept hitting in cloud-native workflows.

developmentseed.org

2 3

Reposted by TEGNicholas.bsky.social

Fernando Pérez @fernandoperez.org · Jul 24

Open source, open science for earth, climate and geospatial science? Coming to #AGU25? Build tools in #Python @jupyter.org?

Submit an abstract for this session and come meet us and like minded scientists!

Wilson Sauthoff @wilsonsauthoff.bsky.social · Jul 22

📣 Call for submissions: AGU25 IN029 "Open Source Geospatial Workflows in the Cloud" 🌍

Please submit your work using cutting-edge practices and tools that are shaping the future of geospatial science.

More info: events.geojupyter.org/conferences/...

#opensource #geospatial #jupyter #AGU25

AGU session advertisement with session title and invited speakers.

4 9

Reposted by TEGNicholas.bsky.social

Emily Atkin @emorwee.bsky.social · Jul 21

Will Alpine co-wrote Microsoft's manifesto on how AI will be a powerful force for good for climate change

In a new interview, Alpine disavows the manifesto, saying he believes Microsoft used his work to distract from the much larger climate harms the company enables through contracts with Big Oil

He helped Microsoft build AI to help the climate. Then Microsoft sold it to Big Oil.

A former Microsoft project manager reveals how the tech giant is using AI to help Big Oil drill—and how he and his partner are now pushing for change.

heated.world

1 55 96

Reposted by TEGNicholas.bsky.social

Henri F. Drake @henrifdrake.bsky.social · Jun 30

This is fucking insane. Closing these NOAA labs would obliterate our ability to observe, understand, and forecast the Earth System, from weather systems tomorrow to sea levels 50 years from now.

Zack Labe @zacklabe.com · Jun 30

What a tragedy this is even being proposed on paper... 💔

NOAA FY2026 Congressional Justification: www.noaa.gov/sites/defaul...

Termination of OAR’s Climate Laboratories and Cooperative Institutes (-$102,292, -204 FTE/ -216 Positions) –NOAA will continue to support high priority ocean and weather research programs in NOS and NWS.

In coordination with the requested terminations for Weather Laboratories and Cooperative Institutes (see OAR-10) and Ocean Laboratories and Cooperative Institutes (see OAR-19), NOAA will close the Atlantic Oceanographic & Meteorological Laboratory (AOML) in Miami, FL; the Air Resources Laboratory (ARL) in College Park, MD, Idaho Falls, ID, and Oak Ridge, TN, as well as a nation-wide network of soil moisture sensors; the Chemical Sciences Laboratory (CSL) in Boulder, CO; the Geophysical Fluid Dynamics Laboratory (GFDL) in Princeton, NJ; the Global Monitoring Laboratory (GML) in Boulder, CO, Utqiaġvik, AK, Mauna Loa, HI, Hilo, HI, Big Island, HI, American Samoa, and the South Pole; the Pacific Marine Environmental Laboratory (PMEL) in Seattle, WA; and the Physical Sciences Laboratory (PSL) in Boulder, CO.

75 140

Reposted by TEGNicholas.bsky.social

Dr. Jessie Christiansen @aussiastronomer.bsky.social · Jun 29

NASA is being told to cancel 19 *active* missions to save $6B, which looks to be less than the ICE *hiring/retention* budget going forward.

I need people to let that sentence sink into their bones for a minute.

Aaron Reichlin-Melnick @reichlinmelnick.bsky.social · Jun 28

If the GOP reconciliation bill passes, ICE gets through FY2029:

- $45 billion for detention, on top of the current annual budget of $3.4 billion
- $14.4 billion for transportation and removal, on top of the current annual budget of $750 million
- $8 billion for hiring/retention
- Billions more.

Jon Cooper @joncooper-us.bsky.social · Jun 28

‼️‼️

73 2.1K 4.2K

Reposted by TEGNicholas.bsky.social

Michele Tobias, PhD @micheletobias.bsky.social · Oct 28

I've been adding new accounts to the Open Source Geospatial starter pack. Who else wants on or off?
#gischat #geosky

go.bsky.app/PGYLmPG

35 22 71

TEGNicholas.bsky.social @tegnicholas.bsky.social · May 22

Oh @jsignell.github.io too!

TEGNicholas.bsky.social @tegnicholas.bsky.social · May 22

I would have suggested Max Jones, Aimee Barciauskas, or Lindsey Nield, but they don't seem to be on BlueSky, so you could instead add @jhamman.bsky.social , @rabernat.bsky.social , or myself.

TEGNicholas.bsky.social @tegnicholas.bsky.social · May 22

There should be some @zarr.dev geospatial representation on here.

Evidence:

earthmover.io/blog/zarr-ta...

Zarr takes Cloud-Native Geospatial by storm - Earthmover

Our takeaways from the Cloud-Native Geospatial conference on Zarr’s surging adoption and its impact on the future of Earth Observation data. Our team just returned from an action-packed week at the Cl...

earthmover.io

1 1 1

Reposted by TEGNicholas.bsky.social

Adam Sobel @adamsobel.bsky.social · May 21

It's outrageous that NASA GISS, one of the best earth & space science labs in the world, is being kicked out of its Columbia home. The outstanding scientists who work there can't say that publicly, but I can. And so can you --- call your reps, esp. (but not only) if you live in NYC or NY state.

Marcus van Lier-Walqui @mvanlierwalq.bsky.social · May 21

Nice Guardian article about us losing our office in Manhattan
www.theguardian.com/environment/...

Godfather of climate science decries Trump plan to shut Nasa lab above Seinfeld diner: ‘It’s crazy’

Over breakfast at Tom’s Restaurant, right below the historic Giss lab, James Hansen calls Doge’s decision a ‘big mistake’

www.theguardian.com

36 62

Reposted by TEGNicholas.bsky.social

Earthmover @earthmover.io · May 14

𝐻𝑜𝑤 𝑑𝑜𝑒𝑠 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝑎𝑣𝑜𝑖𝑑 𝑟𝑒𝑑𝑢𝑛𝑑𝑎𝑛𝑡 𝑠𝑡𝑜𝑟𝑎𝑔𝑒 𝑏𝑒𝑡𝑤𝑒𝑒𝑛 𝑑𝑎𝑡𝑎 𝑣𝑒𝑟𝑠𝑖𝑜𝑛𝑠?

Icechunk stores only new or changed chunks for each version —no redundant copies or rewrites. You get instant time travel, branching, and efficient updates, all with negligible storage overhead.

More: bit.ly/3F1XFST

Icechunk: Efficient storage of versioned array data - Earthmover

We recently got an interesting question in Icechunk’s community Slack channel (thank you Iury Simoes-Sousa for motivating this post): I’m new to Icechunk. How is the storage managed for redundant info...

earthmover.io

4 3

TEGNicholas.bsky.social @tegnicholas.bsky.social · May 12

Excellent post by Brian Davis laying out why doing "Open Science" for data-driven workflows is almost impossible in practice, at least without much better data pipeline tools.

Earthmover @earthmover.io · May 12

Our latest blog post dives into the chaos of the status quo - where every tweak means regenerating the 𝑤ℎ𝑜𝑙𝑒 𝑑𝑎𝑡𝑎𝑠𝑒𝑡 and collaboration and experimentation is often stifled by silos and secret knowledge. Check out the full post: earthmover.io/blog/tensoro...

TensorOps: Scientific Data Doesn't Have to Hurt - Earthmover

Curious how your team scores on the "Data Pain Survey"? Wondering why your teams are building Rube Goldberg machines just to put some data on a map? Or just want to see our plan to bring order to your...

earthmover.io

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 26

nice analogy 😉

1 1

Reposted by TEGNicholas.bsky.social

Abrahm @abrahm.bsky.social · Apr 24

The proposed cuts to NOAA cold have profound consequences not just for climate change, but for our national security and the entire economy. Here's what I learned: www.propublica.org/article/trum...

White House Proposal Could Gut Climate Modeling the World Depends On

Potential funding cuts for NOAA and its research partners threaten irreparable harm not only to climate research but to American safety, competitiveness, and national security.

www.propublica.org

7 210 590

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 23

It's fun to work with real hardcore software engineers like @functionth.bsky.social who can teach you about database consistency and transactions and all that

Scientific data infrastructure should be built on solid foundations like this instead of on piles of janky code written by postdocs...

Earthmover @earthmover.io · Apr 23

1/ 🚨 New Blog Post Alert: "𝐿𝑒𝑎𝑟𝑛𝑖𝑛𝑔 𝐴𝑏𝑜𝑢𝑡 𝐼𝑐𝑒𝑐ℎ𝑢𝑛𝑘 𝐶𝑜𝑛𝑠𝑖𝑠𝑡𝑒𝑛𝑐𝑦 𝑤𝑖𝑡ℎ 𝑎 𝐶𝑙𝑖𝑐ℎ𝑒́𝑑 𝑏𝑢𝑡 𝐼𝑛𝑠𝑡𝑟𝑢𝑐𝑡𝑖𝑣𝑒 𝐸𝑥𝑎𝑚𝑝𝑙𝑒" 🏦🔁

👉 Read it here: earthmover.io/blog/learnin...

Learning about Icechunk consistency with a clichéd but instructive example - Earthmover

In this post we’ll show what can happen when more than one process write to the same Icechunk repository concurrently, and how Icechunk uses transactions and conflict resolution to guarantee consisten...

earthmover.io

Reposted by TEGNicholas.bsky.social

Mike Mahoney @mikemahoney218.com · Apr 19

the fact that I've never once thought about making a range request, and yet make them constantly for extremely targeted data pulls, is absolutely an invisible technical miracle

1 1 4

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 17

Of course- I really wrote this article for my past self! I wish someone had explained this cloud science stuff to me earlier.

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 17

It’s also important for understanding what problem VirtualiZarr solves.

I’ve given this explanation to many people in the past (including at @cworthy.bsky.social), so I hope that this article can serve as a useful reference the next time someone wonders what @zarr.dev actually is.

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 17

I wrote the article I wish I could have read back when I first heard of Zarr and cloud-native science back in 2018.

This explains how object storage and conventional filesystems are different, and the key properties that make @zarr.dev work so well in cloud object storage.

Earthmover @earthmover.io · Apr 17

1/ 💡 Our latest blog post in the fundamentals series, written by @tegnicholas.bsky.social, demystifies cloud-optimized scientific data formats!

Read more: earthmover.io/blog/fundame...

Fundamentals: What is Cloud-Optimized Scientific Data?

What cloud-optimized data really means, and how Zarr and Icechunk enable fast access to massive scientific datasets in cloud object storage.

earthmover.io

1 4

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 16

5/ Almost all organisations working with scientific array data have this kind of data delivery issue, even if it's just internally.

Whilst the Flux integrations today are established geospatial standards, you also see similar patterns in other fields such as Neuroscience.

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 16

4/ Flux's architecture is auto-scaling, so once turned on there is no need to worry about how many users are hitting the data.

As it's not a stateful server like THREDDS, it won't catch fire under pressure.

This is what "Cloud-Native" architectures for scientific data look like.

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 16

3/ Your downstream scientists, GIS users, analysts, and external users can all now forget about file formats!

They just keep using the same GUI or tool or script that they prefer, and don't need any other services or copies of the data made bespoke for them - Flux does that on-demand!

TEGNicholas.bsky.social @tegnicholas.bsky.social · Apr 16

2/ Flux bridges this chasm.

It sits in between your data and the consumers, springing up at a moment's notice to provide subsets of data however your users prefer it.