Just published a new article about the new Python 3.14 module `compression`.
This module offers a neat way to access the 5 compression modules we have in Python:
👉 bz2
👉 gzip
👉 lzma
👉 zlib
👉 zstd
The first four were already available as standalone modules...
This module offers a neat way to access the 5 compression modules we have in Python:
👉 bz2
👉 gzip
👉 lzma
👉 zlib
👉 zstd
The first four were already available as standalone modules...
November 8, 2025 at 12:02 AM
Just published a new article about the new Python 3.14 module `compression`.
This module offers a neat way to access the 5 compression modules we have in Python:
👉 bz2
👉 gzip
👉 lzma
👉 zlib
👉 zstd
The first four were already available as standalone modules...
This module offers a neat way to access the 5 compression modules we have in Python:
👉 bz2
👉 gzip
👉 lzma
👉 zlib
👉 zstd
The first four were already available as standalone modules...
Is the 30GB figure deriving from storing all PLC operations in zstd form, grouped by the 1000s? I'm interested in how you achieved that!
November 5, 2025 at 4:20 PM
Is the 30GB figure deriving from storing all PLC operations in zstd form, grouped by the 1000s? I'm interested in how you achieved that!
~27 seconds to dump PDS endpoints from the *whole* PLC directory - directly from plcbundle files (zstd+jsonl) using Bun
with 2700 MB/s i'm finally happy with the speed
tangled.org/@atscan.net/...
with 2700 MB/s i'm finally happy with the speed
tangled.org/@atscan.net/...
November 1, 2025 at 9:18 AM
~27 seconds to dump PDS endpoints from the *whole* PLC directory - directly from plcbundle files (zstd+jsonl) using Bun
with 2700 MB/s i'm finally happy with the speed
tangled.org/@atscan.net/...
with 2700 MB/s i'm finally happy with the speed
tangled.org/@atscan.net/...
gave it a shot: using streams and swapping zstd library, i'm getting 475.8 MB/sec
October 31, 2025 at 10:06 AM
gave it a shot: using streams and swapping zstd library, i'm getting 475.8 MB/sec
tbh all my approach is very naive :) I admit this is mainly about JSON and zstd implementations
October 30, 2025 at 8:39 PM
tbh all my approach is very naive :) I admit this is mainly about JSON and zstd implementations
Fedora Atomic Desktops are bringing a few helpful changes as the community preps for the big shift to bootc!
* zstd compressed initrds for slightly faster boot time
* 2GB /boot partition for new installs
* wireguard-tools added to make working with VPNs easier
* All desktop environments on […]
* zstd compressed initrds for slightly faster boot time
* 2GB /boot partition for new installs
* wireguard-tools added to make working with VPNs easier
* All desktop environments on […]
Original post on fosstodon.org
fosstodon.org
October 29, 2025 at 3:13 PM
Fedora Atomic Desktops are bringing a few helpful changes as the community preps for the big shift to bootc!
* zstd compressed initrds for slightly faster boot time
* 2GB /boot partition for new installs
* wireguard-tools added to make working with VPNs easier
* All desktop environments on […]
* zstd compressed initrds for slightly faster boot time
* 2GB /boot partition for new installs
* wireguard-tools added to make working with VPNs easier
* All desktop environments on […]
This month in Servo…
☢️🕸️ experimental mode button
🕵🏼♀️🍲 Trusted Types now stable
👻✍️ strokeText() on 2D canvases
🗜️📄 ‘Content-Encoding: zstd’
🧾🥺 new funding request process
servo.org/blog/2025/10...
☢️🕸️ experimental mode button
🕵🏼♀️🍲 Trusted Types now stable
👻✍️ strokeText() on 2D canvases
🗜️📄 ‘Content-Encoding: zstd’
🧾🥺 new funding request process
servo.org/blog/2025/10...
October 25, 2025 at 3:40 AM
This month in Servo…
☢️🕸️ experimental mode button
🕵🏼♀️🍲 Trusted Types now stable
👻✍️ strokeText() on 2D canvases
🗜️📄 ‘Content-Encoding: zstd’
🧾🥺 new funding request process
servo.org/blog/2025/10...
☢️🕸️ experimental mode button
🕵🏼♀️🍲 Trusted Types now stable
👻✍️ strokeText() on 2D canvases
🗜️📄 ‘Content-Encoding: zstd’
🧾🥺 new funding request process
servo.org/blog/2025/10...
All the storage is on @upcloud.com's manged object store coming out to be about $5 for 250gigs, so hoping it won't be too bad.
Everything is also compressed before upload using zstd. Doesn't do a lot for blobs, but did a lot for the repo exports.
Everything is also compressed before upload using zstd. Doesn't do a lot for blobs, but did a lot for the repo exports.
The Perfect Home For Your Data - Managed Object Storage
Store and manage unstructured data effortlessly with UpCloud’s Managed Object Storage. Scalable, secure, and cost-effective for large-scale storage needs.
upcloud.com
October 20, 2025 at 10:35 PM
All the storage is on @upcloud.com's manged object store coming out to be about $5 for 250gigs, so hoping it won't be too bad.
Everything is also compressed before upload using zstd. Doesn't do a lot for blobs, but did a lot for the repo exports.
Everything is also compressed before upload using zstd. Doesn't do a lot for blobs, but did a lot for the repo exports.
This zstd thing is pretty cool
October 11, 2025 at 7:39 PM
This zstd thing is pretty cool
Python 3.14 Released With Performance Improvements, Free-Threading & Zstd - https://www.phoronix.com/news/Python-3.14
Python 3.14 Released With Performance Improvements, Free-Threading & Zstd
Python 3.14 is now available as the newest annual major feature release for the Python programming language...
www.phoronix.com
October 7, 2025 at 5:03 PM
Python 3.14 Released With Performance Improvements, Free-Threading & Zstd - https://www.phoronix.com/news/Python-3.14
I had a quick look but was intimidated by the docs honestly. Nick from the Zstd team mentioned BAMs wrt a forthcoming post on use cases, so hopefully we'll soon have some diverse profiles to learn from. news.ycombinator.com/item?id=4549...
Do you happen to have a pointer to a good open source dataset to look at? Naivel... | Hacker News
news.ycombinator.com
October 7, 2025 at 3:13 PM
I had a quick look but was intimidated by the docs honestly. Nick from the Zstd team mentioned BAMs wrt a forthcoming post on use cases, so hopefully we'll soon have some diverse profiles to learn from. news.ycombinator.com/item?id=4549...
Blogged about how zstd --long fills the gap between fast and slow-but-high-ratio genome compression methods log.bede.im/2025/09/12/z...
September 12, 2025 at 3:07 PM
Blogged about how zstd --long fills the gap between fast and slow-but-high-ratio genome compression methods log.bede.im/2025/09/12/z...
New filing: "U.S. v. Trump (Federal/WDC/1/Indictment/01.08/2023)"
Doc #201: LEAVE TO FILE DENIED- Application for CVRA Relief dated 3/20/2024 as to DONALD J. TRUMP. Although courts have in rare instances exercised their…
[full entry below 👇]
[Download PDF] | [View Full Case]
#CL67656604
Doc #201: LEAVE TO FILE DENIED- Application for CVRA Relief dated 3/20/2024 as to DONALD J. TRUMP. Although courts have in rare instances exercised their…
[full entry below 👇]
[Download PDF] | [View Full Case]
#CL67656604
August 6, 2024 at 12:39 PM
New filing: "U.S. v. Trump (Federal/WDC/1/Indictment/01.08/2023)"
Doc #201: LEAVE TO FILE DENIED- Application for CVRA Relief dated 3/20/2024 as to DONALD J. TRUMP. Although courts have in rare instances exercised their…
[full entry below 👇]
[Download PDF] | [View Full Case]
#CL67656604
Doc #201: LEAVE TO FILE DENIED- Application for CVRA Relief dated 3/20/2024 as to DONALD J. TRUMP. Although courts have in rare instances exercised their…
[full entry below 👇]
[Download PDF] | [View Full Case]
#CL67656604
If you need to send files through e-Mail, but the common formats are disallowed (ZIP, RAR, 7-Zip), try using any of the following ones:
* Zstandard (zstd)
* ZPAQ (zpaq)
* wim
(The latter can be manipulated with 7-Zip.)
#computing #bigfiles #mailhacks
* Zstandard (zstd)
* ZPAQ (zpaq)
* wim
(The latter can be manipulated with 7-Zip.)
#computing #bigfiles #mailhacks
November 3, 2024 at 9:00 AM
If you need to send files through e-Mail, but the common formats are disallowed (ZIP, RAR, 7-Zip), try using any of the following ones:
* Zstandard (zstd)
* ZPAQ (zpaq)
* wim
(The latter can be manipulated with 7-Zip.)
#computing #bigfiles #mailhacks
* Zstandard (zstd)
* ZPAQ (zpaq)
* wim
(The latter can be manipulated with 7-Zip.)
#computing #bigfiles #mailhacks
It would be cool to have some form of content-negotiation where the client requests ZSTD in a header and the server sends back a dictionary to be used for the session somehow. Would be more tolerant of client/server version mismatches if the zstd dictionaries end up seeing some churn over time.
September 22, 2024 at 10:57 PM
It would be cool to have some form of content-negotiation where the client requests ZSTD in a header and the server sends back a dictionary to be used for the session somehow. Would be more tolerant of client/server version mismatches if the zstd dictionaries end up seeing some churn over time.
Zstd bomb thrown into #rstats devel mailing list
I expect either:
* Extinction-level bike-shedding event, or
* A silence not matched since before the big bang
I expect either:
* Extinction-level bike-shedding event, or
* A silence not matched since before the big bang
January 10, 2025 at 11:27 PM
Zstd bomb thrown into #rstats devel mailing list
I expect either:
* Extinction-level bike-shedding event, or
* A silence not matched since before the big bang
I expect either:
* Extinction-level bike-shedding event, or
* A silence not matched since before the big bang
this is good for high impact software but imo it's not sustainable for most things
like imagine if you also had to pay a zstd, zlib, brotli, gzip, h264, h265, av1, mp3, ogg, opus, svg, png, jpg, jxl, avif, etc license with your browser
like imagine if you also had to pay a zstd, zlib, brotli, gzip, h264, h265, av1, mp3, ogg, opus, svg, png, jpg, jxl, avif, etc license with your browser
February 28, 2025 at 3:46 PM
this is good for high impact software but imo it's not sustainable for most things
like imagine if you also had to pay a zstd, zlib, brotli, gzip, h264, h265, av1, mp3, ogg, opus, svg, png, jpg, jxl, avif, etc license with your browser
like imagine if you also had to pay a zstd, zlib, brotli, gzip, h264, h265, av1, mp3, ogg, opus, svg, png, jpg, jxl, avif, etc license with your browser
Okay cool so with some tweaks I've got the per message weights:
482B -> 211B
Using a custom zstd dictionary trained on 100k events
Also got the throughput back up so the compressed vs decompressed clients don't have any difference in load on the server
Requires 2x the storage tho on the server
482B -> 211B
Using a custom zstd dictionary trained on 100k events
Also got the throughput back up so the compressed vs decompressed clients don't have any difference in load on the server
Requires 2x the storage tho on the server
Some preliminary testing with this shows BW savings of ~31.5% using zstd for Jetstream but it also increases the CPU requirements per consumer by an insane amount...
Per-message weight:
- without compression: ~485B
- with compression: ~332B
Max throughput drops by 85.7% though...
Per-message weight:
- without compression: ~485B
- with compression: ~332B
Max throughput drops by 85.7% though...
Makes me wanna experiment a bit with z-std streaming compression on Jetstream... Might be a fun rainy-day project.
September 22, 2024 at 5:52 AM
Okay cool so with some tweaks I've got the per message weights:
482B -> 211B
Using a custom zstd dictionary trained on 100k events
Also got the throughput back up so the compressed vs decompressed clients don't have any difference in load on the server
Requires 2x the storage tho on the server
482B -> 211B
Using a custom zstd dictionary trained on 100k events
Also got the throughput back up so the compressed vs decompressed clients don't have any difference in load on the server
Requires 2x the storage tho on the server
Added a suggestion for #rstats Dev Days on their issue tracker:
Add compression `level` argument to 'saveRDS()' and 'memCompress()'
Zstd is too great/powerful to be stuck with the current hardcoded compression level!
https://github.com/r-devel/r-dev-day/issues/89
Add compression `level` argument to 'saveRDS()' and 'memCompress()'
Zstd is too great/powerful to be stuck with the current hardcoded compression level!
https://github.com/r-devel/r-dev-day/issues/89
Add compression `level` argument to 'saveRDS' and 'memCompress' · Issue #89 · r-devel/r-dev-day
zstd compression has recently been added to R. zstd is highly configurable in terms of speed/compression tradeoffs. With compression set at -1 it is on par with lz4 with very fast compression speed...
github.com
May 15, 2025 at 11:28 AM
Added a suggestion for #rstats Dev Days on their issue tracker:
Add compression `level` argument to 'saveRDS()' and 'memCompress()'
Zstd is too great/powerful to be stuck with the current hardcoded compression level!
https://github.com/r-devel/r-dev-day/issues/89
Add compression `level` argument to 'saveRDS()' and 'memCompress()'
Zstd is too great/powerful to be stuck with the current hardcoded compression level!
https://github.com/r-devel/r-dev-day/issues/89
Zstd 1.5.7 Lands In Linux 6.15 For Better Performance & APIs For Intel QAT Acceleration Phoro […]
[Original post on phoronix.com]
[Original post on phoronix.com]
Original post on phoronix.com
www.phoronix.com
March 28, 2025 at 10:58 AM
Zstd 1.5.7 Lands In Linux 6.15 For Better Performance & APIs For Intel QAT Acceleration Phoro […]
[Original post on phoronix.com]
[Original post on phoronix.com]
Dan does compression - I have three zpools - let's try using it without compression, with lz4, and with zstd.
dan.langille.org/2025/10/06/c...
dan.langille.org/2025/10/06/c...
Compression results – Dan Langille's Other Diary
dan.langille.org
October 6, 2025 at 11:36 AM
Dan does compression - I have three zpools - let's try using it without compression, with lz4, and with zstd.
dan.langille.org/2025/10/06/c...
dan.langille.org/2025/10/06/c...
I didn't do this, but now I will do this
my emulation folder is 2.4tb currently, only 62gb saved on zstd:2 according to compsize
let's find out how much of a difference zstd:19 makes. see you in 1765 days I guess
my emulation folder is 2.4tb currently, only 62gb saved on zstd:2 according to compsize
let's find out how much of a difference zstd:19 makes. see you in 1765 days I guess
October 25, 2025 at 1:14 PM
I didn't do this, but now I will do this
my emulation folder is 2.4tb currently, only 62gb saved on zstd:2 according to compsize
let's find out how much of a difference zstd:19 makes. see you in 1765 days I guess
my emulation folder is 2.4tb currently, only 62gb saved on zstd:2 according to compsize
let's find out how much of a difference zstd:19 makes. see you in 1765 days I guess
First passes: CompressX for images + SpeedifyPress for unused CSS, JS delay, lazy images, font preloads, basic caching, and Cloudflare zstd edge cache. https://adventuresinwordpressing.substack.com/p/woocommerce-can-you-actually-get #PageSpeed #WordPress #WooCommerce
WooCommerce: Can You Actually Get a Speedy Store?
Woo that flies - even for logged-in users. Sounds impossible, right?
adventuresinwordpressing.substack.com
November 4, 2025 at 11:57 PM
First passes: CompressX for images + SpeedifyPress for unused CSS, JS delay, lazy images, font preloads, basic caching, and Cloudflare zstd edge cache. https://adventuresinwordpressing.substack.com/p/woocommerce-can-you-actually-get #PageSpeed #WordPress #WooCommerce