Lightnews — Scholar-powered news

Bede Constantinides

@bedec.bsky.social

900 followers 1.4K following 110 posts

Interested in infectious disease informatics. Research fellow at the University of Birmingham. Also cycling, photography, active travel. https://bede.im

bede.im

Posts Media Videos Starter Packs

Pinned

Bede Constantinides @bedec.bsky.social · Jun 13

New preprint! Deacon is a versatile tool for filtering FASTA/FASTQ files and streams at hundreds of megabases per second using minimizers, built with rapid metagenomic host depletion in mind, but equally useful for search.
github.com/bede/deacon

bioRxiv Bioinfo @biorxiv-bioinfo.bsky.social · Jun 13

Deacon: fast sequence filtering and contaminant depletion https://www.biorxiv.org/content/10.1101/2025.06.09.658732v1

5 40 57

Reposted by Bede Constantinides

Ben Langmead @benlangmead.bsky.social · 1d

I've added 7 videos to my Burrows-Wheeler indexing playlist (www.youtube.com/playlist?lis...), rounding out the r-index series and adding a 5-part series on the move structure. Now 27 videos in that playlist. I aim to add videos on prefix-free parsing, PBWT, Wheeler languages/automata in the future.

Burrows-Wheeler Indexing - YouTube

Videos on : (a) the Burrows-Wheeler Transform (BWT), (b) the FM Index, which uses the BWT to construct a full-text index, (c) Wheeler graphs, (d) r-index, an...

www.youtube.com

2 16 55

Bede Constantinides @bedec.bsky.social · 1d

Yes I got the upload pings and almost posted the same here. A treasure trove of material.

Bede Constantinides @bedec.bsky.social · 1d

No harm in sticking with an old version :-) Results should be identical to 0.10.0 except for seqs containing IUPAC ambiguous bases, changing the classification of exactly one read in my IUPAC benchmark dataset.

1 2

Bede Constantinides @bedec.bsky.social · 1d

Deacon 0.11.0:
- Local server mode
- Ultra-careful handling of non-ACGT
- Faster indexing & index loading
- Denser index now stores k-mers not hashes
- xxHash & FxHash replaced with rapidhash::fast
- Bug fixes

Thanks @curiouscoding.nl (and others!) for contributions
github.com/bede/deacon/...

Release 0.11.0 · bede/deacon

Major release incorporating new features, fixes and peformance optimisations. Includes many PRs from @RagnarGrootKoerkamp, taking advantage of new features in simd-minimizers, packed-seq and parase...

github.com

1 1 6

Bede Constantinides @bedec.bsky.social · 1d

I had a quick look but was intimidated by the docs honestly. Nick from the Zstd team mentioned BAMs wrt a forthcoming post on use cases, so hopefully we'll soon have some diverse profiles to learn from. news.ycombinator.com/item?id=4549...

Do you happen to have a pointer to a good open source dataset to look at? Naivel... | Hacker News

news.ycombinator.com

Bede Constantinides @bedec.bsky.social · 1d

Nick Terrell (Meta): "Looking at the BAM format, it looks like the tokenization portion will be easy. Which means I can focus on the compression side, which is more interesting."

1 1

Bede Constantinides @bedec.bsky.social · 1d

It doesn't compress e.g. FASTA out of the box currently. The bundled profiles are geared towards sensible fixed-length formats like Parquet. I imagine we'll see new profiles emerge quickly openzl.org/getting-star...

Quick Start - OpenZL

openzl.org

Bede Constantinides @bedec.bsky.social · 1d

"It was really hard to resist spilling the beans about OpenZL on this recent HN post about compressing genomic sequence data"
news.ycombinator.com/item?id=4549...

It was really hard to resist spilling the beans about OpenZL on this recent HN p... | Hacker News

news.ycombinator.com

2 3

Bede Constantinides @bedec.bsky.social · 1d

"OpenZL is our answer to the tension between the performance of format-specific compressors and the maintenance simplicity of a single executable binary."
engineering.fb.com/2025/10/06/d...

2 5 12

Bede Constantinides @bedec.bsky.social · 2d

Oh it's a pity if splitting into crates is punished with much worse compile times. Is this the case?

1 1

Bede Constantinides @bedec.bsky.social · 2d

Fun question! Doubt it would noticeably change the plot though. Gzip's 32KB window is a tiny fraction of genome length for anything larger than a virus. Would compress low complexity regions well but miss e.g. gene duplications.

Bede Constantinides @bedec.bsky.social · 6d

I can't bring myself to bet against HashSet given recent experience

1 1

Reposted by Bede Constantinides

Ragnar {Groot Koerkamp} @curiouscoding.nl · 7d

Looking for people to test the latest version of simd-sketch.

It's now 2x as fast at sketching, and supports skipping over kmers containing N and other ambiguous bases (which is only ~35% slower).

'cargo install simd-sketch' is right there under your fingertips ;)

github.com/RagnarGrootK...

GitHub - RagnarGrootKoerkamp/simd-sketch: Compute bottom-s sketches and s-buckets sketches, using simd-minimizers crate.

Compute bottom-s sketches and s-buckets sketches, using simd-minimizers crate. - RagnarGrootKoerkamp/simd-sketch

github.com

2 4 12

Reposted by Bede Constantinides

Ragnar {Groot Koerkamp} @curiouscoding.nl · 9d

FxHashSet::<u32>::contains throughput is wild!

- Up to 4x slowdown for negative queries due to probing.
- Positive queries are fast for small tables, but slow in RAM because they need 2 cache misses.

Lots of variance depending on the load factor, ie whether n is close to 87.5% of a power of 2.

Plot of (inverse) throughput of querying an FxHashSet<u32> of increasing size. 3 lines show the throughput when 1%, 50%, or 99% of queries is present in the set. The 1% and 50% lines show big spikes just before every power of 2, where they are up to 3x slower than in the best case.

2 2

Reposted by Bede Constantinides

Dorottya Nagy @dotnagy.bsky.social · 13d

Pleased to see this pre-printed, highlighting the completeness/accuracy of @nanoporetech.com long-read genome assembly for clinical Enterobacterales: www.biorxiv.org/content/10.1...

Thanks to colleagues @modmedmicro.bsky.social, @ukhsa.bsky.social, @genewiz.bsky.social and @oxfordbrc.bsky.social!

1 12 10

Reposted by Bede Constantinides

Nick Loman @pathogenomenick.bsky.social · 19d

Terrific new feature presented by @theo.io on @pathoplexus.org called SeqSets for generating DOIs for sequence subsets used in publications, that can then be tracked for impact via CrossRef that will allow data generators to track impact! #IMMEMXiV

2 18 45

Bede Constantinides @bedec.bsky.social · 23d

Thanks! Had assumed there'd be a good reason…

Bede Constantinides @bedec.bsky.social · 23d

Oh I didn't know 2bit supported Ns through extra data blocks. I wonder how easily it can be parsed in parallel compared to Binseq – perhaps @noamteyssier.bsky.social can comment?

Bede Constantinides @bedec.bsky.social · 23d

Some applications do need that 5th symbol to represent ambiguity / N, which I assume is why BAM uses 4bit encoding (IIRC). 2bit + bit mask(s) is another way to do it.

2 1

Bede Constantinides @bedec.bsky.social · 23d

Predictably the comments section is mostly horror at the state of FASTA format

2 11

Bede Constantinides @bedec.bsky.social · 23d

Front page of Hacker News 🫨
bsky.app/profile/bede...

4 3 26

Bede Constantinides @bedec.bsky.social · 24d

Thanks for sharing – another related tip is to increase the window size for even higher CR bsky.app/profile/bede...

Bede Constantinides @bedec.bsky.social · 26d

Blogged about how zstd --long fills the gap between fast and slow-but-high-ratio genome compression methods log.bede.im/2025/09/12/z...

Bede Constantinides @bedec.bsky.social · 26d

Blogged about how zstd --long fills the gap between fast and slow-but-high-ratio genome compression methods log.bede.im/2025/09/12/z...

9 18

Bede Constantinides @bedec.bsky.social · 27d

Congratulations both!

1 2

Bede Constantinides @bedec.bsky.social · 28d

First rate BBC journalism that somehow didn't make the cut for #r4today www.bbc.co.uk/news/article...
@andyverity.bsky.social

Anti-Islamic US biker gang members run security at deadly Gaza aid sites

BBC identifies members of Infidels MC gang hired as armed security at US and Israel-backed aid sites.