_Clickbait title: Can you write a bestselling book in Typst?_ A shiny new edition of Hypermedia Systems is out! Apart from incorporating typo and grammar fixes sent in by readers, the book was completely redesigned inside with a new layout. To achieve a much higher fidelity of design, we ported the book to Typst — the hot new LaTeX replacement. ## Why change what works? The first print release of HS was written in AsciiDoc. It seemed like a natural choice at the time – after all, many publishing houses trust AsciiDoc for their books. We learned the hard way that those publishers also have teams of engineers developing proprietary toolchains to produce their books. Vanilla AsciiDoc doesn’t actually have a way to produce good print output: * `asciidoctor-pdf` uses a rudimentary PDF generation library, and the output is just about readable, but not good – and you don’t have any good tools to change that. * the venerable Pandoc can output AsciiDoc, but oddly can’t read it – so we can’t compile our book to LaTeX. Your best option for producing a printable PDF, if you care at all about typesetting, is to compile to HTML, and print that to PDF. `asciidoctor-web-pdf` is a tool that wraps this workflow in a usable package, combined with Paged.js to polyfill the many features of CSS for print media that browsers don’t support. This is how we produced the first edition of HS, and it was a pain for multiple reasons. * Paged.js has bugs. One bug that was nasty to work around was one where using any CSS to modify the page numbers (say, resetting them after the front matter) would cause all the pages to revert to 0. I don’t know if this was fixed. * HTML with Paged.js is slow. The book has a little over 300 pages, and each page has a perceptible delay as it gets laid out. Even worse, I had to manually scroll the browser all the way to the bottom of the book before printing, or the back half of the book would be blank. Afterwards, printing would take what felt like a solid minute, though I never measured how long: Since presumably nobody at Firefox thought printing a page to PDF would be anything but instant, there’s no progress bar or any other indicator of success beyond the PDF appearing in the folder where it’s supposed to be. * The idiosyncrasies of this process meant that it only really worked on my machine, and I had to send PDFs back and forth with my co-authors and editor. The PDFs were absolutely massive and exceeded the Discord upload limit, so I uploaded each one to Google Drive. * Paged.js has more bugs – asides would refuse to break across pages while code blocks were all to eager to create widows and orphans – or disappear completely – which I had to work around in a whack-a-mole fashion. A few bugs seem to have made it into the final release. * How to set the background color of a table cell * AsciiDoc has facilities for indexing, but doesn’t support outputting them in HTML in any way. Worse, the mechanism I could have used to do it myself was deprecated _and removed_ before the replacement was implemented. Our editor ended up creating an index **by hand** , which was appended to the PDF. * You go through all this effort only to be rewarded with very mediocre typesetting, since browsers use greedy algorithms for laying out text for performance reasons, yielding far worse line breaking compared to the dynamic programming algorithms of TeX. Despite all this, we somehow produced a usable PDF. Most readers report being satisfied with the book’s design (Amazon’s failure to print a PDF consistently notwithstanding). The code that generated this PDF is not open source — initially, it was because we thought it might eat into print sales, but these days the real reason is that it might help someone else that tries to do this, and nobody should try to do this. (If you have any “why didn’t you just” recommendations, feel free to post them in the comments on link aggregators — they might help someone else — but don’t send them directly to me. I assure you, I’ve evaluated all options available even if I didn’t mention them here.) Speaking of link aggregators, ## Typst I first stumbled upon Typst when the author’s thesis was posted on the red site. Enthralled by its combination of quality typesetting, lightweight markup and an ergonomic scripting language tightly integrated, I knew that if I was ever part of another book project, I would use Typst. I rewrote my CV in it, as well as some documentation for an old project. This only confirmed my initial impression — Typst is a joy to use, and the scripting language is so good I wish it was a programming language in its own right. But I couldn’t use Typst for Hypermedia Systems, because Typst didn’t support HTML output. Imagine my excitement when a while ago, I realized Pandoc had added Typst support! ## Porting the manuscript As I mentioned before, Pandoc can’t read AsciiDoc. So how can I convert the 104537 words of HS into Typst? I was ready to do it by hand, but I decided to try something else first. One of AsciiDoc’s primary output formats is DocBook XML. I don’t know what it is or who uses it, but Pandoc can read it! I converted the book to DocBook XML, and then to Typst. Except AsciiDoc was buggy and generated invalid XML, so I had to fix that first. And then massage the resulting Typst a bit more. And apply any future changes to the AsciiDoc source to the Typst version. Still better than waiting for the PDF to render, though. ## The code blocks AsciiDoc has a unique style of code blocks where you can place markers in the code that refer to explanatory notes below the block – it looks like this: AsciiDoc code block with callout [source,python] ---- print("Hello, world!") <1> ---- <1> The canonical first program in any language. If you’ve read Hypermedia Systems, now you know how we made those. Typst doesn’t have any such feature, so with the great help of the Typst Discord community, I reimplemented it in about 50 lines of slightly cursed Typst code. ### code-callouts.typ: a Hypermedia Systems literate experience Before implementing code callouts, define a _label_ , which we will use to mark code blocks we have already annotated. This is to prevent an infinite loop that occurs when we use this function in a `show` rule. #let processed-label = <TypstCodeCallout-was-processed> The function `code-with-callouts` takes a code block and a function to render callouts. A simple default implementation returns the annotation number in square brackets in grayed-out sans-serif font. #let code-with-callouts( it /*: content(raw) */, callout-display: default-callout /*: (str) => content */ ) /*: content */ = { ... } Begin by checking if the code block has already been processed. ) /*: content */ = { if it.at("label", default: none) == processed-label { it } else { ... } } The dance with `it.at("label", default: none)` is a way to access the code block’s label without throwing an error if it has none. If it doesn’t have this label, the real fun begins. let (callouts, text: new-text) = parse-callouts(it.text) `parse-callouts` is a function that extracts the callouts from the code block. We pass it the `text` of the code block – the plain, un-highlighted code as a string. It returns that text with the callout markers removed, and the callouts themselves in the form of an array of callout numbers for each line. This is in a way the meat of the whole operation, but it’s also pretty mundane string processing code, so I won’t dwell on it. #let callout-pat = regex("<(\\d+)>(\\n|$)") #let parse-callouts( code-text /*: str */ ) /*: (callouts: array(array(int)), text: str) */ = { let callouts /*: array(array(int)) */ = () let new-text = "" for text-line in code-text.split("\n") { let match = text-line.match(callout-pat) if match != none { callouts.push((int(match.captures.at(0)),)) new-text += text-line.slice(0, match.start) } else { callouts.push(()) new-text += text-line } new-text += "\n" } (callouts: callouts, text: new-text) } Now that we have the callouts, we can render them. We do this by abusing show rules. A `raw.line` (object representing a line of raw text) knows its line number, so we can use that to look up the callouts for that line. We then iterate over the callouts and render them with the provided `callout-display` function. show raw.line: it => { it let callouts-of-line = callouts.at(it.number - 1, default: ()) for callout in callouts-of-line { callout-display(callout) } } You might assume as I did that this would be the end of it and we could just return the code block, but there’s a catch. The code block still has the `<1>` markers in it, and we need to remove them. To do this, we create a new code block with the same attributes as the original, but with our `new-text` as the body. The fact that we create a brand new code block instead of returning the one we were given is what causes the infinite loop I mentioned earlier, so we apply the processed label. let fields = it.fields() let _ = fields.remove("text") let _ = fields.remove("lines") let _ = fields.remove("theme") [#raw(..fields, new-text)#processed-label] This is the code that powers the code blocks in the new edition of Hypermedia Systems. All you need to do is add a `show raw.where(block: true): code-with-callouts` to your Typst document, and you can have code blocks with callouts too! The actual text of the callouts is just a list written below the code block. If I was implementing this from scratch, I might have had the `code-with-callouts` accept the content of the callouts as well and match them up to the code by text labels, but AsciiDoc uses numbers and makes you match them up manually, so I did too. ## The index Typst has no indexing functionality by default, but its scripting and introspection capabilities are powerful enough to implement indexing at the library level. In fact, an indexing library already exists in the form of in-dexter, which is small enough that I made a modified version of it to have full control over the index’s appearance and to add support for hierarchical terms. ## The print book So far, I’ve talked about the most technically interesting parts of the migration and not the most important part: the actual product. Using a capable typesetting system instead of Print to PDF meant I could tweak almost every aspect of the book’s design, and the software would actually cooperate. The new Hypermedia Systems uses indented paragraphs instead of block paragraphs. That is, the first line of a paragraph is indented, and there’s no space between paragraphs. What’s so special about that? You can definitely do that in HTML and CSS, right? Not if you want to do it well. Traditional style and my own preference dictate that indentations _separate_ paragraphs, so the first paragraph after a non-paragraph element (like a heading or a list) should not be indented. This is a common typographic convention, but to do it in CSS is impossible in the general case without dozens, maybe hundreds of selectors to handle every special case. In Typst, you just give paragraphs a `first-line-indent`, and they know when to use it. It’s a similar story with emphasis. The `<em>` element in HTML is italicized by default, and similarly for `emph` in Typst. When emphasis is used in an element that’s already italicized, the emphasis is shown by reversing the italics and making the emphasized text upright. With CSS, you might restyle `<em>` in all those contexts (and remember to revert it as needed). _Container style queries_ might make this easier in CSS. In Typst, it’s the default behavior. Other than styling features, Typst is also just better at typesetting, using line breaking algorithms similar to TeX. Especially compared to browsers with their greedy algorithms and weirdly small hyphenation dictionaries, Typst prose is a joy both to look at and to read. I want HTML and CSS to be as capable in print as they are on screens. Unfortunately, browsers don’t seem as interested – the CSS Paged Media module remains largely unimplemented. While I would love a future where the same HTML code can back both online and print books (and PDF is relegated to an archival format), building separate web and print books from a Typst manuscript seems to be the best strategy for now. ## The web book Typst can produce a wonderful PDF for us, but we also need to publish the book online. While HTML output is a planned feature for the Typst compiler, no implementation currently exists. This is why I didn’t propose Typst while we were producing the first edition, before Pandoc gained support for reading and writing Typst. The old hypermedia.systems was built with the static site generator Lume with an AsciiDoc plugin I wrote. For the new book, I wanted to test out [Müteferrika][], an online book publishing toolkit I’m building. The advantage of Müteferrika is that it understands the structure of a book (parts, chapters, frontmatter etc.). The disadvantage is that it was and still is an unfinished mess, so I had to fix a lot of bugs to get it to work for Hypermedia Systems. The build process had to convert the Typst source to HTML, which I planned to do with Pandoc. Unfortunately, Pandoc doesn’t have full parity with the mainline Typst compiler, and spat out many errors when I first tried to convert the book. The fix for most of these was to replace direct property accesses with `at` calls and add null checks as needed. ## Conclusion The migration from AsciiDoc to Typst was a resounding success. I had to fudge a lot of tooling around book production, since Typst is still a young project and focused more on research papers than books, but the ecosystem is constantly growing and improving. Shiroa is a promising project for publishing web books with Typst, for example. The web book is now live at hypermedia.systems. It has (as far as we could tell) everything the old web book had, and more. There’s a button at the bottom of each page where you can customize the colors of the website to your liking. For the print version, we’re switching from Amazon KDP to Lulu, which will hopefully result in more consistent printing. * Read the new Hypermedia Systems online * Get the EPUB for your e-reader * Buy the new print release on Lulu * Buy the Kindle release on Amazon * Buy the old hardcover while it’s still available * How to set the background color of a table cell

In the latest release of htmx, you can add event listeners to elements with `hx-on`: <form hx-on::beforeRequest="addCsrfToken(event);"> For all the other `hx-` attributes, we use CSS attribute selectors. However, with `hx-on`, the attribute name is not fixed as it contains the event. CSS attribute selectors support wildcards on the _value_ of attributes, but not the name: [hx-trigger] /* common and normal */ [href^="https://dz4k.com"] /* "starts with" operator */ [^ĥx-on:] /* not a thing */ ## XXMLPathPath Language XPath is a query language for extracting information from XML(-like) documents. Its main use cases are XSLT and parsing API responses. The XPath language is significantly more expressive than CSS, making it possible to traverse the XML tree in any direction, filter nodes based on arbitrary predicates, and select any kind of node (including comments, text nodes, and individual attributes). Our non-existent CSS attribute could be written as follows: //@*[starts-with(name(), "hx-on:")] This post is not supposed to be an XPath tutorial, but I’ll break this one down: `//` traverse the document (in CSS, this is the default) `@*` find any attribute (mnemonic: **at** -tribute) `[ ... ]` where… `starts-with(name(), "hx-on:")` its name starts with `"hx-on:"` CSS selectors don’t have these kinds of features, and it has good reasons not to. CSS has strict performance requirements – to the point that “CSS optimization” is generally not a thing – and selectors that offer more control could make slow selectors possible. In addition, CSS has well-defined specificity rules, whereas XPath does not. However, while these features make CSS great for stylesheets, CSS selectors are also the most common way to find DOM elements in JavaScript code and lacking in that regard. Many libraries which extend HTML do so by traversing the entire document and finding elements manually. This is often not needed since, if you didn’t know, **XPath is built into browsers.** ## document.evaluate The `document.evaluate` API is somewhat archaic, partly because it was designed for talking to XML APIs over `XMLHTTPRequest`. Here’s a DOM-friendly wrapper: function* xpath(...args) { let path, root = document; if (args.length > 1) [root, path] = args; else [path] = args; const nodeIterator = document.evaluate( path, root, null, XPathResult.UNORDERED_NODE_ITERATOR_TYPE, null, ); for ( let node = nodeIterator.iterateNext(); node != null; node = nodeIterator.iterateNext() ) { yield node; } } // TypeScript declaration function xpath(path: string): Iterable<Node>; function xpath(root: Element, path: string): Iterable<Node>;

Bamboo Paper is an app released by Wacom in 2014, presumably to promote their Bamboo series of capacitive pens. It features 6 pens, 3 of which need to be unlocked via in-app purchases. Notes are stored in the proprietary WILL format, and can only be synced to Wacom’s own Inkspace cloud service. Otherwise, backing up notes involves exporting each notebook manually to PDF. Due to these restrictions, I don’t use it that often to actually take notes. However, I can’t think of a single note taking app that I enjoy as much. It’s not because of the minimalist interface, or the satisfying action of the pressure-sensitive brushes and pens, though I love both of those things. Only recently have I realized what draws me to this app. Notebooks in Bamboo Paper have the same size as your device’s display. When you rotate the device, the UI rotates, but the notebook itself remains in the same orientation relative to the device. Zooming in is possible, but unwieldy, and you can’t zoom out beyond 100%. All these limitations mean that writing on Bamboo Paper feels like writing on a notebook, whereas using any other app feels like playing a game where your character writes in a notebook. When penstrokes are bound to the specific location on the tablet’s front glass where I put the pen, it makes spatial reasoning more natural, and spatial reasoning is the main advantage of taking handwritten notes over textual notes using handwriting recognition. Using Bamboo Paper with floating windows of other apps in front of it, as Samsung lets me do, is the closest feeling I’ll probably get to using a Microsoft Courier – at least, it would be if Paper had drag-and-drop support. It might be a 10 year old app but it did get an update last August, so come on Wacom… Despite all my searching, I haven’t found another app on Android that imitates Paper’s model. Some apps like Samsung Notes zoom and scroll like Word documents, while other apps have an infinite canvas that makes me feel disoriented. Since I want to store my notes somewhere that’s not too likely to disappear next year, I put up with the Word doc, but I’m putting this out there in case someone points me to the perfect virtual notebook

I’m building Denizen (https://denizen.dz4k.com/), a personal website system that is IndieWeb and Fediverse compatible. Source code: https://codeberg.org/dz4k/denizen

so insane to me that people advocate for distro packaging over flatpak hmm yes i love being unable to use up-and-coming software thet nobody packages yet and making it harder for new distros to get off the ground vertical monopolies are good when open source i guess the concept of software package management existing is a failure one (1) software should be expressible as one thing

I’ve been using a Nothing Phone (2a) for a while now. I love the built in widgets, but there’s not enough of them, and most third-party widgets stick out like a sore thumb. Not even any of the Nothing-themed KWGT packs on the Play Store match. So I tried my own hand at it, and with not much effort, I was able to make these two: To my eyes, that’s an exact match! The date and next event widgets are mine. If you just want to have these widgets, you can download them. If you want to make your own, read on: * VeryFewThings Date * VeryFewThings Next Event (this is my first time using Nextcloud’s sharing links, so let me know if there’s any issues) I’ll probably be updating these widgets and this guide, as well as making more widgets. In particular, some of the formulas in the next event widget are a bit messy. ## Background The background of the widget is a Rectangle. Use formulas for the width, height and paint color (long press each property, and tap the calculator icon). The formulas are: * **Width** : `$si(rwidth)$` * **Height** : `$si(rheight)$` * **Paint** : `$si(syscn1, if(si(darkmode), 10, 93)$` I used a corner radius of 32, though that’s my eyeball measurement and might not be exact. For 2x1 and 1x2 widgets, you can use an absurd number like 9999 to make them pill-shaped. ## Layout The widget should have 32 points of padding. Some Nothing widgets like the digital clock change based on their size. My date widget does the same thing. You can use these formulas to detect if the widget is wide or tall: * **Wide** : `$si(rwidth) > 1.8 * si(rheight)$` * **Tall** : `$si(rheight) > 1.8 * si(rwidth)$` If neither of these are true, the widget is square. ## Text Kustom can access your device’s system fonts, including Nothing’s custom fonts. Dotted NDot (Variant: 57 for big text, 55 for small text) Monospace Lettera Mono Serif Ntype82 Sans-serif Roboto (weight 300) Nothing uses the NDot font exclusively in all caps, so I recommend matching that. * **Text color** : `$si(syscn1, if(si(darkmode), 100, 0))`

I recently published the first pre-release version of Fruit Credits on Flathub. (I then immediately published two more because the reviewer discovered a bug). ## why: boring life stuff After quitting my job and realizing it would be a while before I found a new one1], I realized I might need to be a bit more responsible with money. Naturally, I downloaded [hledger. I’d tried to use hledger before but couldn’t make a habit of it. I was concerned that my new attempt could be a procrastination mechanism, but to my surprise, it was actually massively helpful. Even after one month, I felt more in control of my life than ever before, and it’s kept me in the black since. It was hard for me at first to see the value of keeping your own books when banks already record and present a history of transactions. Right now, the benefits I experience are: * **Everything is in one place.** Most people keep multiple accounts at multiple banks. Banks are somehow all shit at application engineering, or even just providing usable data export. Before picking up hledger, I straight-up had no idea _how much money I had_. * **The data is queryable.** I’m talking super basic stuff – I’ve not even scratched the surface of what one can do with plain text accounting, yet it’s still massively valuable to ask the computer “how much did I spend on A since B?” * **The usual reasons.** It’s free software operating on free file formats. I can see my account balances without popups offering me loans I can’t pay back. I started by typing transactions in the ledger format directly in a text editor. Then, I started using `hledger add`, which has autocompletion capabilities. Unfortunately, both of these were too clunky for me – maybe it takes getting used to, but I kept making typos in `hledger add` and fumbling as I tried to undo them. I imagined a GUI for adding transactions quickly that wouldn’t require me to enter things in order. This vision would eventually feature-creep itself into a GUI version of hledger. ## how: exciting computer stuff I’d heard Tauri was Electron but good, which was an attractive proposition to a web developer. It took me about a week to give up – it turns out making a web app look good as a desktop app is harder than Discord makes it look. Being a GNOME user, I was inspired by the many GNOME Circle apps to build something with GTK4 and libadwaita. I fired up GNOME Builder and spent about 2 days paralyzed by the language selector. After looking at code examples online, I decided Vala was the simplest option. I used GNOME Builder to scaffold the app, and used it to develop for a while. Eventually, I figured out how to build the app without it, and went back to Zed. It took me a while to get everything configured – for example, the default Builder template has a directory for translations with gettext, but there’s extra setup required to actually build and use them. However, between the build headaches, the programming inner loop was quite enjoyable, especially with the new-ish Blueprint UI language. ### vala is pretty cool, imo It seems that Rust is the recommendation for new GTK apps. Unfortunately, I never got into Rust, and I don’t think adding the GObject memory management model into the mix would make it grow on me. For the uninitiated, GObject is a subsystem that was originally part of GTK that provides object-oriented programming and reference-counted memory management features in C. Though GObject has some fun features like signals, there’s little fun to be had in writing classes in C – even with all the macro magic provided, class definitions are full of boilerplate code. In addition, the reference counting mechanism requires you to `g_object_ref ()` and unref your objects manually. One feature of GObject is the ability to generate bindings to other languages, including ones with actual OOP features, which allows GTK apps to be implemented in a variety of languages. However, these bindings often suffer from various impedance mismatches between GObject and the languages’ own object models. Vala, however, is implemented with GObject in mind from the start. It has built-in syntax for features (like signals) Though I’ve fallen in love with Vala, I can see why other people might not enjoy it. It has unfixed compiler bugs, and a standard library that was written for C. Most frustratingly though, refcounting is not as ergonomic as full tracing garbage collection, and when using C libraries (including GObject based ones!) that expect you to be comfortable with creative usage of memory, you can run into hard-to-debug segfaults. The de facto debugger for Vala is gdb, and it has no Vala support. You have to debug through the generated C. I had no gdb experience. Thankfully, GLib has a built in logging framework that makes printf-style debugging quite comfortable. What draws me to Vala is the feeling of consideration that it exudes from every orifice. I would constantly discover that whatever I was trying to do, the Vala designers knew I’d need to do it, and had implemented a feature to make it easier. ### flatpacking haskell A GTK app written in Vala and compiled with Meson is the happiest of happy paths for Flatpak. Unfortunately, hledger, which Fruit Credits necessarily depends on, is written in Haskell. Flathub requires not only that everything is built from source, but also that all necessary sources can be downloaded before the build process (which is sandboxed). This is antithetical to the model used by Haskell (and npm, and cargo, and many others) where a build tool downloads dependencies and discovers transitive dependencies at the same time. After a few days of digging through outdated resources, I understood that a tool called cabal-flatpak was the best way to generate Flatpak manifests that fetch and compile Haskell libraries in the right order. Unfortunately, despite seemingly being under active maintenance, the tool had bugs, and no discernible way to report them. Since I don’t know Haskell, I couldn’t maintain a fork, so I wrote some jq and shell to work around the bugs. Upon submitting my app to Flathub, I found out I was something of a pioneer: > * **bbhtt:** why are these needed and why do you need to do this manually? > * **dz4k:** The haskell modules in the manifest are based on the output of cabal-flatpak and the article How to Flatpak a Haskell App into Flathub. Are there more up-to-date resources for bundling Haskell programs? > * **bbhtt:** No, we don’t have haskell apps. This is probably the second or third one ever. > – Add com.dz4k.FruitCredits by dz4k · Pull Request #5731 · flathub/flathub fun. ## what: thermonuclear war stuff Though Fruit Credits is pretty barebones compared to what hledger can do, it’s crossed from “dogfooding” to actually usable software for my use cases. Queries and transaction input work well. I’m currently working on a setup interface to create a new ledger file from scratch, as well as making the app more beginner-friendly in general. I’d like to add some way of editing and deleting transactions, and some reporting features. My usage of hledger is pretty limited in the grand scheme of things, so I won’t be able to cover every use case by myself. I’d love it if people gave Fruit Credits a try, even if just to tell me how it couldn’t read their journal file. * * * * get it on Flathub * get the code on Codeberg * complain about it on Codeberg * look at the website on the website * * * 1. I’m fine – I live with my parents, have no dependents and still have some income via Hypermedia Systems. I would still very much like a job though, which is why I invested in highly demanded skills as Vala programming and Haskell Flatpak packaging. ↩︎

We went on a walk in the woods today, and Biber came with us. We came across a cat that looks just like her. He was a male cat. Biber soon caught up and they started to interact: I’m not an animal behaviorist and I couldn’t tell you if they were flirting or getting ready to fight.

## Background Denizen is a CMS for personal websites. I want to work on multi-language support – people should be able to make websites and posts in any language they want. (My own blog is in English, Turkish, and Toki Pona – so this is a blocker for eating my dog food). Every aspect of this feels like something other people must have already solved, so I wanted to publicly ask if anyone has the secret right answers. ## Language picker Requirements: Users should be able to set their site to “any language”. I know that’s impossible to define, of course. I’m happy to settle for “every language with an ISO-639 code” including conlangs like Toki Pona (`tok`), which I want for my own site. However, I can’t find a good list anywhere. Wikipedia has one that only lists two-letter codes, and another that’s split across 26 pages with quite a bit of missing data. Ideally, I could just download a big JSON file of every assigned code, with each language’s native and English names, as well as labels for the macrolanguages and non-language codes like ‘zxx’. ## Fallback In addition to marking up the website and the posts in it, the language setting would be used for Denizen’s UI including phrases in the public blog like “reply to” or “last updated”. Since Denizen can’t possibly be translated to every ISO-639 language, it needs a fallback mechanism. There’s three tiers of sophistication for language matching, all of which I’d ideally like to support: * Simple BCP 47 matching, e.g. `en-US` to `en`. * Sister language matching, e.g. Dutch `nl` and Afrikaans `af`, or Turkish `tr` and Azerbaijani `az`. * Unrelated language matching. I could swear I saw Wikipedia using Turkish as a fallback for Laz `lzz`, but I can’t reproduce it. Nevertheless, it would be a desirable feature. For context, Laz and Turkish are not related linguistically (beyond loanwords), but most Laz speakers are bilingual in Turkish. Here, CLDR seems to promise a fix, but I couldn’t figure out how. ## Selecting language for individual posts The big social medias use heuristics to detect language. It’s expensive and goes wrong sometimes. Is there an alternative? If someone uses denizen to post mixed language articles (e.g. language learning material), will they annotate it all correctly? Can I somehow do it for them? Can I help them? Can I encourage them?

_Clickbait title: Can you write a bestselling book in Typst?_ A shiny new edition of Hypermedia Systems is out! Apart from incorporating typo and grammar fixes sent in by readers, the book was completely redesigned inside with a new layout. To achieve a much higher fidelity of design, we ported the book to Typst — the hot new LaTeX replacement. ## Why change what works? The first print release of HS was written in AsciiDoc. It seemed like a natural choice at the time – after all, many publishing houses trust AsciiDoc for their books. We learned the hard way that those publishers also have teams of engineers developing proprietary toolchains to produce their books. Vanilla AsciiDoc doesn’t actually have a way to produce good print output: * `asciidoctor-pdf` uses a rudimentary PDF generation library, and the output is just about readable, but not good – and you don’t have any good tools to change that. * the venerable Pandoc can output AsciiDoc, but oddly can’t read it – so we can’t compile our book to LaTeX. Your best option for producing a printable PDF, if you care at all about typesetting, is to compile to HTML, and print that to PDF. `asciidoctor-web-pdf` is a tool that wraps this workflow in a usable package, combined with Paged.js to polyfill the many features of CSS for print media that browsers don’t support. This is how we produced the first edition of HS, and it was a pain for multiple reasons. * Paged.js has bugs. One bug that was nasty to work around was one where using any CSS to modify the page numbers (say, resetting them after the front matter) would cause all the pages to revert to 0. I don’t know if this was fixed. * HTML with Paged.js is slow. The book has a little over 300 pages, and each page has a perceptible delay as it gets laid out. Even worse, I had to manually scroll the browser all the way to the bottom of the book before printing, or the back half of the book would be blank. Afterwards, printing would take what felt like a solid minute, though I never measured how long: Since presumably nobody at Firefox thought printing a page to PDF would be anything but instant, there’s no progress bar or any other indicator of success beyond the PDF appearing in the folder where it’s supposed to be. * The idiosyncrasies of this process meant that it only really worked on my machine, and I had to send PDFs back and forth with my co-authors and editor. The PDFs were absolutely massive and exceeded the Discord upload limit, so I uploaded each one to Google Drive. * Paged.js has more bugs – asides would refuse to break across pages while code blocks were all to eager to create widows and orphans – or disappear completely – which I had to work around in a whack-a-mole fashion. A few bugs seem to have made it into the final release. * How to set the background color of a table cell * AsciiDoc has facilities for indexing, but doesn’t support outputting them in HTML in any way. Worse, the mechanism I could have used to do it myself was deprecated _and removed_ before the replacement was implemented. Our editor ended up creating an index **by hand** , which was appended to the PDF. * You go through all this effort only to be rewarded with very mediocre typesetting, since browsers use greedy algorithms for laying out text for performance reasons, yielding far worse line breaking compared to the dynamic programming algorithms of TeX. Despite all this, we somehow produced a usable PDF. Most readers report being satisfied with the book’s design (Amazon’s failure to print a PDF consistently notwithstanding). The code that generated this PDF is not open source — initially, it was because we thought it might eat into print sales, but these days the real reason is that it might help someone else that tries to do this, and nobody should try to do this. (If you have any “why didn’t you just” recommendations, feel free to post them in the comments on link aggregators — they might help someone else — but don’t send them directly to me. I assure you, I’ve evaluated all options available even if I didn’t mention them here.) Speaking of link aggregators, ## Typst I first stumbled upon Typst when the author’s thesis was posted on the red site. Enthralled by its combination of quality typesetting, lightweight markup and an ergonomic scripting language tightly integrated, I knew that if I was ever part of another book project, I would use Typst. I rewrote my CV in it, as well as some documentation for an old project. This only confirmed my initial impression — Typst is a joy to use, and the scripting language is so good I wish it was a programming language in its own right. But I couldn’t use Typst for Hypermedia Systems, because Typst didn’t support HTML output. Imagine my excitement when a while ago, I realized Pandoc had added Typst support! ## Porting the manuscript As I mentioned before, Pandoc can’t read AsciiDoc. So how can I convert the 104537 words of HS into Typst? I was ready to do it by hand, but I decided to try something else first. One of AsciiDoc’s primary output formats is DocBook XML. I don’t know what it is or who uses it, but Pandoc can read it! I converted the book to DocBook XML, and then to Typst. Except AsciiDoc was buggy and generated invalid XML, so I had to fix that first. And then massage the resulting Typst a bit more. And apply any future changes to the AsciiDoc source to the Typst version. Still better than waiting for the PDF to render, though. ## The code blocks AsciiDoc has a unique style of code blocks where you can place markers in the code that refer to explanatory notes below the block – it looks like this: AsciiDoc code block with callout [source,python] ---- print("Hello, world!") <1> ---- <1> The canonical first program in any language. If you’ve read Hypermedia Systems, now you know how we made those. Typst doesn’t have any such feature, so with the great help of the Typst Discord community, I reimplemented it in about 50 lines of slightly cursed Typst code. ### code-callouts.typ: a Hypermedia Systems literate experience Before implementing code callouts, define a _label_ , which we will use to mark code blocks we have already annotated. This is to prevent an infinite loop that occurs when we use this function in a `show` rule. #let processed-label = <TypstCodeCallout-was-processed> The function `code-with-callouts` takes a code block and a function to render callouts. A simple default implementation returns the annotation number in square brackets in grayed-out sans-serif font. #let code-with-callouts( it /*: content(raw) */, callout-display: default-callout /*: (str) => content */ ) /*: content */ = { ... } Begin by checking if the code block has already been processed. ) /*: content */ = { if it.at("label", default: none) == processed-label { it } else { ... } } The dance with `it.at("label", default: none)` is a way to access the code block’s label without throwing an error if it has none. If it doesn’t have this label, the real fun begins. let (callouts, text: new-text) = parse-callouts(it.text) `parse-callouts` is a function that extracts the callouts from the code block. We pass it the `text` of the code block – the plain, un-highlighted code as a string. It returns that text with the callout markers removed, and the callouts themselves in the form of an array of callout numbers for each line. This is in a way the meat of the whole operation, but it’s also pretty mundane string processing code, so I won’t dwell on it. #let callout-pat = regex("<(\\d+)>(\\n|$)") #let parse-callouts( code-text /*: str */ ) /*: (callouts: array(array(int)), text: str) */ = { let callouts /*: array(array(int)) */ = () let new-text = "" for text-line in code-text.split("\n") { let match = text-line.match(callout-pat) if match != none { callouts.push((int(match.captures.at(0)),)) new-text += text-line.slice(0, match.start) } else { callouts.push(()) new-text += text-line } new-text += "\n" } (callouts: callouts, text: new-text) } Now that we have the callouts, we can render them. We do this by abusing show rules. A `raw.line` (object representing a line of raw text) knows its line number, so we can use that to look up the callouts for that line. We then iterate over the callouts and render them with the provided `callout-display` function. show raw.line: it => { it let callouts-of-line = callouts.at(it.number - 1, default: ()) for callout in callouts-of-line { callout-display(callout) } } You might assume as I did that this would be the end of it and we could just return the code block, but there’s a catch. The code block still has the `<1>` markers in it, and we need to remove them. To do this, we create a new code block with the same attributes as the original, but with our `new-text` as the body. The fact that we create a brand new code block instead of returning the one we were given is what causes the infinite loop I mentioned earlier, so we apply the processed label. let fields = it.fields() let _ = fields.remove("text") let _ = fields.remove("lines") let _ = fields.remove("theme") [#raw(..fields, new-text)#processed-label] This is the code that powers the code blocks in the new edition of Hypermedia Systems. All you need to do is add a `show raw.where(block: true): code-with-callouts` to your Typst document, and you can have code blocks with callouts too! The actual text of the callouts is just a list written below the code block. If I was implementing this from scratch, I might have had the `code-with-callouts` accept the content of the callouts as well and match them up to the code by text labels, but AsciiDoc uses numbers and makes you match them up manually, so I did too. ## The index Typst has no indexing functionality by default, but its scripting and introspection capabilities are powerful enough to implement indexing at the library level. In fact, an indexing library already exists in the form of in-dexter, which is small enough that I made a modified version of it to have full control over the index’s appearance and to add support for hierarchical terms. ## The print book So far, I’ve talked about the most technically interesting parts of the migration and not the most important part: the actual product. Using a capable typesetting system instead of Print to PDF meant I could tweak almost every aspect of the book’s design, and the software would actually cooperate. The new Hypermedia Systems uses indented paragraphs instead of block paragraphs. That is, the first line of a paragraph is indented, and there’s no space between paragraphs. What’s so special about that? You can definitely do that in HTML and CSS, right? Not if you want to do it well. Traditional style and my own preference dictate that indentations _separate_ paragraphs, so the first paragraph after a non-paragraph element (like a heading or a list) should not be indented. This is a common typographic convention, but to do it in CSS is impossible in the general case without dozens, maybe hundreds of selectors to handle every special case. In Typst, you just give paragraphs a `first-line-indent`, and they know when to use it. It’s a similar story with emphasis. The `<em>` element in HTML is italicized by default, and similarly for `emph` in Typst. When emphasis is used in an element that’s already italicized, the emphasis is shown by reversing the italics and making the emphasized text upright. With CSS, you might restyle `<em>` in all those contexts (and remember to revert it as needed). _Container style queries_ might make this easier in CSS. In Typst, it’s the default behavior. Other than styling features, Typst is also just better at typesetting, using line breaking algorithms similar to TeX. Especially compared to browsers with their greedy algorithms and weirdly small hyphenation dictionaries, Typst prose is a joy both to look at and to read. I want HTML and CSS to be as capable in print as they are on screens. Unfortunately, browsers don’t seem as interested – the CSS Paged Media module remains largely unimplemented. While I would love a future where the same HTML code can back both online and print books (and PDF is relegated to an archival format), building separate web and print books from a Typst manuscript seems to be the best strategy for now. ## The web book Typst can produce a wonderful PDF for us, but we also need to publish the book online. While HTML output is a planned feature for the Typst compiler, no implementation currently exists. This is why I didn’t propose Typst while we were producing the first edition, before Pandoc gained support for reading and writing Typst. The old hypermedia.systems was built with the static site generator Lume with an AsciiDoc plugin I wrote. For the new book, I wanted to test out [Müteferrika][], an online book publishing toolkit I’m building. The advantage of Müteferrika is that it understands the structure of a book (parts, chapters, frontmatter etc.). The disadvantage is that it was and still is an unfinished mess, so I had to fix a lot of bugs to get it to work for Hypermedia Systems. The build process had to convert the Typst source to HTML, which I planned to do with Pandoc. Unfortunately, Pandoc doesn’t have full parity with the mainline Typst compiler, and spat out many errors when I first tried to convert the book. The fix for most of these was to replace direct property accesses with `at` calls and add null checks as needed. ## Conclusion The migration from AsciiDoc to Typst was a resounding success. I had to fudge a lot of tooling around book production, since Typst is still a young project and focused more on research papers than books, but the ecosystem is constantly growing and improving. Shiroa is a promising project for publishing web books with Typst, for example. The web book is now live at hypermedia.systems. It has (as far as we could tell) everything the old web book had, and more. There’s a button at the bottom of each page where you can customize the colors of the website to your liking. For the print version, we’re switching from Amazon KDP to Lulu, which will hopefully result in more consistent printing. * Read the new Hypermedia Systems online * Get the EPUB for your e-reader * Buy the new print release on Lulu * Buy the Kindle release on Amazon * Buy the old hardcover while it’s still available * How to set the background color of a table cell

In the latest release of htmx, you can add event listeners to elements with `hx-on`: <form hx-on::beforeRequest="addCsrfToken(event);"> For all the other `hx-` attributes, we use CSS attribute selectors. However, with `hx-on`, the attribute name is not fixed as it contains the event. CSS attribute selectors support wildcards on the _value_ of attributes, but not the name: [hx-trigger] /* common and normal */ [href^="https://dz4k.com"] /* "starts with" operator */ [^ĥx-on:] /* not a thing */ ## XXMLPathPath Language XPath is a query language for extracting information from XML(-like) documents. Its main use cases are XSLT and parsing API responses. The XPath language is significantly more expressive than CSS, making it possible to traverse the XML tree in any direction, filter nodes based on arbitrary predicates, and select any kind of node (including comments, text nodes, and individual attributes). Our non-existent CSS attribute could be written as follows: //@*[starts-with(name(), "hx-on:")] This post is not supposed to be an XPath tutorial, but I’ll break this one down: `//` traverse the document (in CSS, this is the default) `@*` find any attribute (mnemonic: **at** -tribute) `[ ... ]` where… `starts-with(name(), "hx-on:")` its name starts with `"hx-on:"` CSS selectors don’t have these kinds of features, and it has good reasons not to. CSS has strict performance requirements – to the point that “CSS optimization” is generally not a thing – and selectors that offer more control could make slow selectors possible. In addition, CSS has well-defined specificity rules, whereas XPath does not. However, while these features make CSS great for stylesheets, CSS selectors are also the most common way to find DOM elements in JavaScript code and lacking in that regard. Many libraries which extend HTML do so by traversing the entire document and finding elements manually. This is often not needed since, if you didn’t know, **XPath is built into browsers.** ## document.evaluate The `document.evaluate` API is somewhat archaic, partly because it was designed for talking to XML APIs over `XMLHTTPRequest`. Here’s a DOM-friendly wrapper: function* xpath(...args) { let path, root = document; if (args.length > 1) [root, path] = args; else [path] = args; const nodeIterator = document.evaluate( path, root, null, XPathResult.UNORDERED_NODE_ITERATOR_TYPE, null, ); for ( let node = nodeIterator.iterateNext(); node != null; node = nodeIterator.iterateNext() ) { yield node; } } // TypeScript declaration function xpath(path: string): Iterable<Node>; function xpath(root: Element, path: string): Iterable<Node>;

Bamboo Paper is an app released by Wacom in 2014, presumably to promote their Bamboo series of capacitive pens. It features 6 pens, 3 of which need to be unlocked via in-app purchases. Notes are stored in the proprietary WILL format, and can only be synced to Wacom’s own Inkspace cloud service. Otherwise, backing up notes involves exporting each notebook manually to PDF. Due to these restrictions, I don’t use it that often to actually take notes. However, I can’t think of a single note taking app that I enjoy as much. It’s not because of the minimalist interface, or the satisfying action of the pressure-sensitive brushes and pens, though I love both of those things. Only recently have I realized what draws me to this app. Notebooks in Bamboo Paper have the same size as your device’s display. When you rotate the device, the UI rotates, but the notebook itself remains in the same orientation relative to the device. Zooming in is possible, but unwieldy, and you can’t zoom out beyond 100%. All these limitations mean that writing on Bamboo Paper feels like writing on a notebook, whereas using any other app feels like playing a game where your character writes in a notebook. When penstrokes are bound to the specific location on the tablet’s front glass where I put the pen, it makes spatial reasoning more natural, and spatial reasoning is the main advantage of taking handwritten notes over textual notes using handwriting recognition. Using Bamboo Paper with floating windows of other apps in front of it, as Samsung lets me do, is the closest feeling I’ll probably get to using a Microsoft Courier – at least, it would be if Paper had drag-and-drop support. It might be a 10 year old app but it did get an update last August, so come on Wacom… Despite all my searching, I haven’t found another app on Android that imitates Paper’s model. Some apps like Samsung Notes zoom and scroll like Word documents, while other apps have an infinite canvas that makes me feel disoriented. Since I want to store my notes somewhere that’s not too likely to disappear next year, I put up with the Word doc, but I’m putting this out there in case someone points me to the perfect virtual notebook

We went on a walk in the woods today, and Biber came with us. We came across a cat that looks just like her. He was a male cat. Biber soon caught up and they started to interact: I’m not an animal behaviorist and I couldn’t tell you if they were flirting or getting ready to fight.

I recently published the first pre-release version of Fruit Credits on Flathub. (I then immediately published two more because the reviewer discovered a bug). ## why: boring life stuff After quitting my job and realizing it would be a while before I found a new one1], I realized I might need to be a bit more responsible with money. Naturally, I downloaded [hledger. I’d tried to use hledger before but couldn’t make a habit of it. I was concerned that my new attempt could be a procrastination mechanism, but to my surprise, it was actually massively helpful. Even after one month, I felt more in control of my life than ever before, and it’s kept me in the black since. It was hard for me at first to see the value of keeping your own books when banks already record and present a history of transactions. Right now, the benefits I experience are: * **Everything is in one place.** Most people keep multiple accounts at multiple banks. Banks are somehow all shit at application engineering, or even just providing usable data export. Before picking up hledger, I straight-up had no idea _how much money I had_. * **The data is queryable.** I’m talking super basic stuff – I’ve not even scratched the surface of what one can do with plain text accounting, yet it’s still massively valuable to ask the computer “how much did I spend on A since B?” * **The usual reasons.** It’s free software operating on free file formats. I can see my account balances without popups offering me loans I can’t pay back. I started by typing transactions in the ledger format directly in a text editor. Then, I started using `hledger add`, which has autocompletion capabilities. Unfortunately, both of these were too clunky for me – maybe it takes getting used to, but I kept making typos in `hledger add` and fumbling as I tried to undo them. I imagined a GUI for adding transactions quickly that wouldn’t require me to enter things in order. This vision would eventually feature-creep itself into a GUI version of hledger. ## how: exciting computer stuff I’d heard Tauri was Electron but good, which was an attractive proposition to a web developer. It took me about a week to give up – it turns out making a web app look good as a desktop app is harder than Discord makes it look. Being a GNOME user, I was inspired by the many GNOME Circle apps to build something with GTK4 and libadwaita. I fired up GNOME Builder and spent about 2 days paralyzed by the language selector. After looking at code examples online, I decided Vala was the simplest option. I used GNOME Builder to scaffold the app, and used it to develop for a while. Eventually, I figured out how to build the app without it, and went back to Zed. It took me a while to get everything configured – for example, the default Builder template has a directory for translations with gettext, but there’s extra setup required to actually build and use them. However, between the build headaches, the programming inner loop was quite enjoyable, especially with the new-ish Blueprint UI language. ### vala is pretty cool, imo It seems that Rust is the recommendation for new GTK apps. Unfortunately, I never got into Rust, and I don’t think adding the GObject memory management model into the mix would make it grow on me. For the uninitiated, GObject is a subsystem that was originally part of GTK that provides object-oriented programming and reference-counted memory management features in C. Though GObject has some fun features like signals, there’s little fun to be had in writing classes in C – even with all the macro magic provided, class definitions are full of boilerplate code. In addition, the reference counting mechanism requires you to `g_object_ref ()` and unref your objects manually. One feature of GObject is the ability to generate bindings to other languages, including ones with actual OOP features, which allows GTK apps to be implemented in a variety of languages. However, these bindings often suffer from various impedance mismatches between GObject and the languages’ own object models. Vala, however, is implemented with GObject in mind from the start. It has built-in syntax for features (like signals) Though I’ve fallen in love with Vala, I can see why other people might not enjoy it. It has unfixed compiler bugs, and a standard library that was written for C. Most frustratingly though, refcounting is not as ergonomic as full tracing garbage collection, and when using C libraries (including GObject based ones!) that expect you to be comfortable with creative usage of memory, you can run into hard-to-debug segfaults. The de facto debugger for Vala is gdb, and it has no Vala support. You have to debug through the generated C. I had no gdb experience. Thankfully, GLib has a built in logging framework that makes printf-style debugging quite comfortable. What draws me to Vala is the feeling of consideration that it exudes from every orifice. I would constantly discover that whatever I was trying to do, the Vala designers knew I’d need to do it, and had implemented a feature to make it easier. ### flatpacking haskell A GTK app written in Vala and compiled with Meson is the happiest of happy paths for Flatpak. Unfortunately, hledger, which Fruit Credits necessarily depends on, is written in Haskell. Flathub requires not only that everything is built from source, but also that all necessary sources can be downloaded before the build process (which is sandboxed). This is antithetical to the model used by Haskell (and npm, and cargo, and many others) where a build tool downloads dependencies and discovers transitive dependencies at the same time. After a few days of digging through outdated resources, I understood that a tool called cabal-flatpak was the best way to generate Flatpak manifests that fetch and compile Haskell libraries in the right order. Unfortunately, despite seemingly being under active maintenance, the tool had bugs, and no discernible way to report them. Since I don’t know Haskell, I couldn’t maintain a fork, so I wrote some jq and shell to work around the bugs. Upon submitting my app to Flathub, I found out I was something of a pioneer: > * **bbhtt:** why are these needed and why do you need to do this manually? > * **dz4k:** The haskell modules in the manifest are based on the output of cabal-flatpak and the article How to Flatpak a Haskell App into Flathub. Are there more up-to-date resources for bundling Haskell programs? > * **bbhtt:** No, we don’t have haskell apps. This is probably the second or third one ever. > – Add com.dz4k.FruitCredits by dz4k · Pull Request #5731 · flathub/flathub fun. ## what: thermonuclear war stuff Though Fruit Credits is pretty barebones compared to what hledger can do, it’s crossed from “dogfooding” to actually usable software for my use cases. Queries and transaction input work well. I’m currently working on a setup interface to create a new ledger file from scratch, as well as making the app more beginner-friendly in general. I’d like to add some way of editing and deleting transactions, and some reporting features. My usage of hledger is pretty limited in the grand scheme of things, so I won’t be able to cover every use case by myself. I’d love it if people gave Fruit Credits a try, even if just to tell me how it couldn’t read their journal file. * * * * get it on Flathub * get the code on Codeberg * complain about it on Codeberg * look at the website on the website * * * 1. I’m fine – I live with my parents, have no dependents and still have some income via Hypermedia Systems. I would still very much like a job though, which is why I invested in highly demanded skills as Vala programming and Haskell Flatpak packaging. ↩︎

I’ve been using a Nothing Phone (2a) for a while now. I love the built in widgets, but there’s not enough of them, and most third-party widgets stick out like a sore thumb. Not even any of the Nothing-themed KWGT packs on the Play Store match. So I tried my own hand at it, and with not much effort, I was able to make these two: To my eyes, that’s an exact match! The date and next event widgets are mine. If you just want to have these widgets, you can download them. If you want to make your own, read on: * VeryFewThings Date * VeryFewThings Next Event (this is my first time using Nextcloud’s sharing links, so let me know if there’s any issues) I’ll probably be updating these widgets and this guide, as well as making more widgets. In particular, some of the formulas in the next event widget are a bit messy. ## Background The background of the widget is a Rectangle. Use formulas for the width, height and paint color (long press each property, and tap the calculator icon). The formulas are: * **Width** : `$si(rwidth)$` * **Height** : `$si(rheight)$` * **Paint** : `$si(syscn1, if(si(darkmode), 10, 93)$` I used a corner radius of 32, though that’s my eyeball measurement and might not be exact. For 2x1 and 1x2 widgets, you can use an absurd number like 9999 to make them pill-shaped. ## Layout The widget should have 32 points of padding. Some Nothing widgets like the digital clock change based on their size. My date widget does the same thing. You can use these formulas to detect if the widget is wide or tall: * **Wide** : `$si(rwidth) > 1.8 * si(rheight)$` * **Tall** : `$si(rheight) > 1.8 * si(rwidth)$` If neither of these are true, the widget is square. ## Text Kustom can access your device’s system fonts, including Nothing’s custom fonts. Dotted NDot (Variant: 57 for big text, 55 for small text) Monospace Lettera Mono Serif Ntype82 Sans-serif Roboto (weight 300) Nothing uses the NDot font exclusively in all caps, so I recommend matching that. * **Text color** : `$si(syscn1, if(si(darkmode), 100, 0))`

so insane to me that people advocate for distro packaging over flatpak hmm yes i love being unable to use up-and-coming software thet nobody packages yet and making it harder for new distros to get off the ground vertical monopolies are good when open source i guess the concept of software package management existing is a failure one (1) software should be expressible as one thing

I’m building Denizen (https://denizen.dz4k.com/), a personal website system that is IndieWeb and Fediverse compatible. Source code: https://codeberg.org/dz4k/denizen

## Background Denizen is a CMS for personal websites. I want to work on multi-language support – people should be able to make websites and posts in any language they want. (My own blog is in English, Turkish, and Toki Pona – so this is a blocker for eating my dog food). Every aspect of this feels like something other people must have already solved, so I wanted to publicly ask if anyone has the secret right answers. ## Language picker Requirements: Users should be able to set their site to “any language”. I know that’s impossible to define, of course. I’m happy to settle for “every language with an ISO-639 code” including conlangs like Toki Pona (`tok`), which I want for my own site. However, I can’t find a good list anywhere. Wikipedia has one that only lists two-letter codes, and another that’s split across 26 pages with quite a bit of missing data. Ideally, I could just download a big JSON file of every assigned code, with each language’s native and English names, as well as labels for the macrolanguages and non-language codes like ‘zxx’. ## Fallback In addition to marking up the website and the posts in it, the language setting would be used for Denizen’s UI including phrases in the public blog like “reply to” or “last updated”. Since Denizen can’t possibly be translated to every ISO-639 language, it needs a fallback mechanism. There’s three tiers of sophistication for language matching, all of which I’d ideally like to support: * Simple BCP 47 matching, e.g. `en-US` to `en`. * Sister language matching, e.g. Dutch `nl` and Afrikaans `af`, or Turkish `tr` and Azerbaijani `az`. * Unrelated language matching. I could swear I saw Wikipedia using Turkish as a fallback for Laz `lzz`, but I can’t reproduce it. Nevertheless, it would be a desirable feature. For context, Laz and Turkish are not related linguistically (beyond loanwords), but most Laz speakers are bilingual in Turkish. Here, CLDR seems to promise a fix, but I couldn’t figure out how. ## Selecting language for individual posts The big social medias use heuristics to detect language. It’s expensive and goes wrong sometimes. Is there an alternative? If someone uses denizen to post mixed language articles (e.g. language learning material), will they annotate it all correctly? Can I somehow do it for them? Can I help them? Can I encourage them?

_Clickbait title: Can you write a bestselling book in Typst?_ A shiny new edition of Hypermedia Systems is out! Apart from incorporating typo and grammar fixes sent in by readers, the book was completely redesigned inside with a new layout. To achieve a much higher fidelity of design, we ported the book to Typst — the hot new LaTeX replacement. ## Why change what works? The first print release of HS was written in AsciiDoc. It seemed like a natural choice at the time – after all, many publishing houses trust AsciiDoc for their books. We learned the hard way that those publishers also have teams of engineers developing proprietary toolchains to produce their books. Vanilla AsciiDoc doesn’t actually have a way to produce good print output: * `asciidoctor-pdf` uses a rudimentary PDF generation library, and the output is just about readable, but not good – and you don’t have any good tools to change that. * the venerable Pandoc can output AsciiDoc, but oddly can’t read it – so we can’t compile our book to LaTeX. Your best option for producing a printable PDF, if you care at all about typesetting, is to compile to HTML, and print that to PDF. `asciidoctor-web-pdf` is a tool that wraps this workflow in a usable package, combined with Paged.js to polyfill the many features of CSS for print media that browsers don’t support. This is how we produced the first edition of HS, and it was a pain for multiple reasons. * Paged.js has bugs. One bug that was nasty to work around was one where using any CSS to modify the page numbers (say, resetting them after the front matter) would cause all the pages to revert to 0. I don’t know if this was fixed. * HTML with Paged.js is slow. The book has a little over 300 pages, and each page has a perceptible delay as it gets laid out. Even worse, I had to manually scroll the browser all the way to the bottom of the book before printing, or the back half of the book would be blank. Afterwards, printing would take what felt like a solid minute, though I never measured how long: Since presumably nobody at Firefox thought printing a page to PDF would be anything but instant, there’s no progress bar or any other indicator of success beyond the PDF appearing in the folder where it’s supposed to be. * The idiosyncrasies of this process meant that it only really worked on my machine, and I had to send PDFs back and forth with my co-authors and editor. The PDFs were absolutely massive and exceeded the Discord upload limit, so I uploaded each one to Google Drive. * Paged.js has more bugs – asides would refuse to break across pages while code blocks were all to eager to create widows and orphans – or disappear completely – which I had to work around in a whack-a-mole fashion. A few bugs seem to have made it into the final release. * How to set the background color of a table cell * AsciiDoc has facilities for indexing, but doesn’t support outputting them in HTML in any way. Worse, the mechanism I could have used to do it myself was deprecated _and removed_ before the replacement was implemented. Our editor ended up creating an index **by hand** , which was appended to the PDF. * You go through all this effort only to be rewarded with very mediocre typesetting, since browsers use greedy algorithms for laying out text for performance reasons, yielding far worse line breaking compared to the dynamic programming algorithms of TeX. Despite all this, we somehow produced a usable PDF. Most readers report being satisfied with the book’s design (Amazon’s failure to print a PDF consistently notwithstanding). The code that generated this PDF is not open source — initially, it was because we thought it might eat into print sales, but these days the real reason is that it might help someone else that tries to do this, and nobody should try to do this. (If you have any “why didn’t you just” recommendations, feel free to post them in the comments on link aggregators — they might help someone else — but don’t send them directly to me. I assure you, I’ve evaluated all options available even if I didn’t mention them here.) Speaking of link aggregators, ## Typst I first stumbled upon Typst when the author’s thesis was posted on the red site. Enthralled by its combination of quality typesetting, lightweight markup and an ergonomic scripting language tightly integrated, I knew that if I was ever part of another book project, I would use Typst. I rewrote my CV in it, as well as some documentation for an old project. This only confirmed my initial impression — Typst is a joy to use, and the scripting language is so good I wish it was a programming language in its own right. But I couldn’t use Typst for Hypermedia Systems, because Typst didn’t support HTML output. Imagine my excitement when a while ago, I realized Pandoc had added Typst support! ## Porting the manuscript As I mentioned before, Pandoc can’t read AsciiDoc. So how can I convert the 104537 words of HS into Typst? I was ready to do it by hand, but I decided to try something else first. One of AsciiDoc’s primary output formats is DocBook XML. I don’t know what it is or who uses it, but Pandoc can read it! I converted the book to DocBook XML, and then to Typst. Except AsciiDoc was buggy and generated invalid XML, so I had to fix that first. And then massage the resulting Typst a bit more. And apply any future changes to the AsciiDoc source to the Typst version. Still better than waiting for the PDF to render, though. ## The code blocks AsciiDoc has a unique style of code blocks where you can place markers in the code that refer to explanatory notes below the block – it looks like this: AsciiDoc code block with callout [source,python] ---- print("Hello, world!") <1> ---- <1> The canonical first program in any language. If you’ve read Hypermedia Systems, now you know how we made those. Typst doesn’t have any such feature, so with the great help of the Typst Discord community, I reimplemented it in about 50 lines of slightly cursed Typst code. ### code-callouts.typ: a Hypermedia Systems literate experience Before implementing code callouts, define a _label_ , which we will use to mark code blocks we have already annotated. This is to prevent an infinite loop that occurs when we use this function in a `show` rule. #let processed-label = <TypstCodeCallout-was-processed> The function `code-with-callouts` takes a code block and a function to render callouts. A simple default implementation returns the annotation number in square brackets in grayed-out sans-serif font. #let code-with-callouts( it /*: content(raw) */, callout-display: default-callout /*: (str) => content */ ) /*: content */ = { ... } Begin by checking if the code block has already been processed. ) /*: content */ = { if it.at("label", default: none) == processed-label { it } else { ... } } The dance with `it.at("label", default: none)` is a way to access the code block’s label without throwing an error if it has none. If it doesn’t have this label, the real fun begins. let (callouts, text: new-text) = parse-callouts(it.text) `parse-callouts` is a function that extracts the callouts from the code block. We pass it the `text` of the code block – the plain, un-highlighted code as a string. It returns that text with the callout markers removed, and the callouts themselves in the form of an array of callout numbers for each line. This is in a way the meat of the whole operation, but it’s also pretty mundane string processing code, so I won’t dwell on it. #let callout-pat = regex("<(\\d+)>(\\n|$)") #let parse-callouts( code-text /*: str */ ) /*: (callouts: array(array(int)), text: str) */ = { let callouts /*: array(array(int)) */ = () let new-text = "" for text-line in code-text.split("\n") { let match = text-line.match(callout-pat) if match != none { callouts.push((int(match.captures.at(0)),)) new-text += text-line.slice(0, match.start) } else { callouts.push(()) new-text += text-line } new-text += "\n" } (callouts: callouts, text: new-text) } Now that we have the callouts, we can render them. We do this by abusing show rules. A `raw.line` (object representing a line of raw text) knows its line number, so we can use that to look up the callouts for that line. We then iterate over the callouts and render them with the provided `callout-display` function. show raw.line: it => { it let callouts-of-line = callouts.at(it.number - 1, default: ()) for callout in callouts-of-line { callout-display(callout) } } You might assume as I did that this would be the end of it and we could just return the code block, but there’s a catch. The code block still has the `<1>` markers in it, and we need to remove them. To do this, we create a new code block with the same attributes as the original, but with our `new-text` as the body. The fact that we create a brand new code block instead of returning the one we were given is what causes the infinite loop I mentioned earlier, so we apply the processed label. let fields = it.fields() let _ = fields.remove("text") let _ = fields.remove("lines") let _ = fields.remove("theme") [#raw(..fields, new-text)#processed-label] This is the code that powers the code blocks in the new edition of Hypermedia Systems. All you need to do is add a `show raw.where(block: true): code-with-callouts` to your Typst document, and you can have code blocks with callouts too! The actual text of the callouts is just a list written below the code block. If I was implementing this from scratch, I might have had the `code-with-callouts` accept the content of the callouts as well and match them up to the code by text labels, but AsciiDoc uses numbers and makes you match them up manually, so I did too. ## The index Typst has no indexing functionality by default, but its scripting and introspection capabilities are powerful enough to implement indexing at the library level. In fact, an indexing library already exists in the form of in-dexter, which is small enough that I made a modified version of it to have full control over the index’s appearance and to add support for hierarchical terms. ## The print book So far, I’ve talked about the most technically interesting parts of the migration and not the most important part: the actual product. Using a capable typesetting system instead of Print to PDF meant I could tweak almost every aspect of the book’s design, and the software would actually cooperate. The new Hypermedia Systems uses indented paragraphs instead of block paragraphs. That is, the first line of a paragraph is indented, and there’s no space between paragraphs. What’s so special about that? You can definitely do that in HTML and CSS, right? Not if you want to do it well. Traditional style and my own preference dictate that indentations _separate_ paragraphs, so the first paragraph after a non-paragraph element (like a heading or a list) should not be indented. This is a common typographic convention, but to do it in CSS is impossible in the general case without dozens, maybe hundreds of selectors to handle every special case. In Typst, you just give paragraphs a `first-line-indent`, and they know when to use it. It’s a similar story with emphasis. The `<em>` element in HTML is italicized by default, and similarly for `emph` in Typst. When emphasis is used in an element that’s already italicized, the emphasis is shown by reversing the italics and making the emphasized text upright. With CSS, you might restyle `<em>` in all those contexts (and remember to revert it as needed). _Container style queries_ might make this easier in CSS. In Typst, it’s the default behavior. Other than styling features, Typst is also just better at typesetting, using line breaking algorithms similar to TeX. Especially compared to browsers with their greedy algorithms and weirdly small hyphenation dictionaries, Typst prose is a joy both to look at and to read. I want HTML and CSS to be as capable in print as they are on screens. Unfortunately, browsers don’t seem as interested – the CSS Paged Media module remains largely unimplemented. While I would love a future where the same HTML code can back both online and print books (and PDF is relegated to an archival format), building separate web and print books from a Typst manuscript seems to be the best strategy for now. ## The web book Typst can produce a wonderful PDF for us, but we also need to publish the book online. While HTML output is a planned feature for the Typst compiler, no implementation currently exists. This is why I didn’t propose Typst while we were producing the first edition, before Pandoc gained support for reading and writing Typst. The old hypermedia.systems was built with the static site generator Lume with an AsciiDoc plugin I wrote. For the new book, I wanted to test out [Müteferrika][], an online book publishing toolkit I’m building. The advantage of Müteferrika is that it understands the structure of a book (parts, chapters, frontmatter etc.). The disadvantage is that it was and still is an unfinished mess, so I had to fix a lot of bugs to get it to work for Hypermedia Systems. The build process had to convert the Typst source to HTML, which I planned to do with Pandoc. Unfortunately, Pandoc doesn’t have full parity with the mainline Typst compiler, and spat out many errors when I first tried to convert the book. The fix for most of these was to replace direct property accesses with `at` calls and add null checks as needed. ## Conclusion The migration from AsciiDoc to Typst was a resounding success. I had to fudge a lot of tooling around book production, since Typst is still a young project and focused more on research papers than books, but the ecosystem is constantly growing and improving. Shiroa is a promising project for publishing web books with Typst, for example. The web book is now live at hypermedia.systems. It has (as far as we could tell) everything the old web book had, and more. There’s a button at the bottom of each page where you can customize the colors of the website to your liking. For the print version, we’re switching from Amazon KDP to Lulu, which will hopefully result in more consistent printing. * Read the new Hypermedia Systems online * Get the EPUB for your e-reader * Buy the new print release on Lulu * Buy the Kindle release on Amazon * Buy the old hardcover while it’s still available * How to set the background color of a table cell

I recently published the first pre-release version of Fruit Credits on Flathub. (I then immediately published two more because the reviewer discovered a bug). ## why: boring life stuff After quitting my job and realizing it would be a while before I found a new one1], I realized I might need to be a bit more responsible with money. Naturally, I downloaded [hledger. I’d tried to use hledger before but couldn’t make a habit of it. I was concerned that my new attempt could be a procrastination mechanism, but to my surprise, it was actually massively helpful. Even after one month, I felt more in control of my life than ever before, and it’s kept me in the black since. It was hard for me at first to see the value of keeping your own books when banks already record and present a history of transactions. Right now, the benefits I experience are: * **Everything is in one place.** Most people keep multiple accounts at multiple banks. Banks are somehow all shit at application engineering, or even just providing usable data export. Before picking up hledger, I straight-up had no idea _how much money I had_. * **The data is queryable.** I’m talking super basic stuff – I’ve not even scratched the surface of what one can do with plain text accounting, yet it’s still massively valuable to ask the computer “how much did I spend on A since B?” * **The usual reasons.** It’s free software operating on free file formats. I can see my account balances without popups offering me loans I can’t pay back. I started by typing transactions in the ledger format directly in a text editor. Then, I started using `hledger add`, which has autocompletion capabilities. Unfortunately, both of these were too clunky for me – maybe it takes getting used to, but I kept making typos in `hledger add` and fumbling as I tried to undo them. I imagined a GUI for adding transactions quickly that wouldn’t require me to enter things in order. This vision would eventually feature-creep itself into a GUI version of hledger. ## how: exciting computer stuff I’d heard Tauri was Electron but good, which was an attractive proposition to a web developer. It took me about a week to give up – it turns out making a web app look good as a desktop app is harder than Discord makes it look. Being a GNOME user, I was inspired by the many GNOME Circle apps to build something with GTK4 and libadwaita. I fired up GNOME Builder and spent about 2 days paralyzed by the language selector. After looking at code examples online, I decided Vala was the simplest option. I used GNOME Builder to scaffold the app, and used it to develop for a while. Eventually, I figured out how to build the app without it, and went back to Zed. It took me a while to get everything configured – for example, the default Builder template has a directory for translations with gettext, but there’s extra setup required to actually build and use them. However, between the build headaches, the programming inner loop was quite enjoyable, especially with the new-ish Blueprint UI language. ### vala is pretty cool, imo It seems that Rust is the recommendation for new GTK apps. Unfortunately, I never got into Rust, and I don’t think adding the GObject memory management model into the mix would make it grow on me. For the uninitiated, GObject is a subsystem that was originally part of GTK that provides object-oriented programming and reference-counted memory management features in C. Though GObject has some fun features like signals, there’s little fun to be had in writing classes in C – even with all the macro magic provided, class definitions are full of boilerplate code. In addition, the reference counting mechanism requires you to `g_object_ref ()` and unref your objects manually. One feature of GObject is the ability to generate bindings to other languages, including ones with actual OOP features, which allows GTK apps to be implemented in a variety of languages. However, these bindings often suffer from various impedance mismatches between GObject and the languages’ own object models. Vala, however, is implemented with GObject in mind from the start. It has built-in syntax for features (like signals) Though I’ve fallen in love with Vala, I can see why other people might not enjoy it. It has unfixed compiler bugs, and a standard library that was written for C. Most frustratingly though, refcounting is not as ergonomic as full tracing garbage collection, and when using C libraries (including GObject based ones!) that expect you to be comfortable with creative usage of memory, you can run into hard-to-debug segfaults. The de facto debugger for Vala is gdb, and it has no Vala support. You have to debug through the generated C. I had no gdb experience. Thankfully, GLib has a built in logging framework that makes printf-style debugging quite comfortable. What draws me to Vala is the feeling of consideration that it exudes from every orifice. I would constantly discover that whatever I was trying to do, the Vala designers knew I’d need to do it, and had implemented a feature to make it easier. ### flatpacking haskell A GTK app written in Vala and compiled with Meson is the happiest of happy paths for Flatpak. Unfortunately, hledger, which Fruit Credits necessarily depends on, is written in Haskell. Flathub requires not only that everything is built from source, but also that all necessary sources can be downloaded before the build process (which is sandboxed). This is antithetical to the model used by Haskell (and npm, and cargo, and many others) where a build tool downloads dependencies and discovers transitive dependencies at the same time. After a few days of digging through outdated resources, I understood that a tool called cabal-flatpak was the best way to generate Flatpak manifests that fetch and compile Haskell libraries in the right order. Unfortunately, despite seemingly being under active maintenance, the tool had bugs, and no discernible way to report them. Since I don’t know Haskell, I couldn’t maintain a fork, so I wrote some jq and shell to work around the bugs. Upon submitting my app to Flathub, I found out I was something of a pioneer: > * **bbhtt:** why are these needed and why do you need to do this manually? > * **dz4k:** The haskell modules in the manifest are based on the output of cabal-flatpak and the article How to Flatpak a Haskell App into Flathub. Are there more up-to-date resources for bundling Haskell programs? > * **bbhtt:** No, we don’t have haskell apps. This is probably the second or third one ever. > – Add com.dz4k.FruitCredits by dz4k · Pull Request #5731 · flathub/flathub fun. ## what: thermonuclear war stuff Though Fruit Credits is pretty barebones compared to what hledger can do, it’s crossed from “dogfooding” to actually usable software for my use cases. Queries and transaction input work well. I’m currently working on a setup interface to create a new ledger file from scratch, as well as making the app more beginner-friendly in general. I’d like to add some way of editing and deleting transactions, and some reporting features. My usage of hledger is pretty limited in the grand scheme of things, so I won’t be able to cover every use case by myself. I’d love it if people gave Fruit Credits a try, even if just to tell me how it couldn’t read their journal file. * * * * get it on Flathub * get the code on Codeberg * complain about it on Codeberg * look at the website on the website * * * 1. I’m fine – I live with my parents, have no dependents and still have some income via Hypermedia Systems. I would still very much like a job though, which is why I invested in highly demanded skills as Vala programming and Haskell Flatpak packaging. ↩︎

I’m building Denizen (https://denizen.dz4k.com/), a personal website system that is IndieWeb and Fediverse compatible. Source code: https://codeberg.org/dz4k/denizen

In the latest release of htmx, you can add event listeners to elements with `hx-on`: <form hx-on::beforeRequest="addCsrfToken(event);"> For all the other `hx-` attributes, we use CSS attribute selectors. However, with `hx-on`, the attribute name is not fixed as it contains the event. CSS attribute selectors support wildcards on the _value_ of attributes, but not the name: [hx-trigger] /* common and normal */ [href^="https://dz4k.com"] /* "starts with" operator */ [^ĥx-on:] /* not a thing */ ## XXMLPathPath Language XPath is a query language for extracting information from XML(-like) documents. Its main use cases are XSLT and parsing API responses. The XPath language is significantly more expressive than CSS, making it possible to traverse the XML tree in any direction, filter nodes based on arbitrary predicates, and select any kind of node (including comments, text nodes, and individual attributes). Our non-existent CSS attribute could be written as follows: //@*[starts-with(name(), "hx-on:")] This post is not supposed to be an XPath tutorial, but I’ll break this one down: `//` traverse the document (in CSS, this is the default) `@*` find any attribute (mnemonic: **at** -tribute) `[ ... ]` where… `starts-with(name(), "hx-on:")` its name starts with `"hx-on:"` CSS selectors don’t have these kinds of features, and it has good reasons not to. CSS has strict performance requirements – to the point that “CSS optimization” is generally not a thing – and selectors that offer more control could make slow selectors possible. In addition, CSS has well-defined specificity rules, whereas XPath does not. However, while these features make CSS great for stylesheets, CSS selectors are also the most common way to find DOM elements in JavaScript code and lacking in that regard. Many libraries which extend HTML do so by traversing the entire document and finding elements manually. This is often not needed since, if you didn’t know, **XPath is built into browsers.** ## document.evaluate The `document.evaluate` API is somewhat archaic, partly because it was designed for talking to XML APIs over `XMLHTTPRequest`. Here’s a DOM-friendly wrapper: function* xpath(...args) { let path, root = document; if (args.length > 1) [root, path] = args; else [path] = args; const nodeIterator = document.evaluate( path, root, null, XPathResult.UNORDERED_NODE_ITERATOR_TYPE, null, ); for ( let node = nodeIterator.iterateNext(); node != null; node = nodeIterator.iterateNext() ) { yield node; } } // TypeScript declaration function xpath(path: string): Iterable<Node>; function xpath(root: Element, path: string): Iterable<Node>;

Bamboo Paper is an app released by Wacom in 2014, presumably to promote their Bamboo series of capacitive pens. It features 6 pens, 3 of which need to be unlocked via in-app purchases. Notes are stored in the proprietary WILL format, and can only be synced to Wacom’s own Inkspace cloud service. Otherwise, backing up notes involves exporting each notebook manually to PDF. Due to these restrictions, I don’t use it that often to actually take notes. However, I can’t think of a single note taking app that I enjoy as much. It’s not because of the minimalist interface, or the satisfying action of the pressure-sensitive brushes and pens, though I love both of those things. Only recently have I realized what draws me to this app. Notebooks in Bamboo Paper have the same size as your device’s display. When you rotate the device, the UI rotates, but the notebook itself remains in the same orientation relative to the device. Zooming in is possible, but unwieldy, and you can’t zoom out beyond 100%. All these limitations mean that writing on Bamboo Paper feels like writing on a notebook, whereas using any other app feels like playing a game where your character writes in a notebook. When penstrokes are bound to the specific location on the tablet’s front glass where I put the pen, it makes spatial reasoning more natural, and spatial reasoning is the main advantage of taking handwritten notes over textual notes using handwriting recognition. Using Bamboo Paper with floating windows of other apps in front of it, as Samsung lets me do, is the closest feeling I’ll probably get to using a Microsoft Courier – at least, it would be if Paper had drag-and-drop support. It might be a 10 year old app but it did get an update last August, so come on Wacom… Despite all my searching, I haven’t found another app on Android that imitates Paper’s model. Some apps like Samsung Notes zoom and scroll like Word documents, while other apps have an infinite canvas that makes me feel disoriented. Since I want to store my notes somewhere that’s not too likely to disappear next year, I put up with the Word doc, but I’m putting this out there in case someone points me to the perfect virtual notebook

so insane to me that people advocate for distro packaging over flatpak hmm yes i love being unable to use up-and-coming software thet nobody packages yet and making it harder for new distros to get off the ground vertical monopolies are good when open source i guess the concept of software package management existing is a failure one (1) software should be expressible as one thing

We went on a walk in the woods today, and Biber came with us. We came across a cat that looks just like her. He was a male cat. Biber soon caught up and they started to interact: I’m not an animal behaviorist and I couldn’t tell you if they were flirting or getting ready to fight.