Lightnews — Scholar-powered news

Paddy Mullen @paddymullen.bsky.social · 4d

@marcogorelli.bsky.social Did excellent work on the rust plugin tutorial. The cookiecutter worked and came with an impressive CI setup that runs against MacOS, Windows, Linux, and multiple python versions.

2

Paddy Mullen @paddymullen.bsky.social · 4d

I wrote my first rust code and polars extension - pl_series_hash. It runs xx_hash over an entire series to get a single hash u64. Works on most all Polars data types including nested structs, it's fast. WIll be very useful for caching summary stats. github.com/paddymul/pl_...

GitHub - paddymul/pl_series_hash: a polars plugin to performantly cache series

a polars plugin to performantly cache series. Contribute to paddymul/pl_series_hash development by creating an account on GitHub.

github.com

1 1

Paddy Mullen @paddymullen.bsky.social · Aug 27

Providence RI still has a large jewelry making industry. Lots of small shops around there. Also, look up CNC tool dealers like Method Machine tools, give them a call and ask who they would recommend from their customers.

1

Paddy Mullen @paddymullen.bsky.social · Aug 21

BTW I'm mortified by the preview image that PyData or youtube chose. It was a live screen recording and I had to tab between multiple windows where buckaroo runs (VSCode, Jupyter, Google Colab, Marimo).

Paddy Mullen @paddymullen.bsky.social · Aug 21

Looking at the data with Buckaroo
- My talk from PyData Boston is now up on youtube.
www.youtube.com/watch?v=Htah...

Paddy Mullen - Looking at the Data with Buckaroo (PyData Boston 2025)

YouTube video by PyData

www.youtube.com

1

Paddy Mullen @paddymullen.bsky.social · Aug 11

Getting the script plumbed into consult-mode was a bear. customizing consult-mode requires returning a builder function, that returns another function, that is called by consult. None of the args are documented. #emacs

1

Paddy Mullen @paddymullen.bsky.social · Aug 11

This is really useful because I want to find whatever string i'm looking for, but I'd rather find it in my project's source first.

github.com/minad/consul...

How I plumbed in a custom find-grep scripts · minad consult · Discussion #1250

I wanted to write a custom find-grep script that did the following: returned matches from the preferred file extensions (for python -> py,js, ts, jsx, tsx) and an exhaustive exclude list then retur...

github.com

1

Paddy Mullen @paddymullen.bsky.social · Aug 11

After a bunch of work with prot, I got my custom find-grep script plumbed into emacs.

It searches a python project for matches in preferred files first (not site-packages, node_modules, py, js, tx extensions preferred). Then it runs a much more comprehensive search of those excluded directories

1

Paddy Mullen @paddymullen.bsky.social · Aug 11

Life pro tip. Next time you have to assemble IKEA furniture, buy yourself a 4mm hex t-handle. and a 4mm wera (or wiha) allen key. So much easier. www.kleintools.com/catalog/t-ha...

4 mm Hex Key, Journeyman™ T-Handle, 6-Inch - JTH6M4 | Klein Tools

Klein hex-keys are the tools professionals cannot afford to be without. Heat-treated and tempered for superior strength and durability. Klein hex-keys are designed for a precise fit in sockets, preven...

www.kleintools.com

1

Paddy Mullen @paddymullen.bsky.social · Aug 9

Nope, some how my new enum_dataframe 3-5xd the memory usage in python. Moral of the story is that polars and parquet have some seriously impressive engineering behind them.

Paddy Mullen @paddymullen.bsky.social · Aug 9

After some more work, I outputted the entire dataframe, original columns + sparse where necessary. Then I looked at the size, a couple megs more than the original parquet... No big deal, my python memory usage should be less wihtout all of those strings right?

1

Paddy Mullen @paddymullen.bsky.social · Aug 9

sparse values. When I changed to an enum per column, the file went down to 50 MB. This was suspiciously low. I realized my new dataframe only included sparse columns, not regular columns + sparse columns.

1

Paddy Mullen @paddymullen.bsky.social · Aug 9

UGH, I thought I was so slick. I had a nice function that used polars to convert a csv to parquet with enum columns for sparsely populated columns. I thought it saved about 30% on a 700 meg parquet file (10G csv). Then I dug some more and found out that I was encoding all possible sparse value

1

Paddy Mullen @paddymullen.bsky.social · Aug 1

Some content is on Youtube and few other places (heavy equipment, dirt bikes...). But I realized generally I have been searching youtube for more topics because I trust the results more than SEO spam... And this is after I switched to ddg. Obviously there are shills on yt, but beats listicles.

Paddy Mullen @paddymullen.bsky.social · Jul 5

Link? What is that site

Paddy Mullen @paddymullen.bsky.social · Jul 5

I need to write more. I have basically no online following. Inhale gotten better distribution through medium. I do try to set my articles to not be behind a paywall

Paddy Mullen @paddymullen.bsky.social · Jun 17

There is some good stuff on facebook. Fun post from the mainframers group about an IBM Sytem 370 card for the IBM PS/2. www.facebook.com/share/p/17tx...

Some people may ask, "What is a mainframe?". Well, the answer varies, everything from the multi-ton beasts, to the smaller versions. How small? May I present one of the mainframes I used to operate/administer, an IBM P/370. That's a complete IBM S/370 processor, including a full 16M of memory, on a single MCA card, which fit into a PS/2 model 95 system, and which ran VM/SP 5.
Once, just for grins and giggles, I IPLed a second level VM system. Then, for more grins and giggles, I IPLed a third level VM system. But, it drove me crazy trying to remember which prompt to use for which system. Still, it demonstrated how thorough of an architecture implementation the card was.
https://en.wikipedia.org/.../PC-based_IBM_mainframe...
It was great having my own personal mainframe, which I used for product development and testing. It allowed me to do test installs of software on a real mainframe.
Anyway, since some of y'all are posting really great pictures of machine rooms and big mainframes, I thought I'd offer a glimpse of what may be one of the smallest mainframes. 🙂
Oh, yeah, there was a follow-on product, the P/390, which was a S/390 system on a card. I lusted after one for many years, but was never able to justify it. And, there were a couple of proceeding products, the 7437, which was a full S/370 in a box about the size of a PS/2 model 95. There was also the XT/370, which was a PC/XT, with a card-set in it which was a partial implementation of the S/370 instruction set (one of which I happen to personally own).

I was the team leader for the 7437 project and designed most of the CPU. It used an IBM bit slice (similar to the AMD 2900), and an IBM chip called FLAINE which implemented the floating point instructions. The rest of the CPU was built using mostly 7400F and 7400LS chips. The CPU had a writeable control store of 8K words of 96 bits each. Initially, we implemented dynamic address translation in 7400 logic, but then one of our engineers, Tak Ng, designed a CMOS chip to do the translation, and that went into the product-level release. The chip replaced a whole card worth of 7400 logic. The 7437 consisted of 6 cards, each approximately 9x12". The cards were the Instruction Unit, Execution Unit, Control Store, PS2 Interface and two 8 MB memory cards. Below is a photo of the writeable control store card. It's the only piece of hardware I still have.
The XT/370 was based on a Motorola 68000 that IBM paid Motorola to modify to execute 370 instructions. The chip only could run CMS and not CP as it did not support supervisor state. Instead CP functions were provided by X86 code running on the PC.
The P/370 and the P/390 chips were also designed by Tak Ng.
The 7437, P/370 and P/390 were all done in Bill Beausoleil's IBM Fellow department by a team of 6 to 10 engineers and programmers.
Bill wanted IBM to sell the 7437 for about $5,000. But that would have seriously broken IBM's pricing model vs the cost/performance of traditional mainframes like the 4341 which sold for something like $100,000. VM wanted to charge something like $100K for a 7437 license,
Upper level executives were frightened that customers would buy a slew of 7437s instead of a 3090, costing IBM millions in profit. They swore that they would never let the 7437 go out the door if it meant losing the sale of even one 3090. In the end, they severely restricted who could buy a 7437 by bundling it with a 5080 graphics workstation and pricing it at $50,000.

Paddy Mullen @paddymullen.bsky.social · Jun 13

On the toughest most annoying problems I need to try to put in a half hour to an hour a day, and let ideas percolate. But actually do that work, it's much more effective than hours of grinding. I'm thinking about devops stuff especially

Paddy Mullen @paddymullen.bsky.social · Jun 13

You hear those stories about how garbage collection on lisp machines in the 80s didn't really work and engineers just restarted the machine once a day. Then I realized I do the same thing with firefox/chrome. Need to upgrade to a 32+GB laptop

Paddy Mullen @paddymullen.bsky.social · Jun 12

I want larger wheels because I think they’ll help with surface roughness a lot. Something about rollerblading bothers my knees like skiing, and ice skating don’t.

Paddy Mullen @paddymullen.bsky.social · Jun 11

There are constructions that are very useful that I know I would have avoided because they make the type checkers life difficult.

This is a very bad habit to get into. I don’t know how to generally fix it. Getting more comfortable with typing will help.

Paddy Mullen @paddymullen.bsky.social · Jun 11

I think the typing helps, a bit, but I really worry about the code that I would write if I started writing typed python. There are things I’m doing which are complex to type. The urge and incentives are to make it typed first and useful second.

1

Paddy Mullen @paddymullen.bsky.social · Jun 11

It’s more a matter of seeing the syntax highlighting and wanting to fix it then wanting my code typed.

1

Paddy Mullen @paddymullen.bsky.social · Jun 11

I have started heavily typing my python code. I’m late to the party on this one, I know, but here are some thoughts.

I was prompted by finally getting eglot mode working with basedpyright in eMacs.

1

Paddy Mullen @paddymullen.bsky.social · Jun 9

All of this will be part of my article "so you want to serialize a dataframe to JS". Which is a subsection (probably the largest) of "so you want to write a table viewer"