Grant McDermott
@gmcd.bsky.social
4.9K followers 150 following 550 posts
Economics and data science | Former academic now working in the tech sector | Views my own | 🇿🇦 in 🇺🇸 | https://grantmcdermott.com/
Posts Media Videos Starter Packs
Pinned
gmcd.bsky.social
A few months ago, I gave a workshop on “(Pretty) big data wrangling with DuckDB and Polars”.

Slides, notebooks etc. are all available here: grantmcdermott.com/duckdb-polars/

#EconSky
We just read a ~180 million row dataset (from disk!) and did a group-by aggregation on it.
In < 1 second.
On a laptop. Benchmark plot showing minimal data I/O cost of duckdb and polars relative to other options (alongside very fast compute time)
gmcd.bsky.social
Hmmm, I wouldn't say the problem is the availability of binary-build channels. Alongside r2u (great albeit Ubuntu only), there's rspm and the Posit manylinux repo (both multi-distro). The problem is conda hijacking and/or overriding these channels, so that they don't work like normal :-/
gmcd.bsky.social
Yeah, unfortunately the inconsistency is the killer here. I've also run into a (related?) problem of missing pre-compiled binaries on some Linux distros that we use at work. So you end up having to install + compile everything from source ☠️
gmcd.bsky.social
It's a constant stream of untaxed negative externalities that I can only assume are part of some cunning grand plan for language hegemony.
gmcd.bsky.social
Old ground, but I still cannot believe how poorly 🐍 plays with other languages. Okay, we're going to force you to do everything in a venv (b/c otherwise we break everything...) But that venv is going to auto instantiate and override any system level env vars or processes that you've set ☠️
gmcd.bsky.social
Fair points. Still, if you feel like modifying / playing around... bsky.app/profile/gmcd...
gmcd.bsky.social
Successfully nerdsniped myself into writing a little script for this. gist.github.com/grantmcdermo...

(Longer than it probably needs to be bc of annoying Wiki table formatting, but I'm too lazy to find another source.)
Showing the minimum no. of voters that could have flipped the results from recent US presidential elections.

elec_marg(2016)
#> Winner: Republican 
#> Min votes to flip Democratic: 77,744
#>   0.06% of total votes
#>   0.6% of votes from 3 states (Michigan, Pennsylvania, Wisconsin)

elec_marg(2020)
#> Winner: Democratic 
#> Min votes to flip Republican: 123,473
#>   0.08% of total votes
#>   0.7% of votes from 4 states (Georgia, Arizona, Wisconsin, Pennsylvania)

elec_marg(2024)
#> Winner: Republican 
#> Min votes to flip Democratic: 344,866
#>   0.2% of total votes
#>   2% of votes from 4 states (Wisconsin, Michigan, Pennsylvania, Georgia)
Comment
gmcd.bsky.social
Successfully nerdsniped myself into writing a little script for this. gist.github.com/grantmcdermo...

(Longer than it probably needs to be bc of annoying Wiki table formatting, but I'm too lazy to find another source.)
Showing the minimum no. of voters that could have flipped the results from recent US presidential elections.

elec_marg(2016)
#> Winner: Republican 
#> Min votes to flip Democratic: 77,744
#>   0.06% of total votes
#>   0.6% of votes from 3 states (Michigan, Pennsylvania, Wisconsin)

elec_marg(2020)
#> Winner: Democratic 
#> Min votes to flip Republican: 123,473
#>   0.08% of total votes
#>   0.7% of votes from 4 states (Georgia, Arizona, Wisconsin, Pennsylvania)

elec_marg(2024)
#> Winner: Republican 
#> Min votes to flip Democratic: 344,866
#>   0.2% of total votes
#>   2% of votes from 4 states (Wisconsin, Michigan, Pennsylvania, Georgia)
Comment
gmcd.bsky.social
Hmmm, I don't think so. If the state of the system hinges on a few critical units then I think you want to model those directly. To the OP, if we don't see much volatility of those marginal units over time then I don't think it makes sense to talk about "close" vs "big" victories, no?
gmcd.bsky.social
Not giving CM the inside ball at 1:30 (beautiful line!) was a legitimate gripe. But vanishingly few of those.
Reposted by Grant McDermott
vincentab.bsky.social
Whoa—my book is up for pre-order!

𝐌𝐨𝐝𝐞𝐥 𝐭𝐨 𝐌𝐞𝐚𝐧𝐢𝐧𝐠: 𝐇𝐨𝐰 𝐭𝐨 𝐈𝐧𝐭𝐞𝐫𝐩𝐫𝐞𝐭 𝐒𝐭𝐚𝐭 & 𝐌𝐋 𝐌𝐨𝐝𝐞𝐥𝐬 𝐢𝐧 #Rstats 𝐚𝐧𝐝 #PyData

The book presents an ultra-simple and powerful workflow to make sense of ± any model you fit

The web version will stay free forever and my proceeds go to charity.

tinyurl.com/4fk56fc8
gmcd.bsky.social
Not that I disagree with the sentiment, but isn't the way to evaluate US victory "closeness" by looking at the minimum no. of voters that could have swayed the EC?
gmcd.bsky.social
Clearly not the most important news coming out of Oregon tonight, but I just added a recession bars example to the tinyplot “Tips & tricks” vignette. grantmcdermott.com/tinyplot/vig... #rstats
tinyplot showing US GDP since 1980 with recessions highlighted in the background
Reposted by Grant McDermott
brendannyhan.bsky.social
Ask yourself if this is the kind of country you want to live in
bradlander.bsky.social
I was back at 26 Federal Plaza today, where an ICE agent violently threw this bereft woman to the ground in front of her kids. She had not touched him. She did not pose any threat. She had to be taken to the hospital. (🎥: Elias Eliahu)
gmcd.bsky.social
search.r-project.org/R/refmans/gr...

The real pros (TM) use symbols > 32 or cut straight to Unicode, e.g. pch/shape = "\u2642" 🥴
R: Add Points to a Plot
search.r-project.org
gmcd.bsky.social
We enforce R >= 4.0.0 to make use of some of the newer palettes that became available at the time. But I sometimes wonder how far back `tinyplot` _could_ have been implemented from a purely technical perspective. I think the hard req. would be recordGraphics, which was introduced in R 3.1.0 IINM.
gmcd.bsky.social
It's in the gallery linked in the alt text: grantmcdermott.com/tinyplot/vig...

If you just want the exact code, here's the snippet file. github.com/grantmcdermo...
Gallery – tinyplot
grantmcdermott.com
gmcd.bsky.social
The (g)goat, for sure. Hopefully we're filling a useful niche.
gmcd.bsky.social
+1

Exciting stuff, thanks for sharing @dfalbel.bsky.social.
gmcd.bsky.social
`tinyplot` 0.5.0 is now available from CRAN & R-universe. Alongside bug fixes & internal improvements, this release includes some cool new features like bubble chart + dual legend support, element dodging, & ephemeral themes.

Detailed changelog: grantmcdermott.com/tinyplot/NEW...

#rstats #dataviz
tinyplot bubble chart example with dual legend, using penguins data. Taken from https://grantmcdermott.com/tinyplot/vignettes/gallery.html
gmcd.bsky.social
Update: it won't see us through the winter, but a respectable harvest through the season.
Smorgasbord, of sorts. (A bunch of veg.)
gmcd.bsky.social
Word. Whenever someone asks me for career advice these days, I reply without irony: "Work on your core."
"Whoever designed the human visual system has a lot to answer for. Maybe not as much as the knee team, or the lower back group. But a lot." Quote from @kjhealy.co
gmcd.bsky.social
Now you have my attention!

(Seriously, sounds great. I'd love to see a demo if/when you have one to share.)