Lightnews — Scholar-powered news

LightNews

Marco Zocca

@ocramz.bsky.social

320 followers 700 following 640 posts

ML, λ • language and the machines that understand it • https://ocramz.github.io

ocramz.github.io

Posts Media Videos Starter Packs

Pinned

Marco Zocca @ocramz.bsky.social · Jul 4

CERN for frontier AI >>>

Marco Zocca @ocramz.bsky.social · 2h

audacity v4 looks to becoming a proper DAW so.. perhaps soon?

Marco Zocca @ocramz.bsky.social · 2h

the kids figured out an interesting "pair layback" #climbing technique

Marco Zocca @ocramz.bsky.social · 3h

timeline cleanse

Marco Zocca @ocramz.bsky.social · 5h

but the problem persists as long as there is a public "view" or API from which bots can scrape.

I would love to believe that adding one more layer of tech indirection will "feed the wolf while sparing the chickens" but I simply don't.

Marco Zocca @ocramz.bsky.social · 5h

iiuc atproto supports already private data servers, whereas bsky does not. Would love to learn more about this discrepancy

Marco Zocca @ocramz.bsky.social · 22h

This sounds like a great shared task for the next NeurIPS

Dan Levenstein @dlevenstein.bsky.social · 1d

So I get that a Neuroscientist Couldn’t Understand a Microprocessor, and TBH I’m ok with that. But could a neuroscientist understand a deep RNN? Because that seems like a more pressing issue.

*assuming you think the brain operates through the parallel activity of many connected input/output units

Marco Zocca @ocramz.bsky.social · 1d

days since inspecting the data as a first thing would have saved hours of work: 0

Marco Zocca @ocramz.bsky.social · 1d

what changed in v2 and v3?

Marco Zocca @ocramz.bsky.social · 1d

anyway, @kelseyhightower.com had a rather simpler point around chaining bash commands. We could take a page from probabilistic programming and finally acknowledge that even regular bash commands are nondeterministic if you open a socket or file, and model that at the type level

Marco Zocca @ocramz.bsky.social · 1d

yes but the fundamental shift is for _tools_ to be nondeterministic and not have accountability at the same time.

Marco Zocca @ocramz.bsky.social · 2d

kids figuring out language compositionality eary:

my 4yo is on a swing next to her friend, who speaks little english still.

As I tell my kid to have a nice day at school, she replies "si papà", and the other one "si, <name>'s papa" 🐵🐵

Marco Zocca @ocramz.bsky.social · 2d

shall we control for "entities that are in the pocket or have a direct stake into the companies in question"? bc that's what I feel is the axis of this issue

Marco Zocca @ocramz.bsky.social · 2d

Thank you for elaborating your view; I learned something new today.

Marco Zocca @ocramz.bsky.social · 2d

being able to do all experiments in silico means that we can be more deliberate with interventions, and that much more can be controlled for.

I think MI is still figuring out how to control for one factor at a time.

Marco Zocca @ocramz.bsky.social · 2d

building theory is generally slower than making gpu go brr, you'll agree on that :)

Marco Zocca @ocramz.bsky.social · 2d

as a matter of fact "interpretable ML" is way broader than "mech interp for deep learning", and has an much more complex history of methods and results.

Marco Zocca @ocramz.bsky.social · 2d

the common thread is the vast and barely coherent phenomenology and the lack of a consensus on even the right observation methods.

Marco Zocca @ocramz.bsky.social · 2d

I mean, I care that it doesn't break and that it serves me well, but do I feel for it?

Marco Zocca @ocramz.bsky.social · 2d

this is super interesting, and a thought I also recently had.

I really must ask though whether having an attachment or a functional dependency on some piece of tech is really the same as empathizing or even identifying with it.

Marco Zocca @ocramz.bsky.social · 2d

Circuits as minimal explanations of behaviour is a worthwile goal but very hard to pull off, as eg transformer blocks compensate for ablations.

Finding "knowledge" or little switches like the denial direction or the evil vector are fun tricks but not super interesting.

Marco Zocca @ocramz.bsky.social · 2d

great q. I think currently the answer
is: multiple overlapping problems. A good recent taxonomy is The quest for the right mediator by @amuuueller.bsky.social . A recent review called "Open problems in mech interp" is also a good reference.

As a sci field, it's still figuring out the microscope.

Marco Zocca @ocramz.bsky.social · 2d

please elaborate on how would that work out.

Marco Zocca @ocramz.bsky.social · 3d

I think we're only doing very high-d now :)

Marco Zocca @ocramz.bsky.social · 3d

yes, all added together in the residual stream

Marco Zocca @ocramz.bsky.social · 3d

hackage.haskell.org/package/unbo... this one? looks neat tho

unbound-generics

Support for programming with names and binders using GHC Generics

hackage.haskell.org