Lightnews — Scholar-powered news

Geoffrey Irving

@girving.bsky.social

3.5K followers 110 following 420 posts

Chief Scientist at the UK AI Security Institute (AISI). Previously DeepMind, OpenAI, Google Brain, etc.

Posts Media Videos Starter Packs

Pinned

Geoffrey Irving @girving.bsky.social · Jul 30

I am very excited that AISI is announcing over £15M in funding for AI alignment and control, in partnership with other governments, industry, VCs, and philanthropists!

Here is a 🧵 about why it is important to bring more independent ideas and expertise into this space.

alignmentproject.aisi.gov.uk

The Alignment Project by AISI — The AI Security Institute

The Alignment Project funds groundbreaking AI alignment research to address one of AI’s most urgent challenges: ensuring advanced systems act predictably, safely, and for society’s benefit.

alignmentproject.aisi.gov.uk

Geoffrey Irving @girving.bsky.social · 1d

It will be very funny if AI control doesn't work because the model is able to steganographically encode it's reasoning in references to safety orgs.

Geoffrey Irving @girving.bsky.social · 1d

One fun bit of Google lore is that there was a 6-character limit on internal email addresses. Jeff Dean pulled rank and got [email protected]. The next day he woke up to find 20,000 spam messages in his inbox.

Geoffrey Irving @girving.bsky.social · 7d

Physics would also be good.

Geoffrey Irving @girving.bsky.social · 7d

Is there Matt Levine but for pure mathematics?

Geoffrey Irving @girving.bsky.social · 7d

Impressive that they were able to make a physical book without using electricity. :)

Geoffrey Irving @girving.bsky.social · 9d

Wait for it...

Geoffrey Irving @girving.bsky.social · 9d

Ominous start to a Wikipedia page about a formula...

en.wikipedia.org/wiki/Fa%C3%A...

Geoffrey Irving @girving.bsky.social · 16d

Ug, further nit: radii of the first singularities, since they won't necessarily be poles.

Geoffrey Irving @girving.bsky.social · 16d

Nit: I think it means that unless there is a spectacular coincidence about the radii of the first poles, but there probably isn't that coincidence.

Geoffrey Irving @girving.bsky.social · 16d

In any case, the fact that the generating function is algebraic means that the number of JSON strings grows as c n^ɑ + O(smaller) for some algebraic c and ɑ, but that doesn't make them easy to find.

Geoffrey Irving @girving.bsky.social · 16d

And in case you write it down, then want some machinery to prove it correct and compute with it...

github.com/girving/series

Geoffrey Irving @girving.bsky.social · 16d

One just needs the context free grammar to be unambiguous, which I think works.

en.wikipedia.org/wiki/Chomsky...

(Also GPT-5 Thinking spit out a large formula, but that Wikipedia page is better as a reference. :))

Chomsky–Schützenberger enumeration theorem - Wikipedia

en.wikipedia.org

Geoffrey Irving @girving.bsky.social · 16d

Seems algebraic.

Geoffrey Irving @girving.bsky.social · 16d

What's the generating function?

Reposted by Geoffrey Irving

xenaproject.bsky.social @xenaproject.bsky.social · 20d

Amongst the projects funded is my project www.renaissancephilanthropy.org/a-dataset-of... to create what in 2025 is a super-hard dataset of pairs (informal hard proof, formal statement) of recent results from top journals. The challenge for machine is to formalise the rest of the paper.

www.renaissancephilanthropy.org

Geoffrey Irving @girving.bsky.social · 27d

Complex dynamics and complex analysis in general is a joy (github.com/girving/ray). Lovely mix of local and global: smooth things that are true near a point extrapolate into strong statements that hold everywhere.

GitHub - girving/ray: Formalizing results about the Mandelbrot set in Lean

Formalizing results about the Mandelbrot set in Lean - girving/ray

Geoffrey Irving @girving.bsky.social · Sep 4

Cas is very good and you should hire him as faculty!

Cas (Stephen Casper) @scasper.bsky.social · Sep 4

📌📌📌
I'm excited to be on the faculty job market this fall. I just updated my website with my CV.
stephencasper.com

Visit the post for more.

stephencasper.com

Geoffrey Irving @girving.bsky.social · Aug 30

Reminiscent of naml.us/post/increas...

Increasingly bizarre typos? | naml.us

Geoffrey Irving @girving.bsky.social · Aug 26

Then you should be able to rotate the Feynman diagrams and see particle creation, not just side interaction.

Geoffrey Irving @girving.bsky.social · Aug 26

Yes, but they're wrong! It would show up as additional particles or fields atop the standard model, and we don't see those at the required low energy levels.

Geoffrey Irving @girving.bsky.social · Aug 26

I think it is fair to say we know a piece of glass isn't conscious.

Geoffrey Irving @girving.bsky.social · Aug 23

From near the end of Sleepwalkers, by Christopher Clark, as World War I starts.

An English traveller recalled the reaction in an Altai (Semipalatinsk) Cossack settlement when the 'blue flag' borne aloft by a rider and the noise of bugles playing the alarm brought news of mobilization. The Tsar had spoken, and the Cossacks, with their unique military calling and tradition, 'burned to fight the enemy'. But who was that enemy? Nobody knew. The mobilization telegram provided no details. Rumours abounded. At first everyone imagined that the war must be with China - 'Russia had pushed too far into Mongolia and China had declared war.
Then another rumour did the rounds: 'It is with England, with England. This view prevailed for some time.

> Only after four days did something like the truth come to us, and then nobody believed it.

Geoffrey Irving @girving.bsky.social · Aug 20

I’m excited to work with you!

Geoffrey Irving @girving.bsky.social · Aug 20

Arrows?

Reposted by Geoffrey Irving

Marcelo Mattar @marcelomattar.bsky.social · Aug 20

I'm honored to serve as Expert Advisor for "The Alignment Project", an international initiative dedicated to ensuring AI systems are safe and beneficial. They are providing significant funding, compute, and collaboration opportunities for researchers---including those in cogsci/neuro. Please apply!

Geoffrey Irving @girving.bsky.social · Jul 30

I am very excited that AISI is announcing over £15M in funding for AI alignment and control, in partnership with other governments, industry, VCs, and philanthropists!

Here is a 🧵 about why it is important to bring more independent ideas and expertise into this space.

alignmentproject.aisi.gov.uk

The Alignment Project by AISI — The AI Security Institute

The Alignment Project funds groundbreaking AI alignment research to address one of AI’s most urgent challenges: ensuring advanced systems act predictably, safely, and for society’s benefit.

alignmentproject.aisi.gov.uk