Lightnews — Scholar-powered news

Sam Duffield @samduffield.com · Aug 29

Read more at arxiv.org/abs/2508.20883

Including scaling LRW up to image generation with Stable Diffusion 3.5 🐱

Lattice Random Walk Discretisations of Stochastic Differential Equations

We introduce a lattice random walk discretisation scheme for stochastic differential equations (SDEs) that samples binary or ternary increments at each step, suppressing complex drift and diffusion co...

arxiv.org

5

Sam Duffield @samduffield.com · Aug 29

As described in the paper, LRW provides multiple benefits but the key motivation for us @normalcomputing.com was the co-design with novel stochastic computing hardware which we believe can drastically accelerate general-purpose SDE sampling.

1 2

Sam Duffield @samduffield.com · Aug 29

New paper on arXiv! And I think it's a good'un 😄

Meet the new Lattice Random Walk (LRW) discretisation for SDEs. It’s radically different from traditional methods like Euler-Maruyama (EM) in that each iteration can only move in discrete steps {-δₓ, 0, δₓ}.

1 5 16

Reposted by Sam Duffield

Sam Power @spmontecarlo.bsky.social · May 13

In slides from a recent talk - the { virtuous / vicious } cycle of filtering, smoothing, and parameter estimation in state space models.

2 1 20

Sam Duffield @samduffield.com · Apr 28

Oh you king this is great thanks! I was at Lau Pa Sat the other day but went for shrimp noodles (which were great) because the satay queue was too long

2

Sam Duffield @samduffield.com · Apr 27

Didn’t listen, good decision

1 1

Sam Duffield @samduffield.com · Apr 27

Me: Hey so where’s good to eat round here?
Singapore taxi driver: Malaysia

2 2

Sam Duffield @samduffield.com · Apr 18

However! We’re working on a much broader generalisation of abile which hopefully will be able to share soon 🤞🔜

1 1

Sam Duffield @samduffield.com · Apr 18

Adjacent!

posteriors takes the natural gradient descent viewpoint on EKF arxiv.org/abs/1703.00209

Which is nice for online deep learning but not necessarily bespoke state-space model inference

Online Natural Gradient as a Kalman Filter

We cast Amari's natural gradient in statistical learning as a specific case of Kalman filtering. Namely, applying an extended Kalman filter to estimate a fixed unknown parameter of a probabilistic mod...

arxiv.org

1 4

Sam Duffield @samduffield.com · Apr 18

We've also updated the paper and made some cool updates to the library 😎

Paper: arxiv.org/abs/2406.00104
Repo: github.com/normal-compu...

1 2

Sam Duffield @samduffield.com · Apr 18

📃 Poster #419
🗓️ Sat 26th, 10:00–12:30
📍 #ICLR2025, Singapore

Swing by if you’re into probml, thermodynamic computing or just wanna say hi

2 2

Sam Duffield @samduffield.com · Apr 18

posteriors 𝞡 published at ICLR!

I’ll be in Singapore next week, let’s chat all things scalable Bayesian learning! 🇸🇬👋

1 2 11

Reposted by Sam Duffield

Sam Power @spmontecarlo.bsky.social · Apr 17

A new instalment of office decor:

3 1 20

Sam Duffield @samduffield.com · Apr 16

F

1

Sam Duffield @samduffield.com · Apr 10

Should have said, here h is stepsize 😅

Sam Duffield @samduffield.com · Apr 10

So simple!

Normally we order our minibatches like
a, b, c, ...., [shuffle], new_a, new_b, new_c, ....
but instead, if we do
a, b, c, ...., [reverse], ...., c, b, a, [shuffle], new_a, new_b, ....

The RMSE of stochastic gradient descent reduces from O(h) to O(h²)

arxiv.org/abs/2504.04274

1 2

Reposted by Sam Duffield

Alex Thiery @alexxthiery.bsky.social · Mar 29

Sequential Monte Carlo (aka. Particle Socialism?):

"why send one explorer when you can send a whole army of clueless one"

2 12

Sam Duffield @samduffield.com · Mar 25

Yep! That would be clearer

Sam Duffield @samduffield.com · Mar 24

arxiv.org/abs/1806.07366

Neural Ordinary Differential Equations

We introduce a new family of deep neural network models. Instead of specifying a discrete sequence of hidden layers, we parameterize the derivative of the hidden state using a neural network. The outp...

arxiv.org

Sam Duffield @samduffield.com · Mar 24

Was revisiting the Neural ODEs paper the other day and greatly enjoying.

But I found this super confusing, it’s not an A=B+A statement

3 3

Reposted by Sam Duffield

Sam Power @spmontecarlo.bsky.social · Mar 20

Thrillingly (/s), I have today (lightly) updated my website (sites.google.com/view/sp-mont...).

I highlight that I've added
i) links to several slide decks for talks about my research, and
ii) materials related to the (few) short courses which I've given in the past couple of years.

Enjoy!

Sam Power's site

Hello! My name is Sam, and I am a researcher in Statistics. I am currently Lecturer in Statistical Science at the University of Bristol. Prior to this role, I was a Senior Research Associate (also at...

sites.google.com

3 2 31

Reposted by Sam Duffield

AlgoPerf @algoperf.bsky.social · Mar 14

Hi there! This account will post about the AlgoPerf benchmark and leaderboard updates for faster neural network training via better training algorithms. But let's start with what AlgoPerf is, what we have done so far, and how you can train neural nets ~30% faster.

1 3 6

Sam Duffield @samduffield.com · Feb 28

Thinking about it more, I think the sharp jumps are an artefact of the plotting. The plotting function will linearly interpolate but you can actually probabilistically interpolate using the smoothing equations

1

Sam Duffield @samduffield.com · Feb 28

Oh you are right! Very nice!

1 1

Sam Duffield @samduffield.com · Feb 28

I don’t think so - I can’t see any backward iterations in the code. Also the sharp changes after a result in e.g. the boxing plot are a classic filtering feature - there is a reason smoothing is called smoothing after all 😄

1 1