Lightnews — Scholar-powered news

Alan Amin @alannawzadamin.bsky.social · Jun 25

Many thanks for the award for this work at AI4NA workshop at ICLR!

More experiments and details of our linear algebra in the paper! Come say hi at ICML! 7/7
Paper: arxiv.org/abs/2506.19598
Code: github.com/AlanNawzadAm...

Alan Amin @alannawzadamin.bsky.social · Jun 25

Ablations on model size and number of features show that larger models trained on more features make more accurate predictions with no evidence of plateauing! This suggests further improvements by training across many phenotypes, or across populations in future work! 6/7

1 1

Alan Amin @alannawzadamin.bsky.social · Jun 25

We can now train flexible models on many features. DeepWAS models predict significant enrichment of effect in conserved regions, accessible chromatin, and TF binding sites! And, as shown above, they make better phenotype predictions in practice! 5/7

1

Alan Amin @alannawzadamin.bsky.social · Jun 25

Our idea is to rearrange the linear algebra problem to counter-intuitively increase matrix size, but make it "better conditioned". Iterative algorithms (like CG) converge on huge matrices quickly! By moving to GPU we achieved another order of magnitude speed up! 4/7

1 1 1

Alan Amin @alannawzadamin.bsky.social · Jun 25

But to train on the full likelihood, we must solve big linear algebra problems because variants in the genome are correlated. A naive method would take O(M³) – intractable for many variants! By reformulating the problem, we reduce this to roughly O(M²), enabling large models. 3/7

1 1

Alan Amin @alannawzadamin.bsky.social · Jun 25

Many previous methods used the computationally light “LDSR objective” and saw no benefit from larger models – maybe deep learning isn’t useful here? No! Using the full likelihood, DeepWAS unlocks the potential of deep priors, improving phenotype prediction on UK Biobank! 2/7

1 1

Alan Amin @alannawzadamin.bsky.social · Jun 25

We can make population genetics studies more powerful by building priors of variant effect size from features like binding. But we’ve been stuck on linear models! We introduce DeepWAS to learn deep priors on millions of variants! #ICML2025 Andres Potapczynski, @andrewgwils.bsky.social 1/7

1 3 6

Alan Amin @alannawzadamin.bsky.social · Jun 16

Details and more results demonstrating scaling and on language in the paper, code for training classical and SCUD models on github! Great working with Nate Gruver and @andrewgwils.bsky.social! 7/7
arXiv: arxiv.org/abs/2506.08316
Code: github.com/AlanNawzadAm...

Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion

Discrete diffusion models, like continuous diffusion models, generate high-quality samples by gradually undoing noise applied to datapoints with a Markov process. Gradual generation in theory comes wi...

arxiv.org

1

Alan Amin @alannawzadamin.bsky.social · Jun 16

When applying SCUD to gradual processes like Gaussian (images) and BLOSUM (proteins), we combine masking's scheduling advantage with domain-specific inductive biases, outperforming both masking and classical diffusion! 6/7

1

Alan Amin @alannawzadamin.bsky.social · Jun 16

We show that controlling the amount of information about transition times in SCUD interpolates between uniform noise and masking, clearly illustrating why masking has superior performance. But SCUD also applies to other forward processes!

1

Alan Amin @alannawzadamin.bsky.social · Jun 16

That’s exactly what we do to build schedule-conditioned diffusion (SCUD) models! After some math, training a SCUD model is like training a classical model except time is replaced with the number of transitions at each position, a soft version of how “masked” each position is! 4/7

1

Alan Amin @alannawzadamin.bsky.social · Jun 16

But in practice, SOTA diffusion models have detectable errors in transition times! The exception is masking, which is typically parameterized to bake-in the known distribution of “when”. Why don’t we represent this knowledge in other discrete diffusion models? 3/7

1

Alan Amin @alannawzadamin.bsky.social · Jun 16

In discrete space, the forward noising process involves jump transitions between states. Reversing these paths involves learning when and where to transition. Often the “when” is known in closed form a priori, so it should be easy to learn… 2/7

1

Alan Amin @alannawzadamin.bsky.social · Jun 16

There are many domain-specific noise processes for discrete diffusion, but masking dominates! Why? We show masking exploits a key property of discrete diffusion, which we use to unlock the potential of those structured processes and beat masking! w/ Nate Gruver and @andrewgwils.bsky.social 1/7

1 1 5

Reposted by Alan Amin

Eli Weinstein @eliweinstein.bsky.social · May 29

Thrilled to announce that I am joining DTU in Copenhagen in the fall, as an assistant professor of chemistry.

My research group will focus on fundamental methodology in machine learning for molecules.

8 11 58

Reposted by Alan Amin

Jonathan Frazer @jonnyfrazer.bsky.social · May 26

Want to improve your protein or genomic language model’s performance at zero-shot variant effect prediction? We propose a simple adjustment to likelihood-based predicton

Charlie Pugh @cwjpugh.bsky.social · May 26

New preprint in collaboration with @paulinanunezv.bsky.social supervised by @jonnyfrazer.bsky.social and Mafalda Dias – we propose a simple approach to improving zero-shot variant effect prediction in pre-existing protein and genome language models: 🧶 1/n

www.biorxiv.org/content/10.1...

From Likelihood to Fitness: Improving Variant Effect Prediction in Protein and Genome Language Models

Generative models trained on natural sequences are increasingly used to predict the effects of genetic variation, enabling progress in therapeutic design, disease risk prediction, and synthetic biolog...

www.biorxiv.org

6 24

Alan Amin @alannawzadamin.bsky.social · Dec 17

Details and more results in the paper, trained models on github! Great working with the Big Hat team: Hunter Elliot, Ani Raghu, Calvin Mccarter, Peyton Greenside! (Also got outstanding poster at AIDrugX at Neurips!) 7/7
arXiv: arxiv.org/abs/2412.07763
Code: github.com/AlanNawzadAm...

Bayesian Optimization of Antibodies Informed by a Generative Model of Evolving Sequences

To build effective therapeutics, biologists iteratively mutate antibody sequences to improve binding and stability. Proposed mutations can be informed by previous measurements or by learning from larg...

arxiv.org

6

Alan Amin @alannawzadamin.bsky.social · Dec 17

Finally, we validated CloneBO in vitro! We did one round of designs and tested them in the lab, comparing against the next best method. We see that CloneBO’s designs improve stability and significantly beat LaMBO-Ab in binding. 6/7

1 4

Alan Amin @alannawzadamin.bsky.social · Dec 17

To use our prior to optimize an antibody, we now need to generate clonal families that match measurements in the lab – bad mutations should be unlikely and good mutations likely. We developed a twisted sequential Monte Carlo approach to efficiently sample from this posterior. 5/7

1 2

Alan Amin @alannawzadamin.bsky.social · Dec 17

We train a transformer model to generate entire clonal families – CloneLM. Prompting with a single sequence, CloneLM samples realistic clonal families. These samples represent a prior on possible evolutionary trajectories in the immune system. 4/7

1 1

Alan Amin @alannawzadamin.bsky.social · Dec 17

Our bodies make antibodies by evolving specific portions of their sequences to bind their target strongly and stably, resulting in a set of related sequences known as a clonal family. We leverage modern software and data to build a dataset of nearly a million clonal families! 3/7

1 1

Alan Amin @alannawzadamin.bsky.social · Dec 17

SoTA methods search the space of sequences by iteratively suggesting mutations. But the space of antibodies is huge! CloneBO builds a prior on mutations that make strong and stable binders in our body to optimize antibodies in silico. 2/7

1 1

Alan Amin @alannawzadamin.bsky.social · Dec 17

How do you go from a hit in your antibody screen to a suitable drug? Now introducing CloneBO: we optimize antibodies in the lab by teaching a generative model how we optimize them in our bodies!
w/ Nat Gruver, Yilun Kuang, Lily Li, @andrewgwils.bsky.social and the team at Big Hat! 1/7

1 1 11

Alan Amin @alannawzadamin.bsky.social · Dec 14

New model trained on new dataset of nearly a million evolving antibody families at AIDrugX workshop Sunday at 4:20 pm (#76) #Neurips! Collab between @andrewgwils.bsky.social and BigHatBio. Stay tuned for full thread on how we used the model to optimize antibodies in the lab in coming days!

1 4