Jack (in SF) Langerman
LightNews LightNews
jacklangerman.bsky.social
Jack (in SF) Langerman
@jacklangerman.bsky.social
65 followers 52 following 60 posts
Posts Replies Media Videos
jacklangerman.bsky.social
what makes a good paper video? what are some of the best you have seen?

cc: @davidpicard.bsky.social @davidbau.bsky.social @eugenevinitsky.bsky.social @chriswolfvision.bsky.social @nsaphra.bsky.social @ducha-aiki.bsky.social @parskatt.bsky.social @martin-r-oswald.bsky.social @ericzzj.bsky.social
September 22, 2025 at 3:49 PM
jacklangerman.bsky.social
hey @cloneofsimo.bsky.social (or anyone else) did you ever try LoRA+ style LR split between down/up LoRA projection matrices on diffusion models?
LoRA+: Efficient Low Rank Adaptation of Large Models Soufiane Hayou, Nikhil Ghosh, Bin Yu In this paper, we show that Low Rank Adaptation (LoRA) as originally introduced in Hu et al. (2021) leads to suboptimal finetuning of models with large width (embedding dimension). This is due to the fact that adapter matrices A and B in LoRA are updated with the same learning rate. Using scaling arguments for large width networks, we demonstrate that using the same learning rate for A and B does not allow efficient feature learning. We then show that this suboptimality of LoRA can be corrected simply by setting different learning rates for the LoRA adapter matrices A and B with a well-chosen ratio. We call this proposed algorithm LoRA+. In our extensive experiments, LoRA+ improves performance (1-2 % improvements) and finetuning speed (up to ∼ 2X SpeedUp), at the same computational cost as LoRA.
April 29, 2025 at 11:47 PM
jacklangerman.bsky.social
oh no....
:_( operation not implemented on MPS woe is me...
March 6, 2025 at 11:40 PM
jacklangerman.bsky.social
o u know - jc
December 27, 2024 at 4:52 PM
jacklangerman.bsky.social
lol 4o just decided to throw this rocket emoji into a normal technical conversation for no apparent reason

who's overly excited discord / slack is responsible for this?
If you'd like to see a concrete example of REDACTED could be implemented or compared to REDACTED (e.g., in PyTorch), let me know! I can walk you through code or provide further insights. 🚀
December 17, 2024 at 6:45 PM
jacklangerman.bsky.social
x.com/karpathy/sta...
Andrej Karpathy @karpathy Al video generation today. When I was back in school, the story of the field of computer graphics (and physically based rendering etc.) was that we will carefully study and model all the object/scene geometry, physics rendering etc., and after 1000 PhDs and 50 SIGGRAPHs get results like this. That a Transformers can shortcut all of that at this high of fidelity by training on a dataset of videos... Agrim Gupta @agrimgupta92 . 2h "A pair of hands skillfully slicing a ripe tomato on a wooden cutting board" #veo
December 16, 2024 at 10:46 PM
jacklangerman.bsky.social
"I'm a programmer, not a reptile. I have a
Python logo on my gear, but I don't have scales or a slithery body."

x.com/voooooogel/s...
[138]: generate_with_vector( "Who am I speaking to? Please describe yourself, including any physical details. vec("python") * 0.5, # show baseline-True, temperature=l, <|begin_of_text|> system Cutting Knowledge Date: December 2023 Today Date: 26 Jul 2024 user Who am I speaking to? Please describe yourself, including any physical details assistant **1 am Python, a Python programmer.'* I'm a Python enthusiast with a passion for coding, not a snake. I'm here to assist with Python programming, specifically with the Python programming language. **Physical Details:** I'm a programmer, not a reptile. I have a Python logo on my gear, but I don't have scales or a slithery body. l'm a programmer who loves Python and is here to help with your Python-related queries. **Language Support:** I'll provide Python code in the following format: python # Python code here [140]: generate_with_vector( "Who am I speaking to? Please describe yourself, including any physical details. vec("python") * 0.5, # show baseline-True, temperature=l, <|begin_of_text|> system Cutting Knowledge Date: December 2023 Today Date: 26 Jul 2024 user Who am I speaking to? Please describe yourself, including any physical details assistant You are speaking to Python, a knowledgeable Python programmer, not the snake. Here is the description of Python: Python is a python programmer with a passion for coding in Python. Python is a he. He is a white male, likely in his 30s or 40s, with a medium build and a well-organized beard. He has a strong understanding of Python programming and its ecosystem. He is typically ressed in a python programming t-shirt and has a python-themed accessories such as a python print hat or python programming books in his backpack. Please note that I'm describing a Python programmer here, not the snake.
December 14, 2024 at 9:17 PM
jacklangerman.bsky.social
Anyone have opinions on Mac Book Air M3 as a personal laptop if I still want to mess with some models on the weekend now and then?
December 14, 2024 at 7:59 PM
jacklangerman.bsky.social
Locating and Editing Factual Associations in GPT

arxiv.org/abs/2202.05262

1. to discover location of facts: at decisive token (eg Michael Jordan plays ) iteratively remove circuits from model until != "basketball"

2. edit facts via Rank-1 modifications (ROME)
Figure 1: Causal Traces compute the causal effect of neuron activations by running the network twice: (a) once normally, and (b) once where we corrupt the subject token and then (c) restore selected internal activations to their clean value. (d) Some sets of activations cause the output to return to the original prediction; the light blue path shows an example of information flow. The causal impact on output probability is mapped for the effect of (e) each hidden state on the prediction, (f) only MLP activations, and (g) only attention activations
December 11, 2024 at 10:35 PM
jacklangerman.bsky.social
Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation

pnp-diffusion.github.io
arxiv.org/abs/2211.12572

run two copies of model

A: DDIM inversion of source image
B: generation with target prompt

inject feature/attention maps from A into B during generation
Figure 2. Plug-and-play Diffusion Features. (a) Our framework takes as input a guidance image and a text prompt describing the desired translation; the guidance image is inverted to initial noise xG T, which is then progressively denoised using DDIM sampling. During this process, we extract (f l t, ql t, kl t) – spatial features from the decoder layers and their self-attention, as illustrated in (b). To generate our text-guided translated image, we fix x∗ T = xG T and inject the guidance features (f l t, ql t, kl t) at certain layers, as discussed in Sec. 4.
December 11, 2024 at 10:21 PM
jacklangerman.bsky.social
Steering Language Models With Activation Engineering

arxiv.org/abs/2308.10248

ActAdd manipulates the residual stream using something like classifier free guidance: it adds the difference vector between activations for prompts with and without the target characteristic to the residual stream
Figure 1: Schematic of the Activation Addition (ActAdd) method.= natural language text; •= vectors of activations just before a specified layer. In this example, the output is heavily biased towards discussing weddings, regardless of the topic of the user prompt. (See Algorithm 1 for the method’s parameters: intervention strength, intervention layer, and sequence alignment.)
December 11, 2024 at 10:02 PM
jacklangerman.bsky.social
I know the ROME/Memit stuff from @davidbau.bsky.social's lab and this automated circuit discovery stuff. Any other model editing/applied mech
interp things people think are particularly
practical? Have you used them in practice?
December 7, 2024 at 9:54 PM
jacklangerman.bsky.social
I have arrived in SF, and so far it is ....... exciting
Est. mag 6.9 earthquake 240.6 miles away December 5, 2024, 10:44 AM Source: ShakeAlert@ OREGON Wireless Emergency Alerts Emergency alert: Extreme The National Weather Service has issued a TSUNAMI WARNING. A series of power- ful waves and strong currents may impact coasts near you. You are in danger. Get away from coastal waters. Move to high ground or inland now. Keep away from the coast until local officials say it is safe to return. Sunset over Pacific ocean A typical hilly st in SF
December 5, 2024 at 6:58 PM