Lightnews — Scholar-powered news

Erick Scott @erickscott.bsky.social · 1d

I'd love to see someone specify that function...where does Terence Tao sit on the curve?

Erick Scott @erickscott.bsky.social · 1d

I have been surprised that the first generation is usually more thoughtful with the sycophant warning as a system prompt.

I won't speculate on what's actually happening with matmul/reasoning, but I have found it helpful to counteract the vendor's ingratiating base prompt.

1

Erick Scott @erickscott.bsky.social · 1d

Try adding: 'Don't be a sycophant' to your system prompt.

Gemini is more stubborn...

1 1 1

Erick Scott @erickscott.bsky.social · 1d

I don't think they are useless and deepmind is definitely moving forward with principled quantitative approaches. Moving stochastic outputs into structured models + rapid human error correction is what @travisgerke.bsky.social and I are working on at cStructure. Happy to chat anytime

1 2

Erick Scott @erickscott.bsky.social · 1d

The code that supposedly underpinned the analysis used a fake propensity score (0.1*covariate1 + 0.2*covariate2...) with a comment that a real propensity model should be implemented.

This happens all the time: code syntax was fine, semantics wrong - assoc. text was plausible. User beware.
2/2

1 2

Erick Scott @erickscott.bsky.social · 1d

I have many similar stories. For example, I asked for a propensity score analysis of Lalonde assuming this canonical example is a best case scenario. I provided the dataset. The generated text provided a correct and nuanced description of the estimator and the ATE. 1/2

1 3

Erick Scott @erickscott.bsky.social · May 16

cStructure is proud to collaborate with @BeeKeeperAI_Inc and DREAM on The COVID Causal Diagram DREAM Challenge.

Privacy-preserving compute + collaborative causal modeling -> the future of responsible AI development.

Learn more at cstructure.net

@travisgerke.bsky.social

1 2

Reposted by Erick Scott

Rick @rickylongthread.bsky.social · Apr 19

Tonight is the 250th anniversary of Paul Revere’s midnight ride. May his memory remind us all to resist the tyranny forming in our government.

330 7.6K 23K

Erick Scott @erickscott.bsky.social · Apr 18

Bayesian posterior distributions. So much information packed into the density. If two people disagree on what threshold should be used to make a decision, it's easy to calculate the support for either.

1 4

Erick Scott @erickscott.bsky.social · Apr 17

Reminds me of the difference b/w Efficacy (ITT if properly used, e.g. abstinence for teen pregnancy) vs Effectiveness (PP, outcomes when abstinence is used in practice).

In practice, I think the effectiveness of a causal method is important as unmeasured confounding is ever present in real data.

1 1

Reposted by Erick Scott

Chris Murphy @chrismurphyct.bsky.social · Apr 17

The Trump-Supreme Court battle is not really the crisis.

The crisis is here now. Trump is enacting an insidious coordinated attack on our institutions of democratic accountability, designed to crater democracy before next fall.

1/ A long 🧵to explain the plan & how we stop it.

160 2K 4.6K

Erick Scott @erickscott.bsky.social · Apr 17

I really do believe Gordon et al. offered a best effort assessment. The scale, diversity, and quality of their ground truth is quite impressive.

What would you have done differently to reduce assumption violation?

1 1

Erick Scott @erickscott.bsky.social · Apr 17

I see simulations as a useful tool to assess method performance under various degrees of assumption violation.

I also think the simulations should approximate the magnitude and direction of bias seen in high quality empirical studies.

Erick Scott @erickscott.bsky.social · Apr 17

I love the WeightIt package, thanks for developing it.

How do you interpret these simulation papers in light of large scale empirical benchmarks

www.researchgate.net/publication/...

(PDF) Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement

PDF | Randomized controlled trials (RCTs) have become increasingly popular in both marketing practice and academia. However, RCTs are not always... | Find, read and cite all the research you need on R...

www.researchgate.net

1 2

Erick Scott @erickscott.bsky.social · Apr 16

Am I the only one in industry, that looks at this thread and remembers junior hires showing up to their first stakeholder meeting after throwing "all the x's" into sci-kit learn and then getting absolutely thrashed by the domain experts?

Erick Scott @erickscott.bsky.social · Apr 16

It's like we learned absolutely nothing from the reproducibility crisis, kitchen-sink machine learning models for covid, population/environmental stratification in genomics, a/b testing at scale...sigh.

1

Erick Scott @erickscott.bsky.social · Apr 16

I just named several industries that in practice don't blindly use LASSO. A/B testing is used by any industry with a website/app and small to large companies employ (data) scientists to design and analyze the experiments. Healthcare is a pretty large industry, Computing is a pretty large industry??

Erick Scott @erickscott.bsky.social · Apr 16

Then I am puzzled by the idea that in practice scientists just expect LASSO to select the right variables. Here's SHAP docs describing why that is a bad assumption **in practice**
shap.readthedocs.io/en/latest/ex...

1

Erick Scott @erickscott.bsky.social · Apr 16

You should encourage him to explore causal inference.

Practical applications that share the same concern about LASSO: A/B testing, drug development, electrical engineering, physics

2 1

Reposted by Erick Scott

Chris Murphy @chrismurphyct.bsky.social · Apr 14

Bone chilling.

A court ordered Kilmar Abrego Garcia to stay in the United States.

The Supreme Court ruled 9-0 that he was illegally removed. Trump is pretending he won the ruling 9-0.

1/ You may not think this case means anything to you. But let me tell you why it does.

470 4.3K 13K

Erick Scott @erickscott.bsky.social · Apr 14

Empirical studies

RCT-Duplicate: www.rct-duplicate.org

FDA RWE examples: www.fda.gov/media/146258... &

Northwestern/Meta A/B testing: www.kellogg.northwestern.edu/faculty/gord... and arxiv.org/abs/2201.07055

2

Erick Scott @erickscott.bsky.social · Apr 14

There are several excellent technical books on this subject.

WhatIf by Hernán and Robins: miguelhernan.org/whatifbook

Causal Inference in Statistics by Pearl: www.amazon.com/Causal-Infer...

Causality by Pearl: bayes.cs.ucla.edu/BOOK-2K/

1 1 2

Reposted by Erick Scott

Chris Murphy @chrismurphyct.bsky.social · Apr 3

Those trying to understand the tariffs as economic policy are dangerously naive.

No, the tariffs are a tool to collapse our democracy. A means to compel loyalty from every business that will need to petition Trump for relief.

1/ A 🧵 to explain his plan and how we fight back.

930 14K 28K

Erick Scott @erickscott.bsky.social · Mar 28

Rectangle is amazing for organizing windows
rectangleapp.com

Spaces are also really helpful to keep work streams partitioned. 3 finger up/down

Dbeaver is the best free database GUI for mac

Homebrew for package management is a must

Rectangle

Move and resize windows in macOS using keyboard shortcuts or snap areas. The official page for Rectangle.

rectangleapp.com

1

Reposted by Erick Scott

Akshay Agrawal @akshaykagrawal.bsky.social · Mar 12

🚀 The @marimo.io YouTube channel crossed 1k subscribers today — just two weeks after its launch!

marimo is best understand by seeing it in action. In his latest video, the one and only @koaning.bsky.social gives a bird's eye overview of what sets marimo apart:

www.youtube.com/watch?v=3N6l...

An overview of marimo

YouTube video by marimo

www.youtube.com

4 11