Lightnews — Scholar-powered news

Reposted by Winston Lin

John V. Kane @uptonorwell.bsky.social · 14d

Are you or one of your students considering doing a Ph.D. in a social science? I've spent a lot of time talking about this w/ students & finally wrote something up.

IMO, there are only 3 good reasons to do it. One of them needs to be true--otherwise, don't.

medium.com/the-quantast...

The Only Three Reasons to Do a Ph.D. in the Social Sciences

If none are true, don’t do it.

medium.com

1 4

Winston Lin @linstonwin.bsky.social · Sep 1

This is written for economists, but I think it’s very helpful more generally

www.aeaweb.org/articles?id=...

How to Write an Effective Referee Report and Improve the Scientific Review Process

(Winter 2017) - The review process for academic journals in economics has grown vastly more extensive over time. Journals demand more revisions, and papers have become bloated with numerous robustness...

www.aeaweb.org

2

Reposted by Winston Lin

Noah Greifer @noahgreifer.bsky.social · Jul 9

A nice recent article on why you should abandon hazard ratios.

#statssky #episky

How hazard ratios can mislead and why it matters in practice - European Journal of Epidemiology

Hazard ratios are routinely reported as effect measures in clinical trials and observational studies. However, many methodological works have raised concerns about the interpretation of hazard ratios ...

doi.org

2 21 78

Reposted by Winston Lin

Coalition for Evidence-Based Policy @coalition4evidence.bsky.social · Jun 11

See our No-Spin report on a widely-covered NBER study of Medicaid expansion. In brief: Despite the abstract's claims that expansion reduced adult mortality 2.5%, the study found much smaller effects that fell short of statistical significance in its main preregistered analysis.🧵

1 2 1

Reposted by Winston Lin

Noah Greifer @noahgreifer.bsky.social · Jun 4

Starting to look like I might not be able to work at Harvard anymore due to recent funding cuts. If you know of any open statistical consulting positions that support remote work or are NYC-based, please reach out! 😅

11 97 150

Winston Lin @linstonwin.bsky.social · May 5

In case this is of interest, even ANCOVA I is consistent and asymptotically normal in completely randomized experiments (though II is asymptotically more efficient in imbalanced or multiarm designs)

1 3

Reposted by Winston Lin

Ethan Mollick @emollick.bsky.social · Apr 21

Issues with interpreting p-values haunts even AI, which is prone to same biases as human researchers. ChatGPT, Gemini & Claude all fall prey to "dichotomania" - treating p=0.049 & p=0.051 as categorically different, and paying too much attention to significance. www.cambridge.org/core/journal...

2 6 41

Reposted by Winston Lin

Brennan Kahan @brennankahan.bsky.social · Apr 15

NEW: CONSORT 2025 now published!

Some notable changes:
-items on analysis populations, missing data methods, and sensitivity analyses
-reporting of non-adherence and concomitant care
-reporting of changes to any study methods, not just outcomes
-and lots of other things

www.bmj.com/content/389/...

CONSORT 2025 explanation and elaboration: updated guideline for reporting randomised trials

Critical appraisal of the quality of randomised trials is possible only if their design, conduct, analysis, and results are completely and accurately reported. Without transparent reporting of the met...

www.bmj.com

10 15

Reposted by Winston Lin

Andrew Vickers @vickersbiostats.bsky.social · Apr 14

How to write a response to reviewers. www.sciencedirect.com/science/arti...

How To Write a Response to Reviewers

www.sciencedirect.com

1 6 9

Winston Lin @linstonwin.bsky.social · Apr 9

Very nice explainer by @economictricks.bsky.social

www.econometrics.blog/post/why-eco...

Why Econometrics is Confusing Part 1: The Error Term | econometrics.blog

“Suppose that \(Y = \alpha + \beta X + U\).” A sentence like this is bound to come up dozens of times in an introductory econometrics course, but if I had my way it would be stamped out completely.

www.econometrics.blog

1 12

Reposted by Winston Lin

Rafe Meager (they/them) @economeager.bsky.social · Apr 6

today we will all read imbens 2021 on statistical significance and p values, which is a strong contender for having the best opening paragraph of any stats paper

pubs.aeaweb.org/doi/pdf/10.1...

27 110 720

Winston Lin @linstonwin.bsky.social · Apr 6

Btw here's an email I sent Stata in 2012, suggesting a clarification to their descriptions of the "unequal" and "welch" ttest options. Got a polite reply but I don't think they changed it :)

Email to Stata explaining that Welch (1949) "welched" on Welch (1947), but refraining from asking if their co-founder Finis Welch was related to B. L. Welch. :) Continued in next screenshot

Second half of email, suggesting that the "unequal" option be described as using "the formula of Satterthwaite (1946) and Welch (1949)" (because what many people call the Welch test is what Stata's "unequal" option does, not what their "welch" option does)

2

Winston Lin @linstonwin.bsky.social · Apr 6

Imbens & Kolesar give a nice overview and simulations

www.jstor.org/stable/24917...

ROBUST STANDARD ERRORS IN SMALL SAMPLES: SOME PRACTICAL ADVICE on JSTOR

Guido W. Imbens, Michal Kolesár, ROBUST STANDARD ERRORS IN SMALL SAMPLES: SOME PRACTICAL ADVICE, The Review of Economics and Statistics, Vol. 98, No. 4 (October 2016), pp. 701-712

www.jstor.org

1 1 7

Reposted by Winston Lin

Carlisle Rainey 👨‍💻📊📚 @carlislerainey.bsky.social · Mar 10

Here's some older, related stuff (from me) aimed at political scientists.

Related paper #1

"Arguing for a Negligible Effect"

Journal: onlinelibrary.wiley....

PDF: www.carlislerainey.c...

1 1 3

Reposted by Winston Lin

Carlisle Rainey 👨‍💻📊📚 @carlislerainey.bsky.social · Mar 10

"The Need for Equivalence Testing in Economics"

from Jack Fitzgerald (@jackfitzgerald.bsky.social)

Preprint: osf.io/preprints/met...

We know that "not significant" does not imply evidence for "no effect," but I still see papers make this leap.

Good to see more work making this point forcefully!

4 9 29

Winston Lin @linstonwin.bsky.social · Mar 9

www.researchgate.net/publication/...

The impact of a Hausman pretest on the size of a hypothesis test: The panel data case | Request PDF

Request PDF | The impact of a Hausman pretest on the size of a hypothesis test: The panel data case | The size properties of a two-stage test in a panel data model are investigated where in the first ...

www.researchgate.net

1 1

Winston Lin @linstonwin.bsky.social · Mar 9

Isn’t the Hausman approach likely to lead to undercoverage, similar to what @jondr44.bsky.social wrote about in a different context?

www.aeaweb.org/articles?id=...

Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends

(September 2022) - This paper discusses two important limitations of the common practice of testing for preexisting differences in trends ("pre-trends") when using difference-in-differences and relate...

www.aeaweb.org

2 1

Winston Lin @linstonwin.bsky.social · Mar 8

For RCTs, another reference with simulation evidence on the robustness of OLS is Judkins & Porter (2016). But average marginal effects from logit are also robust

doi.org/10.1002/sim....

Robustness of ordinary least squares in randomized clinical trials

There has been a series of occasional papers in this journal about semiparametric methods for robust covariate control in the analysis of clinical trials. These methods are fairly easy to apply on cu....

doi.org

11

Winston Lin @linstonwin.bsky.social · Mar 8

In the '90s when I worked at Abt and MDRC, I wrote an email that initially had the subject header "OLS without apology". I shared a later version with Freedman, who cited it as "Lim (1999)" in his "Randomization does not justify logistic regression"

1 10

Winston Lin @linstonwin.bsky.social · Mar 8

Clarification: my paper doesn’t advocate a specific estimator. That’s one of the meanings of “agnostic” in the title :)

2

Reposted by Winston Lin

Elea McDonnell Feit @eleafeit.bsky.social · Feb 19

An important plea from @lizstuart.bsky.social in today's SCI-OCIS Special Webinar Series:

A warning and a plea
- As fields start to use more advanced quantitative / "cause" methods, there is a desire to help consumers of the research (four journal reviewers, editor) to easily assess the study quality and validity (e.g. JAMA causality language)
...
- We need to push against this - need ways to help people understand and assess the (inherently untestable) assumptions in many studies.

2 2 9

Winston Lin @linstonwin.bsky.social · Feb 10

I don't wanna put words in Rosenbaum's mouth ("spectrum" is just a word that came to my mind for a quick Bluesky reply) and I'd really encourage anyone interested to read his papers and the Stat Sci discussion in full, and then critique them. :) But here's a screenshot from his reply to Manski

2

Winston Lin @linstonwin.bsky.social · Feb 10

competing theories. He has an interesting debate with Manski on external validity in the comments on the Stat Sci paper (I'll send you some excellent responses that my undergrad students at Yale wrote).

1 1

Winston Lin @linstonwin.bsky.social · Feb 10

can lead to badly misleading literatures; (3) we can sometimes learn from collections of studies with different designs & weaknesses (I think he's partly influenced by the literature on smoking & lung cancer, which he cites elsewhere); (4) we should try to falsify or corroborate predictions of 2/

1 2

Winston Lin @linstonwin.bsky.social · Feb 10

Thanks, Alex! I think that's a small part of his message. It's hard for me to do justice to these papers in a short thread, but I think he's also saying (1) credibility is on a spectrum and we should try to learn from all sorts of designs; (2) repeating the same design with the same weaknesses 1/

1 2