Lightnews — Scholar-powered news

Pausal Zivference @pausalz.bsky.social · 34m

I haven't really paid attention to the lalonde data much, so what is the positivity violation that occurs?

1

Pausal Zivference @pausalz.bsky.social · 3h

Feel free, I can also send any additional details you need

1

Pausal Zivference @pausalz.bsky.social · 3h

Give it a read, there are some fun visualization in there, like this one

1

Pausal Zivference @pausalz.bsky.social · 3h

For years I had trouble following some of the discussion about confidence bands, but at ACIC this year @noahgreifer.bsky.social pointed me to a helpful paper

So you don't have to be as perplexed as I once was, we have a new pre-print introducing the key ideas
arxiv.org/abs/2510.07076

Confidence Regions for Multiple Outcomes, Effect Modifiers, and Other Multiple Comparisons

In epidemiology, some have argued that multiple comparison corrections are not necessary as there is rarely interest in the universal null hypothesis. From a parameter estimation perspective, epidemio...

arxiv.org

1 1 6

Pausal Zivference @pausalz.bsky.social · 4h

Glad I get to continue my anti-R (and associated R products) persona

1

Pausal Zivference @pausalz.bsky.social · 4h

The correct version would have instead re-sampled the age distribution form the data

This error is subtle (because it doesn't crash the program, I also missed it on my first glance) and anyone using LLMs like the quoted person is making these errors

2 1

Pausal Zivference @pausalz.bsky.social · 4h

Now the trick is that the code it output is actually wrong in a subtle way. I actually hadn't noticed the error it introduce because it won't output an error. The red boxes are the error.

I won't get into the finer details, but this causes the whole procedure to under-estimate the variance

1 2

Pausal Zivference @pausalz.bsky.social · 4h

and like the summary was fine. It was very basic and cursory, but no errors

It didn't highlight that I had code provided as part of the paper. So, I followed up by asking it for code. It generated a new example (using the details from the paper example)

1 1

Pausal Zivference @pausalz.bsky.social · 4h

I have an interesting case study on this actually. So at the beginning of the semester, I was preparing a bit on the (mis)use of LLMs for the course I co-teach

One of things I did was have it summarize one of my own papers, since people say "it's so good at it"
arxiv.org/abs/2503.02789

Accounting for Missing Data in Public Health Research Using a Synthesis of Statistical and Mathematical Models

Introduction: Missing data is a challenge to medical research. Accounting for missing data by imputing or weighting conditional on covariates relies on the variable with missingness being observed at ...

arxiv.org

2 2 5

Pausal Zivference @pausalz.bsky.social · 5h

I wish he posted the code, so we could also see that it was just a bunch of regressions (which makes the mysticism of the blog post about it much funnier to me)

Whenever I ask for estimators with several steps (eg TMLE) from scratch, it doesn't work

2 12

Pausal Zivference @pausalz.bsky.social · 20d

could be a cool mascot though

4

Pausal Zivference @pausalz.bsky.social · 21d

lol it's so bad that bsky doesn't even want to show the screenshot

1

Pausal Zivference @pausalz.bsky.social · 21d

Aptos is such a bad font, let along one to set as the default...

Microsoft Word document failing to render the default font, Aptos, in a legible way.

1

Reposted by Pausal Zivference

Jamie Cummins @jamiecummins.bsky.social · 21d

Can large language models stand in for human participants?
Many social scientists seem to think so, and are already using "silicon samples" in research.

One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want.

THREAD 🧵

The threat of analytic flexibility in using large language models to simulate human data: A call to attention

Social scientists are now using large language models to create "silicon samples" - synthetic datasets intended to stand in for human respondents, aimed at revolutionising human subjects research. How...

arxiv.org

12 150 330

Pausal Zivference @pausalz.bsky.social · Sep 9

The code above then fits the pooled logistic model, computes the censoring weights, and then estimates the risk function at the unique event times

This algorithm incorporates both aspects of computing IPCW

Finally, it gives us a consistent estimator of the variance without needing to bootstrap

Pausal Zivference @pausalz.bsky.social · Sep 9

I do this with the following line from above. What is does is it takes the original risk set matrix (which includes events in their interval) and then I subtract off a matrix that indicates where the events happened in time

That zeroes out the contributions to the pooled logit model for events

1

Pausal Zivference @pausalz.bsky.social · Sep 9

Okay so now we can look at how the elements from the EF are constructed. We construct 2 separate time design matrices (one for censoring times, the other for event times)

Here is where we exclude the events from contributing to the model (for the rows that correspond to the event times)

1

Pausal Zivference @pausalz.bsky.social · Sep 9

Then a product-limit (discrete-time Kaplan-Meier) is applied using the IPCW

1

Pausal Zivference @pausalz.bsky.social · Sep 9

For IPCW, we need to ensure they are aligned with the events times. To do this, I do a loop over the matrices and the unique event times. From the IPCW matrix, I create a new matrix that assigns the weights for each corresponding time by looking up the nearest time

(there is likely a better way)

1

Pausal Zivference @pausalz.bsky.social · Sep 9

I'll skip ahead to defining the EF, so we can see the forest a bit before looking at trees

We fit a pooled logit model (given some input design matrices) and then generate predictions. To lag, we simply add a row of 1's at the top of the predicted Pr of uncensored and drop the last row

1

Pausal Zivference @pausalz.bsky.social · Sep 9

The pooled logit algorithm in the pre-print thus needs to be modified so both of these are incorporated

The following is some Python code that does this process. First, is the setup

Python code that loads in packages and a few custom functions from a file called `efuncs`. That file simply contains some estimating functions that implement the pooled logistic procedure

The second part creates a simple data set to implement this procedure with

1

Pausal Zivference @pausalz.bsky.social · Sep 9

In long data sets, censoring usually happens at the end of the interval. Because of that, events happen before censoring, so they need to be excluded from the model

Further, the computed weights actually are important for the *next* interval since censoring is the last thing that happens

1

Pausal Zivference @pausalz.bsky.social · Sep 9

By convention, we events are assumed to happen before censoring in survival analysis. This convention has a few important (but sometimes overlook) implications when constructing IPCW

1. events in an interval do not contribute to the risk set at the event time
2. weights need to be lagged

1

Pausal Zivference @pausalz.bsky.social · Sep 9

I'm going to do a quick #MEstimatorMonday (36/52) to show how the pooled logit EE can be used to construct inverse probability of censoring weights (IPCW)

This is based on the algorithm in my pre-print, but needs a few tweaks

arxiv.org/abs/2504.13291

Estimating equations for survival analysis with pooled logistic regression

Pooled logistic regression models are commonly applied in survival analysis. However, the standard implementation can be computationally demanding, which is further exacerbated when using the nonparam...

arxiv.org

1 1 3

Pausal Zivference @pausalz.bsky.social · Sep 2

A well-spent $10m per year on Bill Belichick. Glad our university leadership really knows where to invest wisely

1 2