Though LLMs as persuasion tools are (rightly) getting attention, their most valuable civic use may be retrieval: grounded answers with links to the source material using language that is accessible to users. Positive use cases aren’t dominating the discourse, but that’s where I think the upside is.

In ongoing work w/ Alec Ewig, we explore how RAG can enhance measures of representation by retrieving relevant legislation from a 15,000+ bill database mapped to voter preferences. RAG + agentic workflows can uncover info buried in dense policy docs and answer complex queries with citations.

We used retrieval-augmented generation (RAG), a method that pulls relevant text from curated sources directly into prompts. A vector database, multiple API calls, and a lot of trial and error later, it substantially reduced errors and preserved issue-specific language from party platforms.

When we started this project, standard LLMs weren't up to the task. They produced plausible-sounding answers but regularly hallucinated policy stances, especially on niche or novel issues. For a voting advice application, accuracy was critical, so we had to build a more involved approach.

Congrats, Damien!

I have a 2024 pre-election Verasight survey I carried out with Alec Ewig that used an adaptive survey method to create a kind of dynamic CES. Happy to chat offline if you think it would be useful!

All of this is to say that I hope I’m also invited to the party, not only because I care about identifying causal effects, but because I also care about measuring theoretical constructs with a level of precision that quasi-experimental and field experimental designs simply can’t deliver.

As we increasingly interact with GUIs via LLMs, targeted ads, and social media, survey experiments are *even more* fit for the task of understanding human behavior, and I hope we continue relying on them to sort through thorny causal questions.

That’s a useful distinction. Another way to put it is whether a treatment *intervenes* on an outcome. On its face, your standard persuasion experiment looks a lot like “do you prefer [Joe/Jose] for this job?,” but the latter design isn’t trying to shift Y. It’s a measurement exercise.

It's likely not a problem *yet*

Thanks for flagging! I know Brave has several privacy layers. Will look into it.

Thanks! Have to channel my existential panic into something!

Surveys are going to need new ways to establish human presence as AI gets faster and harder to detect. Pulse is one step in that direction.

Special thanks to @seanwestwood1.bsky.social who flagged several spoofing paths. This is exactly the kind of scrutiny we’ll need as we adapt to this new era.

This doesn’t solve the “verify-then-handoff” problem. But it does filter out the most aggressive headless-browser attacks and gives us a first layer of defense in an era where attention checks and gotcha questions fail.

Demo and QSF download here: columbiauniversity.qualtrics.com/jfe/form/SV_...

I'm currently working on Pulse, a “proof-of-life” approach compatible with Qualtrics that uses a simple finger-on-lens verification task to confirm human presence. This approach avoids privacy concerns tied to capturing faces or living environments, and it works on laptops and smartphones.

I used to believe that survey aesthetics have minimal effects on completion rates and attentiveness… until I saw that slime-green color scheme