Lightnews — Scholar-powered news

Reposted by Anka Reuel ➡️ NeurIPS

Cas (Stephen Casper) @scasper.bsky.social · Apr 21

🚨New paper:

Current reports on AI audits/evals often omit crucial details, and there are huge disparities between the thoroughness of different reports. Even technically rigorous evals can offer little useful insight if reported selectively or obscurely.

Audit cards can help.

1 2 2

Reposted by Anka Reuel ➡️ NeurIPS

Stanford HAI @stanfordhai.bsky.social · Mar 25

A recent Stanford paper reveals that many popular AI benchmarks are fundamentally flawed: They can be outdated, easily gamed, or inaccurate. Stanford HAI Graduate Fellow
@ankareuel.bsky.social talks about how researchers are rethinking AI benchmarks: www.emergingtechbrew.com/stories/2025...

Some researchers are rethinking how to measure AI intelligence

www.emergingtechbrew.com

1 3 9

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 28

Hey Kabir! A lot of it is applicable for different types of evals, especially when it comes to reporting considerations. Would you mind sharing more infos here or via DM on the hackathon? Sounds like this would be a cool opportunity to extend the BetterBench work!

1 3

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 27

Submitting a benchmark to
ICML? Check out our NeurIPS Spotlight paper BetterBench! We outline best practices for benchmark design, implementation & reporting to help shift community norms. Be part of the change! 🙌

+ Add your benchmark to our database for visibility: betterbench.stanford.edu

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Nov 25

🚨 NeurIPS 2024 Spotlight
Did you know we lack standards for AI benchmarks, despite their role in tracking progress, comparing models, and shaping policy? 🤯 Enter BetterBench–our framework with 46 criteria to assess benchmark quality: betterbench.stanford.edu 1/x

1 2 11

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 5

This is such a hard one :D And I think it extends beyond being patient with the students but also being patient with yourself knowing that you won't get everything perfect the first time around (or ever 🥲)

1 6

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 5

🔄 Sharing is caring! Help us reach as wide of an audience as possible by spreading the word. Your support is key in crafting an insightful, community-driven chapter and help key researchers in the field get their work promoted! Thank you! 🙏#StanfordHAI #AIIndex x/

2 5

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 5

The AI Index is an initiative by @stanfordhai.bsky.social. The annual report showcases AI research to enable decision-makers to advance AI responsibly. Previous versions have been cited 300+ times; it's been featured in top media outlets like the @nytimes.com & the @financialtimes.com. 4/

1 5

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 5

Our chapter will cover fairness & non-discrimination, transparency, explainability, data governance & privacy, security, societal impact, and more. Plus, a special subchapter on responsible AI agents! 🤖 3/

1

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 5

We're seeking impactful AI ethics/safety research from 2024/2025 for inclusion in Stanford's 2025 #AIIndex. Submit your papers or nominate others’ work through our Google Form👇

forms.gle/Hgrzvsi9Yb2B... 2/

2025 Stanford AI Index – Responsible AI Chapter

Thank you for submitting research for consideration for the responsible AI chapter for this year's Stanford AI Index by Stanford HAI! The AI Index is an independent initiative at the Stanford Instit...

forms.gle

1 1 1

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 5

📢 Excited to share: I'm again leading the efforts for the Responsible AI chapter for Stanford's 2025 AI Index, curated by @stanfordhai.bsky.social. As last year, we're asking you to submit your favorite papers on the topic for consideration (including your own!) 🧵 1/

1 8 13

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 4

This is all awesome advice, thank you so much for sharing! This is an in-person course but we’ll make all lectures publicly available.

1

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Jan 4

I‘m teaching my first own course starting next week (Intro to AI Governance at Stanford). Super proud but also nervous 🥹 Any advice from more seasoned instructors? 😬 #AcademicTwitter #AcademicChatter #TeachingTips #AcademicAdvice

2 11

Reposted by Anka Reuel ➡️ NeurIPS

Stephan Geering @stephangeering.bsky.social · Dec 24

The regular reminder of my starter packs full of amazing folks / accounts to follow. I am trying to keep them up to date but let me know if I missed you.

Stephan Geering @stephangeering.bsky.social · Nov 30

I have updated my starter packs with lots of follow-worthy accounts since I last shared them. Take a look and follow generously! #PrivacySky #ResponsibleAI

Privacy and security part 1: go.bsky.app/6ApBSmA

1 5

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Dec 19

Thank you, Stefanie! ❤️

Anka Reuel ➡️ NeurIPS @ankareuel.bsky.social · Dec 19

As one of the vice chairs of the EU GPAI Code of Practice process, I co-wrote the second draft which just went online – feedback is open until mid-January, please let me know your thoughts, especially on the internal governance section!

digital-strategy.ec.europa.eu/en/library/s...

Second Draft of the General-Purpose AI Code of Practice published, written by independent experts

Independent experts present the second draft of the General-Purpose AI Code of Practice, based on the feedback received on the first draft, published on 14 November 2024.

digital-strategy.ec.europa.eu

5 14

Reposted by Anka Reuel ➡️ NeurIPS

Stanford HAI @stanfordhai.bsky.social · Dec 11

In our latest brief, Stanford scholars present a novel assessment framework for evaluating the quality of AI benchmarks and share best practices for minimum quality assurance. @ankareuel.bsky.social @chansmi.bsky.social @mlamparth.bsky.social hai.stanford.edu/what-makes-g...