Charles Foster
@cfoster.bsky.social
Twitter: @CFGeek
Mastodon: @[email protected]
When I choose to speak, I speak for myself.
🪄 Tensor-enjoyer 🧪
Mastodon: @[email protected]
When I choose to speak, I speak for myself.
🪄 Tensor-enjoyer 🧪
Update for those who’ve left the other app:
I’m now on the policy team at Model Evaluation and Threat Research (METR). Excited to be “doing AI policy” full-time.
I’m now on the policy team at Model Evaluation and Threat Research (METR). Excited to be “doing AI policy” full-time.
March 7, 2025 at 3:19 AM
Update for those who’ve left the other app:
I’m now on the policy team at Model Evaluation and Threat Research (METR). Excited to be “doing AI policy” full-time.
I’m now on the policy team at Model Evaluation and Threat Research (METR). Excited to be “doing AI policy” full-time.
Why aren’t our AI evaluations better? AFAICT a key reason is that the incentives around them are kinda bad.
In a new post, I explain how the standardized testing industry works and write about lessons it may have for the AI evals ecosystem.
open.substack.com/pub/contextw...
In a new post, I explain how the standardized testing industry works and write about lessons it may have for the AI evals ecosystem.
open.substack.com/pub/contextw...
February 26, 2025 at 4:35 AM
Why aren’t our AI evaluations better? AFAICT a key reason is that the incentives around them are kinda bad.
In a new post, I explain how the standardized testing industry works and write about lessons it may have for the AI evals ecosystem.
open.substack.com/pub/contextw...
In a new post, I explain how the standardized testing industry works and write about lessons it may have for the AI evals ecosystem.
open.substack.com/pub/contextw...
You want traders to give you cheap but calibrated estimates for all the claims. The randomization reduces the expected size of payouts they’d receive for their bets, since each market only has a 10% chance of getting audited & resolved, but it preserves the incentive to bet their true probabilities.
December 24, 2024 at 8:58 PM
You want traders to give you cheap but calibrated estimates for all the claims. The randomization reduces the expected size of payouts they’d receive for their bets, since each market only has a 10% chance of getting audited & resolved, but it preserves the incentive to bet their true probabilities.
Neat thing about real-money prediction markets is that you can get paid for doing this.
November 30, 2024 at 4:40 PM
Neat thing about real-money prediction markets is that you can get paid for doing this.
If you’re driving from Chicago, it can make sense to talk of racing to NYC. But once in NYC, that aim no longer has clear meaning.
IMO this “near vs. far” distinction explains why previous AI goals now seem vague even though they seemed clear when farther on the horizon. (4/6)
IMO this “near vs. far” distinction explains why previous AI goals now seem vague even though they seemed clear when farther on the horizon. (4/6)
November 27, 2024 at 7:01 AM
If you’re driving from Chicago, it can make sense to talk of racing to NYC. But once in NYC, that aim no longer has clear meaning.
IMO this “near vs. far” distinction explains why previous AI goals now seem vague even though they seemed clear when farther on the horizon. (4/6)
IMO this “near vs. far” distinction explains why previous AI goals now seem vague even though they seemed clear when farther on the horizon. (4/6)
Still gathering my thoughts on @TheCurveConf, but for now, a short reflection on why I like “the curve” as a way of thinking about the future of AI. (1/6)
November 27, 2024 at 7:01 AM
Still gathering my thoughts on @TheCurveConf, but for now, a short reflection on why I like “the curve” as a way of thinking about the future of AI. (1/6)
“I had worked hard for nearly two years, for the sole purpose of infusing life into an inanimate body. For this I had deprived myself of rest and health […] but now that I had finished, the beauty of the dream vanished, and breathless horror and disgust filled my heart.”
- M. Shelley in Frankenstein
- M. Shelley in Frankenstein
November 13, 2024 at 10:27 PM
“I had worked hard for nearly two years, for the sole purpose of infusing life into an inanimate body. For this I had deprived myself of rest and health […] but now that I had finished, the beauty of the dream vanished, and breathless horror and disgust filled my heart.”
- M. Shelley in Frankenstein
- M. Shelley in Frankenstein