andrewbean.bsky.social
@andrewbean.bsky.social
PRISM (Oral Session 1b, Wednesday 10:00am) led by Hannah R Kirk, asks 'to whom' are we aligning LLMs. By collecting a global dataset of preferences through interactive dialogues, we highlight the importance of including a wide range if viewpoints in model alignment.
arxiv.org/abs/2404.16019
#NeurIPS
arxiv.org/abs/2404.16019
#NeurIPS
The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models
Human feedback is central to the alignment of Large Language Models (LLMs). However, open questions remain about methods (how), domains (where), people (who) and objectives (to what end) of feedback p...
arxiv.org
December 10, 2024 at 11:00 AM
PRISM (Oral Session 1b, Wednesday 10:00am) led by Hannah R Kirk, asks 'to whom' are we aligning LLMs. By collecting a global dataset of preferences through interactive dialogues, we highlight the importance of including a wide range if viewpoints in model alignment.
arxiv.org/abs/2404.16019
#NeurIPS
arxiv.org/abs/2404.16019
#NeurIPS
LingOly (Oral Session 4a, Thursday 3:30pm) is a new benchmark for reasoning in LLMs based on puzzles about low-resource langauges. We carefully control for memorised responses and find that top LLMs struggle to solve multi-step reasoning puzzles.
arxiv.org/abs/2406.06196
#NeurIPS
arxiv.org/abs/2406.06196
#NeurIPS
LINGOLY: A Benchmark of Olympiad-Level Linguistic Reasoning Puzzles in Low-Resource and Extinct Languages
In this paper, we present the LingOly benchmark, a novel benchmark for advanced reasoning abilities in large language models. Using challenging Linguistic Olympiad puzzles, we evaluate (i) capabilitie...
arxiv.org
December 10, 2024 at 11:00 AM
LingOly (Oral Session 4a, Thursday 3:30pm) is a new benchmark for reasoning in LLMs based on puzzles about low-resource langauges. We carefully control for memorised responses and find that top LLMs struggle to solve multi-step reasoning puzzles.
arxiv.org/abs/2406.06196
#NeurIPS
arxiv.org/abs/2406.06196
#NeurIPS