Author | Lightnews

David Nelson @thedavenelson.bsky.social · 2d

Repression breeds resistance.

David Nelson @thedavenelson.bsky.social · 5d

After multiple months using Copilot pro on trial (normally $30/month per user) I would say its ability to search all your Microsoft stuff at once is the killer app. Its chatbot inference and output abilities are as bad as an airline phone bot that makes you say all your questions.

David Nelson @thedavenelson.bsky.social · 6d

Bear the F Down!!!

David Nelson @thedavenelson.bsky.social · 12d

This is an awesome small-scale look at how sycophantic LLMs lead learners astray in problem-solving. Next, I'd love to look at what type of sycophancy actually attracts students if given a choice of bots. arxiv.org/abs/2510.03667

Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks

Sycophancy, the tendency of LLM-based chatbots to express excessive enthusiasm, agreement, flattery, and a lack of disagreement, is emerging as a significant risk in human-AI interactions. However, th...

arxiv.org

2

David Nelson @thedavenelson.bsky.social · 13d

Sycophancy in bots is an inimical part of AI in Teaching and Learning. When the bot wants to tell you that you are right, high dependence almost certainly means you will inculcate incorrect knowledge. Love papers like this who explore sycophancy in the discipline arxiv.org/abs/2510.04721

BrokenMath: A Benchmark for Sycophancy in Theorem Proving with LLMs

Large language models (LLMs) have recently shown strong performance on mathematical benchmarks. At the same time, they are prone to hallucination and sycophancy, often providing convincing but flawed ...

arxiv.org

David Nelson @thedavenelson.bsky.social · 14d

I also brought them up multiple times, to multiple students in class and in reflection feedback.

1

David Nelson @thedavenelson.bsky.social · 14d

I was surprisingly disappointed (after I library reserved and even bought landmark SciFi books on AI for suggested readings in my AI in Teaching and Learning Course) when not a single student bothered with Murderbot, Moon is a Harsh Mistress or others. Need to assign them next time. :(

1 4

David Nelson @thedavenelson.bsky.social · 14d

How well does it work when prompted only to assess or evaluate the root cause? Is tuning or exemplar code helpful at all? Are the edge cases simply so varied that an LLM as root causal evaluator is doomed to fail?

1

David Nelson @thedavenelson.bsky.social · 21d

Skeptical of most 3rd party IT company analyses of other software, but this SCALE AI paper outlines many of the limits of creating and evaluating "AI as Tutor" studies. Too many edge cases. arxiv.org/abs/2510.02663

TutorBench: A Benchmark To Assess Tutoring Capabilities Of Large Language Models

As students increasingly adopt large language models (LLMs) as learning aids, it is crucial to build models that are adept at handling the nuances of tutoring: they need to identify the core needs of ...

arxiv.org

David Nelson @thedavenelson.bsky.social · Sep 18

Try Copilot 365 if that is not sufficiently inaccurate or opaque for you. That $30/user/month fee pays for itself when your goal is to project existential dread into rage against a machine. Bonus points - it chides you for expletive use in emails :)

1

David Nelson @thedavenelson.bsky.social · Sep 17

Balatro is the answer

David Nelson @thedavenelson.bsky.social · Sep 16

what did it sound like?

1

David Nelson @thedavenelson.bsky.social · Sep 15

Lots of guesses about student use of AI are turning out as expected: Students use them to solve/complete homework; they ignore LLM-posted questions guiding them to think deeper, single-prompts dominate interactions with little dynamic engagement.

arxiv.org/abs/2509.08862

Investigating Student Interaction Patterns with Large Language Model-Powered Course Assistants in Computer Science Courses

Providing students with flexible and timely academic support is a challenge at most colleges and universities, leaving many students without help outside scheduled hours. Large language models (LLMs) ...

arxiv.org

1

David Nelson @thedavenelson.bsky.social · Sep 1

So, we can just keep on thinking Isak and Wirtz just wont work out for some reason, right???

David Nelson @thedavenelson.bsky.social · Aug 21

Want more studies on small-level feedback mechanisms around disciplinary understandings. No need to have an LLM measure "everything". Start small, start with a need area.
arxiv.org/abs/2508.14823

Using an LLM to Investigate Students' Explanations on Conceptual Physics Questions

Analyzing students' written solutions to physics questions is a major area in PER. However, gauging student understanding in college courses is bottlenecked by large class sizes, which limits assessme...

arxiv.org

David Nelson @thedavenelson.bsky.social · Aug 20

Love it. But I think you mispelled "Quadruple"

5

David Nelson @thedavenelson.bsky.social · Aug 19

1) AI claims fast learning; but learning is slow
2) Inauthentic or irrelevant work -> students outsourcing thinking work
3) We have MUCH less institutional infrastructure conveying value, authenticity and utility of what we teach
4) Chatbots do not care if we learn
5) AI detectors are horrid judges

1

David Nelson @thedavenelson.bsky.social · Aug 19

Our five big assumptions that shaped the week /

David Nelson @thedavenelson.bsky.social · Aug 19

1

David Nelson @thedavenelson.bsky.social · Aug 19

Purdue's AI Academy finished with 70+ instructors creating projects, plans, tools or critical approaches around and in response to AI. I was particularly enthused when multiple participants said "I thought I was gonna learn the tech, but I learned about learning"

3 1

Reposted by David Nelson

Derek Bruff @derekbruff.bsky.social · Aug 19

Today on the podcast: Study Hall! @leaton01.bsky.social @michellemillerphd.bsky.social and @thedavenelson.bsky.social and I discuss three recent studies exploring the intersection of AI and teaching. Cognitive offloading, chatbot sycophancy, & more! intentionalteaching.buzzsprout.com/2069949/epis...

3 6

David Nelson @thedavenelson.bsky.social · Aug 17

Democratizing prompt for LLMs:

Read and review new terms of service for X company. Compare and contrast with previous versions. What should I be aware of? What might any consumer be wary of or concerned about?

David Nelson @thedavenelson.bsky.social · Aug 9

Reminds me of the specter of internet throttling before net neutrality. Don’t want to pay us extra for our tech? Fine. You just might not like what you get. No transparency, varying quality of an information commodity on every use. I’d guess an upswing in paid subs.

David Nelson @thedavenelson.bsky.social · Aug 7

Likely Microsoft wants to gain benefits of Chat but not replace any of its own software that could then hinder enterprise level negotiations

1

David Nelson @thedavenelson.bsky.social · Aug 5

Article

arxiv.org/abs/2508.01503

A Theory of Adaptive Scaffolding for LLM-Based Pedagogical Agents

Large language models (LLMs) present new opportunities for creating pedagogical agents that engage in meaningful dialogue to support student learning. However, the current use of LLM systems like Chat...

arxiv.org