Unless you study and read what it does.

Given the test (which i would have written myself) it was able to identify a number of issues, and now we're solving those.

The LLM needs a human to approach the problem in a solvable way.

I've pasted it error messages from the web page "inspect" console window. No joy.

So I gave it the advice of capturing and saving the actual data, and writing a set of tests using that data to evaluate the creation of the graph data, the 'communities' output, and the visualisation.

I do that also, even when I have it write something (a tool, usually). 'Examine these tools together as a set and suggest simplifications and common utilities'

Usually there are tools that exist for this, so they only have to be tailored to my operating systems and editors. Sometimes I don't know what tools are available, and I have an LLM create scripts and configure the tools for me.

it depends. Sometimes it's checking blogposts for bad links, sometimes it's checking markdown documents that are specific to a particular publisher. Sometimes it's data formats.

I've had it quickly convert primitive types to NamedTuple or DataClass types in Python, which is small work that is easily done but tedious if there is much to it.

Sometimes I will start a dialogue about an output checker, and it will bring up tools that I didn't know, some of which are irrelevant or inappropriate, but I learn and get output checkers sooner.

"Yeah, but they were just writing a few functions and they were okay at the end."

They could have been okay the whole time.
Look, I'm not demanding TDD with microcommits.
Just don't ignore error marks. Fix that before going on.

Using it to generate quality assessment tools, link checkers and file formats, and to help spot the impact of fixing or not fixing some issues.

Sometimes there are things I wouldn't have done because it takes too long by hand.

I walk because I can.

I know they're not uncommon, but it's always another nano-task when reading them.

And, no, I don't suppose that you will hit it with the couch on move-in day and destroy the door and the lock. It's not in the way. It will be fine. Don't worry about things like that.
No, the kids won't be tempted to hang from it! Where do you get this nonsense?

????

My ide has 2 LLMs in it, and one at the command line.
And a linter, and a test auto-runner, and...

Where have you been that you don't have your LLMs integrated?