The article is 1.5 years old and unfortunately doesn’t mention the versions used. Is the observation still vali? Can it be generalized to LLMs can’t even (i.e. will never be able to) summarize well?

I guess you got that from the artwork. Well it's what rebar3 does: build, deploy and run. rebar3 compile, rebar3 hex, rebar3 shell