Lightnews — Scholar-powered news

LightNews

Gabriel

@morecoffeeplz.bsky.social

1K followers 460 following 370 posts

AI Research scientist. Former OpenAI, Apple infosec. “Professor” at John’s Hopkins SAIS Alperovitch Institute. Great deceiver of hike length and difficulty.

Posts Media Videos Starter Packs

Reposted by Gabriel

sina @rejectionking.bsky.social · 23h

“What if you could fuck the singularity?” is the apotheosis of technofuturism (2025)

Dr McCoy in peak bisexual lighting

The VibeCamp Archipelago

So the nightmwate begins

NSA and FBI just shocked at what they are seeing

Reposted by Gabriel

Sheryl Gay Stolberg @sherylnyt.bsky.social · 4d

BREAKING: Friday night massacre underway at CDC. Doznes of "disease detectives," high-level scientists, entire Washington staff and editors of the MMWR (Morbidity and Mortality Weekly Report) have all been RIFed and received the following notice:

Gabriel @morecoffeeplz.bsky.social · 10d

👋

Gabriel @morecoffeeplz.bsky.social · 14d

Some research from my team!

SentinelOne @sentinelone.com · 26d

🔎 Attackers are embedding LLMs directly into malware, creating code that can generate malicious logic at runtime rather than embedded in code.

🔥New @sentinellabs.bsky.social research by @alex.leetnoob.com, @vkamluk.bsky.social, and Gabriel Bernadett-Shapiro at #LABScon 2025. 🔥 s1.ai/llm-mw

Gabriel @morecoffeeplz.bsky.social · 14d

@sentinelone.com social team I am also on bluesky 😂

Reposted by Gabriel

sina @rejectionking.bsky.social · 19d

james comey (2025)

I had failed everyone

Nothing left to do but vibe

Why American TV is so vulnerable to foreign disinformation

James Comey, ghostface, and method man

Gabriel @morecoffeeplz.bsky.social · 20d

Not the BPO report we need, but definitely the one we deserve.

John Hultquist @hultquist.bsky.social · 21d

We are releasing details on BRICKSTORM malware activity, a China-based threat hitting US tech to potentially target downstream customers and hunt for data on vulnerabilities in products. This actor is stealthy, and we've provided a tool to hunt for them. cloud.google.com/blog/topics/...

Another BRICKSTORM: Stealthy Backdoor Enabling Espionage into Tech and Legal Sectors | Google Cloud Blog

BRICKSTORM is a stealthy backdoor used by suspected China-nexus actors for long-term espionage.

cloud.google.com

Gabriel @morecoffeeplz.bsky.social · 22d

3. What additional constraints do LLMs produce for adversaries? Hunting with the contraints of our adversaries was our initial premise. We've been doing it for years, LLMs simply present a new dimension for us to explore. If you'd like to work with us on this please let us know!

Gabriel @morecoffeeplz.bsky.social · 22d

Malware that can run simple instructions, identify the target device, important files, and provide summaries back to a C2 would eliminate or streamline a significant amount of adversary workload.

Gabriel @morecoffeeplz.bsky.social · 22d

2. LLM-enabled malware is interesting and (we believe) important to study, but it is unclear exactly what the operational advances are. Assuming we get to the point of LLMs running natively on endpoints malware that could hijack that process may be extremely useful.

Gabriel @morecoffeeplz.bsky.social · 22d

Ok some questions that this research posed for us:

1. Hunting for prompts and API keys works, but it is a brittle detection. Eventually adversaries will move to proxy services that provide some level of obfuscation. What do we do then?

Gabriel @morecoffeeplz.bsky.social · 22d

If we want to understand LLM risks, we should align expectations with risks we can observe and measure, not hype.

Gabriel @morecoffeeplz.bsky.social · 22d

Understanding how capable LLMs are wrt hacking is important work, but setting that aside for the moment, in a year of analysis we did not observe the capabilities that labs are concerned with being deployed by malicious actors in the wild.

Gabriel @morecoffeeplz.bsky.social · 22d

We noted that the capabilities we observed in LLM-enabled malware were operational, that is they helped adversaries with specific tasks.

That aligns with current LLM capabilities in software development and how they’re deployed.

Gabriel @morecoffeeplz.bsky.social · 22d

Traditionally, malware analysis starts at a disadvantage, you work backward from development assumptions.

With prompts, intent is immediately visible. No need to second-guess the adversary’s aim.

Gabriel @morecoffeeplz.bsky.social · 22d

This was by far our most successful technique.

We quickly identified prompts for agentic computer-network exploitation, vulnerability injectors, shellcode generators, WormGPT copycats, apps designed to control Android screens, and red-teaming tools for LLM-agent benchmarking.

Gabriel @morecoffeeplz.bsky.social · 22d

In our case, we knew the prompts were hardcoded and that they followed certain formats, structures, or keywords.

This isn’t so different from hunting code patterns, but instead we’re hunting strings and patterns in hardcoded prompts.

Gabriel @morecoffeeplz.bsky.social · 22d

There’s a lot to improve here, but we were excited the classification method worked at scale. Still we wanted to find more samples which led us to our final method.

In traditional malware, we hunt code; in LLM-enabled malware, we hunt prompts.

Gabriel @morecoffeeplz.bsky.social · 22d

For Java files, we built a custom solution: extract potential prompts via regex, identify/sanitize API keys, and send file content to an LLM to summarize class behavior and assess suspicious/malicious/benign. As a lightweight classifier, it was efficient, but flagged only one file as malicious.

Gabriel @morecoffeeplz.bsky.social · 22d

We used regex to spot LLM provider indicators (API keys, domains, prompt syntax), then decompiled and identified classes referencing LLM artifacts or behaviors.

Java files and .so binaries were the most promising since, at this stage, code is decompiled and readable.

Gabriel @morecoffeeplz.bsky.social · 22d

Back to our original catch: of nearly 7k samples, ~4k were Android. @alex.leetnoob.com built a GCP pipeline to upload APK/DEX to a bucket, then processed them with a script that decompiled each APK.

Gabriel @morecoffeeplz.bsky.social · 22d

So, hunting by API keys and simple clustering works... but it isn’t the most effective.

There were many false positives, and sorting required significant manual review.

Gabriel @morecoffeeplz.bsky.social · 22d

Then something interesting, TestMal3 ironically a defensive tool (“FalconShield”).

This is a brittle scanner checking for “import openai” + “exec(” patterns in a target Python file. It asks GPT to judge if code is malicious, writes a “malware analysis” report, and (claims to) hand off to VT.

Gabriel @morecoffeeplz.bsky.social · 22d

TestMal2 has a more structured builder/loader, more nuanced ransomware/reverse-shell menu options with IP/Port, cleans the LLM response, writes to a separate script, then executes it.

Some obfuscation attempts are present, but still a straight malware-generation pipeline.

Gabriel @morecoffeeplz.bsky.social · 22d

So MalTerminal.exe is the compiled malware.

testAPI (1) & (2) are functionally identical Python loaders offering Ransomware or Reverse Shell options.

TestMal2 is a more advanced version of testAPI likely an early Python version of the compiled Malterminal.