Logging the AI: One Engineer’s Push for Practical Oversight

In August 2026, inside DEF CON’s AI Village in Las Vegas, Anshumaan Mishra stood beside a poster with a title that sounded more like an internal work request than a big conference headline: “Detecting Unauthorized Tool Calls Using Ollama and Splunk.” There was no stage and no spotlight. People wandered by between sessions, some already weighed down with tote bags from vendor booths, others still adjusting to the bright hallways after hours in windowless rooms. Mishra stayed put, ready to explain his idea to anyone who stopped.

Mishra ended up giving the same overview again and again, but the discussions never felt identical. Some attendees wanted to understand how the detection worked. Others cared less about the “how” and more about whether it could fit into systems they already used. By the end of the day, Mishra said he realized he had not only been sharing research. He had been learning something more practical: if you want people to adopt a new security idea, you have to understand what matters to them and describe it in their language.

DEF CON is famous for big moments: dramatic demos, live hacks, and the sense that someone has already found a weakness in what the rest of the world still relies on. Mishra’s message was more practical than flashy. He wanted security teams to treat activity from locally run AI models like any other routine security record, something that can be collected, searched, and reviewed alongside logs from laptops, servers, and email systems. His poster highlighted PromptMon, an open-source Python library he first released in May 2026. Now at version 0.2.2, it remains an early-stage project: functional, focused, and still evolving rather than packaged as a commercial product. Mishra also emphasizes that PromptMon is independent work, separate from his role as a cyber automation engineer at North Dakota Information Technology.

The problem he is trying to address is increasingly common as AI tools spread through workplaces. More organizations are running language models on their own machines instead of sending every question to a remote service. Tools like Ollama make it easier to host models locally. But once an AI model is allowed to do more than write text, once it can connect to other programs to retrieve a file, search internal content, or trigger a workflow, there is a new risk. A malicious message can try to trick the model into taking actions no one intended or approved.

PromptMon is built around a straightforward idea: treat AI actions like events that can be tracked. The library collects records such as the prompt a user entered, the tools the model attempted to use, the information passed into those tools, the outputs that came back, and the time the activity occurred. It can also work with a separate text-classification model that scores prompts for suspicious behavior, allowing teams to set their own thresholds for what should be flagged.

What mattered most in the poster, though, was not the novelty of the data. It was where that data goes. Mishra designed PromptMon to feed those records into Splunk, a system many security teams already use to store and search logs. His reasoning was practical: security organizations already have established routines for how they investigate problems. Introducing a brand-new security tool, even a clever one, can create friction, training requirements, and yet another place to look when something goes wrong. Logging AI activity into existing systems can help teams see it alongside everything else they already monitor.

Mishra does not present PromptMon as a complete solution to AI risk. Any system that scores prompts depends on the quality of the model doing that scoring, and simply collecting logs only helps if teams turn them into real alerts and response procedures. His point is that routing AI activity into the tools security teams already use creates a clear path to those next steps, even if it does not deliver them automatically.

That difference came up again and again throughout the day at DEF CON. Security tools often fall short for practical reasons, not because the idea is wrong. They stall when they do not fit existing workflows, when the person expected to install them is not the person who will rely on them, or when even a small change gets stuck in approvals and competing priorities. At the poster, Mishra saw the split firsthand. Some attendees wanted to dig into how detection worked; others wanted to know how easily it could plug into what they already run. Underneath both questions was the same concern: would this create more work, or help reduce it?

The poster format fits that reality. A talk delivers one message to one room, but a poster becomes a chain of brief conversations, shaped by the concerns of whoever stops. Mishra has said it was his first real experience trying to convince strangers to adopt something he built. PromptMon is free to use, but winning attention and trust takes work. At its core, the project is a push to treat AI activity as a real source of security information, not something to ignore.

Skeptics can also point to what is still unknown. PromptMon’s first public release came in May 2026, and a DEF CON poster is not the same as long-term field testing. So far, there is no public data showing how widely the library has been deployed, how often it triggers false alarms, or how it holds up under everyday conditions when inputs and outputs get unpredictable. Those open questions do not make the project unserious. They place it where many useful tools begin: early, promising, and still being proven in the real world.

Many security teams are now facing a broader challenge: as AI systems move from answering questions to taking actions, those actions need to be visible. If the people responsible for investigating incidents never see what an AI system attempted to do, organizations risk creating a new blind spot and mistaking it for efficiency. Mishra’s contribution, as presented in Las Vegas, is not a sweeping prediction about where AI is headed. It is a practical way to capture AI activity and place it where security teams already look.

He left DEF CON with a takeaway that was more about adoption than technology. Identifying a real risk is not always enough to change behavior. For a new signal to matter, it has to show up in the tools people already trust, and it has to be explained in terms that match the listener’s priorities. It is not glamorous work, but it is often how change happens, one conversation at a time.

Also Read: Google Will Cut Off Search Access for Students at Schools That Reject AI