Amazon honors ASU researcher for agentic AI security work

W. P. Carey information systems professor received the honor for research focused on securing increasingly autonomous artificial intelligence systems.

Every spring, Amazon recognizes outstanding research by academics worldwide. This year, Assistant Professor of Information Systems Xiyang Hu received an Amazon Research Award (ARA) for his work on AI for information security. Hu is one of 70 recipients representing 49 universities across 11 countries.

For Hu, the ARA is not only an honor but a validation of W. P. Carey's position as a leader in AI research.

"As AI systems become more autonomous, the conversation must shift from just making them capable to making them safe and reliable. This award provides crucial support for our team to tackle these emerging security challenges," Hu says. "It also represents a fantastic opportunity to bridge academic research with real-world, industry-level infrastructure, ensuring that the safety frameworks we develop can actually be deployed to protect enterprise systems and user data."

Agentic AI — the next step in the evolution of artificial intelligence — can reason, create step-by-step plans, and take autonomous actions. When several AI agents collaborate, they form powerful multi-agent systems, but their autonomy creates blind spots that can be exploited, leading to leaked data or unauthorized actions.

Hu's research, titled "Securing Agentic AI: From Local Detection to Global Assurance," proposes a two-layered defense system to mitigate these risks.

"Locally, we use lightweight monitors to double-check an agent's reasoning and tool use for any abnormal behavior. Globally, we map out how different agents interact with each other over time using what we call a 'Global Interaction Graph,'" Hu explains. "If our system detects a hidden chain of risky actions, it automatically limits the AI's privileges or pauses its tasks. Ultimately, we are building the guardrails that allow companies to use autonomous AI safely and predictably."

Hu first worked with Amazon as a research intern while earning his PhD. The experience exposed him to the organization's extensive cloud infrastructure and the ARA program, which provides academics with research funding and Amazon Web Services promotional credits for academic research. Amazon released a call for proposals on agentic AI and AI for information security last year, which he describes as "a perfect full-circle moment."

"The challenges they are looking to solve align exactly with our mission to build secure, reliable frameworks for autonomous systems," says Hu, who is a member of W. P. Carey's Center for AI and Data Analytics (AIDA) for Business and Society.

Through the center, Hu focuses on the safety, robustness, and reliability of autonomous AI. His current research explores how communication structures within multi-agent systems can lead to unintended memory leakage, helping researchers measure, understand, and prevent accidental data exposure as these systems become more complex.

Hu is also studying how AI agents use external tools and software by evaluating "over-privilege" in agent skills to ensure an AI cannot access more system permissions than necessary, and researching the robustness of tools so AI doesn't break down in multilingual environments.

"We are continuously working on foundational model robustness, such as mitigating sim-to-real gaps, LLM hallucinations, and more," says Hu. "It is an exciting time for our group as we pull all these threads together to build the next generation of mindful AI."

Learn more about Hu's research.