Agencies across the federal government adopting artificial intelligence (AI) need to limit what AI agents can access and do, government and industry officials said Oct. 21, as increasingly autonomous tools test how agencies apply zero trust security and human oversight.
Speaking at FedInsider’s Innovation Summit 2026 in Washington, D.C., the experts discussed extending existing security practices to AI systems that retrieve information, call tools, and act on employees’ behalf. Their recommendations included restricting agents’ privileges, monitoring their inputs and outputs, and giving people a way to stop them.
The discussion also exposed differing views on human oversight: A Customs and Border Protection (CBP) official argued that people must remain involved, while an Agriculture Department (USDA) official said resource constraints make constant human involvement difficult to sustain.
Least privilege must extend to AI
Rudolf Rojas, IT program manager and security lead at USDA, said AI agents should receive minimal privileges rather than automatically inherit the access rights of the employees using them.
“I would not want to give an agent my privileges or roles,” Rojas said. “The AI models today are just not mature enough unless you’ve got a really good risk management framework, you’ve done analysis of the networks, and you’ve got a great demilitarized zone within your agency, like we do at USDA.”
USDA’s zero trust work has helped it prepare for AI security challenges, he said. Rojas described employee training, multifactor authentication, vetting of AI tools, defined access privileges, and continuous network monitoring as elements of the department’s approach.
Felipe Fernandez, chief technology officer at Fortinet Federal, identified prompt injection as a realistic threat agencies face as models and agents interact with external information and tools. He described how material retrieved from a website or returned by a tool can become part of the context a model uses to carry out a task.
That creates risks beyond what employees type into an AI application or receive directly in its answers, he said. That’s why agencies need visibility into the interactions that happen as agents work.
“You can’t expect the model to protect itself,” Fernandez said.
Fernandez recommended placing security and governance controls outside the agent and model. He described an AI gateway that monitors inputs and outputs, providing visibility into communications and a way to interrupt access to resources.
For workflows involving multiple agents, he recommended dividing tasks among agents and giving each only the privileges needed for its specific actions. Employees responsible for an agent also need a way to stop it, he said.
“They need to have a kill switch,” Fernandez said.
Experts debate human involvement in AI
Rojas questioned whether agencies can keep a person involved at every step as they expand their use of AI, citing cutbacks and limited contract support. He described a balance in which tools perform work and report back to employees, supported by restricted privileges and continuous monitoring.
“We’re going to have to get smarter and utilize these tools so we can have them do the work, and then report back to us,” Rojas said. “And then, also using the privileges of zero trust, we can do continuous monitoring of those networks and the tools.”
Ajay Phogat, senior advisor to the chief information officer at CBP, emphasized maintaining human oversight as AI pursues assigned goals.
“[The] human has always to be in the loop, and that’s the approach we’re taking,” Phogat said.
Phogat also stressed using approved models and protecting personally identifiable information and law enforcement sensitive information from leaving agency networks through AI use. Agencies need clear policies and workflow definitions before moving forward with AI, he said.
Security policy should be consistent and span human users, devices, workloads, and AI agents, advised Judson Walker, senior system engineering director at Versa Networks. He advocated centralized policy control with enforcement close to the systems being protected, including devices operating far from an agency’s core infrastructure.
Dan Carroll, field chief technology officer at Dell Federal, said agencies need to treat security as an organizational culture that shapes how employees evaluate requests and handle sensitive information.
“Security is often looked at as a compliance goal, not a culture goal,” Carroll said.
As AI makes impersonation more convincing, employees need to question interactions they once would have accepted as routine, he said. A familiar voice or face on a video call may no longer establish someone’s identity. Requests involving sensitive data or access should prompt additional verification, such as calling back through a known number, Carroll advised.