Federal agencies are shifting their artificial intelligence (AI) efforts from isolated pilots to mission-specific applications, but scaling those tools remains a challenge, government and industry leaders said at FedInsider’s Innovation Summit 2026 on Oct. 21 in Washington, D.C.
Officials from the National Institutes of Health (NIH), State Department, and Library of Congress outlined how their agencies are tackling that challenge – from identifying workflow problems and training employees to testing applications before committing to broader deployment.
NIH shifts focus to mission-specific applications
Joe Croghan, associate director of informatics for NIH’s Office of Cyber Infrastructure and Computational Biology, said agencies should begin by understanding employees’ workflow challenges rather than asking them how they want to use AI.
“I think one of the responsibilities we have is to not ask people, ‘How do you think you could use AI?’ I don’t think they know,” Croghan said.
Instead, he said, technical teams should identify where workflows break down, what data employees manage, and which tools could address those problems.
“[NIH] has pivoted away from the personal productivity that a lot of companies are providing [… and] focused on purpose-built applications,” Croghan said.
Supporting those applications requires more than individual development efforts. Croghan said NIH is working toward shared infrastructure that incorporates policy, data governance, and financial oversight, allowing the agency to support promising projects without creating disconnected tools that are difficult to manage or scale.
Cole Feely, a sales engineer for OEM Federal at Dell Technologies, noted, however, that the data challenge in the federal government could prevent AI applications from performing reliably.
“Data is extremely chaotic … you can’t put it into like a Formula One engine [and] expect it to operate properly,” Feely said.
He said agencies need to organize relevant datasets and establish security, monitoring, and access controls early in development, particularly for AI agents that may retrieve information or perform tasks on employees’ behalf.
State emphasizes training and measurable results
At the State Department, the challenge includes preparing a large, geographically dispersed workforce to use AI effectively.
Gary Barret, acting director of Super Intelligence Workforce Enablement for the Bureau of Diplomatic Technology’s Center of Analytics, said State supports more than 70,000 employees across more than 200 overseas missions.
Barret said AI training must be continuous, rather than a one-time exercise. Agencies need to update guidance as tools evolve and establish feedback channels so employees can influence how applications are developed and deployed.
“We need to not just listen once but keep listening and actually take that feedback and relay it back to the people, leadership, IT boards who actually make those decisions and ultimately the product owners as well,” Barret said.
State is also assessing how AI affects daily operations. For instance, Barret estimated that State Chat, the department’s in-house enterprise AI tool, saves users approximately two hours per week on average. He also highlighted Northstar, a media-monitoring application that helps employees prepare news summaries and brief senior officials more quickly.
But Barret cautioned against treating time savings as the only measure of success.
“Agencies also need qualitative feedback to determine whether AI applications improve employees’ work and support broader mission objectives,” he said.
Library of Congress builds flexibility into AI testing
The Library of Congress is approaching AI adoption by giving employees room to experiment before committing to production.
Natalie Buda Smith, the library’s director of digital strategy, said the organization uses an AI sandbox where employees can test proposed applications.
“We have an AI sandbox place for people to experiment, prove feasibility, and it does allow them to fail. If it’s not feasible, then we’re not moving forward,” Smith said, explaining that the approach allows the library to identify limitations early rather than investing in applications that may not meet operational needs.
Fred Millon, head of government go-to-market at OpenAI, concurred with Smith, adding that employees learn best by using approved AI tools directly, particularly on familiar tasks where they can independently check the results.
“The way you learn about AI is by using AI. These tools exist, and this is why the bottoms-up compliant access is important,” Millon said.
That emphasis on experimentation also shapes the library’s approach to technology acquisition. Smith said the agency needs procurement arrangements that can accommodate changes in AI technology rather than locking them into tools that may quickly become outdated.
“Vendors that have built-in flexibility in what they propose are very helpful,” Smith said. “That’s really what we’re looking for, mainly because this technology is changing really, really quickly.”