As the General Services Administration (GSA) expands its use of artificial intelligence (AI), the agency plans to rely on extensive telemetry data to determine whether AI investments continue delivering value once vendor discounts, free pilots, and other subsidies begin to disappear.
Speaking June 11 at the Government Service Delivery conference in Washington, D.C., GSA Chief Information Officer (CIO) David Shive said agencies need visibility into both operational and financial metrics to evaluate whether AI deployments are generating measurable mission and productivity gains.
The issue surfaced during a discussion about how agencies should assess AI projects once favorable pilot-phase economics disappear.
“I don’t have all the answers,” Shive acknowledged, “but I have all the telemetry.”
According to Shive, GSA maintains full observability into human-machine interactions, machine-to-machine communications, application programming interface activity, infrastructure spending, and generative AI token consumption.
That data allows the agency to connect AI usage to specific workflows and measure outcomes through traditional business metrics, including time saved, process efficiency, and service quality.
Shive said telemetry also helps identify situations where AI-generated productivity gains fail to translate into broader organizational improvements. For example, AI may enable developers to write code faster, but it might not accelerate the delivery of digital services if bottlenecks remain elsewhere in the development process.
The discussion highlights a growing focus across government and industry on measuring AI return on investment as organizations transition from pilot projects to enterprise-scale deployments.
Shive said GSA is working with academic institutions and private-sector partners to develop early methodologies for measuring AI value in government operations.
Until those approaches mature, he advised agencies to rely on established business metrics when evaluating AI deployments.
“If you’re applying AI and you’re not seeing positive value generation through those traditional business measures,” Shive said, “You should ask yourself, why are you doing that?”