FITARA Scorecards Could Better Reflect Progress, CIOs Say

The third iteration of FITARA scorecards released by Congress showed room for improvement for agencies—none scored an "A"—but some chief information officers believe the scorecards themselves ought to be improved.

Congress assesses grades to agencies based on metrics like data center consolidation, PortfolioStat reviews, transparency and risk management and now CIO authorities, with data from agencies and the Government Accountability Office. In theory, the scorecards are great tools to motivate agencies to perform better, but they may not always accurately capture much of actual progress.

“I just don’t think it really reflects the work and accomplishments we’ve made,” Carlene Ileto, executive director of the Homeland Security Department's Enterprise Business Management Office, said of the scorecards.

» Get the best federal technology news and ideas delivered right to your inbox. Sign up here.

Ileto, speaking at a Nextgov event Thursday alongside Environmental Protection Agency CIO Ann Dunkin, praised FITARA as key legislation that “is truly changing DHS across the board,” promoting collaboration between the CIO, chief financial officer and component CIOs. But the scorecards aren’t telling that story.

“When I look at the FITARA law, it talks about acquisition … it talks about budget, and making sure the CIO knows what component CIOs are doing with their money," Ileto said. “I don’t see that in the scorecard. As I look at the scorecard, it may be an indirect measure, but it’s definitely not a direct measure of what we are actually doing.”

DHS went from a C to a B after significantly exceeding its goal to save $570 million, or 10 percent of its $5.7 billion IT spend. Instead, DHS saved $1.5 billion—nearly tripling its goal—by shuttering data centers and exploring cloud-based solutions, Ileto said.

Yet, DHS still got an F in incremental development, another issue the scorecard will be challenged to address.

“I know each and every one of you have a different definition of agile,” Ileto said. “Every program probably doesn’t need to be in the agile arena, and probably can’t. As we push forward with FITARA, one of the things we need to ask ourselves is, does that scorecard represent what we’re doing in FITARA and FITARA law?”

Dunkin, one of a dozen or so politically appointed CIOs, offered additional ways the scorecard could better reflect agency progress.

In risk and transparency, for example, agencies are currently rewarded for telling the truth about risky IT investments. That makes sense—at least initially—Dunkin said, because GAO and the Office of Management and Budget want accurate risk ratings showing up on the IT Dashboard. But it’s not a long-term solution.

“Out of the gate, I did really well, and I’ve been hard on some projects and said they had a lot of risks,” said Dunkin, whose agency scored an A in risk and transparency. “However, if I work with projects and I get the risk level down and improve my posture, my grade will go down. That is not a sustainable metric if you want to fix underlying problems in addressing risk.”

Speaking at the same event, Richard McKinney, CIO for the Transportation Department, highlighted this point. The Federal Aviation Administration, which commands a large chunk of DOT’s IT budget, reports most of its investments as low risk. That actually hurts DOT’s transparency score. McKinney, by the way, does not have authority over FAA IT investments.

Dunkin said the scorecard’s agile development metric isn’t foolproof, either.

“I always have a joke that says I have a stamp that certifies I’m agile,” Dunkin said. “I guarantee you every project can change to deliver something every six months.”

Though meeting its federally mandated goal of closing 40 percent of its data centers, EPA did not report cost savings and thus received its lowest score—an F—in data center consolidation. Dunkin said of 80-plus data centers, three were classified as “tiered,” and—set up in prime geographic locations at the coastlines—would not be closing. Many of EPA’s data centers that did close, Dunkin explained, were essentially closets with networking equipment, a file server, printer server and perhaps lab management information.

“Closing those, it costs us money—it doesn’t save us money,” Dunkin said, noting EPA would then have to buy bandwidth to connect a server elsewhere. The scorecards don't mention improvements to data center operations, bolstering power utilization and efficiency, Dunkin said, nor do they consider that one of EPA's large data center in Raleigh, North Carolina, just brought in the General Services Administration as a tenant.

“Unless that (metric) is rebaselined, I’m going to get an F forever,” she said.

But Dunkin was clear it was the scorecard she took issue with, not FITARA itself. While Dunkin is the only CIO at EPA, the components are helmed by senior IT leaders who now feel empowered to exercise authority over IT in their regions.

FITARA "is making our whole organization more effective, driving conversations in governance across the agency,” Dunkin said. “They all feel incredibly empowered, and they’re finding things they never would have found before, making sure we’re not doing things wastefully or duplicatively.”

Rep. Gerry Connolly, D-Va., one of the founding fathers of FITARA, and others, including Rep. Will Hurd, R-Texas, have vowed to continue their oversight roles on FITARA implementation. Both have shown a willingness to listen to CIOs on these issues, and each iteration of the scorecard has evolved with new adaptations, such as measuring CIO authorities.

“We want to capture data the best we can, reflect progress in agencies and support CIOs in implementing,” Connolly said Thursday.