Mohana Ravindranath | Nextgov | March 22, 2017 | 0 Comments

IARPA Leader Rethinks How to Get the Biggest Bang for Research Funds

Flickr user Pictures of Money

President Donald Trump’s budget blueprint, dubbed a “hard-power budget” by Office of Management and Budget Director Mick Mulvaney, slashes federal research and investment dollars at various agencies, especially at the Energy Department.

The administration plans to outright to eliminate the Advanced Research Projects Agency-Energy, an energy research and development unit. And though it’s not mentioned directly in the White House’s preliminary requests, one intelligence community group is bracing for tighter federal research budgets across government.

“We’re very interested in the science of science,” Jason Matheny, Intelligence Advanced Research Projects Activity director, said at a Government Analytics Breakfast event in Washington on Wednesday. He asked audience members for help understanding how investments into areas such as high-performance computing, machine learning and other fields have on technological and economic growth rates.

» Get the best federal technology news and ideas delivered right to your inbox. Sign up here.

“How can we disentangle the impact that those research investments have ... compared to baseline growth?” Matheny asked, emphasizing it's a "hugely important question" as the government contemplates how much funding to provide to the different R&D agencies.

In the realm of intelligence, understanding the relationship between research funding and the emergence of technology could predict “where the United States may lag in the future if we don’t invest where other countries may lead because of patterns of historical investment," he said.

Internally, IARPA evaluates which of its own research projects should be funded by reviewing them every six months. The group spends about 25 percent of its budget on testing and evaluation, and last year hired a chief of testing and evaluation to oversee that process.

It’s challenging to assess research quality, Matheny said. The metrics IARPA submits to Congress, including the number of research contracts awarded or the number of publications resulting from the research, aren’t always the most important, he added.

Instead, IARPA assesses the success of its predictive systems by comparing their predictions to what happens in the future, instead of retrofitting models to reflect what’s already happened in the past, he explained. The forecasting projects vary in focus, anticipating the outcomes of foreign elections, interstate conflicts or whether a treaty is signed.

“That kind of evaluation allows us to see how well systems perform against real events ... seeing how well they predict events that aren’t in the data set that are truly out of sample," Matheny said.


Thank you for subscribing to newsletters from
We think these reports might interest you:

  • Modernizing IT for Mission Success

    Surveying Federal and Defense Leaders on Priorities and Challenges at the Tactical Edge

  • Communicating Innovation in Federal Government

    Federal Government spending on ‘obsolete technology’ continues to increase. Supporting the twin pillars of improved digital service delivery for citizens on the one hand, and the increasingly optimized and flexible working practices for federal employees on the other, are neither easy nor inexpensive tasks. This whitepaper explores how federal agencies can leverage the value of existing agency technology assets while offering IT leaders the ability to implement the kind of employee productivity, citizen service improvements and security demanded by federal oversight.

  • Effective Ransomware Response

    This whitepaper provides an overview and understanding of ransomware and how to successfully combat it.

  • Forecasting Cloud's Future

    Conversations with Federal, State, and Local Technology Leaders on Cloud-Driven Digital Transformation

  • IT Transformation Trends: Flash Storage as a Strategic IT Asset

    MIT Technology Review: Flash Storage As a Strategic IT Asset For the first time in decades, IT leaders now consider all-flash storage as a strategic IT asset. IT has become a new operating model that enables self-service with high performance, density and resiliency. It also offers the self-service agility of the public cloud combined with the security, performance, and cost-effectiveness of a private cloud. Download this MIT Technology Review paper to learn more about how all-flash storage is transforming the data center.


When you download a report, your information may be shared with the underwriters of that document.