GAO's Review of NARA's FY 2011 Expenditure Plan for the Electronic Records Archive

From the GAO Report, Electronic Government: National Archives and Records Administration's Fiscal Year 2011 Expenditure Plan (GAO-11-299, March 4, 2011).

GAO Findings:

NARA’s fiscal year 2010 expenditure plan satisfies four of the six legislative conditions. Specifically, NARA’s plan complies with the agency’s enterprise architecture, conforms to the agency’s enterprise life-cycle methodology, reflects certain system acquisition best practices, and was reviewed by GAO. The expenditure plan partially satisfies the other two conditions:

• NARA partially satisfies the condition that it develop capital planning and investment control review processes designed to help ensure that projects are being implemented at an acceptable cost and within expected time frames and that they are contributing to observable improvements in mission performance. While NARA has conducted regular meetings with senior-level agency management to review ERA progress, these groups did not document approval of important schedule and scope changes to a recent ERA increment. Further, NARA has not conducted post-implementation reviews of deployed ERA capabilities, such as the recently deployed congressional records component, to validate estimated benefits and costs. As a result, NARA has limited ability to ensure that the system is being implemented at an acceptable cost and within expected time frames, and that deployed capabilities are contributing to observable improvements in mission performance.
• NARA partially satisfies the condition that the expenditure plan be approved by the agency and OMB. NARA approved the expenditure plan in October 2010, but the plan was not approved by OMB because, according to NARA officials, OMB will only review the expenditure plan if there is an existing appropriation with language requiring its review. Without approval from OMB, Congress will have limited assurance that the plan is accurate and reliable.

NARA has fully implemented one of our prior recommendations and partially implemented three:

• NARA implemented our recommendation to ensure that ERA’s requirements are managed using a disciplined process by updating the ERA requirements document and developing a process to ensure that the ERA requirements are kept current following significant changes to the program, such as significant modifications to the contract.
• NARA has partially implemented three other recommendations. First, in response to our recommendation that NARA ensure that its investment review process has adequate executive-level oversight by maintaining documentation of the results of reviews, NARA documented meetings that its senior management attended where the ERA program was discussed, but the agency did not document approval of important changes to a recent increment through its review process. Second, in response to our recommendation that NARA provide additional information in the fiscal year 2011 expenditure plan on what was spent and delivered for deployed increments, NARA added information on delivered functionality and additional capabilities planned for fiscal year 2011. However, the expenditure plan lacked other important information, such as the estimated fiscal year 2011 costs and the expected completion date of an ongoing increment. Third, in response to our recommendation to strengthen its use of earned value processes, NARA documented an action plan to improve its earned value processes, but continues to have weaknesses in this area. Among other things, we found that ERA’s earned value data trends do not accurately portray program status, and future cost overruns would likely be between $195 and $433 million if the full ERA system were to be completed as originally designed.

We made three observations related to the ERA program and fiscal year 2011 expenditure plan and summary:

• NARA’s fiscal year 2011 ERA expenditure plan does not provide a reliable basis for informed investment decision making. Specifically, NARA’s plan does not clearly show what functions have been delivered to date or how actual costs compare to planned costs. For example, the plan does not clearly show the system functions of an ongoing increment that have been delivered and how the actual costs of this increment compare to the amount NARA planned to spend, as reported in the fiscal year 2010 expenditure plan. Further, even though NARA plans to end development in fiscal year 2011, the fiscal year 2011 expenditure plan does not clearly show what functionality is planned to be delivered, by when, and at what cost during this period. For example, the plan does not discuss the delivery date of an ongoing increment, nor the fiscal year 2011 costs associated with completing this increment. Lastly, NARA’s cost estimates, used as the basis for determining its fiscal year 2011 funding requests, do not reliably reflect the work to be completed because of weaknesses in the supporting methodology. For example, while NARA’s estimates were based on information provided by the contractor, including estimates of the total lines of software code required and the related costs, these data could not be traced back to their original sources, and NARA did not validate the information when preparing its cost estimates.
• NARA’s fiscal year 2011 expenditure plan does not address how remaining multiyear funds from fiscal year 2010 will be allocated. Specifically, in fiscal year 2010, NARA spent $41.7 million of the $61.8 million in accumulated multiyear funds that it was provided by Congress to develop the ERA system, which resulted in a balance of $20.1 million that is available for obligation in fiscal year 2011. According to NARA officials, the ERA program plans to use the $20.1 million to fund development efforts needed to make the system available to all federal agencies, as well as to fund other operations and maintenance costs. However, NARA’s plans for using these funds and their supporting details are not included in the fiscal year 2011 expenditure plan or the revised summary. Until NARA specifies how it will use the multiyear funds remaining from fiscal year 2010, appropriators will lack information important for evaluating NARA’s fiscal year 2011 request.
• NARA has not yet determined which ERA requirements will be implemented in fiscal year 2011, nor fully prioritized the remaining requirements. In June 2010, we reported that NARA had developed and documented a set of 853 high-level business requirements for ERA and planned to complete about 57 percent of them by the end of fiscal year 2010. Subsequently, in July 2010, NARA updated its requirements document. According to NARA, the original 853 requirements were decomposed to a more detailed set of 1,577 requirements, portions of which were allocated to Increments 1 through 5, deferred to post-Increment 5 (i.e., post-fiscal year 2011), or removed from the scope of the ERA program. However, NARA could not determine which of the remaining requirements would be addressed before the end of fiscal year 2011 because, according to NARA officials, the requirements were subject to ongoing negotiations with the development contractor. This uncertainty is further exacerbated because NARA has not fully prioritized its requirements. Maintaining a prioritized set of system requirements can better ensure that the requirements that are most critical to the customer and other stakeholders are addressed quickly. Until the ERA program fully prioritizes the ERA requirements to be development work is planned to be completed and whether it will result in functionality that is most critical to NARA’s customers and other stakeholders.

GAO's Conclusions:

While NARA’s fiscal year 2011 expenditure plan meets four of the six legislative conditions, the lack of critical capital planning and oversight steps—including documentation demonstrating approval of significant changes to a recent ERA increment, post-implementation reviews of deployed capabilities, and OMB’s approval of the expenditure plan—limits NARA’s ability to ensure that the system is being implemented at an acceptable cost and within expected time frames and contributes to observable improvements in mission performance. These issues are further exacerbated by the agency’s partial implementation of several open GAO recommendations, such as those related to improving investment oversight and earned value processes. With significant weaknesses in many basic oversight and management processes, as well as continued delays in completing Increment 3, NARA’s ability to make significant development progress in the remainder of the fiscal year will be challenged.

In addition, without a reliable ERA expenditure plan, NARA has not provided adequate information to assist congressional oversight and informed decision making related to the use of appropriated funds. When these weaknesses are combined with the lack of prioritization of the remaining requirements under negotiation for fiscal year 2011, Congress has little assurance that additional funds allocated to ERA will result in significant benefits to potential users. With OMB’s direction to stop development after 2011, it is unclear whether NARA will be able to effectively address the full range of weaknesses we identified and still have adequate time to complete significant development efforts.

GAO's Recommendations for Executive Action:

We are recommending that the Archivist of the United States immediately take the following two actions while the current system development contract is active:

• Report to Congress on the specific outcomes to be achieved with the balance of any previous multiyear funds in fiscal year 2011.
• Ensure that the ERA requirements planned for fiscal year 2011 are fully prioritized so that those most critical to NARA’s customers and other stakeholders are addressed.

To ensure that any future efforts are completed within reasonable funding and time constraints, we are recommending that the Archivist of the United States take the following four actions:

• Ensure that significant changes to ERA’s program’s cost, schedule, and scope are approved through NARA’s investment review process.
• Conduct post-implementation reviews of deployed ERA capabilities to validate estimated benefits and costs.
• Submit ERA expenditure plans to OMB for review and approval prior to submitting to Congress.
• Update the ERA Requirements Management Plan and related guidance to mandate requirements prioritization throughout the project’s life-cycle.

See also the GAO's Electronic Records Archive: National Archives Needs to Strengthen Its Capacity to Use Earned Value Techniques to Manage and Oversee Development (GAO-11-86 January 2011) and GAO Critical on NARA Digitization Efforts.

ERA Will Launch This Month. In Stripped-down digital records archive set to debut, the Federal Times reports:

The National Archives and Records Administration's Electronic Records Archive (ERA), after years of delays, will launch at three agencies this month, but will have fewer capabilities than originally planned.

The State, Justice and Health and Human Services departments will be the first to fully access the system to transfer electronic documents to NARA for preservation. Some of the records will be accessible to the public through an existing online portal.

[JH]