General considerations for agile software acquisition
Generally speaking, what is Agile software development, and how does it fit into the acquisition development lifecycle?
Agile software development is a method of software development that utilizes an iterative development process, designs services based on real user needs, and constantly improves software from user feedback. Agile software development principles apply to both pre-award and post-award contexts.
Agile software development is a method of software development that is based on iterative and incremental processes and collaboration among a team. It is a methodology for the creative process that anticipates the need for flexibility and applies a level of pragmatism into the delivery of the finished product. The focus is on keeping code simple, testing often, and delivering functional bits of the application as soon as they are ready. There are various Agile methodologies (DSDM, Scrum, XP, Kanban, etc.), which all adhere to the same values: individuals and interactions, working software, customer collaboration, and responding to change.
Per the Digital Services Playbook, Agile software development is the preferred methodology for software development contracts that contribute to the creation and maintenance of digital services, whether they are websites, mobile applications, or other digital channels. It supports frequent changes, updates, and enhancements to the software. By breaking up the development process into small, manageable pieces – each with desired segments of functionality, and having end users involved throughout the process – and guided by the Product Vision, users receive software that better meets their needs (in terms of both functionality and usability) without wasting money and time on unused or unusable features. The following chart compares traditional software development with Agile software development by describing the key events in the pre-award and post-award contexts.
First and foremost, when using Agile processes, the acquisition community must differentiate between contract requirements and system requirements. Agile software development starts with a Product Vision and does not specify exact system features, but addresses the desired high-level functionality of the system (not the specific system features to achieve that functionality). Deliverables are the functional, working software (deployable code) that are produced through a repeatable iterative delivery process in a production environment. Cost and schedule are adhered to because there is a set schedule of releases in which the contractor must produce deployable code.
The graphic below depicts a typical Agile process. With an Agile software development process, the Product Vision feeds into the product backlog, which contains a prioritized list of user stories (technical functionality for the system). The sprint backlog contains user stories that have been reviewed, scrubbed, and selected by the team to be worked on during a sprint. The 1-4 week sprint cycle turns user stories into implementable and shippable code. This code is ready for production, but is usually held or bundled into a larger software release.
A key feature of Agile software development is the emphasis on the team approach. The Integrated Product Team (IPT) is integral to the success and administration of Agile software development contracts. The team includes key stakeholders, and should be led by a program manager holding the appropriate level of Federal Acquisition Certification in Program/Project Management. It is essential that the IPT members are champions for Agile, as the cultural change from traditional methods to Agile may be challenging. To make this shift happen as seamlessly as possible, before conducting pre-award activities, the key members of the IPT should be trained in Agile software development. Contracting officers and contracting officer’s representatives need not be trained as software developers, but should have a working knowledge of the concepts and processes associated with an Agile IPT.After contract award, the IPT is responsible for communication and brainstorming the system requirements.
To increase the probability of a successful Agile software development, a focus on the entire acquisition lifecycle is critical specifically to past performance, past experience, and market research. Early vendor engagement, including conducting industry days and releasing Requests for Information (RFIs) and draft Requests for Proposal (RFPs) or draft Requests for Quotation (RFQ), is important for both the vendors and the Government. Early vendor engagement informs the vendor community about the Government’s desire to utilize Agile processes and provides an avenue for the vendors to ask questions to ensure they understand the process and what the Government seeking to procure. The Government benefits by identifying vendors who are capable of supporting the specific software and have Agile software development experience.
As detailed in this document, Agile software development expects and anticipates changing technical requirements within the agency’s high-level vision or need, which remains constant. Therefore, it is imperative throughout the Agile process to give end users an opportunity to use the system or features to determine what should drive future technical features. This will allow the Government to benefit from the Agile process and shape a product that is responsive to the needs of the users.
Should Agile software development be used to address all IT needs?
No. Agile software development is intended for activities that require significant software design and development. Many IT needs can be met with commercially available off-the-shelf items and commoditized services, such as subscription services for software licenses, with little or no development work. In those cases when development is not needed, the Government is best served by purchasing commercially available off-the-shelf items.
Like most tools, Agile is not a one-size-fits-all strategy. Digital services, especially those intended for use by the public (e.g., irs.gov, healthcare.gov or recreation.gov), generally require significant software design and development and will benefit from Agile approaches. Agile is beneficial for software development projects, or ones that involve the configuration and modification of commercially available off-the-shelf items. But if IT needs can be met with commercially available off-the-shelf items without configuration, design, or development, there may be no need to apply Agile processes.
Are agencies authorized to shape their IT software acquisitions around Agile principles? The FAR does not expressly speak to Agile concepts such as refining technical solutions after contract award based on testing and customer feedback or buying a product with a process rather than an identified solution.
The principles of Agile software development are consistent with modular contracting, which is discussed in FAR Part 39, Acquisition of Information Technology. In addition, as a general matter, an agency may pursue acquisition practices that are not expressly endorsed in the FAR, including Agile software development, as long as they are not expressly prohibited by law.
Acquisition policies and procedures for acquiring digital services (which are Information Technology services) are addressed in FAR Part 39. Although Part 39 does not directly speak to Agile software development practices, it endorses modular contracting principles where information technology systems are acquired in successive, interoperable increments to reduce overall risk and support rapid delivery of incremental new functionality. See FAR 39.103. In 2012, Office of Management and Budget’s (OMB) Office of Federal Procurement Policy and Office of E-Government and Information Technology issued Contracting Guidance to Support Modular Development to promote greater use of modular IT development and contracting.
Shared Goals of Modular Contracting and Agile Software Development
- Improvement in investment manageability and budgetary feasibility
- Reduction of overall risk
- Frequent delivery of usable capabilities that provide value to customers more rapidly
- Increased flexibility
- Creation of new opportunities for small businesses
- Greater visibility into contractor performance
A number of agencies have used (or are in the process of adopting) Agile software development methodologies to improve investment manageability and reduce the time to value. As explained in this document, FAR authorities can be used to support IT projects that involve contractors performing in accordance with Agile principles. In some cases, these practices will require agencies to think beyond traditional approaches to more efficient ones, but ones that still reflect a commitment to the core values of public procurement, including competition, impartiality, accountability for results, and providing the best value to the taxpayer. Agile is not a method of procurement, but a methodology on how the contractor performs the work. The FAR specifically encourages agencies to pursue business process innovations and makes clear that they should not be constrained by the lack of specific endorsement for a particular practice. In particular, FAR 1.102-4(e) states that if a policy or procedure is not specifically addressed in the FAR nor prohibited by law, Executive Order or other regulation, agencies should not assume it is prohibited. “Rather, absence of direction should be interpreted as permitting the Team to innovate and use sound business judgment that is otherwise consistent with law and within the limits of their authority.”
Because Agile software development is a new process, doesn’t it entail more risk?
Even though Agile software development is a new practice to the Government, it dates back to the 1970s and has been used in private industry for many years (as well as some Government agencies) where it repeatedly demonstrated a shortened time to value, enabled better adaption to changing need, and lowered the risk of project failure. These results make a compelling case to adopt Agile in federal IT projects and ensure acquisitions support these efforts. Moreover, careful selection of contract type will mitigate risk as discussed in Section E.
For many years, private industry has realized benefits from Agile software development, and some Government agencies have begun to benefit from Agile for their software development contracts. Agile software development methods promote effective risk management through continuous testing and short feedback loops to improve outcomes on a recurring basis. Because the Agile process forces problems to be identified early, failure occurs on a small scale with time for corrective action. This reduces the risk of the Government ending up with a product after a long design period that may not meet its needs or that contains unused features, or of an outright failure. In addition, use of Agile software development methods will increase focus on the development of mobile solutions, which are advantageous to today’s workforce.
Use of Agile Processes by DOD’s Defense Health Service Systems to Improve Medical Readiness
Starting in 2012, DOD used Agile (Scrum) processes to enhance the Defense Medical Human Resources System-internet (DMHRSi). This system helps to manage current and future medical human resources needs. DOD’s use of Agile processes resulted in:
- A reduction in the average number of days required to deliver a new software release from 419 days to 58 days (representing ~87% improvement in time to market)
- A massive reduction in Help Desk tickets by 87%
- Increased customer satisfaction
- Production of a better product
Multiple studies of the traditional “waterfall” process, where requirements are defined and documented in full detail before any testing or customer input is received, indicate a low rate of success and a high incidence of paying for features that are never used. There is also a statutory mandate to use modular contracting to the maximum extent practicable for software development that is a major system. Successful application of Agile software development, like all strategies that are newly adopted in the Government, will require training and a support structure to share best practices and lessons learned. The TechFAR Handbook and Smarter Digital Services Playbook are just one step. OMB, in close collaboration with the Chief Acquisition Officers Council, the Chief Information Officers Council, and the Acquisition Center of Excellence (ACE), will work with agencies to support these efforts.
Acquisition personnel – and other stakeholders – should expect that Agile is a cross-functional and highly interactive process – both between agency team members and with the contractor. Therefore, IPT members – the Program/Project Managers (P/PM), IT Specialists, the Contracting Officer and/or Contract Specialist, and the Contracting Officer’s Representative (COR) – should be collocated, either virtually or in person.
Agile requires that the IPT learns better what the end user needs through testing and a culture that supports this approach. Therefore, P/PMs, COs, CORs, and other stakeholders need to be aware that Agile software development will not produce the final expected result with the first iteration, but instead will provide usable, working software that can be iteratively improved; it requires “touch and feel” and user feedback to get the technical features right and arrive at the final product.
Generally speaking, how does privacy and disclosure apply to Agile software development?
Agile software development must adhere to existing privacy and disclosure statutory and regulatory requirements.
FAR 1.602-1(b), provides that no contract shall be entered into unless the Contracting Officer ensures that all requirements of law, executive orders, regulations, and all other applicable procedures, including clearances and approvals, have been met. A contract utilizing an Agile software development methodology is no different and must comply with the law. Furthermore, the proper use of such methodology should not increase the risk of privacy issues.
Part 24 of the FAR implements the provisions of the Privacy Act. Contractors and their employees who contract for the design, development, operation, or maintenance of a system of records on individuals, on behalf of the agency to accomplish an agency function, are subject to the civil and criminal provisions of the Privacy Act. Pursuant to FAR 24.103(b)(1), the Contracting Officer will ensure that the contract work statement specifically identifies the system of records on individuals and the design, development, or operation work to be performed. Procurement actions involving the design, development or operation of a system of records as defined by the Privacy Act will contain the Privacy Act Notification and a clause entitled “PRIVACY ACT,” both of which are at FAR 52.224-1 and FAR 52.224-2, respectively.
Additionally, FAR 39.107 states that FAR 52.239-1, Privacy or Security Safeguards or other similar language, is to be included in all procurement actions for information technology which require security of information technology, and/or are for the design, development, or operation of a system of records using Commercial information technology services or support services. See also Fair Information Practice Principles.