Use of Competition
FAR 11.002 states that agencies should specify needs in a manner that promotes competition. Given that requirements may not be fully defined when the agency solicits offers and that not every offeror knows how to perform Agile software development, what is the best way to ensure effective use of competition?
The Government ensures effective competition by applying a similar process used for performance-based contracting by identifying the desired outcome rather than the details of the design for how to perform the work. There are many vendors that are well-versed in Agile software development, and that number will likely increase as agencies become more familiar with Agile processes and gain experience.
To ensure effective use of competition, the Government must identify a Product Vision with enough information to describe the scope of the project and allow offerors to make a reasoned business judgment whether they want to compete. This is similar to the process used for performance-based contracting where the Government develops an SOO that describes the desired outcome for the effort rather than provides instructions on how the work is to be performed.
As in any other acquisition, requirements development should be coupled with appropriate market research and early vendor engagement (e.g., issuance of an request for information to inform the development of a solicitation, issuance of draft RFPs/RFQs, holding industry days and other mechanisms that allow prospective offerors to ask questions).
Simply because every vendor may not perform Agile software development does not mean that the process should be viewed as unduly restrictive of competition. As long as the agency’s strategy is justified, it should not be at risk of such a challenge being successful. As explained throughout this document, there are many reasons why Agile software development is beneficial in helping to manage risk and drive results. In addition, the number of contractors that are skilled in this methodology is likely to increase as agencies become more familiar with Agile software development processes and gain experience.
If system requirements are refined after the contract has been awarded, how can an agency ensure work was evaluated as part of the initial competition and is not considered an out of scope modification in violation of FAR 6.001(c)?
To ensure that all work is within the scope of the contract, as requirements are refined, the software releases (including the end product) must fall within the scope of the Product Vision described in the statement of work, and the agency must give offerors reasonable notice that the scope of the project includes using Agile techniques.
FAR generally requires competing the work that is called for after contract award if the work is not within the scope of the contract. Under long-standing case law, a modification falls within the scope of the original procurement if potential offerors would have reasonably anticipated such a change prior to initial award. In the context of Agile software development, this ultimately means that the MVP, which emerges out of user testing, must fall within the scope of the Product Vision described in the statement of work or statement of objectives. As explained above in Section C, to give offerors reasonable notice, the agency should describe the requirements sufficiently so potential offerors can understand the scope of the project (which should be enough to produce a high level budgetary estimate) and make clear that Agile techniques will be used. This will alert offerors that refinement of technical requirements will occur post-award using a highly-disciplined process of testing and customer feedback. Furthermore, the Government should alert offerors to technical constraints (e.g., platforms) that may be applicable to the effort.
Software requirement changes are expected to be refined and managed as part of the process agreed to up front, addressed in the solicitation, and reflected in the resulting contract. The mere fact that system requirements are refined to reflect the experience from completed iterations does not mean they are out of scope.
To ensure work remains in scope, the Government and contractor should agree to performance standards and the method for assessing the contractor’s work against these standards – which should be aligned with the Product Vision. A service level agreement will be used to specify the levels of service (e.g., minimum acceptable service level, target service level, performance standards applicable to each level of service, how service will be measured, the weight assigned to each measure, the frequency of measurement, and the office responsible for measurement). As long as the features support the SOO and are delivered within the set schedule and budget, they should be considered within the scope of the contract. This process is analogous to that used for performance-based acquisitions in FAR 37.6 where the Government issues a statement of objective defining its outcomes that allows for industry to provide innovative solutions that are measured under a quality assurance surveillance plan.
FAR requires that the Government must describe the general scope, nature, complexity and purpose of what it is acquiring so potential offers can make an informed decision as to whether they want to submit an offer. To enable a prospective offeror to decide whether to submit an offer, the Government needs to clearly state in the SOO that Agile methodology sprint/release framework will be used and that the contractor must propose a method for planning and sizing the work.
The scope should state broad goals and also include functional areas. The Government should fix the schedule/timeframe in which the work is to be completed, so the contractor can propose the number of sprints and the work required in the sprints. If a baseline is available from prior similar work, the Government may be able to propose the set number of sprints to be completed in the schedule/timeframe. To ensure that the prospective offerors understand the environment in which they may be working, the SOO should contain all applicable IT and environmental constraints, including legacy software, privacy, and security requirements or policies that may impact the development of the software. See Appendix C: Sample Language for Government Contracts for Agile Software Development Services.
Doesn’t the fact that technical requirements are not defined substantially increase the risk of a protest?
The fact that technical requirements are developed through an Agile process should not increase the risk of a protest.
Tips for Successful Solicitation and Evaluation
- Use presentations as part of the technical evaluation. Consider including language in the solicitation that the Government intends to require oral presentations as part of the offeror’s technical portion of its quote or proposal. This will enable the Government to determine whether an offeror truly knows Agile software development. This is not mandatory, but has proven to be effective for some agencies. Of note, oral presentations need to be tightly controlled and recorded to ensure that all offerors are treated equally, that the Government does not inadvertently open discussions, and to create a defendable record of the agency’s actions. If using oral presentations, consider using them after the competitive range is established. The Government should clearly spell out the intended use of oral presentations in the Evaluation Criteria if it chooses to use them.
Integrate agile into the technical factors in the RFQ. Example:
Factor 1 – Performance Work Statement (“Offerors shall provide a Performance Work Statement (PWS) in response to the Statement of Objectives and this RFQ. The proposed solution shall include an explanation of how project and contract management, communication/collaboration with the Government, security and privacy requirements, documentation, and reporting will function in conjunction with the proposed Agile methodology.”);
Factor 2 – Product Development Roadmap “Offerors shall propose an Agile product development roadmap which correlates how the stated objective aligns with the timeframe for implementation and the offeror’s proposed Agile methodology. The product development roadmap shall demonstrate where testing, training, security, privacy, and cut over planning, will be included.”;
Factor 3 – Notional Quality Control Plan “Offerors shall describe the QC and Performance Measurement approach, including how proposed performance standards will be monitored, evaluated, and reported. The purpose of the notional QCP is to provide evaluators with an understanding of how measures and metrics will be applied based on the proposed technical solution.”
- Request agile software development-specific information from offerors. As part of the technical evaluation, request information from the offerors addressing how they manage Agile implementation, techniques for release planning, plans for engaging end users, methods for capturing and applying lessons learned, testing processes, reasons behind the composition of their Agile teams and the rationale behind the proposed development talent and project oversight (tied to Product Vision), how they will make resources available within schedule and budget constraints, and their approach to configuration management.
- Evaluate demonstrated experience with agile. As part of the past experience evaluation criterion, include demonstrated experience with successfully developing software using an Agile approach.
The contract requirements description for Agile software development contracts and the fact that technical requirements are developed through the Agile process should not increases the risk of protest. Requirements must be defined to the SOO level to allow for vendors to know if they want to compete, and the Government must evaluate based on stated evaluation factors and apply the evaluation criteria consistently among offerors. This is really no different than long-standing requirements applicable to all competitively awarded contracts. Of course, a good debriefing may also help to ward off protests by helping an unsuccessful offeror to understand its weaknesses and how it can be more competitive in future competitions.
Will small businesses be disadvantaged because they will not know Agile software development and will not be able to submit a proposal for Agile software development contracts?
The opportunity to award to small businesses exists and many small businesses have the expertise and capacity to perform Agile software development.
FAR generally requires agencies to provide maximum practicable opportunities in its acquisitions to small businesses. Agencies have small business goals and they should give strong consideration to those goals when determining acquisition strategy. Agile software development is consistent with this long-standing policy. Many small businesses in this field have the expertise and capacity to perform Agile software development services, and they are as capable as large businesses to perform this type of work. Methods of increasing small business utilization are encouraged, such as mentor-protégé relationships or having subcontracting arrangements to be required and submitted with the respective proposals.