Skip to main content

Technical Challenge Playbook

V.1 March 2021

The government benefits when companies demonstrate how capable they are during contract competitions, rather than writing about their capabilities in a traditional, written proposal. “Tech challenges” are one form of a capability demonstration. Tech challenges are primarily used to test a company’s ability to design, develop, and deploy software applications and systems.

For the most part, the ability to code software is binary. You can either code or you can’t. Yes, there are a variety of technologies and languages to code with, but the ability to code is still binary. Additionally, coding capability has recently become more of a commodity in the federal marketplace.

Therefore, assessing a company’s ability to simply code software is not an effective approach to picking a partner that will successfully support your government software project. For these reasons, “Can they code?” is not the right question to ask during a tech challenge evaluation. “Can they solve technical problems?” is a much better question to focus on when assessing a company’s software engineering capability.

Download PDF

ABOUT THIS PLAYBOOK

The U.S. Digital Service (USDS) has helped federal agencies plan and execute a variety of coding and tech challenges. This Tech Challenge Playbook is a compilation of what we’ve learned. It provides a reusable roadmap for planning and executing technical challenge evaluations.

Before diving in and crafting your tech challenge, first go in with your eyes open by considering your situation:

Know it’s an investment. Technical challenges can be fruitful but they are a resource-intensive evaluation method to plan and execute.

Determine number of Offerors. Because tech challenges are resource-intensive, consider how many Offerors you want to participate. More Offerors results in a longer evaluation.

Use in Phase 2. In most cases, it makes the most sense to employ tech challenges during Phase 2 of multi-phase evaluations. Use after a Phase 1 down-select which should be an easy-lift and highly discriminating.

Decide multiple vs single award. For a multiple award vehicle, you’re evaluating corporate capability to support a variety of projects. For a single award contract, you’re evaluating the Offeror’s ability to support a specific project.

1. Involve the right people at the right time.

Technical challenges require early and frequent collaboration from the government team involved, from the planning phase all the way through contract award.

Checklist

Key questions

2. Identify and prioritize the discriminators.

Discuss the scope of work to identify & prioritize future objectives and past/current problems that companies will tackle on this contract. Next, identify aspects of the work that are not only high priority but will also allow the government technical evaluation team to clearly discriminate between Offerors as you test them via the technical challenge.

Checklist

Key questions

3. Turn discriminators into eval criteria.

Translate those high priority discriminators related to the work into 2-3 specific evaluation criteria that will be used to test and assess each Offeror’s capabilities.

Checklist

Key questions

4. Select a challenge format and author a simple scenario.

The challenge format will depend on your evaluation criteria. Consider how much the criteria dictates a format that evaluates the Offeror’s journey during the tech challenge versus a format that evaluates the final outcome produced/submitted by the Offeror. Author a simple scenario; elaborate scenarios aren’t necessarily more discriminating. Consider whether the scenario is real or fictional. Real scenarios can favor companies that have already worked with that Agency or program office.

Checklist

Key questions

5. Plan logistical details and constraints.

Plan how the challenge will run, the artifacts to collect, and any communications with the Offerors.

Checklist

Key questions

6. Prepare for and execute a test run.

Create the artifacts for the tech challenge. Test the feasibility and effectiveness of your tech challenge by having a team of feds perform the challenge while the evaluation team performs a mock evaluation.

Checklist

Key questions

7. Refine & share the tech challenge.

After the test run is complete, use the lessons learned in the test run to improve the tech challenge and the draft solicitation. Share the revised draft solicitation internally and with industry.

Checklist

Key questions

8. Execute and evaluate the tech challenge.

The time has come to administer the tech challenge! You have prepared and it will go well.

Checklist

Key questions

9. Defend the award decision.

Tech challenge evaluations typically produce very detailed and technical feedback on the Offeror’s proposal/performance. A transparent debriefing process that includes specific technical feedback on the Offeror’s performance will significantly reduce the risk of a sustained protest while also helping the Offeror to learn from their experiences in the tech challenge.

Checklist

Key questions

10. Review and retrospective.

Reflection and feedback are good. Take time to solicit and then listen to non-attributional feedback from both Agency participants and from Offerors.

Checklist

Key questions