1 min read

Run more pilot tests

Run more pilot tests
Photo by Avel Chuklanov / Unsplash

It is often tempting to dive headfirst into implementing an entire system, without first testing out the underlying idea. As someone who works at a big company, I often find myself falling into this tendency. There's a desire to start building the entire system, before validating the core product idea.

I think this happens when teams are predominantly comprised of engineers. We tend to solve for giving engineers complex technical challenges, rather than prioritizing the development of a truly outstanding product.

This is where pilot testing can be useful. A pilot test involves conducting a small scale experiment, before embarking on the larger project. The goal is to gather key information that can help inform the trajectory of the project. Pilot tests help us evaluate the overall feasibility and cost of the larger project, and identify any potential challenges at an early stage.

Reflecting on my projects on the Go team, I believe that running pilot tests before building pkg.go.dev could have significantly improved our approach. Rather than immediately diving into building the entire package management platform, we could have learned a lot more by either revamping existing platforms like godoc.org, or developing pkg.go.dev for few select packages to evaluate usability. Instead, we ended up having to redesign and build the site twice.

Similarly, rather than integrating symbol detection into Go's vulnerability management system as part of the initial launch, we could have expedited the development cycle by only supporting vulnerability detection based on the import path in the first iteration. Instead, we incorporated numerous ntegrations right from the start, and had to rewrite the tool for v1.0.0.

My team has been working on our product roadmap for the upcoming year, and I've been thinking about adopting more of a pilot testing mindset. Rather than tackling the entire system in one swoop, I think that breaking them down into manageable components could allow for a faster iteration cycle. I believe that running pilot tests first could potentially help us adopt a more agile and user-centric approach to product development.