As I’ve been a part of several fast growing Engineering teams I’ve seen a number of approaches to building out a Quality Assurance organization. Sometimes this has gone well, and others it has been less than effective, which will be the focus of this post. I talked about these in an Ignite style presentation today at DevOps Days Austin, but thought these would be good to share here as well.

The first anti-pattern is hiring a “pile of testers.” While manual testers have their place, and can be a useful at times, only having testers will not dramatically help you produce better quality software. The thought process goes that this is outsourcing work that otherwise your developers would be doing, which is in itself a good thing. However, simply having testers produces typically produces two things: test plans & manual tests. The bad news here is that this means that each release you want to ship the manual regression suite will grow, as will the cost of testing/number of testers required will increase. Also, my experience was that the detail oriented nature of testers produces a lot of miniscule bugs, minor UI issues, and not the more systemic release issues.

To lower these costs you can fall into the second trap, of outsourcing it to another team and/or another country. While your team overseas can be cheaper on the hourly rate, all of our validation showed that the total cost of ownership on this work amounted to an equivalent Senior Engineer based in the United States. Additionally, if you choose to outsource, they often don’t have the same level of commitment to your project. If you are just expecting a bolt-on team to add quality to your product, you’ll likely find that it doesn’t work.

The final anti-pattern is similar, throwing the process, tools, and responsibility for creating quality over to another team, whether your quality team or your DevOps team. The truth is that you need your entire company, and certainly your entire engineering team committed to quality. Engineers building software and throwing it over the wall to another team is an opportunity for confusion, frustration and lower quality.

I’ll cover some best practices in building out quality at scale in a future post, but this airbnb post provides some thoughts to get started with. I’m not so simple as to believe that you can boil quality down to only one thing, and of course your mileage may vary with these tactics above, but if I had to focus on one and only one thing it would be that every engineer owns every line of code they write.