Updated: January 24, 2024- 5 min read
Editor’s note: the following was written by a guest blogger. If you have Product Management/tech industry experience, and would like to contribute to the blog, please contact [email protected]
When my co-founder, Jon Perl, and I started working on QA Wolf back in 2019, we didn’t realize that we’d eventually pioneer a whole new business category. Our goal was to give engineering teams all the time back that they wasted with flaky and unstable automated tests. As engineering leaders ourselves, we knew how much time and energy engineers spend writing and maintaining end-to-end tests, the time it takes to run comprehensive test suites, and how frequently they flaked.
Our solution doesn’t look anything like it did when we started, but we found Product-Market Fit. Today, QA Wolf is helping companies like MailChimp, Gumroad, and MakersPace ship faster and with fewer bugs.
This is how customer feedback and user behavior led us to the problem behind the problem and helped us become the leading provider of QA as a Service.
Lesson 1: Obsess over problems, not technology
Jon and I are enamored with artificial intelligence (AI) and the potential it has to solve all sorts of problems. So our first instinct for a faster, easier, more efficient way to do QA was to build an AI. Our vision was a robot that could explore an application, determine the intended behavior, and identify bugs all on its own. But our imagination got in the way of progress.
After months of research and experimentation (and carefully monitoring the limited savings we set aside to launch our company) we had barely made any progress on the AI. And we certainly weren’t any closer to solving a problem for a customer.
It was a hard lesson but an important one. Building a business around the technology you love instead of the problem to solve is like planning a vacation around your favorite shirt. We decided to switch gears and ship something—anything—that people could use.
Lesson 2: What customers want isn’t always what they’re asking for
The first usable version of QA Wolf was an open-source, command line interface (CLI) for writing test scripts on Microsoft Playwright. It wasn’t great. In fact, it was terrible. But it had one thing that the AI never did: users. Users with valuable feedback. Which we would respond to as quickly as we possibly could.
We were so eager to attract users that we would build almost any off-the-wall feature someone requested. Some were useful, some were a total waste of time—like a series of custom settings for a company that later told us they’d never subscribe to a paid solution. ¯\_(ツ)_/¯
But we also noticed a pattern in the questions and feedback that we received from customers. The three most common issues were:
Trouble installing the Node module. We’d built on top of the Microsoft Playwright test framework, but users were having trouble configuring the setup.
CI integration. This was difficult to troubleshoot because every team had their own systems and set ups.
Personal troubleshooting. People would have questions about their unique test case.
What we realized was that each of these could be solved with a hosted solution because there wouldn’t be anything to install, or integrate into the CI pipeline. And if you were having trouble with your tests, you could simply send us the test’s URL and we could look at it with you.
When you’re building a business for scale, one of the most important things you can do is to prioritize scalable features—not just features that lots of people can use, but features that solve multiple issues all at once.
Lesson 3: Solutions are more valuable than tools
With a hosted version of QA Wolf (and some funding in the bank) we started to turn our attention to sales. Which was when we made one of our most valuable discoveries and finally found Product-Market Fit.
Up until this point, Jon and I were leaning on our own experience with QA and automated testing to guide us. We had both worked as developers and engineering leaders, and we had first-hand experience using the frameworks and testing tools available on the market. We thought we understood what the market needed, because we thought that we were the market.
But the market didn’t seem to want QA Wolf as much as we wanted them to. Our users were only writing a few test cases for applications that needed hundreds. When we asked them why it was, they revealed the problem behind the problem: As long as teams needed to dedicate people and time to QA, it was never going to get the attention that it needed. There were too many other priorities and too few resources.
But for the right price, they would pay for it to be done. We made one more pivot to providing QA as a Service. Not just as an outsourced developer writing automated tests, but an integrated QA concierge integrated directly into the development process.
Today, QA Wolf maintains thousands of test cases for companies of all sizes. Since QA Wolf has taken QA off their plate, they can focus on innovating and making an impact for their customers.
Want to know more about QA Wolf?
Send us a message. We’re at [email protected]. Or schedule a demo and see how QA Wolf can take your team to 80% coverage in four months and keep you there.
Our combination of technology and full-time QA engineers, 24-hour support, test maintenance, and bug reporting enables teams to ship faster and more confidently for a fraction of the cost of an in-house team.
Updated: January 24, 2024
