loading words...

Apr 12, 2019 17:15:56

Running beta tests

by @hum | 610 words | 🐣 | 215💌

Sarah Hum

Current day streak: 0🐣
Total posts: 215💌
Total words: 107318 (429 pages 📄)

Beta tests are great. They give your team the opportunity to test the waters with a new feature. 

  • Is it what people were expecting?
  • Are there crucial missing pieces?
  • What bugs are there?

It's great to answer these questions before rolling the feature out to your entire user base. The more polished you can get it, the better off you'll be when you launch it.

What we've found successful is scoping down the feature to its most basic components. Essentially, an MVP for the feature. Which pieces are a must?

When the MVP is ready, we invite people to beta test the feature. Letting them in at a controlled pace allows you to make incremental improvements without being too overwhelmed and rushed.

People who are willing to beta test are often very willing to give feedback. This is your chance to see how the feature is being received. There are probably things you can iron out in the overall experience. You might want to increase the scope to cover new use cases that come up from your betas.

We love building Canny because it's something we use all the time. We found it quite useful for our latest beta test.

Beta testing with Canny

Canny makes running beta tests a breeze. Assuming you're building a popular feature request, you already have a list of people who want it. 

When the feature is ready for beta testing, just leave a comment on the post. As an admin, your comments get sent out to everyone who voted. This is a highly targeted message. It's also a great way to thank customers who gave feedback by giving them early access.

If the voters list isn't enough, send a wider message to your user base looking for people interested in trying the beta. We send in-app messages via Intercom. Describe what the feature is, explain it's in beta, and include instructions on how to join.

Once you have the beta running, make sure you keep track of who is in the beta. We create tags in Intercom. You can send out additional requests for feedback and see how people are using the new feature. You can also keep them updated with any improvements.

Your main goals of the beta should be to gauge usage and gather feedback. Sometimes, we make a private Canny board specifically for beta testers. They can leave feedback specific to the beta on that board. It keeps things organized. 

Rolling out the new feature

After the beta is successful, you're ready for a wider release! This means doing some product marketing.

Not doing basic product marketing is a big missed opportunity! Your customers want to see that your team is continually improving your product.

This is what we do for feature releases:

  1. Mark the Canny post as "complete"
    Since they gave us feedback, we let them know first. This is great for closing the feedback loop and encouraging more feedback down the line. It's also great for your brand as it shows you're really listening to your customers.
  2. Add the update to our changelog
    Here's our changelog powered by Canny: https://feedback.canny.io/changelog. This is to let active customers know about the update. It also serves as a nice history of changes.
  3. Announce on social media
    We post the update to our company Twitter account. It helps us add a bit of Canny buzz to the social media universe.
  4. Email our broader customer base
    We only do this for bigger releases. Don't email all your customers about a couple bug fixes. Sometimes, a feature is enough to reactivate an account. 

That's it! Not every feature needs months of launch planning.

From Sarah Hum's collection:

contact: email - twitter / Terms / Privacy