How to Run a User Interview

Emmett Shear, Founder and CEO of Twitch talks about the importance of interviewing users to find out what they want and what features you need to build.

This talk covers important topics regarding getting user feedback through face-to-face interviews. The talk becomes more relevant if you realize their first startup faltered because they built a product in a competitive space without talking to users and then went on to build a product for themselves with Justin.tv that, after talking to users, pivoted to Twitch.tv, which was sold to Amazon for $970 million.

Emmett Shear – @eshear

Notes

• First startup was Kiko calendar with Justin Kan out of college
• didn’t listen to users, weren’t calendar users; built nice product but eneded up selling company on ebay
• 1m10s – 2nd startup: Justin.tv; they were own user; reality tv show around Justin Kan’s life
• building startup for 1 user only works if that user representative of large group
• Justin.tv was good for very specific use case; didn’t have expertise to broaden appeal
• pivot: mobile and gaming
• for gaming went to users and talked to gamers and broadcasting
• ran large number of user interviews and used data for next 3 years
• built division to talk to users continually
• determined broadcasters were most important people
• who you talk to as imporant as what you ask and what you pull away from it
• 5m56s – interactive portion: who would you talk to for these startup ideas?
• notetaking app (college students with different majors; college IT administrators; parents)
• get familiar with people important to success of startup not just the most obvious users
• 12m21s – User interview with student
• 16m20s – review: not asking about features or user workflows; drill down to get sense of real problem
• talk to extremes of people; 6 or 7 similar people will probably run out of new material
• if building new product to replace existing solution what would be one thing to add to improve it
• class example: what one feature would you add to google docs?
• merge; postit notes in google docs
• if can build quickly and validate then do that, otherwise draw workflow and show to ask
• just asking about a new feature or product isn’t enough; they will say yes but might not buy
• hack together bare minimum to test; browser extension; find way to cheat
• find people who will pay and that validates it; money test
• 25m40s – Twitch feedback
• talking to existing users can be tricky because these are people already using product and willing to put up with shortcomings
• compare groups of people
• Twitch talked to broadcasters on competitor products
• Listened to broadcasters on competitor solutions because these were people with problems big enough not to use justin.tv
• 30m03s – talked to non-broadcasters: why people not using any solution; to expand market
• combine interviews and sort through feature requests into goals
• In past with Justin.tv spent lots of time looking at raw data and analytics; good but without talking to someone doesn’t tell you precisely where to go/ what to do
• 34m55s – Q&A
• 35m01s – # What is most common mistake?
• Don’t show them your product and put things in their head. Instead ask what they need. Asking specifically about pet feature. “Would you pay for feature x?” Talking to who’s available rather than finding right people, even if it’s hard.
• 36m32s – # How hard is it to get buy in by company?
• Get interview and then back up your position. Actually showing interviews. Video record so don’t interrupt with notetaking and you can play recording for rest of company.
• 37m34s – # Do you recommend face-to-face over email or text only interviews?
• Want to avoid non-interactive feedback. Skype or in-person better to get better feedback.
• 38m40s – # What about user interviews in international markets?
• Hard problem. Twitch works better in English speaking countries as a result.
• 39m46s – # What channels do you use to reach people to interview and do you ever compensate them?
• Onsite messaging when on another website and invite them to skype call. Get to know people at events. Tend not to compensate. Never had trouble getting people to talk.
• 40m57s – # Do you use onsite user feedback tools?
• Getting feedback online for existing products/features very useful to iron out kinks but doesn’t tell you what to build in first place.
• 42m06s – # Did you focus on a particular type of user to interview?
• Yes, focused on people using competing products. Those people aready used a service so goal was to find out what would get them to switch. Easier than trying to create new behavior. Needed quick win.
• 43m10s – # What about game publishers and industry?
• Game publishers or established players typically won’t talk to you as a small startup. As Twitch grew, game publishers more willing to talk and more important to talk to now. Pool of people you talk to will change. Keep talking to new people.
• 45m11s – # How do you give good user feedback as a user?
• Get user to ramble and talk about their life to get a picture of who they are and why they want what they want.

Additional Reading

• Transcript

Series

• Next: How to Design Hardware Products

• Previous: How to Manage

• About Series: How to Start a Startup