Running Lean: Problem Interviews

The following post is part of a series of posts related to my new project, currently code named CB Reader. For the latest info, please consider joining the CB Reader mailing list.

When I last posted, I introduced the lean canvas and explained how it helped to defined some user problem hypothesizes. Today we’ll start a new phase of Running Lean, the Problem Interviews, where you’ll interview people to verify if your problem hypothesizes and more than likely discover aspects and existing solutions you never knew existed.

Problem Interviews

With your falsifiable hypothesizes in hand it’s time to start talking to people. The book recommends 10-15 face-to-face interviews that last around 20 minutes each. It provides the following suggested interview format:

Personal Execution and Results

I reached out to the IndyHall community and the Philly CocoaHeads. I did 7 problem interviews, 4 face-to-face and 3 over Skype. I’ve had a few other people say they were up for the interview but they haven’t materialized (yet).

First, the face-to-face interviews went way better than Skype ones. Skype tended to disconnect and get laggy during the calls. This was annoying for the two people whom I’m previously friendly with but for the one interviewee, whom was a new connection for me, it really made the call awkward. Comparatively the face-to-face interviews went a lot smother (even brand new introductions) and I think generated better overall data. Lesson: by all means do face-to-face if possible.

I tried hard to follow the recommended interview format but I found it a bit off focus. For starters that first 10 minutes becomes a lot of you talking with very short bursts of answers from the customer. Part of this is needed as you need to set some context for the interview, however I just felt it was too long considering the target of 20 minutes. The meat of the interview is clearly “the customer’s worldview” section, where they explain what they view as the problems and the solutions they are using currently to solve them (along with their pros and cons). Moving forward I want to rework things to get to that as fast as possible.

Regarding the collection of demographics, the book suggests using tools like Google Forms which can help you graph and measure the various responses. I did this early on but found using the web form during the interview itself to be a bit of a pain as I needed to take notes on things the person was saying that didn’t fit the form. For the most recent interview I sticked to a simple plain text editor, typing in notes manually and then later punching it into Google Forms where appropriate. That said, the biggest value from all this is the free form stuff. Plotting 10-15 points of data isn’t usually that interesting for me so I’d recommend against the forms for this part.

What did I learn?

Before I started I wrote down the following goals:

Problem interviews will confirm pain points:

• Multiple apps for RSS, Read Later, Pinboard, Twitter is painful / annoying.
• RSS content can take to long to manually browse.
• Regrets that you may not be reading enough.
• Regrets that the time you do spend reading isn’t the most efficient.
• You get value from Twitter but don’t want to monitor it 24/7.

Problem interviews will confirm use of the following alternative solutions:

• Instapaper / Pocket / Safari “Read Later”
• Reeder for iPad
• Google Reader
• Pinboard
• Twitter

Problem interviews will confirm Entrepreneurs and Blog authors as valid customer segments.

For pain points, the biggest amongst my interview base was clearly the quantity of content being hard to manage. All seemed to share a sense that they weren’t getting the most from their reading time. Very few people considered switch apps to be a problem.

All of my suspected alternative solutions (minus Safari’s Read Later) were mentioned somehow. A few new ones popped up including: Zite, Flipboard (was previously known but forgot to list), FLUD, Umano, Fever, Yammer.

During this phase the Google Reader shutdown was announced as well, which made me aware of even more clients (some current, some in-development). I’ve blogged some thoughts on Google Reader in case you are curious.

Finally, my market segment was verified to include entrepreneurs, blog authors as well as programmers.

What’s next?

With our problem verified we move on to the solution interviews. This is where you mockup your solution, show it to people and gauge their reaction. The goals here are to continue to verify your problem and early adopters, figure our the minimum feature set needed to launch, evaluate if people are willing to pay for your solution and then what price they will bear.

I’m breaking away from the book here and doing this in two phases. One is a very digital phase. Part of this is the release of my new intro video and feature survey. The second part will come with more face-to-face interviews, re-interviewing people from before (now with solution demos in hand) and new people — preferably people whom I’m not previously connected to (which frankly I’ve very worried about finding).

Next up: Solution Interviews