GrowthX

We hosted a deepdive on user research with Dharmesh Ba and Abhinav Krishna, where we covered the nuances of how to conduct effective user research, how do you align stakeholders and saw some real life examples of research being conducted. 

This document is a summary of the conversation.

Before we begin, a little about our guests: 

Dharmesh is the founder of 1990 Research Labs, where he helps startups like Cred, Jar, Plum with their user research. He comes with a 12+ years of experience being a part of design & research teams at ClearTax, Rupeek and Setu. Apart from this, he runs an amazing newsletter called The India Notes. It's full of insights on the Indian consumer. 

Abhinav is an OG GrowthX member and an OIR. He has 11+ years of experience of working in design & product teams at ClearTax, AgriFi and Microsoft. He’s currently the head of design at RazorpayX and Razorpay Capital.

We covered 2 main topics in the session: 

In this document, we're going the cover the questions asked, and a summary of answers given by panelists. 

How do you go about forming research objectives?

Before going into any research, conduct your primary and secondary research. You should never go with a clean slate. Always do your homework, research and form some hypothesis on why something might be happening. You should:

For the topic / problem at hand, try and get to the core of it by just data and speaking to customer first teams. Keep asking whys. When you reach a point where you're not able to figure out further, that's when you start research. 

But how do you know you can't figure it out further? When you start getting multiple perspectives. That's when you know, you need more truth. 

How do you ensure you are not broad enough while starting your research?

Abhinav says: Never take a research objective based document to a meeting

Always take a business problem to a meeting and understand how many business leaders are vested in the problem. 

For eg. swiggy wants to increase revenue in swiggy one. Get everyone who's OKRs are related on this in the room. Now you might have a research objective in your mind, there are 2 ways to go about it: 

This needs to supported by a lot of secondary data, like mentioned above. 

There are 3 kinds of research questions: 

Given this, to understand the broadness of research objectives, you need to ask your stakeholders: 

Pushback is when the question that “If this succeeds, what changes?” is not clear, or has too many answers. 

For eg. an edtech app wanted to understand why English language learners drop-off in the onboarding. They had observed that the retention of any other language learner was higher than the English language. That seemed very counter-intuitive to them, and they said, “If this succeeds, we will be able to bring up the retention rate to other languages”. Now this is a very well defined problem statement.

To keep it in plain words, do not try to do too many things at the same time.

When the objective is clear, how do you start deriving questions from it?

Dharmesh says: by but thinking about categories of questions.

Typically in one hour, you will be able to 40 good questions. You put everything in your mind on an excel sheet. These will be a lot more than 40 questions. What you should focus on is not asking all the questions but rather cover all the categories of questions.

Now based on the last question, “When was the last time you walked into a branch?” you put a weightage to the answer they gave 2 questions ago about : “Why do you think a branch being near your home is important?”

What this means is if someone says that they opened a bank account in a particular bank because there was a branch nearby, but they say they have never visited that branch, there is some discrepancy in the answers. Then you try to understand “How do they usually resolve their queries/conflicts?”

This means every answer they say, you don’t take it at face value.

So the way to go here is: for the questions that are very close to the focus of the research, you:

Question: How do you go about conducting the interview?

You need to get them to be vulnerable and get you to tell things. How organically can you get the conversation to start flowing, because that gives to the opportunity to identify the topics you want to deep dive into.

How many questions are too many questions?

The no. of questions do not matter in a user research. What matters is getting stories and understanding the truth. Doesn’t matter if it takes 10, 20, 100, 500 questions to get there.

Always try to get to the third persona. But don’t try too hard. If you can’t get there, leave it and walk away. You are most likely not getting any insights from that conversation

How do you deduce the trust factor in whatever the participant is saying?

Once the participant believes that you’re asking questions for a genuine reason and not being used against them, that’s a starting point

One way to deduce this is by asking good questions

If you ask any number of whys within a truth, if flows because it’s a part of a narrative. If you have to continuously lie, that’s hard. You can catch the lie in a couple of whys.

Once you identify this, you don’t implicitly mention that you’re lying. You understand, move on and not give much weightage to those questions.

How do you go about conducting the user interview?

Rule 1: Don’t look for the answer. Look for an answer. The only difference is, you need to genuinely interested in your life. Empathise with the participants

Your first job is to make the participant feel comfortable. Don’t directly get to the questions. Try to understand the trivial details about their life like their family, are they native to this place or are they immigrants etc.

Dharmesh says, “When they’re in the field, they try to pick a lot from the surroundings. Maybe there’s a picture of a god, it could be a TV etc”

Before every interview, you make an interview guide that you share with the participants

That interview guide contains the context of the interview, why are you taking the interview and some pointers:

You start by asking if they have any questions. Same at the end.

Priming is very essential. Specially in the initial 10-15 mins. In the first 15 mins, you need to:

If you’re working in a product company, it’s very essential to establish that “We’re not testing you, we are having you test our product”. This is because when you establish the conversation as an interview, they try to portray like they know your product which might bias the responses.

Second is: establish the context why you are having this conversation and what will it used for. For ex. if you want to know about their financial details, tell them that hey we are not going to extract any exact details, we are just going to do a proportionality where we say “If you have Rs. X, what are you going to spend it on?”

Third is: get them to a state of vulnerability when they start answering questions. Try to pick a conversation in their area of comfort, and find common talking points to start with.

TO not break the comfort, have only 2-3 people to speak to a participant

Patience is of the utmost importance during your interview

There is a lot of cultural difference when it comes to folks from tier 2 and tier 3 towns who let’s say are a maid, autodrivers etc. These people are very transactional, you pay them they get your job done and move on. They don’t really need to respect you as a higher human being.

There are a lot of cultural and societal cues that can differentiate you from someone who’s in these towns. It could be the way you speak, talk, dress, behave etc. You need to convey the fact that you’re the same. They think of you as a big deal because you’ve come from the city, and then they’re continuously trying to guess what you want and give you that.

You need to be cognisant about what you wear, how you speak and behave in order to make them feel comfortable.

How do you know your interview has gone well?

As a general rule of thumb, interviewer should speak for only 20% of the time.

You should get a lot of descriptive stories. Descriptive stories give you layer of informations and hence you get different level of insights. Let’s understand this using an example: If I share my amazon pay account history, how do you figure out my motivation to use it?

You should get a lot of counter-intuitive insights in the first 15-20 mins. That is counter-intuitive because you don’t have context about that point. That means your interview is going right

How should you be collating research findings?

90-95% insights come after the interview, when you sit down and start going through the interviews again.

For Dharmesh, because he comes form a consultation background, he follows a purist form of analysis. Most of the conversation are not in english, so that process here is:

Every user interview has 2 conversations:

Along with the lingual conversation, you also need to focus on body language. Understand how a point lands on you and with what intensity has the participant said that point. That’ll help you understand what really is important for the participant.

How do you get from findings to insights?

You need to get every stakeholder in a room, sit and go through all videos together and see how they felt about the above 4 questions. Then you prioritise the insights to chase after.

Apart from this, in every research call you’ll find only 40% relevant insights. The rest 60% is not that relevant. Those 60% insights need to go in a separate database, because it might be a visionary insight.

Dharmesh says: At the start of the conversation, we spoke about how research should be conducted when you don’t have answer to certain whys. After the research, you should be able to get that answer and the answer to 3-4 more whys. You need to convince yourself that you are getting closer to the truth. You should be able to convince other stakeholders as well. This will come not just from one conversation, but from a pattern across all conversations.