Sunday, June 18, 2017

How To Build A Minimum Viable Product

Earlier this year, my employer Castlight Health acquired another company called Jiff. I took over the product responsibility for machine learning driven and user input driven personalization for the combined companies. The team that built personalization at Jiff is a capable team with many ideas, some customer implementation under their belt and tens of customers who have bought into those ideas. Looking at the clear need in the market, I decided to focus on executing on those ideas to accelerate delivery of functionality to customers who were paying us millions of dollars every year. Since then the team has delivered their first major functionality release which personalizes Jiff home page and prescribes benefits based on machine learning driven prediction intelligence from Castlight. We are now working towards the next release focusing on user input driven personalization and benefits prescription.

When I did my initial analysis of the team and Jiff personalization product, I noticed that like any young and ambitious team, my colleagues were trying to build too many things too fast. I introduced two principles. The first one is to design and build minimum viable products rather than release too much functionality at one time. The second principle is to deliver value early and often.  I will talk about the first principle in this post.

Minimum Viable Product (MVP)
I shared the diagram below with the team and explained how we need to build a minimum viable product. For example, I explained to them that if we are building Survey functionality, we should not release it without building at least a simple reporting functionality.  It is not only a product error, but also a business error. If we release a Survey feature without associated reporting, customers will reach out to us for reporting, increasing our cost of customer operations. Since manual reporting will involve data analysts and engineers, we will also lose the opportunity to build new valuable features. All these things will make the customer wait for days and weeks and will most certainly decrease their satisfaction with our product. It will also slow down the delivery of future innovation.


I also asked the team to think like a landlord rather than like an architect. The team responded very well. We started defining our products in a very disciplined manner. We consciously reduced features that were not necessary for the MVP.  We started thinking about total cost of operations rather than just the first release of the product. Sometimes, sales teams and even business executives lose track of this fundamental principle. It is the responsibility of the product manager to ensure that this principle is understood and adhered to.

Deliver Value Early and Often
We also followed the principle of delivering value early and often to customers. We have large customers who have hundreds of thousands of users. Such customers go live with our benefits management platform usually once a year. These implementation projects are large, take months to execute and have tens of work streams involving multiple teams.  I notice a tendency among implementation teams to not show any product until very late in the process. I am putting down a process in place where we show and even release functionality to customers months ahead of their requested delivery dates. I believe it has several advantages. I will write more about this principle in a future post.

Wednesday, March 22, 2017

Road-mapping User Journeys Is Better Than Road-mapping Features

Heads of product will face this problem in most software product companies. During every planning period, every product team comes with a long list of features they want to build. Most features will have cryptic names that few people understand and investments are made without even a clear idea about what is being built let along the return on investments. All well-meaning product teams clammer to get their features in without realizing how they fit into the overall objectives of the company. They build the features. The features don't fit well together. Even if they are good, they never get promoted to the right users at the right time. Even in the event the products are used, the reporting teams don't have an idea that the features are getting used because they did not built the tracking instrumentation. Product marketing teams or aggressive sales teams make up their own stories about what the product can do with little input from the teams that built the product. Once the product is made available, disgruntled customers complain bitterly and refuse to renew their contract. The sales team comes to the product team and requests for features. The product team build features with no purpose other than to keep the contract from getting cancelled. Everybody loses including the customer.

Such behavior of product teams might be overlooked for a while in a large company with a very successful product that is already a market leader and makes huge profits. However if you are a company trying to build a product for a new market or if you are a small company working hard to keep your customers, this sort of behavior could kill your company or at the very least your product managers our of work in a couple of years. Slowly but steadily, product managers who build features without knowledge of user journeys and without tracking usage will slowly lose investment and their value to the organization and eventually their job. In other words if one wants to succeed as a product manager of product leader in the long run, user focused and data driven roadmap planning is probably the only way to go.

Changing behavior of product teams is very hard. But there is a framework that might work. In my early experiments with a user journey driven framework for roadmap planning, I see acceptance from various teams and acknowledgment of value from product managers. This is the framework. Ask your product teams, including product managers who build features, product managers who are custodians of existing features, user marketing teams that promote your products and analytics teams that instrument your product for tracking and reporting to think about all their features, assignments and campaigns in the context of a user's journey. Here is an example of a user journey [1] where multiple teams contribute.

 

Ask them to build a collection of the top 5 users journeys that their features form a part of and estimate the impact of those user journeys on product objectives. To do this first, they have to think about the user. Second, they have to think about the other product managers and teams that they need to rely on and collaborate with. Third they need to focus on how much engagement does their feature bring today. Fourth, they need to collaborate with the analytics teams to estimate the potential impact of the user journey on product objectives. Fifth, they need to identify instrumentation gaps in their features and think about building just the right instrumentation for reporting purposes.

During this process product teams will realize that a feature cannot become successful on its own. It required some one such as a user marketing teams and other upstream features such as home page or mobile channel to promote it. User marketing teams will realize that there are good features that bring value for customers not being promoted. Product managers will recognize that their feature usage is not being tracked by the reporting teams. This will help them convince the reporting instrumentation teams to build just the necessary instrumentation. Product teams can communicate a collection of user stories to product leaders, product marketing and sales so that they sell what has been built. Not what they cooked up. Customer success teams will be happy because they will get accurate reporting on user journeys that matter without having to wrestle with the analytics team to dig up data every time they have to report progress to a customer.

I am not saying this is easy. However it has worked in the past for me and my new experiments are showing signs of more promise.  If done right, this approach could be the difference between your small software company staying in business or going out of business.

If you don't have a framework for evaluating features, product teams will use phrases such as 'customer commits',  'table stakes' or 'strategic project' to justify the investment. While customer specific features are part of every product team's work, when you hear such phrases from more than 25% of your product managers, as a head of product, you should be very concerned because you will end up with a hodgepodge of features that bring no value for your users. If you are a CEO, and you hear phrases like these from your product team very often, you should be very concerned because your investment is being squandered by a team that is inexperienced and has no framework for execution towards your business goals.

[1] The user journey discussed above  is for a business to business to consumer (B2B2C) product company. Many enterprise software companies fall into this category. I am making the assumption here that you are a cloud product company and your customers pay you if and only if you can prove to them that their employees are using your product to either stay compliant, save money or help you make money.

Thursday, January 12, 2017

Product Managers Could Explain Data Science Results In Plain English

One of the key responsibilities of a product manager working with a data science team could be to articulate the results of a data science project in plain English to other team members and stakeholders. I take the following approach to do this.

First, I request the data science team to aim for a small success within three months of work. In collaboration with the data scientists and a subject matter expert, I create a concept story (1) that outlines the specific results we aim to achieve. We aim for modest results in a short period rather than aim for very ambitious results in a year (4).

Second, I sit down and have a conversation with one of the data scientists (2) to understand the results of the data science project. I do this during the research phase of the project as soon as the team reaches the projects desired research goals (3). The data scientist will usually share a data file with the results of the data science project. I normally request the data scientist to point out the top three highlights of the research. We then verbalize the results and convert the results into a plain english sentence in a short work session. For example, in a data science project to match data sets, the plain English sentence might say "We were able to improve the match rate between dataset A and dataset B from about 2000 to about 10,000." I then build on the sentence by stating what it means for an end user. For example, the plain English statement might be "When a person looks at a doctor, she is five times more likely to see a hospital affiliation compared to before."

Third, I provide a screen shot of the application area where the data manifests itself to make the data easier to understand for all team members and stakeholders.

A product manager who takes on these responsibilities in the data product team can play a meaningful role (4). It is also a good way to gain credibility not only with the data scientists but also with stakeholders who may not always have a data science background. It might take 6 months and a couple of successful releases for the data scientists and stakeholders to appreciate the role of a product manager. Don't let that stop you. Keep at it and you will succeed.

_______________________________________________________

1. I might share a sample concept story in a future post, if possible.
2. Experienced data scientists are good at articulating the results achieved.
3. Data science research outcome is later turned into scalable code by a data engineering team.
4. Overstating the scope and impact of a data science project is a common mistake.

Keep Application Teams Posted About Data Science Projects

In some cases the results of a data science project might lead to the creation of vast quantities of useful data. If the resulting data is used by applications, it is necessary to keep application engineering teams informed in advance so that they can be prepared for the increase in available data. They may have to invest in improving their infrastructure and performance to accommodate the new data.

Think of data as water and applications as the hydro electric dam that uses the water to generate electricity. A sudden unexpected deluge of water might overwhelm the turbines. So keep those responsible informed about the possible deluge.

This could be a key responsibility of a product manager working with a data science team.

Wednesday, January 11, 2017

Gene - An Intimate History - Book

I took a few days off from work recently. During that time I read the book, The Gene - An Intimate History by Siddhartha Mukherjee.   I read that the gene is going to change the world, the same way the atom and the bit changed the world.

Advances in technology have enabled human beings to not just read but also write into a gene. The idea of debugging a human being by editing the gene is cause for celebration and cause for concern at the same time. You can read the New York Times review of the book here.

Related Posts Plugin for WordPress, Blogger...