What should a PM do once in every quarter?
A short story of product management!
On one fine evening, our Engineering director (my good friend) and I were having a casual chat. (Actual intent for me was to negotiate on a launch date for one of my product launches)
During the awesome conversation, naturally I started flowing through my new project thoughts and he understood where the conversation is destined to be. He then asked me a question ‘How much do you estimate this project will improve our daily revenues?’
I instantly answered as I’ve done my homework ‘4–5%’
Then he started ‘In the last 1 year, all the Product managers in our team had launched at the least 7 new feature launches each and all of them were giving result in A/B test anywhere between 2–4% attribution to daily revenues. If all that is true why we are not at +28% compared to an year back?’ (He was very quick in calculating things real on top of his mind)
I was puzzled!
Later he agreed on helping me with the launch and we left the conversation. But, the question never left me for a quite long.
Then I started dwelling into my own launches and tried to compare the current KPIs of my earlier feature launches with their actual A/B test results.
During A/B test, we conclude results on earlier agreed success criteria by testing against null hypothesis and we reject H-null. After an A/B test is concluded, we make the change launched to 100% population and put that to health check at a top level.
- What we do not really consider is, influence of the other feature, seasonality if any associated to the feature and change in the user economy due to change in times, etc…(in short cannibalization)
- Some of the older features could turn to be obsolete. Or to be specific some earlier user buckets might have progressed to next level and the latest cohorts might not value the feature/solution anymore. (In short, relevancy)
- Sometimes, just small changes to older features could make a great impact on the top and bottom lines. (As small as tweaking just the constructs and updating the skins. They are quick wins and possibly big opportunities to trade off for)
After a series of realizations, I felt a bit unproductive suddenly. And, took some time to recover and think what best can be done now. Then I did what is called in my world ‘Product Audit’
I call this with that name because one must go through each aspect of products/features (UX, Functionality, Economy, tech, Data, consistency with other experiences in the app etc). In a way, deconstruct them to see what is their contribution to overall value chain.
- This gave me a fresh perspective using which I had to just fix some of them and remove some of them and also identified other big wins on the way which really did not needed much of an effort from team.
- I could find how the recent launches we made as a group resulted in cannibalization of other earlier features/solutions and helped us build consensus on how we can achieve better as a group and what to trade off.
- Some of the UX which was done years ago saw the light of new revamps and so resulted in great experience across the app.
After all, I started to strongly believe that there is no need for 100 different solutions to same problem. But, a few compelling solutions to each of the problems. And, keep improving them.
As an end result, I made this as a practice to do audit periodically(quarterly) for the old features. After all, we spend a could of days for planning a quarter. Why, not spend a few days in a quarter to make things better.
I followed this simple method to come up with fruitful outcomes from this activity:
- Refer early pitch of the product/feature
- Look for what metrics were promised/estimated.
- Check if that was delivered during old times
- And, Check again the same with existing times.
- As a practice, take help from experts (Design, Tech and Data) into this process to add more value.
Have a nice product journey!
