The Oracle of Delphi

Why interpreting gross customer-analytics can be a bad idea

A<n age old story goes like this: A king is about to attack a neighboring nation, both of which are roughly equal in strength. So before making an important decision, he goes to see the Oracle of Delphi for some advice. The Oracle, being her, gives a wonderful piece of advice A great empire shall fall. That’s all he manages to get out of her, then the king goes back to his people and gathers his soldiers to attack. He lost that battle and his empire fell.

The main problem was misinterpreting the advice given to him. In the world of web-applications, generalizations about group-behavior obtained from analytics can be grossly misleading just like the prediction of the Oracle. Using data obtained from such analytics can make it difficult to measure up to a meaningful goal. Instead a much better approach is to work top-down by focusing on per-customer behavior. These trends will help illustrate the problems that you have in the early-adoption of the application. What does it mean by focusing on per-customer behavior? Let me give an example, the number of page views that your application got or the number of downloads that you got from your app on a app-store is not an accurate measure of how you will succeed. Sustaining that growth is what matters, otherwise after the hype has died off, your product will suffer the fate that the Oracle’s predicted. On the other hand, focusing on something like churn rate brings us to per-customer behavior. Here the idea is that you have to dig past the gross-analytics and see how much time a user is spending with your app and how many times they use your app in a day? However just knowing the churn rate is not enough, before you even look at the churn rate have an idea in mind of what you want that number to be. That will guide your behavior to make features and fix bugs that lead you to reducing churn.

Making a prediction about how a change in your product will cause your customers to respond is the best possible scenario. It’s a straight-forward extrapolation of understanding and defining a customer profile. Once you have that, it is easy to think about how they would respond. Then you can find someone who fits that profile, just one person and use them are your model customer. Run the changes by them and just get some input and make more changes based around that. This will give you a wonderful starting point and it’s all based on your profile. Making sustainable growth requires adaptability and a deep understanding of what your customers want, and as you get closer to showing it to more people, you can always increase the number of profiles that you have. Multiple profiles also allow for profile switching to run dry-testing with the data available and given a prediction of how one of those profiles will react to the changes in the app will allow to extrapolate the data with some confidence. As your predictions improve, introducing updates will become safer and growth will become more organic. That’s the kind of sustainability that we can achieve from per-customer behavior.