If I were PM for Adobe Analytics, how could I define metrics with my Adobe Analytics Challenge experience?
Intro
The story began from a quick impulse. Among all the hustle of the fall semester, two of my classmates and I decided to participate in Adobe Analytics Challenge 2018. The challenge is yet to reach the end, but we are quite struggling with the tool and are not able to perform a skillful and satisfying results from Adobe Analytics.
It is precious experience that we try to get familiar with this unknown tools and endeavor to master it in a short period of time. And it just came to my mind, with some product analytics perspective, that maybe I can jump into the shoes of Adobe Analytics’ product team and make some recommendation based on my challenge experience. And I bet the product team of Adobe must collect the participants’ data of using Adobe Analytics. And based on the result of product analytics, they may optimize this tool for sure.
Analytics Process
1. Defining the Persona
2. Defining the Problems
3. Defining observational metrics for Project Page
4. Defining observational metrics for Analytics Page
Defining the Persona
Adobe Analytics is a B2B product. Unlike B2C products, the users of this tool are more limited, and the profile of them will be more homogeneous. Adobe Analytics is mainly used for analyzing the web traffic, namely the user behaviors online. Its clients might be the companies have online channels, either on PC or on mobile devices.
To go one step further, I will define that the persona for this research is “us.” We are college or graduate students who are using Adobe Analytics for the first time. To expand this persona and make analogy, we can represent the freshman analyst of any big companies that utilize Adobe Analytics.
For this group of persona, they have some background knowledge of business analytics and web analytics, they might use Google Analytics before, and they don’t equip with much domain knowledge. Furthermore, they are assigned with an analytics project with tight time constraints. They really need to get familiar with the tool very quickly.
Defining the Problem
The goal and problem for product team of Adobe Analytics is relatively simple, but crucial at the same time. The freshman analysts are the majority of the user of this tool. If they are not satisfied and find the tool really hard to use, the retention for the tool subscription is at risk.
I will divide the problem into two parts.
1) How to increase satisfaction for the persona
2) How to increase efficiency for the persona when they are using Adobe Analytics
The two problems can’t not be answer directly, but I will try to imagine and recommend which metrics to observe for better getting insight and further provide some suggestion for Adobe product team.
Defining observational metrics for Project Page
For this page, it is the starting page for every analyst. I will further classify the persona into two group: 1) first time analyst & 2) return analyst.
1) First time analyst
Metric 1: Occurrence of ‘Create New Project’ and Occurrence of templates selection
It can tell the behavior of user first starting the new project. And within all the templates, which one is the most popular one? Furthermore, within the templates selection page, there are 20 templates for selection. Are they too many or too less? How is the usability for the first time analysts. By defining these two metrics and closely monitoring the number, we can optimize the first user experience here.
Metric 2: Bounce rate of templates selection page
It is another important metric for the first time analyst. How much percent of first time user bounce out of the template creation page without selecting any templates? As soon as we define the metrics here, the product team can further look into the reason why they exit, and modify the design of pages to lead to the click on creating project.
2) Return analyst
Metric 1: The frequency of creating project
This metrics can vary from company to company. However, by monitoring this metric, I think it is a good alert of usability for product team. If the frequency is below a threshold, it will be suggested the sales team or customer service team to reach out the company for make sure if there are some of issues when using the tools.
Defining observational metrics for Analytics Page
For the freshman analyst, Adobe Analytics is definitely not an easy too to use. There are many sections which might cause confusion. With the well-defined metrics, product team might observe the hesitation of users and optimize user interface to better respond user’s need. It will lead to the increase in satisfaction in the long run.
Metric 1: Click/Drags number of components
Adobe Analytics consists of four important components: Dimensions, Metrics, Segments, and Time. With these four components, freshman analysts are able to perform deep as well as complicated analyses. It will be insightful to observe how they drag the components from the four categories and within categories.
And further, product analyst of Adobe product team might also be curious about how often analysts click ‘Show all’ button for more request of components.
Metric 2: Number of clicks on Panel options
Adobe Analytics provide 7 options for visualization. It is obvious that the product team might want to know how frequency each of option is used. It is strongly connected to the R&D team I think. Each tool is well-constructed with complex algorithm. If a tool is widely used, it should be first priority to develop more feature to satisfy user’s need; whereas, if a tool is less used, PM team should investigate the reason and decide if it should be replace with more useful tools.
In the challenge, I found my most-used options are Freeform Table and Flow. As a challenge taker, I found seldom did I click on other options for data exploration or analyses.
Metric 3: Gap between actual and expected tools usage
This is metric is more conceptual, and I am not so sure if it can be tracked easily. For product team, the features for each option are sequential. Take Freeform Table for example, we first drag the dimension, then we might filter the data or date, change metrics, and visualize the table. There are several steps for an analyst to reach the satisfactory and presentable results.
And there must be an expected steps for each options. Freeform might have 15 steps set at very beginning. However, 90% of analyst explore and finish the analysis within 5 steps. The 10 gaps left here indicate two message. One is the remaining 10 steps are redundant. Analysts don’t have to go deeper and still can have insightful findings. The other is that the rest of 10 steps are not aware by the analyst. Then the strategy will be more training or more obvious UI that leads to more usage.
Conclusion
From my experience in participating Adobe Analytics Challenge, I try to imagine I was a member in Adobe product team. With the massive usage data from participants. I tried to set the persona who will use Adobe Analytics in their daily tasks.
And I define the goal and problems for the team. I take one step forward. By defining metrics based on my experience in utilizing the tools, I recommend if the product team is able to track these metrics with meticulous care, it will be a great chance to reflect on this B2B product and optimize it to meet the clients’ need.
If you like the article, feel free to give me 5+ claps
If you want to read more articles like this, give me 10+ claps
If you want to read articles with different topics, give me 15+ claps and leave the comment hereThank for the reading
