Overview
In 2018 I started working at OnlinePajak and have a primary focus to increase the user acquisition and help tax-payer simplify their tax management. So we started with redesign the invoice application of OnlinePajak (called eFaktur), and even though we redesigned the application and user likes it, but it’s only impacting the completion rate from creation invoice to approving invoice not increasing user acquisition.
We started to analyse the flow and step from our funnel and to check which step that make a biggest dropping point in our funnel, we found out that the registration of government certificate (called eNofa) is the biggest drop. After discussing and brainstorming with the team we agree to try to improve those step by redesigning the flow.
Role: Product Designer
Duration: March to May 2018
Company: OnlinePajak
Analyse
Here I started with analyse our internal funnel in this phase to really understand what is actually the biggest problem that hold new user to be acquired. While I do analysing, I invite the team (Product Owner, Engineer, Sales, Customer Success and CEO) to do the workshop for discussing the finding from previous redesign project, problems, ideas, possibilities and analyse report together.
Research insights
I finally gather all the insight from users constraints, analytic tools and mapping the current flow.
After looking at the funnel result of the last experiment (Interface Redesign of OnlinePajak Invoice), we notice there is a steps that didn’t increased. Those step is when users arrived on the application and start to setup and connecting their government certificate with OnlinePajak till they complete the setup.
We also collected the feedbacks from user directly when they didn’t connect their certificate with OnlinePajak using hotjar apps. And here are the most common feedback list from the users.
We went to mapping out the current user flow to really understand the system and also to understand where we can improve it.
Challenge: How might we make user trying our application?
After all the research and analysis we had, we come up with one big question that might the real reason why we cannot achieve our objective. We start to review why our drop points is always on the step when user need to register their certificate, try to understand why we need the certificate first before user can try to create invoice, and there is some pain point from the users feedback that seems not really trust OnlinePajak when it come to register their certificate (even without knowing how the application work).
In order to define user is acquired is to make user approve their tax invoice in eFaktur OnlinePajak application. But if we try to dig more deep, user need to try the application first before they decided to use it or not.
So I come out the idea of changing a little bit on the flow, where user can access our eFaktur application and try to create the invoice before they decided to fully use OnlinePajak and register their certificate. After having a discussion with engineers and PO about the business constraints, technical constraints (internal and government’s side) we are agree to have this idea to be build & tested.
Testing
After we finalise the new flows, it’s time to test the flows as we want to know the new flows performance on increasing the acquisition, we set-up the A/B testing where we compare the old flow (control) and the new flow (variant).
To running this test we use google optimize for the analytics tools and to make a user separation, in this case we use 50:50 (where variant will have the same amount of user visit as control).
We running this A/B testing started from 27 march 2018 until 15 may 2018.
Result
After almost two months we running the testing, we agree to stop, analyze and summarize the result.
On google optimize it’s clear that the new flow is lose based on the government certificate registration. But I need to analyse from another tools (mixpanel) to have a deep understanding, if the new flow bring something valuable to the users or not, to check which step is broken on the new one, and to find out what things need to be improved for the next iteration. So we analyse the funnel on each variation.
After seeing the funnel, I get something that meet a little bit of my expectation, where the completion rate from user create tax invoice draft to approve is increasing by 0.35%. We know that percentage not big enough to decide that the new flow is better and can help to achieve the objective but I believe this flow can bring something if we run it. After some discussion and negotiation with some stakeholder (engineer, PO and the CEO) we agree to use the new flow for the whole month of may and see the result if it’s increasing the user acquisition or not.
After we switch all the new users to try the new flow in the whole month of may we got 484 user acquired in that month, and we switch again the next two month (June and July) to see the difference. We finally get to know that the new flow is help us increase the user acquisition more than 50%.
What I learned
Not all product problem will get solved with redesigning the application, but simply changing a bit on the flows can make a big difference.
Sometimes we need to be more confidence with our solution, just because our first testing result was bad doesn’t mean we cannot go with that solution. We just need one more and more testing.
As I iterated and got feedback I learn some principles that I knew my final solution should have. These helped dictate design direction and shaped the feeling of the final product.
