Piloting FullStory as a Customer Experience Observability tool
Observability is something that was new to me in 2020. When I got to grips with customer experience observability, I became really intrigued with just how powerful it could be.
In Bazaarvoice, we wanted more people to get involved in observability, but realised that the time for gaining insights was really lengthy. This was due to having to jump between different tools and overall, it became more of a hindrance and not a seamless experience. We want to make it as easy as possible for people to onboard and for people to get the insights they need without having to go searching. This is where a pilot with FullStory comes in to help us understand if this tool can help us achieve these three goals:
- Decrease time to uncover conversion trends, pain points, and bottlenecks
- Understand friction error messages and user frustration
- Integrate with some of our current tools so we can have a holistic view
FullStory
We got a demo of FullStory — I was amazed. I almost didn’t think it was real. With very little setup we are able to gain a lot out of the box — rage clicks, frustrated user sessions and segment health to name a few. FullStory has customer analytics & session replay while also allowing NPS & alerting software to be integrated. This means that engineers don’t need to go between multiple tools to get the insights they need, they can do it all in one place or have a seamless link to the integrated tools. It also means our colleagues in product and design can gain insights which gives them the opportunity to have a conversation with engineering and vice versa. Something which we didn’t have a lot of before — all round this is a very inclusive tool.
Once you see the benefits you can gain out of the box — you can also create your own segments, funnels and dashboards to help you gain more tailored insights. In this blog I will touch on how you can make use of customer analytic features, session replay and tool integration.
Customer Analytics
There are a number of ways in which we gain customer analytics from FullStory, I have listed the features below and what my thoughts are on them.
Segments
Segments have been my favourite feature in FullStory. Segments really allow you to filter in so many ways so you get exact the data that you need to see. They separate your users into different groups making it so easy to filter by them in funnels, metrics and dashboards.
Segments can be used if you are looking at a particular funnel, for example the amount of people who visited the login page and actually logged in. You can create a segment for users who are external and federated vs external and non-federated. On the funnel you can select both segments and compare.
An extremely useful segment we have created is one to search for a user and an activity. The session replay function will take you to the point in time that the user done that activity. At Bazaarvoice we don’t recognise the email address until the user has successfully logged in. The users who have not yet logged in all appear as User 1001, User 1002 etc. This would be very time consuming if you had to watch each unidentified user session to try and find them entering in the email that you are trying to debug a ticket for. This is where the segment comes in very handy! In the image below you can see we are searching for Any user, who Visited URL where the path is /sso and the Changed Text is exactly (enter user in question email). This brought us to the user in question session and from there we were able to find exactly what we needed. The amount of time this segment saved in a particular issue was impressive. With the use of dev tools we are able to see the users ajax calls therefore meaning we would not need to set up a call with the client saving a lot of time for them and the time to insights and resolution being hugely decreased!
Funnels
Funnels are a really useful way of seeing users dropping off the expected journey and therefore allows us to measure conversion rates. I touched on the uses of funnels in my last blog so I will highlight the added extras I have seen using funnels in FullStory. With FullStory funnels for example, you can add a filter to select up to 4 segments for comparison or you can group by a particular property e.g browser, clientId or URL. The funnel feature also gives you the conversion rate and the number of users with statistics if it is an increase or decrease — very useful for a quick glance.
Metrics
Metrics are a way of having an event such as error clicked, an aggregation type such as add and then the format you want that outcome to be such as percentage. This then plots it onto a graph. In the image below we use metrics to add the number of error clicks and rage clicks in one url and it is output as a percentage on a graph. If you scroll on past the graph in FullStory you will also see sessions ready to replay of all the users who have experienced that — an example of how powerful FullStory is!
Dashboards
Dashboards allow us to bring all our funnels and metrics together so we can view them in one place. What I really like about FullStory’s dashboard is that you can add a title to group different metrics or funnels which are easily moved about the screen and you can use dynamic colour to compare different metrics to see at a quick glance. If something is red you know you need to take action!
Session Replay
FullStory’s session replay has a lot of extra functionality which engineers greatly appreciate. There is the ability to view the network tab of the user in the session, so you can see the requests and the response headers at the time of the incident. You can then also create a note at the point in time where the error happened and share the link, so that other engineers can go straight there and watch the issue. This greatly cuts the time to both insights and resolution. What I love about session replay is that it appears at the bottom of all funnels and metrics so you can always view sessions related to your search.
Notes
This is another feature which I highly rate. It allows you to add a note to a session that you are watching at any stage of the replay and then create a link which can then be shared via JIRA or slack etc. This is really useful for a quick link to the exact moment you are pointing someone to, with notes added for context. If you have integrated FullStory with JIRA, you can also just send this straight to JIRA from the replay screen.
Dev tools
Dev tools during session replay is extremely handy for engineers, with a console and network tab. We are able to use the AJAX feature to see the request and response headers being sent for each request. If Sentry is integrated then any errors that appear will have a link straight out to sentry.
Page Insights
Page Insights gives us click maps, scroll maps and inspect mode. Click maps as you can see in the image below gives us a number of different engagement metrics we can select including most error clicked or rage clicked.
The image below on the right hand side shows the statistics which we can get for each element clicked on screen.
Tool Integration
FullStory allows for a large list of integrations which makes this tool even more powerful. Some useful integrations include JIRA, Slack, Salesforce and Qualtrics. We have integrated with Sentry to link our sentry errors. This allows us when we are watching a session and the error occurs to link us out straight out to that link in Sentry.
Overall, I really enjoyed piloting FullStory. The out of the box insights, the features which give endless insights as well as having customer analytics and session replay in the one place has been so beneficial. The user interface is very easy to move between and is really user friendly. We have seen a reduction in time to insights as well as time to resolution and been able to understand our user friction points all very quickly. The pilot has been a great success. I’m looking forward to continuing to use FullStory to see how we can improve our customer’s experience when they use our platform!