We know everything you’re doing, and when you’re doing it…

June 11, 2015 / Sam Smith

A few months ago, I wrote about an experiment I conducted where I tracked how often our users were clicking on the date picker buttons in our reporting platform. The results to this experiment were so profound that they led to a significant change in our product. Since then, I have become fascinated with the idea of tracking usage statistics within our products.

Over the past two months, I have built a full-featured product analytics tool that tracks every single thing that a user is doing when logged in to our platform, when they are doing it, and even how long it takes them to do it. Here is a screenshot of what that data looks like on our end:

As you can see that this data is organized onto a timeline where the date range can be adjusted. There are four categories, each of which is expandable. These categories are Configuration, Lead Box, Reports, and General. In the screenshot, you can see that I have expanded the Configuration category, so that I can view when users in this account have set up automated reports, tracking lines, staff, and users. I have also expanded the Reports category, so that I can view when users are actively viewing each specific report, and for how long. If the user’s mouse stops moving for two minutes, we stop tracking data until their mouse moves again. If I want to view data for a single specific user, I can select an individual user from the dropdown in the top right. Lastly, I can hover over each data point for more detailed information.

All of this new data will be critical in shaping the future of our product development, as well as the manner in which our consultants train our users to use our products.

To take things a step further, we recognized that there are certain one-time events and actions that are critical to the progression of each user’s understanding of our product. By tracking these events, we can identify the next steps required to ensure that each user is getting the maximum value out of our product. Here is an example of what that data looks like on our end:

You can see that there are four types of events – Critical Events, Secondary Events, AC Events, and Support Cases. There are only about six Critical Events in total, and these events are the steps required for a brand new user to get to the WOW moment within our product (the moment that they realize how awesome and valuable our product is). You can see in the screenshot that I am hovering over the second Critical Event – “provisioned first line”. Behind Critical Events are Secondary Events. There are many more Secondary Events than Critical Events, and while these are important, they aren’t quite as urgent for our consultants’ attention. And that brings me to AC Events. Our Associate Consultants (ACs) spend most of their time reaching out to our clients, training on our products and ensuring that each customer is receiving maximum value. And lastly, we log when a support case has been created in association with this account.

Altogether, this event timeline paints a great picture of the lifespan and progression of every single one of our clients. You can be sure that in the coming months, there will be some great changes coming to our product as a result of this new data that we have just begin tracking.

Functionality: Keeping Your Users Happy

March 6, 2015March 6, 2015 / Jashona Young

Picture this: You receive a text message saying that your bank account is overdrawn and you are in the middle of procuring your life’s necessities (also known as grocery shopping). You know you have money in your savings, so you’ll just transfer $200.00 until your next payday. So you pull up your bank’s app/website and enter your username and password on the log in screen and press enter, nothing happens. How do you feel? Most likely a bit ticked off that you can’t access your hard earned money, correct?

Here is where a website’s functionality comes into play. The design of the website/report can make is rain silver and gold, but if you click a button that is supposed to log you in or load up another date on a report, the design of the website means nothing to you as a user. As a back end developer it is my job to make sure that every time you click, drag, or drop that the information you need is accessible and loads as quickly as humanely possible. To keep our users happy and smiling, I implement a functionality goal list in all of my programming projects:

When a user creates an event on a page:
1. Does it load correctly?
2. Does it load quickly?
3. Does it display the correct information?
If a page is to error out:
1. Does it display a message that says we will quickly fix this issue?
2. Does it display any coding that a user can use to hack our system?
Does it load correctly on all devices? (Meaning no cut off words, etc.)
1. Computer?
2. Mobile Phone?
3. Tablet?
Can something be done to speed up the final queries just a little bit more?
Can (2-3) users other than myself confirm that all of this works correctly as well?

If you can confirm that all of these actions pass your expectations, you will have a functional website or report on your hands.

Want to see this list in action? Check out any of our reporting pods in Car Wars, Service Setter, or any of our other platforms!

Usability: Designing with Data

February 20, 2015February 20, 2015 / Sam Smith

Last summer, I added a fancy date picker to the top of all of CI’s major reports. This date picker included buttons to quickly access data for a variety of date ranges. This feature enhanced the usability of our software, but we recently realized a problem – the date picker was taking up too much vertical space on the top of the page, pushing each report’s meaningful data down and out of view when the page loads. To remedy this problem, we have collapsed the date picker behind an expandable button – which bumps up the meat of the report upwards a good 150 pixels when the page loads.

This was a quick change that was really just a solution to a longstanding (and rational) complaint from our CEO about the date picker being too big. But before making this change, I wanted to track the impact the change would have on our users, so I built an analytics script that tracked how often each button was being clicked within the date picker on a report-by-report basis.

After a week of data collection, I noticed a very curious trend. Of the thirty or so unique reports that we have across our various platforms, nearly 80% of all report views were for the Outbound Activity report. The Outbound Activity report is a useful report, but is certainly not one of our flagship reports.

Even more interesting, of the people who accessed the Outbound Activity report, 65% clicked the “Day” button on the date picker (which shows yesterday’s data), and 82% clicked the “Next Range” button.

I was at baffled by this data. Why are our users so obsessed with the Outbound Activity report? Why are they clicking the “Day” and “Next Range” buttons so often?

After a discussion with one of our consultants, the answer became blindingly obvious. The vast majority of our users use this Outbound Activity report to monitor their live outbound calling activity on a per-staff member basis, ensuring that everyone is meeting their daily OB call quotas.

Not having known this, we still had the default date range for this Outbound Activity report set to the previous week. Thus, our users were (apparently every hour or so) having to load the OB report, then click “Day” to load yesterday’s data, then click “Next Range” to load today’s data. Convoluted by the fact that the date picker was about to be collapsed behind a button, I realized that the process for accessing live outbound call data in our product was not at all easy. In fact, it really sucked.

Solving this problem was as simple as adjusting the OB Activity report’s date range to default to today’s date. The real challenge in the scenario was identifying that there was a problem in the first place, and this would never have happened had I not built an analytics script to gather data on our users’ behavior.

This experiment has undoubtedly inspired me to integrate user data collection into future design decisions. Improving usability always starts with understanding the user, and I look forward to using data collection to supplement my own intuition in my journey to get inside the minds of our users.