Alternative tools for product analytics

Jun 17, 2020 by Robert LacokTwitter icon

Picking the best analytics tool without breaking the bank.

Alternative tools for product analytics - preview image
Product analytics is at the heart of our business at Deepnote. We’re trying to build software people love to use, and user feedback tells us whether we are doing a good job. We ask our users for a lot of qualitative feedback with surveys but that only captures a part of the story. The second part is told by metrics, such as retention, engagement and feature adoption. ## Analytics tools are really expensive A few months ago we hit the ceiling of the free-tier limit on our tool of choice, and we found ourselves presented with a nasty surprise. The cheapest tier was going to cost about $12,000 a year, and even more shortly after. So we decided to shop around, looked at several competitors, and asked for their pricing. It was the same story everywhere we looked. Some free tiers were more permissive than others, and the price varied a little, but the order of magnitude was still the same — thousands of dollars per month. We are in the software business and we have no problem paying for software which brings us a lot of value. However, it seems like the core of the value here is generated by five charts with the most important metrics, and some ad-hoc queries whenever we’re curious about something — and those price tags just seem a bit too much for our stage of the company right now. > "It seems like the core of the value here is generated by five charts with the most important metrics." In the end, we decided to go with the tool with the best startup program, which helps us scale for another year — Amplitude. We’re very thankful for that, it is a great tool. Yet, they mention our current package would cost us close to $40,000 a year once the trial is over, and I am still not convinced that is the best use of the money. ## Alternatives to Amplitude I briefly looked at a few open-source applications we could self-host, but those have two caveats. They are not that great: some look awesome but are a bit too simple in what they can do, and others appeared like having plenty of features, but lack a coordinating designer to make it pleasant to use. Secondly, they bring the extra burden of another service to manage, update and secure. The reddit community seems like a cool place though. ## Solution: a warehouse with visualisation tools <img src="https://storage.googleapis.com/published-content/analytics/tools.png"/> ### Capture There are a few ways to capture events and send them to the warehouse and right now we are sticking to Segment. I really like it because it made switching from Heap to Amplitude a question of several clicks, rather than fixing the entire tracking code, and it was just as easy to start sending the data to a warehouse. ### Store We decided that going forward we will simply store our events in BigQuery — a fully managed solution that’s cheap and simple to connect to other tools, and probably my favourite part of Google Cloud. ### Query and share The last piece in our puzzle is Deepnote. A collaborative notebook with one-click setup makes it easy to query, model and plot the data, and share the results with the rest of the company. A short reflection — this might seem extremely obvious to you, and you might wonder why it warrants a blog post because you’ve done it this way a decade before the analytics tool got popular. Fair point, but when I was searching for a solution I never found a single suggestion, I just assumed I needed a dedicated tool. ## Integrations A nice property of this solution is that you can plug and play. You can write your own solution to capture events if you do not like Segment. You can choose Postgres, Redshift or any other DB/warehouse instead of BigQuery. And you can pick a visualisation tool with buttons if you prefer not to write python — if you have Tableau or Looker then great, but Data Studio is free and also does the job pretty well. ## Bringing analytics where people are Equally important as having analytics set up is having the company actually look at the numbers to inform their decisions. If the charts and dashboards are hidden in tools where people do not look very often, it might be difficult to form a new habit. Also, even tools as mature as Amplitude just make it really difficult to display the information I need in scheduled emails or on the TV screen in our kitchen. So far it is only my hypothesis, but I feel like embedding these charts into Notion could help spread the awareness more. Notion centralizes knowledge and project management across our company and everyone comes here on a daily basis already. Embedding a live a chart from Data Studio is a matter of pasting a link, and it looks great. We're also toying with the idea of creating embeddable cell outputs - would you be interested in using that?
from google.cloud import bigquery import pandas as pd import plotly.graph_objects as go client = bigquery.Client() query = """ SELECT DATE(timestamp) AS date, COUNT(1) as metric FROM `node_js.execute_view` WHERE DATE(timestamp) > DATE_SUB(CURRENT_DATE(), INTERVAL 30 day) GROUP BY date ORDER BY date """ query_job = client.query(query) # Make an API request. df = query_job.to_dataframe() fig = go.Figure(data=[ go.Scatter(name='Count (execute)', x=df['date'], y=df['metric']) ]) fig.update_layout(barmode='group', template='simple_white', title="Count (execute)", xaxis_title="Date", yaxis_title="Execute",) fig.show()
## Alternative uses of event data Even if you love your analytics tool and are not looking to switch, exporting your data to a warehouse might be a good idea anyway. It can enable alternative use cases, such as some ML fuelled predictions or product recommendations in e-commerce. ## Drawbacks There is a reason analytics tools are popular — they democratize the ability to work with data. Writing the correct SQL for retention analysis might not be everyone’s cup of tea. We are lucky not to have this problem because most of us are technical, but I can see how bigger companies might be reluctant. Good tools also bring a layer of access management over the data, which might be desirable in bigger companies. It is probably a good argument against something like Postgres, but BigQuery supports granular access to individual datasets, so it is just another problem that can be solved. Lastly, they have some nice features — like exploring causality between events, which might be more difficult to replicate. In my opinion, this seems like the last 510% of the value, and on its own is not enough to justify the purchase. ## Final thoughts We have not fully migrated from Amplitude and are still exploring this solution. I would love to hear more opinions. What do you think?

Run this article as a notebook

Deepnote is a new kind of data science notebook. Jupyter-compatible and with real-time collaboration.

Sign-up for the waitlist below, or find out more here.

To be continued...
RL
Robert LacokTwitter icon