And why we need to rethink business intelligence

Going for a dive. Photo by Joe Pohle from Unsplashed.

The real meaty questions about the business such as: “what contributed to the sudden growth in revenues last month?” cannot be answered by a generic dashboard with revenue and traffic numbers. You need to do a deep dive into the problem. You need to unravel the question into specific queries.

  • Query revenue by month for the past twelve months
  • Check for seasonality patterns by cross-checking monthly revenue, spaced 12 months apart
  • Query marketing spend data
  • Query marketing traffic data
  • Query product usage data and cut by different actions taken by users
  • Look for differential product actions taken by all users

With, adding context to everything in your data stack is a breeze 💨

Imagine a science lab that has a bunch of test tubes with all sorts of technically sophisticated biological experiments (I’m talkin’ CRISPR, immuno-oncology, mRNA therapeutics). Now imagine that none of the test tubes have any labels, so you don’t know the context of any of the experiments.

Does this sound crazy? Good, because it is. And this is the state of the modern data stack today (cloud-native data warehouses as the central source of truth for data applications). There is no documentation to be seen for tables, columns, and queries in your data warehouse. The method du jour is frantically…

Robert Yi, Ph.D.

San Mateo, CA

I really like mechanical keyboards.

I became a data scientist after my Ph.D. program at MIT. I ended up getting into data science when a coworker prodded me to switch from MATLAB to python, and it all went downhill from there.

In the past, I’d say I’m most proud of the work I’ve done on uplift modeling (I open-sourced the library pylift at Wayfair) and on CUPED (I came up with some cool, novel variations to drastically reduce experiment runtimes at Airbnb). But that was a small part of my jobs in both cases — I ran…

Keeping the main thing the main thing

One of my favorite aphorisms in management and technology: keep the main thing the main thing. It’s a succinct reminder that we all get carried away by unimportant time-suck from time to time. If I’m being honest, I get carried away the majority of the time. Figuring out what the most important things are and letting everything else burn is the primary skill-set that a CEO or a manager needs to have, especially at a small organization with very little guardrails or resources.

The mental model of juggling

Another helpful mental model — which is essentially isomorphic to (the same, but viewed with a different… brings queries, docs, and metadata to a single workspace so you’ll never have to context switch again. Image courtesy of

(Promo: Join our lifetime membership for $100)

The All-In-One Workspace for Data Analysts

Today, I’m excited to announce our invite-only release of, the first all-in-one workspace for data analysts. brings queries, docs, and metadata to a single workspace. We built to address two decades of frustration around data productivity at data-driven companies such as Airbnb, Bird, and Wayfair. With, you’ll never have to context switch between multiple apps to do your data work.

Supercharging the SQL Workflow

One of the core workflows of a data analyst is writing SQL. We believe that the tools currently in the market do not address the unique needs of a…

Or why a consolidation is coming your way

My take: the current tools to work with data in the market are ineffective and will face a wave of consolidation. Why? There are too many tools, and these tools are not great. Read on to learn more.👇

Problem 1/ there are too many tools 🤯

We analysts are using a half dozen different tools to get our job done. The native Snowflake IDE to write queries, Notion to document queries and business context, Google Sheets to document tables and metrics, Slack to bug people to make sure documentation is not out-of-date, and Tableau to visualize some data and to publish results. If you are at a…

Data developers can use to automatically map data context in an organization.

One of the most important aspects of data science and data engineering is understanding and managing the context of the disparate data assets in an organization. Data context 🔎 is about using the right data for the right purpose. Using the right data for the right purpose is generally more important than building sophisticated models because models are only as good as the data that you feed them. …

As I get ready to turn the pages on the second chapter of my life and get ready for my 30's, I wanted to lay out some of my learnings from my 20's. My 20's passed on by so much more quickly than I had imagined, and I suspect that my 30's will be even shorter-lived. I hope these learnings can be helpful to those of you who are fighting your own battles. I hope to reflect back in ten years to see what has changed, and what has not. Some of these learnings are universal, some are idiosyncratic to…

As 2020 draws to a close, I wanted to quickly share this year’s learnings at Dataframe with our friends, community members, and supporters. It’s been a strange year to say the least, with many surprises along the way.

  • Pandemics are terrible. But technology workers are incredibly lucky in that the Covid-19 pandemic is accelerating the digitization of everything. Digital work enables remote work, which is restructuring how teams operate (across geographies and timezones). Tools and processes matter more than ever. …

Today, the team and I are excited to share what we’ve been working on for the past year. We’re launching our invite-only release of 🧪Dataframe, the simplest Data Discovery and Documentation tool for your data warehouse.

About a month ago, we launched 🐳 Whale, our terminal-centric open-source data discovery tool, dubbed “the stupidly simple data discovery tool”. We were pleasantly surprised by the love and support we received from the data community — we surpassed 350 stars on Github and gathered a tightly-knit group of active contributors and early adopters.

In my career as a data scientist, I’ve built data…

Joseph Moon

Data Scientist, Entrepreneur, Investor. Harvard & MIT. @josephmoon_ai

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store