Self-Service BI 2.0 – Offloading self-service BI to business end-users with Data Catalog + + Self-Service BI Tools

Self-Service BI 2.0 (SSBI) – Offloading self-service BI to end users with Data Catalog + SSBI Tools Self-service BI has been around for decades. The original intent was to offload developing reports / analytics from centralized IT to business functions.  

Companies invested in Self-Service BI tools, and hired specialized Data Analysts and Scientists. With the explosion of Big Data and the need for faster, data-driven decisions, we are again running into bottlenecks, where Analysts and Scientists cannot keep pace with the Analytics demanded by the business. In this article, we look at how enterprises can offload self-service to business end-users using a combination of Data Catalog and Self-Service BI Tools. Thereby unleashing data agility, enabling a data-driven culture, and allowing their prized Analyst and Scientists to focus on Advanced Analytics and AI/ML to drive business innovation. 

What is Self-Service BI? 

As the name suggests, it means the ability for business users to do analysis without relying or depending on other parties. 

For Data Analysts who are the primary doers of Self-Service BI, it means being able to do their analysis without relying on IT or Data Engineering. 

At a high level, self-service BI comprises 3 steps. 

1. Data Discovery: This is the stage where based on the need, the user needs to be able to figure out which data to use, where it exists, what it looks like, what its quality, and how to join the data. This is a crucial step. Because raw data lacks business semantics knowledge. Lack of data profile and quality information to make informed decisions and confidence in using the data. Additionally, data stored in data warehouses has a structure. The most common structure is star schema which is optimized for querying. In order to use the data for any sort of slicing, dicing, filtering, it requires joining transactional data (information, history, fact tables) to attribute data (dimension tables or master tables).  
2. Data Preparation: Once you have understood the data, then you extract the data, shape it, and apply transformations.  One of the most basic transformations is joining transactional data using SQL with attribute data to be able to filter, aggregate, slice and dice.  Other common operations are formulas. 
3. Data Analysis: The final step is to visualize the data, finding patterns, trends, pareto analysis, and finding the root causes or insights that will drive decisions. 

Need for off-loading Self-Service BI to business analysts and users 

In a digital economy, data is the new currency. It is being created and consumed at an ever-increasing speed.  

Executives in every company recognize that data needs to be harnessed to fuel business initiatives. And the need for finding insights from the analysis is outpacing the ability of Analysts and Scientists to churn out the analysis. And not all demand for analysis comes in the form of on-going operational analytics. They come in the form of “Ad-hoc requests” that are different each day. Each of these requests may be driving a small or micro-decision. But when you add all that up, the limited pool of Analysts and Scientists cannot deliver the analyses in a timely fashion. It is the theory of constraints ( i.e., the limiting factor in achieving scalability)!

So how do you address this problem? 

You can’t easily and simply hire more Analysts and Scientists. They are in short supply and they are expensive.  

Instead, you can empower and enable your decision makers like Business Analysts and users to do much of the basic analysis themselves. 

Enterprises recognize this and are setting up data literacy initiatives to enable citizen data analysts and scientists.  

Skills gap to address in enabling business analysts and users to do self-service BI 

If we go back to our three steps, the gaps are steps 1 and 2.  Business Analysts and users know their business; they know how to understand trends and find insights. After all, they do it from canned reports and Excel all day. The skill gap to address is Discovery and Prep. Training them on Self-Service BI Tools is a vital part of upskilling them. However, just teaching them on how to use Tableau is not going to address the whole problem. 

Why Self-Service BI Tools like Tableau, and Power BI are by themselves are not enough 

Tools like Tableau and Power BI assume the user knows what data to pull, how to write SQL to join, and what transformations to apply. But business analysts don’t know this and don’t have those skills, therefore – we haven’t addressed that gap yet. 

How does the right Data Catalog integrated with Self-Service BI tools enable BI Self-Service for business users? 

A typical Data Catalog enables a centralized enriched inventory of data across the enterprise by collecting and allowing metadata enrichment. This information is then available through Search. An agile Data Catalog like DvSum ADC also brings these additional capabilities. 

1. It profiles the data and captures the statistics that help with understanding the contents of the data sources 
2. It captures relationships between data objects either modeled in the source databases or infers them automatically 
3. It also creates an inventory of Data Models and Reports. These would be for example: Published Data Sources, Embedded Data Sources, Workbooks in Tableau. Or Datasets and Reports in Power BI 
4. It provides a no-SQL drag-n-drop interface to select and join the data and generates the Self-Service BI file like Tableau .twb file or Power BI .pbix file 

With these capabilities, you have the right tools to allow business users to do Self-Service BI. A step by step approach would be: 

1. Business Analyst starts with analysis to be performed
2. Search for existing data models and reports that already exist. If something relevant is found, then use that as a template. 
3. Otherwise, search for relevant data in the Data Catalog by using business speak. (Semantics) 
4. Explore and evaluate the data without extracting data or writing SQL – definition, data profile, quality 
5. Choose the datasets  
6. The Catalog automatically shows related datasets. (e.g. Dimensional, Attribute or Reference Tables related to the chosen Fact or Transactional Tables) 
7. Choose the relevant attributes and measures and filters 
8. The Catalog can automate creation of the SSBI starter file complete with Data Connection, SQL code 
9. User can now leverage SSBI features to do additional data transformation and preparation 
10. User has the final data available to visualize and analyze 

Summary 

Enterprises that want to unleash data agility have to empower their business users to do Self-Service BI.s  Business Users lack skills and tools for doing Data Discovery and Data Preparation which are pre-requisites before doing Data Analysis. Training those users to use Self-Service BI Tools is a necessary step but is not sufficient.  

Augmenting Self-Service BI Tools with an agile data catalog provides the right tools to enable self-service BI for business users. Consequently, the organization unleashes data agility, fosters a data-driven culture, and allows their prized Analyst and Scientists to focus on Advanced Analytics and AI/ML to drive business innovation.