power query

1. Are you new to Power BI?
2. Been hearing a lot about Power Query, Power Pivot or Power BI lately?
3. Has your management or HQ asked you to quickly learn Power BI?

If the answer to any of the above questions is Yes, then it is time you learned somethings about the latest offerings from Microsoft for Business Intelligence.

After all, Microsoft is way ahead of the competition in terms of Vision, Strategy & Speed of Execution in the space of Business Intelligence. See the latest Gartner Research on Business Intelligence for yourself.

Microsoft is now miles ahead of Tableau or Qlik.

No wonder companies are ditching such software and migrating their Dashboards & entire reporting environment to Microsoft Power BI.

Current State of Enterprise Reporting

Most likely your current state of Corporate Reporting is confined to Analysis in Excel, Conversion of the analysis results to a Line or Bar Chart, and then Pasting the charts into PowerPoint for presentation to the management & clients. This presents a static view of the data, shown in board rooms all over the world.

However, the key thing lacking in such reports is the interactivity. Suppose your customer suddenly asks you to compare the last quarter with the same quarter a couple of years back. While you might have the data, you don’t have the chart ready-made, right now. Chances are high that you’d have to apologise and promise to show them the requested report or chart in a subsequent meeting.

But Business can’t wait! By the time you show them the report next month, it may be useless, and people would have even forgotten about it too.

Plus, sharing data with your users is a big problem. You’d have to send huge Excel files, that take time to load, and are full of VLookups to different prices and master codes. If someone were to tinker and make a change in a place, chances are that the whole thing might collapse, and render the reporting useless.

Additionally, Reports don’t refresh automatically each month. Someone needs to load the next month’s data, and refresh the reports manually, month after month. This wastes so much corporate time & resources.

The ability to show any data, from any month, quarter or year, on the fly, can help the business answer any question they may have, and react faster!

And it would work wonders if the reports could refresh automatically, month after month, without anyone’s intervention.

Enter Power BI Suite of Products…

What is PowerBI Suite of Products?

Power BI is a brand new product from Microsoft. It was launched in 2015, and in less than 5 years, it has gained supremacy in the Business Intelligence space.

Power BI enables the common users to build stunning reports, dashboards, and make them available to all users, without having to download any expensive software. Users can consume the reports anytime, anywhere, even with just a phone or on an iPad, on any browser.

The reports are a visual treat, and make playing with data a breeze. It is extremely user-friendly and has hardly any learning curve.

If someone can use a web page, they can consume a dashboard done in Power BI and analyze data to their heart’s content – slice it, dice it, export it, print it, compare it with another month, year on any business segment, any category, any zone.

The Power BI Desktop is free to download.  It can fetch data from over 70 different sources, including Excel, any SQL or native Database, Web, SalesForce, SAP, Azure, Google Analytics, Mailchimp, ZenDesk, Twilio, SurveyMonkey and several other cloud services. Power BI is able to clean the data, process it, and get it ready for consumption, all in a simple, easy to use software.

Everything you need is in-built, ready to use.

There are 4 major components – Power Query, Power Pivot, Power BI & the Power BI Online Services.

They help you to Get Data, Analyze Data, Visualize Data, and then Share Data with your users. Let’s delve into these components a bit deeper…

What is Power Query?

Power Query allows us to bring in data from almost any source, clean it, fill empty values, replace nulls, remove empty rows, and do Vlookup type operations into other tables to pick any reference values, all without writing a single piece of VBA code, or any formula.

Everything is done through the elaborate options, buttons, settings and features of Power Query.

I was able to clean very dirty data files in just 12 minutes, which would have taken me at least 3-4 hours to do manually in Excel.

And I don’t have to do it again! Next week, when the new data file comes from the ERP system, I just have to drop it in the correct folder, and it will be cleaned up automatically.

PowerQuery was released as an Add-On component in Excel 2010 & Excel 2013, but it is now embedded into Excel 2016, 2019 and Office 365. No need to install or enable anything.

It is ready, enabled by default, and available at a click of the button, already existing in the Microsoft Excel DATA tab, as well as in Power BI Desktop.

Have fun with it… it is an absolute delight to load and clean data using Power Query. Once the data is loaded into Power BI, we then have to learn how to create a data model in Power BI.

What is Power Pivot?

PowerPivot is the engine that powers the data visualizations in Power BI. It runs in the background in Power BI, and as an add-on within Microsoft Excel.

ower

Loading Millions of Rows, Fast!

Each Excel worksheet has a limit of just over a million rows. 1,048,576 Rows to be exact. This is a logical limit. However, most of the time if I only load 500,000 rows of transactional data, the Excel file becomes quite large and takes forever to open.

Plus, multiple Vlookups, complex Formulas etc. can slow things down, and Excel becomes unresponsive for long periods as we make changes on large worksheets and workbooks.

Enter Power Pivot, which is an Add-on to Excel (Yes, it is still available as an Add-On on Office 2013, 2016, 2019 & Office 365).

With Power Pivot, this limit of just 1 million rows is easily eliminated. Now we can load millions of rows, and the file size does not grow considerably. Plus the Excel files are quite responsive and able to handle things quite fast.

This is because PowerPivot does not store the data in the traditional Excel way. It uses the Vertipaq Columnar Database, which compresses the data, and loads what is needed for any calculations in the RAM only.

The speed is blazing fast and allows you to work with Excel freely. Plus, it removes all the limitations that came with Excel Pivot Tables.

Now we can extend the normal pivots by loading data from multiple Tables, Multiple Files, Multiple Sources, and combine them, merge them, mash them up and report using a Data Model, which can have relationships with the different entities.

The ability to load from multiple sources, and create pivot table reports that use Big Data (1 Million Plus… usually 40-60 million rows is not a problem), and is still quite fast.

I haven’t seen Excel ever hang or blue screen on me even with this huge sized data, which is what I use most of the time. It is like working off our Corporate Oracle Database, which has over 500 Million+ rows of transactions and is over 80TB in size.

Data Model View

I can view the entire data model visually, and see the relationships… something that I couldn’t do in several other databases or reporting software. This allows you to see the tables and their relationship with other tables easily. A visual data model shows the relationships clearly.

Data

DAX – A New Language For Writing Amazing Formulas For Visualization

With PowerPivot, Microsoft has introduced a plethora of new formulas, a completely new language of writing formulas – called DAX (Data Analysis Expressions).

I simply love writing DAX to calculate things which would have taken me complex formulas to compute, with a lot of helper columns, tables and worksheets.

Things like measuring Revenue from the same period last year, last quarter and available in a simple function.  Counting Distinct Rows, Calculating things over multiple tables, with multiple complex conditions is handled so seamlessly and easily, that I am amazed.

I used to be a die-hard SQL fan, being able to extract any data from any database using SQL, but with DAX, it puts my SQL skills to shame. The DAX calculations functions are aplenty and make any calculation a matter of a few minutes to write.

However, learning DAX does take time. There’s Row Context, Filter Context, and the ability to alter context on the fly take a while to understand. Newbies often get stumped in understanding these concepts and it does take time to get a good handle on writing good DAX.

Even though learning DAX is complex and takes time to master, the effort is simply worth it.

I never regretted the time I spent in learning and writing good quality DAX. It has allowed me to calculate complex things for myself and my clients.

In fact, almost all of my consulting time is spent in helping clients read, understand or write complex DAX measures. They love it, and I love teaching it too.

Once the DAX functions are written, it is time to visualize the data. This can be done in a normal Pivot Table in Excel, but I prefer to visualize this in Power BI – which has a number of chart types to visualize the information easily.

What is Power BI?

Power BI is the beautiful, sexy, outer world, where the clients see amazing column charts, bar charts, pie charts, maps, slicers, matrix reports, KPIs in Dynamic, self-updating Dashboards.

PowerBI is simply a class apart!

In no other BI software have I seen so many ways to visualize, slice, dice, and navigate the data, so easily. I seldom have to teach the interface to any client – because it is so intuitive, easy to use, completely user-friendly, yet extremely powerful.

Power BI lets you create multiple ways to visualize the same information. It’s a breeze to create any visual, just by dragging and dropping the different measures, and slicing them by any dimension – by country, by geography, by business unit,  by category, sub-category, by zone, by year, by quarter, by sector, by Product… almost anything you have in your data.

It can come from any dimension in any table within the data model.

The best part of Power BI Visualizations is that it automatically filters other visuals immediately as you touch any bar or value in any other visual.

This allows us to see things in its entirety, without having to write any extra code or effort. Plus you can a good view of the pie of the pie or how much impact does one item have on the overall value?

We can see the data, sort it, export it, and see only one visual in the Focus mode.

Once you have analyzed the data in Power BI Dashboards to your heart’s content, it is time to spread the love, and share it with other colleagues and stakeholders who could benefit from the dashboard data analysis and visualization to make better, more informed decisions.

You don’t have to send huge, heavy files by zipping to anyone. Simply use Power BI Online Services.

What is Power BI Online

Once the Power BI Visualization Dashboard is completed, it is time to Publish it Online, publically (Free), or to your Private Group of People within the Department or Division or Company (Paid) through the Power BI Online Services.

You can share your dashboard with others by emailing them a link, and then they can consume it whenever, wherever, without installing any software. They can browse, slice and dice, visualize in any way, on any device, using any browser.

There is even a Phone View, which fits all the visuals perfectly on the phone, and makes it easy to check the KPIs on the fly.

The paid Power BI Services allow you to refresh the data every 3 hours, and you can even go to refresh it every half hour in the Enterprise server option (that’s akin to 48 refreshes each day)

For those with Write or Edit access, they can even make changes to the reports and dashboards and save another copy. You can publish as many dashboards, in different workspaces, and it works seamlessly with Sharepoint, Web, Azure and all other Online platforms, refreshing data on the fly and showing you the latest numbers.

A Perfect Package of Power, Simplicity & Elegance

All these features packaged together make the whole thing work seamlessly. You don’t even realize when you moved from Power Query to Power Pivot to Power BI to the Online Services. It just feels one simple to use package that does it all.

Microsoft has put in a lot of effort to design a state of the art, cutting edge Business Intelligence Software for the information-hungry business world.

Unlike Microsoft Office, which only gets updated every 2-3 years, Power BI suite gets updated each month, with multiple features, and even new DAX formulas being released each month.

Start the Exciting Journey of Gaining Business Insights With Microsoft Power BI

It’s time to embark on the journey to understand, use and implement Power BI in your business. Help the business make better decisions with updated information available to the decision-makers. Give them the ability to slice and dice data without waiting for IT or analysts to prepare the reports manually.

Be future-ready. Don’t wait till all your competitors are using it to gain an edge.

Be the force leading change in your business. Get Started Today!

Intellisoft Systems offers 2 day hands-on workshops for Power Query, Power Pivot, Power BI & Online Services, that have been extremely popular in Singapore for SME and MNC companies embarking on the Power BI suite of products.

Written By: Vinai Prakash

Vinai is the founder & Managing Director of Intellisoft Systems, a leading Training company based in Singapore. Vinai writes regularly for the Straits Times, leading magazines and newspapers, and conducts several workshops around the world sharing his knowledge in Business Intelligence, Data Warehousing, Data Mining & Data Analysis.

At Intellisoft, Vinai conducts workshops and seminars on several topics like Power BI, Building Dashboard with Excel, Data Analysis & Project Management.

Contact us to attend a training or to organize a workshop for your entire department or company to benefit from Vinai’s impactful data analysis techniques and practical, hands-on approach that has won the accolades from thousands of delegates from around the world.

Article Written by Vinai Prakash, MBA, PMP, GAP, ACTA Certified

Additional Resources for Power BI

Training Courses

Data Analytics & Visualization with Power BI

Learn Microsoft Power BI Suite For Better Data Analysis & Reporting

Power BI Tips, Tricks & Video Tutorials

Power BI Tip #2: Reference Query Results in Another Query With Power Query [Video Tutorial]

Microsoft Power BI: Super Charge Your Data Analysis Process

Power BI Tip #6: Fixing The Vertical Axis in Power BI Visualisations

Power BI Tip #5: All About Slicer Controls in Power BI

Power BI Tip#4: Enter Data Into Power BI Quickly [Video]

Power BI Tip #3: Quick Formatting of Power BI Visuals