Best Practices for Building Analytics-Ready Data Lakes

This session guided users through the process of evolving from a raw data into an analytics ready state. The audience was led through the Data and Analytics Supply Chain which is broken into three groups: Data, Analytics, and Insights.

1. Data (e.g. Analytics Data Preparation, Data Warehouse Automation, Data Lake Pipelines)

2. Analytics (e.g. Analytics Calculation, Geo Spatial, Data Science & Predictive)

3. Insight (e.g. Guided dashboards, Governed Self Service, Mobile Analytics)

It was shared that some of the reasons why some companies fail to deliver a great data product is due to speed leading to data dumping, poor data lineage, governance or outdated data and performance issues. Then it was discussed how Qlik’s suite of products could help address these issues. The Data Catalog, a centralized place where all data is searchable by users but still governed and secured by IT has the ability to have multiple personas (roles). For example:

• Data Engineers can onboard data once they built their operational pipeline and then use the data catalog to measure the quality of the data.
• Data Quality Analysts can then use Data Catalyst to ensure naming conventions are followed, masking data, or provision the use of it.
• Business users have the ability to look for the data they need, can reuse and collaborate, and publish to Qlik or another BI tool.

The presentation was wrapped up by offering some key advice:

1. Architectures are evolving, reinforce platform independence

2. Automation is key to delivering real-time, analytics-ready data

3. Create processes that heighten trust of data governance and access

4. Assure alignment between IT and Business; and make data available to everyone!

Visual Studio Code

Qlik this year also explored new ways of development, whether it be with its new integration of scripting using Visual Studio Code, or it’s new cloud-based command line interface. Visual Studio Code is Microsoft’s new open source text editor and being able to leverage its features in Qlik would be immensely helpful for Qlik development. The CLI tools also help speed development as you are no longer tied to just using Qlik’s own UI but instead could use simple scripting commands to perform migration, reload, version control and testing tasks from the console.

Qlik Engine

Qlik also demonstrated an insider look into The Qlik Engine, which details how their backend works and how it powers their Qlik products. From script execution, to Selection to the client side, its worth checking out if you’re at all interested and may provide insights into how your data works on the inside.

Links to sessions:

• Best Practices for Building Analytics-Ready Data Lakes
• VSCode as a full Qlik Sense Editor
• Qlik Engine – Under the Hood
• Modernize your workflows – An insight into the world of scripting

We were impressed by Qlik’s ability in migrating QlikWorld online and their free breakout sessions for all of us this year. Now that these sessions are offered on-demand, check out all the new advancements Qlik has to offer. The best of QlikWorld Online is only a qlik away!

Toby Baker, Sr BI Consulant
Christian Haro, BI Consultant