Professional Documents
Culture Documents
TECHNICAL BRIEF
A QlikView Technical Brief
September 2013
qlikview.com
Table of Contents
Introduction 3
Overview 3
Data Sourcing
Provisioning Data
10
12
Learn More
13
Introduction
Business Discovery relies on the connection, transformation, distribution and ultimately,
analysis of data. This paper provides an introductory overview of the data flows through a
typical QlikView deployment and describes the role of individual systems. We explain how
data is sourced from multiple, heterogeneous sources, how it is manipulated to make it
consistent and logical, and how it is distributed where users can interact with the QlikView
applications.
Overview
There are four main systems involved in building a QlikView enterprise system: QlikView
Desktop, QlikView Publisher, QlikView Server, and Clients. To understand the data flow, we
need to understand the role of these systems and where they are situated in the system
architecture (see Figure 1).
SAN Storage
SMTP service
QlikView Desktop: This is the main tool for creating QlikView applications. The
application designer uses this tool to specify where data is sourced, how it is manipulated,
and how it is displayed. The application presentation is handled by Clients, but application
data processing is managed by the Server.
Clients: This is where users use the QlikView application to view and interact with data.
The application can be part of a standalone executable or part of a web page. The client
side of applications is designed to consume few computational resources.
QlikView Server: This system serves applications and data to clients, performs application
calculations, and manages security.
QlikView Publisher: This system provides a means of controlling how the data used by
applications is updated.
The sections of this paper will follow how data is sourced, loaded and modeled, provisioned,
and then analyzed in QlikView. Additionally, discussion of how data is governed and secured
is included.
Data
Sourcing
Loading &
Modeling Data
Data for
Analytics
Provisioning
Data
Governance
and Security
Data Sourcing
QlikView extracts data from multiple, heterogeneous sources (e.g. databases, spreadsheets,
web pages and Big Data sources such as Hadoop and Google BigQuery) and creates a
homogenous data set suitable for analysis and visualization.
QlikView
Direct Query
SFDC
Web
pages
Spread
sheets
QVX
SFDC
SAP
Other
(Direct Discovery)
ODBC
SAP
OLEDB
Big Data
Data
Warehouse
Standard
Databases
Once data is loaded into a QlikView application, its held in-memory. What this means is
that QlikView applications require one-time data source access to read in a dataset and
store that historical data. For new data (delta or updated data), QlikView can simply
load this new data and append it to the historical data without having to do a full reload.
In addition, QlikView utilizes sophisticated algorithms to compress the data (sometimes
up to 90% from the size on disk in a database) to make optimal use of the in-memory
store. For more information, please see blog post: http://community.qlikview.com/blogs/
qlikviewdesignblog/2012/11/20/symbol-tables-and-bit-stuffed-pointers
Application developers also use the Load Script to model data from the various source
systems prior to inserting it into the in-memory engine. In reality, business intelligence tools
must cope with data that is incomplete, poorly labeled, or duplicated across multiple sources.
Linking data from different data sources requires the use of a key, but the same data can be
labeled in different way across different sources (e.g. Sales, Sales Revenue, and Sales
Numbers might all be the same data see Figure 4). QlikView can easily merge these
similar data fields from different tables into a single, consolidated view (e.g. converting the 3
Sales fields into a single field called Sales $ see Figure 4).
A more subtle problem is slightly different data formats for the same underlying data, for
example one data source might store dates in a single YYYY-MM-DD field while another
might have separate Year, Month, and Date fields. The application designer must be able to
consolidate all date fields into a single, representative view.
The Load Script allows fields to be renamed, separated, joined, or otherwise manipulated.
For example, the developer can do table joins, or create a Name field by combining First
Name and Last Name fields. Because QlikView directly reads in data sources, its possible to
manipulate fields across multiple data sources, for example the user could conditionally read in
sales person data (HR database) where the sales person has made a sale (Sales database).
QlikView provides a data model viewer (see Figure 5) that makes it easy to see the
associations that have been made within the engine as well as providing information about
the data such as density, field names, table names, and so on. It can also find data model
problems to fix them with the scripting environment.
The QlikView engine provides a unique associative capability to the data that has been
loaded. This means that data that is sourced from multiple systems can be treated as a
single data entity within the engine for the purpose of analytics, regardless of where the
data came from. QlikView applies associations between the data from the various systems
by automatically mapping fields that have the same name and same data type. This allows
users to interrogate and make discoveries in their data as if it were a single table of data,
rather than data coming from a variety of disparate and unconnected systems. In Figure
5, one can see the automatic associations are made between the Facts table and the
Employees table, for example, via the EmployeeID field.
Provisioning Data
QlikView offers a set of file-based data persistence options. In fact, every QlikView
application (a .qvw file) itself contains all the data needed for the application. This data
within the .qvw file on disk, which is binary encoded, represents the data that was loaded
during the previous execution of the Load Script. The Load Script is also contained within
the .qvw file, as is the entire presentation layer.
Larger deployments typically use a data staging layer. This is to a) provide atomic data
packages that are optimized for a particular analytic need (e.g. a Finance package that
contains data from various Finance and Ops systems), and b) provide an optimized data
loading environment for QlikView. QlikView developers can create a .qvd file which is an
optimized QlikView data file that can be loaded rapidly into a .qvw application.
Typical deployments of QlikView include a QVD Layer containing a number of .qvd files
(e.g. a Finance QVD, Sales QVD, Q1 QVD and so on) that application developers can use
off the shelf to build their own specific QlikView applications and promote the reuse of
consistent data across many QlikView apps. See Figure 6.
Learn More
QlikView Architectural Overview
QlikView Governance Overview
QlikView Security Overview
QlikView Design Blog Post: Logical Inference and Aggregations
QlikView Design Blog Post: Symbol Tables and Bit Stuffed Pointers
2013 QlikTech International AB. All rights reserved. QlikTech, QlikView, Qlik, Q, Simplifying Analysis for Everyone, Power of Simplicity, New Rules, The Uncontrollable Smile and
other QlikTech products and services as well as their respective logos are trademarks or registered trademarks of QlikTech International AB. All other company names, products
and services used herein are trademarks or registered trademarks of their respective owners. The information published herein is subject to change without notice. This publication
is for informational purposes only, without representation or warranty of any kind, and QlikTech shall not be liable for errors or omissions with respect to this publication. The only
warranties for QlikTech products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should
be construed as constituting any additional warranty.