Resident Load, Preceding Load and Incremental Load in QlikView

Last updated on Nov 01 2021
Harshal Shah

Table of Contents

Resident Load, Preceding Load and Incremental Load in QlikView

QlikView can load data from tables already existing in its RAM, which is already processed by a script. This requirement arises when you want to create a table deriving data from an already existing table in the same script. Please note that both the new table and the existing table should be in the same script.

Creating the Load Script

Open the script editor (or use Control+E) and mention the following script. Here we create an inline table named Regions with sales data for different regions. Then we create another table named Total to calculate the total sales by Region Names. Finally we drop the table Regions, as in this .qvw file we only need the table named Total for data analysis.

image1 21

Table Box Data

On creating a Table Box Sheet Object, we see the data that is read from the resident data load option.

image2 18

QlikView – Preceding Load

QlikView Preceding load is a load type in which we use a load statement, which takes the columns of another load statement present in the same script. The data read by the first Load statement, which is at the bottom of the script editor window and then used by the load statements above it.

Load Script

The below given screen shot shows the script for data, which is loaded as Inline data and then the max function is applied to one of the columns. The load statement at the bottom makes the data available in QlikView’s memory, which is used by the second load statement above the first load statement. The second load statement applies the max function with group by clause.

image3 15

Table Box Data

On creating a Table Box Sheet Object, we see the data that is read from the Inline data load option.

image4 14

QlikView – Incremental Load

As the volume of data in the data source of a QlikView document increases, the time taken to load the file also increases which slows down the process of analysis. One approach to minimize this time taken to load data is to load only the records that are new in the source or the updated ones. This concept of loading only the new or changed records from the source into the QlikView document is called Incremental Load.

To identify the new records from source, we use either a sequential unique key or a date time stamp for each row. These values of unique key or data time field has to flow from the source file to QlikView document.

Let us consider the following source file containing product details in a retail store. Save this as a .csv file in the local system where it is accessible by QlikView. Over a period of time some more products are added and the description of some product changes.

Product_Id,Product_Line,Product_category,Product_Subcategory

1,Sporting Goods,Outdoor Recreation,Winter Sports & Activities

2,"Food, Beverages & Tobacco",Food Items,Fruits & Vegetables

3,Apparel & Accessories,Clothing,Uniforms

4,Sporting Goods,Athletics,Rugby

5,Health & Beauty,Personal Care

6,Arts & Entertainment,Hobbies & Creative Arts,Musical Instruments

7,Arts & Entertainment,Hobbies & Creative Arts,Orchestra Accessories

8,Arts & Entertainment,Hobbies & Creative Arts,Crafting Materials

9,Hardware,Tool Accessories,Power Tool Batteries

10,Home & Garden,Bathroom Accessories,Bath Caddies

11,"Food, Beverages & Tobacco",Food Items,Frozen Vegetables

12,Home & Garden,Lawn & Garden,Power Equipment

Loading the Data into QlikView

image5 10  

We will load the above CSV file using the script editor (Control+E) by choosing the Table Files option as shown below. Here we also save the data into a QVD file in the local system. Save the QlikView document as a .qvw file.

Verifying the Data Loaded.

We can check the data loaded to QlikView document by creating a sheet object called Table Box. This is available in the Layout menu and New Sheet Objects sub-menu.

image6 8

Creating the Table Layout

On selecting the Table Box sheet object, we get to the next screen, which is used to select the columns and their positions in the table to be created. We choose the following columns and their positions and click Finish.

image7 7  

Viewing the Existing Data

The following chart showing the data as laid out in the previous step appears.

image8 7

Updating the Source Data

Let us add the following three more records to the source data. Here, the Product IDs are the unique numbers, which represent new records.

13,Office Supplies,Presentation Supplies,Display

14,Hardware,Tool Accessories,Jigs

15,Baby & Toddler,Diapering,Baby Wipes

Incremental load script

Now, we write the script to pull only the new records form the source.

// Load the data from the stored qvd.

Stored_Products:

LOAD Product_Id,

     Product_Line,

     Product_category,

     Product_Subcategory

FROM

[E:\Qlikview\data\products.qvd]

(qvd);

//Select the maximum value of Product ID.

Max_Product_ID:

Load max(Product_Id) as MaxId

resident Stored_Products;

//Store the Maximum value of product Id in a variable.

Let MaxId = peek('MaxId',-1);

  drop table Stored_Products;

//Pull the rows that are new.             

NewProducts:

LOAD Product_Id,Product_Line, Product_category,Product_Subcategory

                from [E:\Qlikview\data\product_categories.csv]

                (txt, codepage is 1252, embedded labels, delimiter is ',', msq)

                where Product_Id > $(MaxId);                

//Concatenate the new values with existing qvd.

Concatenate

LOAD Product_Id,Product_Line, Product_category,

     Product_Subcategory

FROM [E:\Qlikview\data\products.qvd](qvd);
//Store the values in qvd.
store NewProducts into [E:\Qlikview\data\products.qvd](qvd);

The above script fetches only the new records, which are loaded and stored into the qvd file. As we see the records with the new Product IDs 13, 14 and 15.

image9 5

So, this brings us to the end of blog. This Tecklearn ‘Resident Load, Preceding Load and Incremental Load in QlikView’ blog helps you with commonly asked questions if you are looking out for a job in QlikView BI. If you wish to learn QlikView and build a career in Business Intelligence domain, then check out our interactive, QlikView Certification Training, that comes with 24*7 support to guide you throughout your learning period. Please find the link for course details:

https://www.tecklearn.com/course/qlik-view-certification-training/

QlikView Certification Training

About the Course

Tecklearn’s QlikView Certification Training will help you become an expert in Data Visualization with QlikView. Learn all the basics and advanced features of QlikView such as data modelling, reports and dashboards, visualization, object formatting, system table etc. Along with this, you will be given hands-on working experience on real-time projects that will help you pass the QlikView certification exam. You also get hands-on experience in QlikView applications.

Why Should you take QlikView Training?

  • The average annual pay for a QlikView Professional is $122,000. -PayScale.com.
  • Deloitte, Cisco, Qualcomm, Sony, AON & other top Fortune 500 companies use QlikView
  • QlikView – A Leader in 2017 Gartner Magic Quadrant for Business Intelligence & Analytics Platforms (For 7th Consecutive Year).

 What you will Learn in this Course?

Introduction to QlikView

  • Concept of Business Intelligence
  • Features and components of QlikView
  • Comparison with other BI tools
  • Architecture
  • Installation & Navigation
  • QVS, QVW and .log files
  • Sheet Objects
  • Dimensions and Expressions
  • Various file types and extensions

Various QlikView Products

  • Overview of the various QlikView products

Introduction to Data and Scripting

  • Structuring the Script
  • Create tabs in the scripts, Debugging of scripts
  • Hands On

Data Model

  • QlikView Data File Types: QVD, QVX
  • System fields
  • Star schema
  • Synthetic Key Tables
  • Data Modelling Considerations
  • Straight and Pivot Table
  • Hands On

Components of Qlik View

  • Sheets and Sheet Objects
  • The List Box
  • The Table Box
  • The Multi Box
  • The Button & Text Object
  • Basic Charts
  • Hands On

Set Analysis

  • Set analysis
  • Working with modifiers, identifiers, operators, example of expressions,
  • Indirect set analysis
  • YTD & MTD

QlikView Functions, Reports and Charts

  • Uses of Expressions
  • Chart Properties: Dimensions
  • Qlikview Functions
  • Adhoc Reports
  • Incremental Load using QVD Files
  • Bookmarks
  • Hands On

Data Analysis

  • What-if Analysis
  • Comparative Analysis
  • Dynamic Reporting or Ad-hoc Reporting
  • Document Analyzer

QlikView Security

  • QlikView security
  • Defining the access controls and levels of access
  • Maintaining the access control database

QlikView Administration

  • QMC (QlikView Management Console)
  • Server Installation & Deployment
  • QlikView Project

Got a question for us? Please mention it in the comments section and we will get back to you.

 

0 responses on "Resident Load, Preceding Load and Incremental Load in QlikView"

Leave a Message

Your email address will not be published. Required fields are marked *