• The Data Mix
  • Posts
  • 📊 Why load it all when 2% is all you need?

📊 Why load it all when 2% is all you need?

In partnership with

Hey, it’s Brian again 👋

Last week’s Qlik Cloud Data Prep edition - Script-Free, Stress-Free - struck a chord with analysts and data leads alike.

We explored how Qlik Cloud Data Flow lets you shape, join, and transform data without writing a single line of script.

If you missed it, definitely go back and give it a read 👆

Because this week? We’re building on that theme of simplifying complexity…

🔎 ODAG (On-Demand App Generation) - Qlik’s enterprise-grade solution for working with massive datasets without overloading memory.

If you’ve ever tried to load everything into one app and watched it crawl (or crash), ODAG is your escape hatch. We’ll walk through how it works, why it matters, and how setup differs between Cloud and On-Prem.

🔴 And don’t miss this week’s double But First - one includes tequila and live podcasting, and the other is a personal request for support on your next Qlik challenge.

Let’s get into it.

In Today’s Issue

I’m going to cover these things with you today!

  1. Cool Tool - Deel - navigating how to build your best global workforce.

  2. But First… Drinks ‘n Data? - Apparently they DO go together and make an awesome podcast.

  3. But First (2) - A request for your support - Have Qlik requirements? Drop me a line.

  4. Cool Tool (2) - 1404 Media - Fact-based news without bias.

  5. Deep Dive - ODAG - When you just don’t need all of the data, all of the time.

  6. Before You Go - Other ways that I can help and work with you

Finally - at the end of my newsletter, you’ll get a poll to tell me how I did and feedback what you’d like to see in the future and do for YOU! Let’s hope it’s not a 🥧 chart!

First time reading? ⏬⏬

⚒️ Cool Tool - Deel

You found global talent. Deel’s here to help you onboard them

Deel’s simplified a whole planet’s worth of information. It’s time you got your hands on our international compliance handbook where you’ll learn about:

  • Attracting global talent

  • Labor laws to consider when hiring

  • Processing international payroll on time

  • Staying compliant with employment & tax laws abroad

With 150+ countries right at your fingertips, growing your team with Deel is easier than ever.

But First…Tequila with Shawn and Kanon!

Recently, I was live in Orlando for Qlik Connect 2025 – not just as a media partner, but as a….tequila taste tester??

🧊 A mic.

🥃 A few bottles of tequila.

🎙️ And some of the realest Qlik Connect chat you’ll hear this year.

I joined Shawn Helwig and Kanon Cozad live at the Broadcast Zone for a special episode of the Drinks 'n Data Podcast at Qlik Connect 2025.

Yes, we covered the serious stuff:

  • The Upsolver + Iceberg moves and what they signal for open data lakes

  • Why trust scoring is becoming mission-critical inside Qlik

  • How Agentic AI is shaking up the platform

  • The wild pace of innovation across Qlik Cloud

But there was also tequila.

And no script.

And a lot of laughter.

It’s hard to stay on track with your analysis when the tequila kicks in but t was so much fun!

⚒️ Cool Tool (2) - 1440 Media

Fact-based news without bias awaits. Make 1440 your choice today.

Overwhelmed by biased news? Cut through the clutter and get straight facts with your daily 1440 digest. From politics to sports, join millions who start their day informed.

But First (2)…I need your help

Quick note before we dive into today’s issue 👇

With a major Qlik Cloud engagement just wrapped, I’ve got some new room in the calendar.

So if you’ve been:
→ Wrestling with reporting chaos
→ Exploring a new Qlik Cloud rollout
→ Wondering how WhatsApp for Qlik could work in your business
→ Or just need a second pair of eyes on your dashboards or automation setup…

Now’s a great time to connect.

Whether it’s advisory, hands-on development, or something experimental – I’d love to hear what you’re working on.

📱 Drop me a quick message on WhatsApp: +44 131 560 6887

Let’s talk Qlik, data, and getting things unstuck.

🔎 Deep Dive: ODAG (On-Demand App Generation) in Qlik Sense - Cloud vs. On-Prem

Struggling to explore massive datasets in Qlik without overwhelming memory limits? That’s where ODAG comes in.

ODAG (On-Demand App Generation) is Qlik’s built-in mechanism for working with large-scale data without compromising performance.

It helps users drill into huge datasets - like multi-terabyte transaction logs or high-volume IoT records - by generating apps on the fly with just the data they need.

This week, we’re breaking down how ODAG works in both Qlik Sense Cloud and Qlik Sense Enterprise (On-Prem).

You’ll learn where it fits best, how to set it up, what’s different between cloud and on-prem, and how to put it to work in your enterprise environment.

🔍 Why This Matters: App Generation at Enterprise Scale

ODAG was built for one specific job: reducing data volume by segmenting analysis workflows.

Instead of trying to load everything into memory, ODAG allows users to select a subset of data (e.g., a region, date range, or customer segment) and generate a new Qlik app on demand that contains just that slice of data.

This keeps memory usage lean, performance high, and the user experience focused.

Common ODAG use cases:

  • High-volume sales or transaction detail reporting

  • Drill-downs from summary dashboards into record-level views

  • Exploring longitudinal data (e.g., patient journeys, customer behavior)

  • Reducing memory usage when working with wide or deep fact tables

  • Supporting data lakes or warehouse offloading scenarios

🛠️ How ODAG Works (At a High Level)

  1. The Selection App: This app presents users with high-level filters and KPIs (e.g., total sales by region and product).

  2. The Navigation Link: Users make selections and trigger an ODAG link, which passes those parameters to a template app.

  3. The Template App: This app contains a load script with bind variables. When ODAG is triggered, Qlik generates a new app using the selections and the script logic.

  4. The Generated App: A brand-new Qlik app is created with just the relevant data subset. Users can explore it freely, share it, or schedule it.

⚖️ Cloud vs. On-Prem: Key Differences

While ODAG core concepts are consistent, the setup and experience diverge significantly between Qlik Sense Cloud and On-Prem deployments.

Feature/Aspect

Qlik Sense Cloud

Qlik Sense Enterprise (On-Prem)

Setup Interface

GUI-driven in the Qlik SaaS Hub

Requires QMC configuration and scripting

Link Creation

Drag-and-drop interface for ODAG navigation

Manual link definition in QMC

Template App Management

Managed directly in cloud hub

Requires app import/export

Data Source Access

Uses Qlik Data Gateway for on-prem sources

Direct access to database or QVD

Security Rules

Managed via Qlik Cloud roles & spaces

Custom security rules in QMC

Reloading Generated Apps

Supports auto-reload via Qlik Automate (formerly Qlik Application Automation)

Requires task chaining or external triggers

Monitoring & Governance

Centralized monitoring in Management Console

Limited logging unless customized

🔧 ODAG Setup: Step-by-Step (Cloud)

  1. Build the Selection App: Load the summary data and build filters.

  2. Create Template App: Define the full data load with binding variables (e.g., SET vRegion = '$(odag_Region)').

  3. Create ODAG Link:

    1. Open the selection app in Qlik Cloud.

    2. Click the "..." menu and select “ODAG links.”

    3. Click "Create new ODAG link," choose your template app, and define field mappings (e.g., odag_Region).

    4. Optionally configure naming, tags, and target space.

    5. Save the link to activate navigation.

  4. Test Navigation: Make a selection and launch the link - a new app should be generated based on your criteria.

  5. Automate (Optional): Use Qlik Automate to reload the app or notify users.

🔧 ODAG Setup: Step-by-Step (On-Prem)

  1. Create Selection App: Includes basic filters and KPIs.

  2. Configure QMC and ODAG Link:

    1. A. In the QMC, go to ODAG settings and define a new navigation link.

    2. B. Choose the selection app and target template app.

    3. C. Map selection fields to bind variables (e.g., odag_DateFrom).

  3. Create Template App: Include variables in the load script (e.g., LET vCountry = '$(odag_Country)').

  4. Publish and Test: Publish both apps to streams, make selections, and verify that the ODAG link correctly generates a filtered child app.

⚠️ Limitations & Considerations

  • ODAG does not support associative exploration across datasets - it creates a standalone app.

  • Template apps must be designed carefully with parameterization in mind.

  • Qlik Cloud simplifies much of the process, but doesn’t yet expose all the advanced controls available on-prem.

  • Consider storage and governance: ODAG can generate a large number of apps over time if not managed.

  • ODAG requires stable bind variables and must avoid script loops and circular logic.

🏬 Enterprise Use Case: Regional Drilldowns in Retail

A global retailer uses ODAG to allow regional managers to drill into transactional data. The selection app contains sales KPIs by region and week. Managers can select their region and generate a detailed app that loads SKU-level sales for just their territory. This avoids loading 100M+ rows across all regions every time.

  • Selection app size: < 200 MB

  • Generated app size: 300 MB to 2 GB depending on region

  • Speed: Apps generated in < 30 seconds with filters applied

💡 Top Tips for ODAG Success (Cloud & On-Prem)

For Qlik Cloud:

  • Use automation to clean up old generated apps to avoid clutter and storage issues.

  • Use ODAG tags and metadata to organize generated apps by user or purpose.

  • Test bind variable logic with static inputs before deploying in production.

For Qlik On-Prem:

  • Build PowerShell scripts or use the QRS API to monitor generated apps and clean up periodically.

  • Consider dedicated streams or app folders to manage ODAG outputs.

  • Implement ODAG testing as part of your deployment QA - especially where reload triggers are external.

General ODAG Advice:

  • Keep selection apps lightweight and visually simple.

  • Limit the number of ODAG fields passed in a single link to reduce script complexity.

  • Always validate selections with preview KPIs before triggering navigation.

🧠 Final Thoughts

ODAG remains one of the most underused yet powerful features in Qlik’s stack.

In both Cloud and On-Prem environments, it empowers business users to explore massive datasets without the technical overhead of full-scale app design or constant IT intervention.

If you're designing enterprise Qlik solutions, especially those involving heavy datasets, ODAG should absolutely be part of your toolbox.

Before you go - here’s 3 ways I can help you

  1. LinkedIn - I’m always talking about Data, Qlik on there and would love to connect with you!

  2. WhatsApp for Qlik - Want to talk to your data in Qlik using WhatsApp? I built that as a service and you can check out the demo.

  3. The Data Mix Podcast on YouTube - Every fortnight, I deliver a cordial interview with the best guests in the world of Data and Analytics.

P.S. If you’re interested in sponsoring the newsletter, please drop me a reply and let me know. Let’s talk!

Reply

or to participate.