Legacy Pipeline Atlas worker-1snapshot 2026-07-15

MDClarity · task primitive

UpdateDashboardTables

Overview

UpdateDashboardTablesTask regenerates pre-aggregated tables for all dashboard elements defined in the system. It iterates through every dashboard report, creates optimized aggregate tables for each element (charts and measures), and updates the dashboard definitions with the new table references.

Use this task when: You need to refresh all dashboard data after source data changes, on a scheduled basis (nightly/hourly), or after adding/modifying dashboard configurations.

Don't use this task when:


Parameters

Parameter Type Required Default Description
None - - - This task has no configurable parameters. It operates on all dashboards in the system.

Execution Flow

NoYesNoYesYesNoYesNoYesNoStartUpdateDashboardTablesCreate DimensionManagerand DashboardRepoGet all DashboardReportentities from databaseAny dashboards exist?End - No dashboards toprocessIterate through eachDashboardReportGet dashboard elementsfrom reportAny elements indashboard?Save dashboard andcontinue to nextIterate through eachDashboardElementGenerate table name withtimestampDrop old GeneratedTable ifexistsBuild pre-aggregation SQLfrom element configExecute SELECT INTO tocreate new tableCreation successful?Updateelement.GeneratedTableand element.TimestampLog error,element.GeneratedTableremains nullMore elements?More dashboards?

Processing Algorithm

The task creates pre-aggregated tables to optimize dashboard performance. For each dashboard element, it:

Table Generation Process

  1. Generate Table Name: Creates a unique table name using the pattern:

    Dashboard_{ReportKey}_{ElementName}_{Timestamp}
    

    All special characters are converted to safe column names.

  2. Build Aggregation SQL: Constructs a SELECT ... INTO query that:

    • Selects all global filters (from DashboardReport.Filters)
    • Selects the time dimension (if specified)
    • Selects element-specific filters/dimensions (from DashboardChart.Dimensions or DashboardMeasure.Filters)
    • Aggregates measures using expressions (e.g., count(*), sum(Amount))
    • Groups by all dimension columns
    • Sources data from the element's DataQuery
  3. Execute and Update: Runs the SQL, creates the table, and updates the element's GeneratedTable property

Example Execution

For a dashboard element "Visit Count by Provider" with:

The generated SQL would be:

SELECT * INTO Dashboard_ProviderReport_VisitCount_2025_01_15T14_30_00
FROM (
    SELECT [Facility], [ScheduledDate], [Provider], count(*) as [VisitCount]
    FROM (SELECT * FROM Visit) as dQ
    GROUP BY [Facility], [ScheduledDate], [Provider]
) as preAgg

This pre-aggregated table is then used for all dashboard queries, dramatically improving performance.


Usage Examples

Example 1: Basic scheduled refresh

// Run nightly to refresh all dashboard data
var task = new UpdateDashboardTablesTask();
// No parameters needed - processes all dashboards automatically

Example 2: Part of a data loading pipeline

// After loading new visit data, refresh dashboards
var loadTask = new LoadDataFileTask
{
    DataFileSpecKey = "DailyVisits",
    FilePath = @"C:\imports\visits.csv",
    ImportMode = ImportMode.Append
};

// Then update all dashboards to reflect new data
var dashboardTask = new UpdateDashboardTablesTask();

Example 3: Manual refresh after configuration changes

// After adding new dashboard elements or modifying existing ones,
// regenerate all pre-aggregated tables
var task = new UpdateDashboardTablesTask();
// This will create tables for new elements and recreate existing ones

Related Tasks Comparison

Task Scope Use Case Performance Impact
UpdateDashboardTables All dashboards in system Scheduled refresh of all dashboard data High (processes every dashboard)
No single-dashboard equivalent - - -
No selective element refresh - - -

Key Characteristics:


Performance Considerations

Database Impact

Optimization Tips

Scenario Recommendation Rationale
Many dashboards Run during off-peak hours (nightly) High resource usage for complex dashboards
Large source tables Ensure indexes exist on dimension columns Speeds up GROUP BY operations
Complex DataQuery Optimize the source query independently Pre-aggregation is only as fast as the base query
Frequent refreshes Consider incremental updates for large datasets Full refresh may be unnecessary if only recent data changed
Stale dashboards Remove unused dashboards to reduce processing time Fewer dashboards = faster execution