Legacy Pipeline Atlas worker-1snapshot 2026-07-15

MDClarity · task primitive

SyncBatch

Overview

SyncBatchTask synchronizes data from batch processing tables to their corresponding production tables. It performs a delete-and-insert operation to replace existing records in production tables with the updated records from batch tables, ensuring that batch-processed data (visits, quotes, custom properties, GFE compliance, and change events) is promoted to production.

Use this task when: You need to commit batch-processed data from staging/batch tables to production tables after completing batch calculations or updates.

Don't use this task when: You need to process individual records incrementally (use direct repository save methods), need to preserve historical production data without replacement, or when batch tables haven't been populated yet.

Parameters

Parameter Type Required Default Description
SyncQuote bool No true Synchronizes Quote records from QuoteBatch to Quote table and syncs auto-benefits status maps
SyncVisit bool No true Synchronizes Visit records from VisitBatch to Visit table and syncs auto-benefits status maps
SyncCustomPropertyMap bool No true Synchronizes custom property mappings from CustomPropertyMapBatch to CustomPropertyMap table
SyncVisitGFECompliance bool No true Synchronizes GFE compliance records from VisitGFEComplianceBatch to VisitGFECompliance (only for GFE-enabled customers)
SyncRemovedCustomPropertyMaps bool No true Removes custom property maps from production that are missing in batch (cleanup operation)
SyncVisitChangeEvent bool No true Synchronizes visit change events from VisitChangeEventBatch to VisitChangeEvent table

Execution Flow

YesNoYesNoYesNoYesYesNoNoYesNoYesNoStart SyncBatchSyncQuote?Sync QuoteBatch to QuoteSync Quote AutoBenefitsStatusSyncVisit?Sync VisitBatch to VisitSync Visit AutoBenefitsStatusSyncCustomPropertyMap?SyncCustomPropertyMapBatchto CustomPropertyMapSyncVisitGFECompliance?Is GFE Enabled?SyncVisitGFEComplianceBatchto VisitGFEComplianceSyncRemovedCustomPropertyMaps?Remove MissingCustomPropertyMapsSyncVisitChangeEvent?SyncVisitChangeEventBatch toVisitChangeEventComplete

Processing Algorithm

The task executes a two-phase synchronization algorithm for each entity type:

Phase 1: Delete Matching Records

Delete all records from the production table where the join key matches a record in the batch table.

Phase 2: Insert Batch Records

Insert all records from the batch table into the production table.

Both operations are wrapped in a database transaction to ensure data consistency.

Example Execution

For Visit synchronization:

  1. Delete all Visit records where Visit.Key matches any VisitBatch.Key
  2. Insert all records from VisitBatch into Visit
  3. Update AutoApplyBenefitsStatusMap for synced visits

If VisitBatch contains 100 visits and Visit contains 500 visits:

Usage Examples

Example 1: Full sync after batch processing

var task = new SyncBatchTask
{
    SyncQuote = true,
    SyncVisit = true,
    SyncCustomPropertyMap = true,
    SyncVisitGFECompliance = true,
    SyncRemovedCustomPropertyMaps = true,
    SyncVisitChangeEvent = true
};
// All batch tables will be synced to production

Example 2: Sync only visits and quotes

var task = new SyncBatchTask
{
    SyncQuote = true,
    SyncVisit = true,
    SyncCustomPropertyMap = false,
    SyncVisitGFECompliance = false,
    SyncRemovedCustomPropertyMaps = false,
    SyncVisitChangeEvent = false
};
// Only Quote and Visit batch tables synced, custom properties remain unchanged

Example 3: Sync custom properties only

var task = new SyncBatchTask
{
    SyncQuote = false,
    SyncVisit = false,
    SyncCustomPropertyMap = true,
    SyncVisitGFECompliance = false,
    SyncRemovedCustomPropertyMaps = true, // Also cleanup removed properties
    SyncVisitChangeEvent = false
};
// Only custom property batch data is synced and cleaned up

Example 4: Part of a batch processing pipeline

// Step 1: Load visits into batch table
var loadTask = new LoadVisitsTask { /* config */ };
loadTask.Execute(dataStore);

// Step 2: Process visits in batch
var processTask = new ProcessVisitsTask { /* config */ };
processTask.Execute(dataStore);

// Step 3: Sync batch results to production
var syncTask = new SyncBatchTask(); // All defaults = true
syncTask.Execute(dataStore);

Related Tasks Comparison

Task Use Case Scope Data Flow Transaction Safety
SyncBatch Promote batch data to production Multiple entity types Batch tables → Production tables Transactional per entity
LoadVisits Load visits into batch table Visits only SQL query → VisitBatch Bulk insert
AutoApplyBundlesToVisitBatch Apply bundles to batch visits Visits in batch VisitBatch → QuoteBatch Batch processing
AutoSendLetters Send letters from batch visits Letters for batch visits VisitBatch → Email/SMS Batch processing

Performance Considerations

Database Impact

Optimization Tips

Scenario Recommendation Rationale
Large batch sizes Sync during off-peak hours DELETE and INSERT operations can lock tables temporarily
Selective sync Disable unused entity syncs Reduces database load and execution time
GFE not used Leave SyncVisitGFECompliance enabled Task checks GFEEnabled flag and skips automatically
After import errors Validate batch data before sync Production data is replaced; errors propagate to production

Bulk Operations

The task uses standard SQL DELETE and INSERT statements without explicit bulk copy operations. For large batch sizes (>10,000 records), consider:

Error Handling

The task uses transactional SQL statements wrapped by SqlGenerator.MakeTransactionFromQuery() to ensure atomicity.

// Each sync operation is wrapped in a transaction
BEGIN TRANSACTION
    DELETE [Entity] FROM [Entity] INNER JOIN [EntityBatch] ON ...
    INSERT INTO [Entity] SELECT * FROM [EntityBatch]
COMMIT TRANSACTION

Important:

Common Pitfalls

Issue Problem Solution
Empty batch tables Syncing with empty batch tables deletes matching production records without replacement Always verify batch tables contain expected data before syncing
Foreign key violations Quote sync fails if VisitKey references non-existent Visit Sync Visits before Quotes, or ensure referenced entities exist
Missing GFE compliance GFE sync skipped silently if customer not GFE-enabled Verify Customer.GFEEnabled flag matches expectations
Partial data loss Only syncing some entities leaves inconsistent state Sync all related entities together (Visit + Quote + CustomPropertyMap)
AutoBenefits status maps Status maps not synced if only custom properties are updated Enable SyncQuote or SyncVisit to sync corresponding status maps
Removed properties not cleaned Custom properties persist in production after batch removal Enable SyncRemovedCustomPropertyMaps to clean up deleted properties

Code References