Legacy Pipeline Atlas worker-1snapshot 2026-07-15

sub-job

HeatonEye · Daily-Visit-Processing-Flow

End to end daily visit processing job

called by Daily-Processing

sub-job HeatonEye/Daily-Visit-Processing-Flow · db HeatonEye_ProdMirror · updated 2025-09-17

task tree

What the sub-job runs

Tasks in order, each with what it does — typed stepsfrom the MDClarity task docs (click to open the reference), sub-jobsyou can expand inline (▶) to reveal their own tasks, and SQL steps summarized by Gemini (✨). Every step shows the tables it writes/reads and which earlier step it feeds from(sqlglot lineage).

sub-job

Daily-Visit-Processing-Flow

5 tasks · HeatonEye
sub-job / Flow action SQL / native step▶ expand a sub-job inline · ↗ open it as its own page · a typed step opens its doc
  1. ▶Load-NextGen-Files-Flowsub-job · 2↗Job Description: Downloads and imports NextGen data -Flow
    1. DownloadDownload
      Downloads files from remote FTP or SFTP servers to local directories with support for archiving and parallel transfers.
      server=HeatonEye_ProdMirror-mdcssh-sftpremote directory=/Flow/local directory={SourceDataFolderFlow}remote archive directory=/Flow/Archive/
    2. ▶Load-ExportAppointments-Flowsub-job · 1↗Loads appointment data
      1. Load-ExportAppointmentLoad-ExportAppointment
        writes ImportAppointmentSourceAppointment
        file path={FilePath}archive directory={SourceDataArchiveFlow}base table name=Appointmentrow terminator=Newlinecolumn separator=Pipe
  2. ▶Load-Visit-Transform-Tables-Flowsub-job · 5↗Loads transform tables for Visits
    1. Load TempVisitDeltasqlload_visit_transform_tables_flow__load_tempvisitdelta.sql
      ✨ Identifies surgical visits and insurance updates that require processing by comparing recent source changes against previously transformed records.
    2. ▶Load-SourceAppointmentChain-Flowsub-job · 1↗
      1. Load SourceAppointmentChainsqlload_sourceappointmentchain_flow__load_sourceappointmentchain.sql
        ✨ Builds a history of rescheduled appointments to track the progression and relationship between original and updated visit records.
        writes SourceAppointmentChain · reads SourceAppointmentVisitDelta
    3. Load TransformVisitsqlload_visit_transform_tables_flow__load_transformvisit.sqldisabled
      ✨ Builds the transformed visit table by mapping appointment details, statuses, and service types from source data for processing.
      writes TransformVisit · reads FacilitySourceAppointmentSourceAppointmentChainSourceEventVisitDelta
    4. ExecuteMergeSqlExecuteMergeSql
      Synchronizes data from external sources into target tables by performing batched insert, update, and delete operations based on keys.
      writes TransformVisit · reads FacilitySourceAppointmentSourceAppointmentChainVisitDelta
      merge data source type=ThisDatabasetable name=TransformVisitmerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock
    5. Load TransformVisitInsurancesqlload_visit_transform_tables_flow__load_transformvisitinsurance.sql
      ✨ Loads insurance coverage details for visits, determining coordination of benefits and mapping source payers to standardized insurance categories.
      writes TransformVisitInsurance · reads InsuranceMapSourcePatientSourcePatientInsuranceTransformPatientTransformVisitVisitDelta
      ↳ feeds from ExecuteMergeSql
  3. ▶Load-Dimension-Members-Flowsub-job · 2↗Loads Dimension Members
    1. Load VisitStatussqlload_dimension_members_flow__load_visitstatus.sql
      ✨ Loads visit status members into the dimension table using appointment status values extracted from transformed visit data.
      writes DimensionMember · reads TransformVisit
    2. Load VisitType Dimensionsqlload_dimension_members_flow__load_visittype_dimension.sql
      ✨ Populates the visit type dimension by mapping unique event identifiers and practice-specific descriptions from source event records.
      writes DimensionMember · reads SourceEvent
  4. ▶Load-Visits-Flowsub-job · 20↗
    1. ReprocessChangedConfigurationsReprocessChangedConfigurations
      Identify and queue visits or accounts affected by configuration changes to ensure historical data reflects updated pricing or insurance rules.
      reprocess visits
    2. Visits to Reprocesssqlload_visits_flow__visits_to_reprocess.sql
      ✨ Flags visits from the reprocess queue for inclusion in the current data processing delta.
      writes VisitDelta · reads VisitVisitDeltaVisitReprocessQueueItem
      ↳ feeds from Sync Visits and Quotes
    3. LoadVisitBatchLoadVisitBatch
      Load scheduled visits and associated charges into the processing pipeline using custom SQL queries for pricing estimation and quote generation.
      writes QuoteBatchVisitBatch · reads InsuranceMapPatientTransformVisitTransformVisitInsuranceVisitVisitDelta
      ↳ feeds from Sync Visits and Quotes, Visits to Reprocess
      custom visit queryauto charge querycustom service type querycustom coverage queryadd zero charge
    4. ▶Load-Custom-Properties-Flowsub-job · 3↗Load custom properties from Load-Visits
      1. UpdateVisitNoveltyStatusUpdateVisitNoveltyStatus
        · reads PatientSourcePatientVisitBatch
        entity=Visitcustom property key=NewVisitStatusauto querybatch mode
      2. Surgical TypeUpdateCustomPropertyFromQuery
        · reads SourceAppointmentSourceAppointmentChainVisitBatch
        entity=Visitcustom property key=CustomProperty-[id]custom querybatch mode
      3. Referral RequiredUpdateCustomPropertyFromQuery
        · reads CustomPropertyMapDimensionMemberSourceAppointmentChainTransformVisitInsuranceVisitBatch
        entity=Visitcustom property key=CustomProperty-[id]custom querybatch mode
    5. AutoApplyBundlesToVisitBatchAutoApplyBundlesToVisitBatch
      Applies matching charge bundles to batch visits based on facility and insurance to ensure consistent pricing across service types.
    6. CheckBenefits OLDflowdisabled
      Verifies insurance eligibility and retrieves detailed benefits information for visits by making real-time API calls to insurance providers.
      · reads TempBatchVisitVisitBatch
      custom queryDISABLED
    7. CheckBenefitsflow
      Verifies insurance eligibility and retrieves detailed benefits information for visits by making real-time API calls to insurance providers.
      auto query
    8. Re-Check Benefitsflowdisabled
      Verifies insurance eligibility and retrieves detailed benefits information for visits by making real-time API calls to insurance providers.
      · reads EligibilityStatusMapEligibilityTxnTempBatchVisitVisitBatch
      custom queryDISABLED
    9. AutoApplyBenefitsAutoApplyBenefits
      Populates insurance benefits like copays and deductibles on batch visits using data from eligibility verification transactions.
    10. CalculateVisitsflow
      Compute the good-faith estimate (CalculateVisits).
      · reads DimensionMemberVisitBatch
      ↳ feeds from LoadVisitBatch
      custom querycalculate chargesbatch mode
    11. AutoSendLettersAutoSendLetters
      Generates and sends patient cost estimate letters for visits in the batch table based on configured qualification and communication rules.
    12. LogVisitChangeEventsBatchLogVisitChangeEventsBatch
      Compare batch visits against production records to detect and log specific property changes for auditing and compliance tracking.
    13. Load TransformVisitInsuranceCoverageStatussqlload_visits_flow__load_transformvisitinsurancecoveragestatus.sqldisabled
      ✨ Updates insurance coverage status for visits by analyzing the latest eligibility transactions to determine if coverage is active or failed.
      writes TempEligibilityStatsTransformVisitInsuranceCoverageStatus · reads TempEligibilityStatsTransformVisitInsuranceVisitBatch
    14. ▶Load-CustomProperty-CoverageStatus-Flowsub-job · 3↗
      1. Load Primary Coverage StatusUpdateCustomPropertyFromQuery
        · reads CustomPropertyMapEligibilityTxnVisitBatchVisitCoverageBatch
        entity=Visitcustom property key=PrimaryCoverageStatuscustom querybatch mode
      2. Load Secondary Coverage StatusUpdateCustomPropertyFromQuery
        · reads CustomPropertyMapEligibilityTxnVisitBatchVisitCoverageBatch
        entity=Visitcustom property key=SecondaryCoverageStatuscustom querybatch mode
      3. Load Tertiary Coverage StatusUpdateCustomPropertyFromQuery
        · reads CustomPropertyMapEligibilityTxnVisitBatchVisitCoverageBatch
        entity=Visitcustom property key=TertiaryCoverageStatuscustom querybatch mode
    15. Sync Visits and QuotesSyncBatch
      Synchronizes processed data from batch staging tables to production tables, updating visits, quotes, and custom property mappings.
      writes CustomPropertyMapQuoteVisitVisitChangeEvent · reads CustomPropertyMapBatchQuoteBatchVisitBatchVisitChangeEventBatch
      ↳ feeds from LoadVisitBatch
      sync quotesync visitsync custom property mapsync visit change event
    16. ExportBatchVisitEligibilityflowdisabled
      Push the batch to the app / export results.
      · reads EligibilityTxnVisitBatch
      custom querytable name=BatchEligibility_Resultsarchive table name=BatchEligibility_ResultsArchiveDISABLED
    17. SearchForInsuranceflowdisabled
      Find coverage for visits missing insurance.
      · reads EligibilityTxnBatch
      custom querytable name=InsuranceSearch_Resultbatch modeDISABLED
    18. Generate-Reportsub-jobdisabled
    19. ▶Load-Visit-Fact-Flowsub-job · 1↗
      1. Load VisitFactsqlload_visit_fact_flow__load_visitfact.sql
        ✨ Updates the visit fact table by refreshing records that have recent activity in payments, eligibility checks, or patient communications.
        writes VisitFact · reads EligibilityStatusMapEligibilityTxnPatientLetterPaymentTxnVisit
    20. Update-Dashboard-Tablessub-jobdisabled
  5. Data-Integrity-Reportsub-job