Legacy Pipeline Atlas worker-1snapshot 2026-07-15

Batch / file feed · ClarityFlow pipeline

OneGI · Flow-MM-Processing

Overall Processing job for HL7 pipeline

pipeline OneGI/Flow-MM-Processing · db OneGI_ProdMirror · updated 2026-06-04
classification: flow · 10 Flow markers

trigger

How it's scheduled

Daily batch — once at 03:00

The Batch / file feed arrives as a scheduled drop, so one overnight pass is enough: it ingests the day's file and works the whole batch to check eligibility, price the good-faith estimate, and send the patient letter. A tighter loop would just re-scan the same file.

Scheduled taskCadenceServerStateCoordination
\MDC\Daily - Processing - ONEGI-MMdaily at 03:00prod-db-3enabledstandalone

shape

Against the canonical Flow spine

Every Flow pipeline is a variation on one shape. Lit stages are the ones this pipeline runs; the labels match the capability lens.

Ingest
Batch / file feed
→
Eligibility & benefits
EligibilityBenefitsDiscovery
→
Estimate
BundlesEstimate
→
Letters
LettersHL7
→
Work queue & sync
Work queueDashboardsSync

computed concepts · from the SQL

What it computes

Business concepts mined from this pipeline's SQL text — the meaning behind the typed steps. Each links to the concept lens (who else computes it).

Coverage statusCPT / procedure logic

task tree

What the pipeline runs

Tasks in order, each with what it does — typed stepsfrom the MDClarity task docs (click to open the reference), sub-jobsyou can expand inline (▶) to reveal their own tasks, and SQL steps summarized by Gemini (✨). Every step shows the tables it writes/reads and which earlier step it feeds from(sqlglot lineage).

flow

Flow-MM-Processing

8 tasks · OneGI
sub-job / Flow action SQL / native step▶ expand a sub-job inline · ↗ open it as its own page · a typed step opens its doc
  1. ▶Flow-MM-LoadEntitiessub-job · 3↗Load Facility, Provider, Patients
    1. Load FacilityLoadEntities
      writes Facility · reads FacilitySourceLocationMMSourceOrganizationMMSourceUsaAddressMM
      entity type=Facilitycustom querykey prefix=F
    2. Merge ProvidersMergeSql
      writes Provider · reads SourceOrganizationMMSourceProviderMM
      merge data source type=ThisDatabasetable name=Providermerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock
    3. Load PatientsLoadPatientsdisabled
      Upsert patient records into the patient dimension.
      · reads PatientTransformPatient
      custom patient queryDISABLED
  2. ▶Flow-MM-Transformsub-job · 8↗HL7 Transformation Layer
    1. TruncateVisitDeltasqlflow_mm_transform__truncatevisitdelta.sql
      ✨ Clears the visit delta table to prepare for a fresh set of changed records in the current pipeline run.
      writes VisitDelta
    2. TransformPatientTransformPatient
      writes TransformPatient · reads SourceOrganizationMMSourcePatientMM
      merge data source type=ThisDatabasetable name=TransformPatientmerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock
    3. TransformPatientInsurance PatientDeltasqlflow_mm_transform__transformpatientinsurance_patientdelta.sql
      ✨ Identifies new or updated patient insurance records and configuration changes to flag patients requiring data transformation and reprocessing.
      writes FlowPatientDelta · reads FlowLogLastProcessingTimeFlowPatientDeltaInsuranceMapSourceOrganizationMMSourcePatientInsuranceMMSourcePatientMMTransformPatientInsurance
      ↳ feeds from Merge TransformPatientInsurance
    4. Merge TransformPatientInsuranceMergeSql
      writes TransformPatientInsurance · reads FlowPatientDeltaInsuranceMapSourcePatientInsuranceMM
      ↳ feeds from TransformPatientInsurance PatientDelta
      merge data source type=ThisDatabasetable name=TransformPatientInsurancemerge mode=InsertUpdateHardDeletebulk copy timeout seconds=300bulk copy use table lock
    5. TransformPatientInsurance VisitDeltasqlflow_mm_transform__transformpatientinsurance_visitdelta.sql
      ✨ Flags future appointments for reprocessing when associated patient insurance information has been updated or modified in the system.
      writes VisitDelta · reads FlowPatientDeltaTransformVisit
      ↳ feeds from MergeTransformVisit, TransformPatientInsurance PatientDelta
    6. VisitDeltasqlflow_mm_transform__visitdelta.sql
      ✨ Identifies new or updated colonoscopy appointments from source data to queue them for transformation and processing in the pipeline.
    7. MergeTransformVisitsqlflow_mm_transform__mergetransformvisit.sql
      ✨ Updates transformed visit records with current appointment, provider, and diagnosis details for visits flagged for reprocessing.
      writes TransformVisit · reads FacilitySourceAppointmentChainMMSourceColonoscopyTypePerAppointmentMMVisitDelta
      ↳ feeds from TransformPatientInsurance VisitDelta, TruncateVisitDelta
    8. MergeTransformVisitInsurancesqlflow_mm_transform__mergetransformvisitinsurance.sql
      ✨ Maps and loads patient insurance information into the transformed visit insurance table for visits identified in the delta set.
      writes TransformVisitInsurance · reads TransformPatientInsuranceTransformVisitVisitDelta
      ↳ feeds from Merge TransformPatientInsurance, MergeTransformVisit, TransformPatientInsurance VisitDelta, TruncateVisitDelta
  3. LoadPatientsLoadPatientsdisabled
    Upsert patient records into the patient dimension.
    · reads PatientTransformPatient
    custom patient queryDISABLED
  4. ▶Flow-MM-Processing-LoadDimensionMemberssub-job · 16↗
    1. LoadDimensionMembersLoadDimensionMembers
      Synchronize facility, provider, and insurance metadata into the dimension member table to refresh dashboard filters and reporting categories.
      writes DimensionMember
      load insuranceload quote insuranceload eligibility insuranceload facilityload provider
    2. Load Insurance Dimensionsqlflow_mm_processing_loaddimensionmembers__load_insurance_dimension.sqldisabled
      ✨ Populates the insurance dimension with unique carrier keys to enable filtering and analysis by payer.
      writes DimensionMember · reads Insurance
    3. Load EligibilityInsurance Dimensionsqlflow_mm_processing_loaddimensionmembers__load_eligibilityinsurance_dimension.sqldisabled
      ✨ Loads insurance members that support electronic eligibility and benefits verification via API integration.
      writes DimensionMember · reads DimensionMemberInsurance
    4. Load QuoteInsurance Dimensionsqlflow_mm_processing_loaddimensionmembers__load_quoteinsurance_dimension.sqldisabled
      ✨ Builds a dimension of insurance providers associated with active contracts to facilitate accurate patient cost estimates.
      writes DimensionMember · reads ContractMapInsurance
    5. LoadInsuranceDimensionMembersLoadInsuranceDimensionMembersdisabled
      Synchronize insurance-specific dimension members from the insurance table to update contract and eligibility filtering options across the application.
      writes DimensionMember
      load insuranceload quote insuranceload eligibility insuranceDISABLED
    6. Load Time Dimensionssqlflow_mm_processing_loaddimensionmembers__load_time_dimensions.sql
      ✨ Populates time dimension hierarchies for days, months, and years using appointment timestamps to support temporal data analysis.
      writes DimensionMember · reads TransformVisit
    7. Load VisitStatus Dimensionsqlflow_mm_processing_loaddimensionmembers__load_visitstatus_dimension.sql
      ✨ Builds the visit status dimension by extracting unique appointment statuses from transformed visit data.
      writes DimensionMember · reads TransformVisit
    8. Load Provider Dimensionsqlflow_mm_processing_loaddimensionmembers__load_provider_dimension.sqldisabled
      ✨ Synchronizes provider details into the dimension member table to enable provider-based filtering and grouping in reports.
      writes DimensionMember · reads Provider
    9. Load Facility Dimensionssqlflow_mm_processing_loaddimensionmembers__load_facility_dimensions.sqldisabled
      ✨ Builds facility and facility group hierarchies, linking clinics to their parent groups and tax identification numbers for financial reporting.
      writes DimensionMember · reads DimensionMemberFacilitysourceFacilityGroupToTIN
    10. Load VisitType Dimensionssqlflow_mm_processing_loaddimensionmembers__load_visittype_dimensions.sql
      ✨ Builds and updates visit type dimension members using appointment types from transformed visit data.
      writes DimensionMember · reads TransformVisit
    11. Load Diagnosis Code Dimensionsqlflow_mm_processing_loaddimensionmembers__load_diagnosis_code_dimension.sqldisabled
      ✨ Loads and updates the DiagnosisCode dimension by extracting unique primary, secondary, and tertiary diagnosis codes from charge data.
      writes DimensionMember · reads TransformCharge
    12. Load RadioIsotopes Code Dimensionsqlflow_mm_processing_loaddimensionmembers__load_radioisotopes_code_dimension.sqldisabled
      ✨ Flags and updates the RadioIsotopes dimension to indicate the presence of radioisotope codes within the source risk data.
      writes DimensionMember · reads SourceCareRiskRadioIsotopeCodes
    13. Load Modifier Dimensionsqlflow_mm_processing_loaddimensionmembers__load_modifier_dimension.sqldisabled
      ✨ Loads and updates the Modifier dimension by collecting unique modifier codes from multiple charge record fields.
      writes DimensionMember · reads TransformCharge
    14. Load Place of Service Dimensionsqlflow_mm_processing_loaddimensionmembers__load_place_of_service_dimension.sqldisabled
      ✨ Builds and updates the PlaceOfService dimension using unique location codes extracted from charge transaction records.
      writes DimensionMember · reads TransformCharge
    15. Load SourceInsurance Dimensionsqlflow_mm_processing_loaddimensionmembers__load_sourceinsurance_dimension.sqldisabled
      ✨ Loads and updates the SourceInsurance dimension with mapped insurance provider details for specific data sources and active accounts.
      writes DimensionMember · reads AccountInsuranceInsuranceMap
    16. Load State Dimensionssqlflow_mm_processing_loaddimensionmembers__load_state_dimensions.sql
      ✨ Updates state dimensions using facility location data to allow for geographic segmentation of healthcare services and revenue.
      writes DimensionMember · reads Facility
  5. ▶Flow-MM-Processing-LoadVisitssub-job · 21↗
    1. Load TempBatchVisitsqlflow_mm_processing_loadvisits__load_tempbatchvisit.sqldisabled
      ✨ Builds a temporary batch table to identify upcoming visits requiring initial or follow-up insurance eligibility and benefit verification.
      writes TempBatchVisitVisitEligServiceTypesVisitServiceTypes · reads DimensionMemberTempBatchVisitTransformVisitTransformVisitInsuranceTransformVisitInsuranceCoverageStatusVisitEligServiceTypesVisitServiceTypes
    2. Augment Visit Deltasqlflow_mm_processing_loadvisits__augment_visit_delta.sqldisabled
      ✨ Updates the visit delta table by adding records flagged as requiring new insurance eligibility lookups from the batch process.
      writes VisitDelta · reads TempBatchVisitVisitDelta
    3. ReprocessChangedConfigurationsReprocessChangedConfigurations
      Identify and queue visits or accounts affected by configuration changes to ensure historical data reflects updated pricing or insurance rules.
      reprocess visits
    4. Augment VisitDelta - ReprocessChangedConfigssqlflow_mm_processing_loadvisits__augment_visitdelta_reprocesschangedconfigs.sql
      ✨ Builds a visit lookup table to update insurance mappings and flags modified visits for reprocessing in the data pipeline.
      writes VisitDeltaVisitKeyLookup · reads SourceOrganizationMMTransformVisitVisitDeltaVisitKeyLookup
    5. LoadVisitBatchLoadVisitBatch
      Load scheduled visits and associated charges into the processing pipeline using custom SQL queries for pricing estimation and quote generation.
      writes QuoteBatchVisitBatch · reads InsuranceMapPatientTransformVisitTransformVisitInsuranceVisitVisitDelta
      ↳ feeds from Augment VisitDelta - ReprocessChangedConfigs, Sync Visits and Quotes
      custom visit queryauto charge queryauto service type querycustom coverage queryadd zero charge
    6. Flow-HL7-Processing-LoadCustomPropertiessub-jobdisabled
    7. AutoApplyBundlesToVisitBatchAutoApplyBundlesToVisitBatch
      Applies matching charge bundles to batch visits based on facility and insurance to ensure consistent pricing across service types.
    8. CheckBenefitsflow
      Verifies insurance eligibility and retrieves detailed benefits information for visits by making real-time API calls to insurance providers.
      auto query
    9. Re-Check Benefitsflowdisabled
      Verifies insurance eligibility and retrieves detailed benefits information for visits by making real-time API calls to insurance providers.
      · reads EligibilityStatusMapEligibilityTxnTempBatchVisitVisitBatch
      custom queryDISABLED
    10. AutoApplyBenefitsAutoApplyBenefits
      Populates insurance benefits like copays and deductibles on batch visits using data from eligibility verification transactions.
    11. CalculateVisitsflow
      Compute the good-faith estimate (CalculateVisits).
      · reads DimensionMemberVisitBatch
      ↳ feeds from LoadVisitBatch
      custom querycalculate chargesbatch mode
    12. AutoSendLettersAutoSendLettersdisabled
      Generates and sends patient cost estimate letters for visits in the batch table based on configured qualification and communication rules.
      DISABLED
    13. LogVisitChangeEventsBatchLogVisitChangeEventsBatch
      Compare batch visits against production records to detect and log specific property changes for auditing and compliance tracking.
    14. Load TransformVisitInsuranceCoverageStatussqlflow_mm_processing_loadvisits__load_transformvisitinsurancecoveragestatus.sql
      ✨ Loads insurance coverage status by calculating the latest eligibility transaction results for visits in the current batch.
      writes TempEligibilityStatsTransformVisitInsuranceCoverageStatus · reads TempEligibilityStatsTransformVisitInsuranceVisitBatch
      ↳ feeds from LoadVisitBatch
    15. ▶Flow-MM-Processing-CustomPropertysub-job · 4↗
      1. Load Primary Coverage StatusUpdateCustomPropertyFromQuery
        · reads TransformVisitInsuranceCoverageStatusVisitBatch
        entity=Visitcustom property key=PrimaryCoverageStatuscustom querybatch mode
      2. Load Secondary Coverage StatusUpdateCustomPropertyFromQuery
        · reads TransformVisitInsuranceCoverageStatusVisitBatch
        entity=Visitcustom property key=SecondaryCoverageStatuscustom querybatch mode
      3. Load Tertiary Coverage StatusUpdateCustomPropertyFromQuery
        · reads TransformVisitInsuranceCoverageStatusVisitBatch
        entity=Visitcustom property key=TertiaryCoverageStatuscustom querybatch mode
      4. Load Colonoscopy TypeUpdateCustomPropertyFromQuery
        · reads TransformVisitVisitBatch
        entity=Visitcustom property key=ColonoscopyTypecustom querybatch mode
    16. Sync Visits and QuotesSyncBatch
      Synchronizes processed data from batch staging tables to production tables, updating visits, quotes, and custom property mappings.
      writes CustomPropertyMapQuoteVisitVisitChangeEvent · reads CustomPropertyMapBatchQuoteBatchVisitBatchVisitChangeEventBatch
      ↳ feeds from LoadVisitBatch
      sync quotesync visitsync custom property mapsync visit change event
    17. ExportBatchVisitEligibilityflowdisabled
      Push the batch to the app / export results.
      · reads EligibilityTxnVisitBatch
      custom querytable name=BatchEligibility_Resultsarchive table name=BatchEligibility_ResultsArchiveDISABLED
    18. SearchForInsuranceflowdisabled
      Find coverage for visits missing insurance.
      · reads EligibilityTxnBatch
      custom querytable name=InsuranceSearch_Resultbatch modeDISABLED
    19. Generate-Reportsub-jobdisabled
    20. Load-Visit-Factsub-jobdisabled
    21. Update-Dashboard-Tablessub-jobdisabled
  6. Update HL7ProcessLogsqlflow_mm_processing__update_hl7processlog.sql
    ✨ Updates the timestamp for the last successful processing run of the ModMed data flow.
    writes FlowLogLastProcessingTime
  7. WorkqueueAssignmentWorkqueueAssignment
    Assigns accounts or visits to workqueue stacks by evaluating matching criteria and updating item lists based on search terms.
  8. SendHL7Lettersflowdisabled
    Exports pending patient letters as HL7 messages with PDF attachments to a local directory for transmission to external systems.
    output file path=D:\ClientData\RIA\ORUpublish directory=D:\ClientData\RIA\ORUsending application=MDClaritysending facility=MDClarityreceiving application=Abbadox

markers

Why it classified as flow

Front-of-house actions found anywhere in the tree (including nested sub-jobs).

CalculateVisits
Compute the good-faith estimate
CheckBenefits
Real-time eligibility (270/271) benefit lookup
ExportBatchVisitEligibility
Export eligibility results
Load TransformVisitInsuranceCoverageStatus
ClarityFlow front-of-house step
LoadVisitBatch
ClarityFlow front-of-house step
SearchForInsurance
Insurance discovery for visits missing coverage
SendHL7Letters
Emit letters back over the HL7 interface
auto_apply_bundles
Apply procedure bundles for the estimate
auto_benefits
Auto-apply parsed benefits to visits
auto_send_letters
Send GFE / patient letters