Batch / file feed · RevFind pipeline
SGP · RevFind-Daily-Processing
Daily RevFind Processing main job. Added 'RevFind-Process-Load-ChargeType' as step 4 as this previously wasn't being run at all - CW [date] Moved 'Revfind-Source-LoadDataFile' to 'Cutover-Revfind-Source-LoadDataFile'. the old job is disabled above it - SF [date]
trigger
How it's scheduled
The Batch / file feed arrives as a scheduled drop, so one overnight pass is enough: it ingests the day's file and works the whole batch to process the visits. A tighter loop would just re-scan the same file.
| Scheduled task | Cadence | Server | State | Coordination |
|---|---|---|---|---|
\MDC\Daily - Processing - SGP | daily at 01:00 | prod-db-1 | enabled | standalone |
shape
The RevFind back-office spine
RevFind pipelines share a back-office shape: ingest → load accounts → reconcile remits → model contracts → surface underpayments. Lit stages are the ones this pipeline runs.
computed concepts · from the SQL
What it computes
Business concepts mined from this pipeline's SQL text — the meaning behind the typed steps. Each links to the concept lens (who else computes it).
Payers in SQLMedicare
task tree
What the pipeline runs
Tasks in order, each with what it does — typed stepsfrom the MDClarity task docs (click to open the reference), sub-jobsyou can expand inline (▶) to reveal their own tasks, and SQL steps summarized by Gemini (✨). Every step shows the tables it writes/reads and which earlier step it feeds from(sqlglot lineage).
RevFind-Daily-Processing
▶RevFind-Source-Import-QuickSightsub-job · 14↗
- Downloads files from remote FTP or SFTP servers to local directories with support for archiving and parallel transfers.server=SGP_ProdMirror-mdcssh-sftpremote directory=/local directory={SourceDataFolder}DISABLED
- S3DownloadS3Downloadcommand=powershell.exe
- Patient Encounter Ins File CleanupExecuteProcesscommand=powershell.exe
- Encounter File CleanupExecuteProcesscommand=powershell.exe
- Load ChargeMasterProcessFilewrites
ImportChargemasterSourceChargemasterfile path={SourceDataFolder}\chargemaster_data000archive directory={SourceDataArchive}base table name=Chargemasterrow terminator=Linefeedcolumn separator=Comma - Load EncountersProcessFilewrites
ImportEncounterSourceEncounterfile path={SourceDataFolder}\encounter_data000archive directory={SourceDataArchive}base table name=Encounterrow terminator=Newlinecolumn separator=Comma - Load PatientEncounterInsuranceProcessFilewrites
ImportPatientEncounterInsuranceSourcePatientEncounterInsurancearchive directory={SourceDataArchive}base table name=PatientEncounterInsurancerow terminator=Newlinecolumn separator=Comma - Load PatientProcessFilewrites
ImportPatientSourcePatientfile path={SourceDataFolder}\patient_data000archive directory={SourceDataArchive}base table name=Patientrow terminator=Linefeedcolumn separator=Comma - Load ProviderProcessFilewrites
ImportProviderSourceProviderfile path={SourceDataFolder}\provider_data000archive directory={SourceDataArchive}base table name=Providerrow terminator=Linefeedcolumn separator=Comma - Load LocationProcessFilewrites
ImportLocationSourceLocationfile path={SourceDataFolder}\location_data000archive directory={SourceDataArchive}base table name=Locationrow terminator=Linefeedcolumn separator=Comma - Load ChargeProcessFilewrites
ImportChargeSourceChargefile path={SourceDataFolder}\charges000archive directory={SourceDataArchive}base table name=Chargerow terminator=Linefeedcolumn separator=Comma - Load TransactionProcessFilewrites
ImportTransactionSourceTransactionfile path={SourceDataFolder}\payments_adj000archive directory={SourceDataArchive}base table name=Transactionrow terminator=Linefeedcolumn separator=Comma - Load InsuranceProcessFilewrites
ImportInsuranceSourceInsurancefile path={SourceDataFolder}\insurance__data000archive directory={SourceDataArchive}base table name=Insurancerow terminator=Linefeedcolumn separator=Comma - Load Patient Encounter Insurance ExceptionsProcessFiledisabledwrites
ImportPatientEncounterInsuranceExceptionsSourcePatientEncounterInsuranceExceptionsarchive directory={SourceDataArchive}base table name=PatientEncounterInsuranceExceptionsrow terminator=Newlinecolumn separator=CommaDISABLED
▶RevFind-Source-LoadDataFilesub-job · 16↗
- S3DownloadS3Downloadcommand=powershell.exe
- Patient Encounter Ins File CleanupExecuteProcesscommand=powershell.exe
- Encounter File CleanupExecuteProcesscommand=powershell.exe
- Patient File CleanupExecuteProcesscommand=powershell.exe
- LoadDataFile-ChargeMasterLoadDataFile-ChargeMasterwrites
ChargemasterFiledata file spec=ChargemasterFilefile path={SourceDataFolder}\chargemaster_data000import mode=Refillarchive file path={SourceDataArchive} - LoadDataFile-EncountersLoadDataFile-Encounterswrites
EncounterFiledata file spec=EncounterFilefile path={SourceDataFolder}\encounter_data000import mode=Refillarchive file path={SourceDataArchive} - LoadDataFile-PatientEncounterInsuranceLoadDataFile-PatientEncounterInsurancedisabledwrites
PatientEncounterInsuranceFiledata file spec=PatientEncounterInsuranceFileimport mode=Refillarchive file path={SourceDataArchive}DISABLED - Load PatientEncounterInsuranceProcessFilewrites
ImportPatientEncounterInsuranceSourcePatientEncounterInsurancearchive directory={SourceDataArchive}base table name=PatientEncounterInsurancerow terminator=Newlinecolumn separator=Comma - LoadDataFile-PatientLoadDataFile-Patientwrites
PatientFiledata file spec=PatientFilefile path={SourceDataFolder}\patient_data000import mode=Refillarchive file path={SourceDataArchive} - LoadDataFile-ProviderLoadDataFile-Providerwrites
ProviderFiledata file spec=ProviderFilefile path={SourceDataFolder}\provider_data000import mode=Refillarchive file path={SourceDataArchive} - LoadDataFile-LocationLoadDataFile-Locationwrites
LocationFiledata file spec=LocationFilefile path={SourceDataFolder}\location_data000import mode=Refillarchive file path={SourceDataArchive} - LoadDataFile-ChargeLoadDataFile-Chargedisabledwrites
ChargeFiledata file spec=ChargeFilefile path={SourceDataFolder}\charges000import mode=Refillarchive file path={SourceDataArchive}DISABLED - Load ChargeProcessFilewrites
ImportChargeSourceChargefile path={SourceDataFolder}\charges000archive directory={SourceDataArchive}base table name=Chargerow terminator=Linefeedcolumn separator=Comma - LoadDataFile-TransactionLoadDataFile-Transactionwrites
TransactionFiledata file spec=TransactionFilefile path={SourceDataFolder}\payments_adj000import mode=Refillarchive file path={SourceDataArchive} - LoadDataFile-InsuranceLoadDataFile-Insurancewrites
InsuranceFiledata file spec=InsuranceFilefile path={SourceDataFolder}\insurance__data000import mode=Refillarchive file path={SourceDataArchive} - LoadDataFile-InsuranceExceptionsLoadDataFile-InsuranceExceptionsdisabledwrites
PatientEncounterInsuranceExceptionsFiledata file spec=PatientEncounterInsuranceExceptionsFileimport mode=Refillarchive file path={SourceDataArchive}DISABLED
▶RevFind-Source-MergeImportToSourcesub-job · 12↗
- Merge ChargeMasterMergeSqlwrites
SourceChargemaster· readsImportChargemastermerge data source type=ThisDatabasetable name=SourceChargemastermerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock - TEMPORARY Fix for Dupe Encounterssql
revfind_source_mergeimporttosource__temporary_fix_for_dupe_encounters.sql✨ Removes duplicate encounter records from the import table to ensure data integrity before merging into source tables.writescte· readsImportEncounter - Merge EncountersMergeSqlwrites
SourceEncounter· readsImportEncountermerge data source type=ThisDatabasetable name=SourceEncountermerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock - Merge PatientEncounterInsuranceMergeSqldisabledwrites
SourcePatientEncounterInsurance· readsImportPatientEncounterInsurancemerge data source type=ThisDatabasetable name=SourcePatientEncounterInsurancemerge mode=InsertUpdateHardDeletebulk copy timeout seconds=300bulk copy use table lock - Merge PatientMergeSqlwrites
SourcePatient· readsImportPatientmerge data source type=ThisDatabasetable name=SourcePatientmerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock - Merge ProviderMergeSqlwrites
SourceProvider· readsImportProvidermerge data source type=ThisDatabasetable name=SourceProvidermerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock - Merge LocationMergeSqlwrites
SourceLocation· readsImportLocationmerge data source type=ThisDatabasetable name=SourceLocationmerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock - Merge ChargeMergeSqlwrites
SourceCharge· readsImportChargemerge data source type=ThisDatabasetable name=SourceChargemerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock - TEMPORARY Fix for dupe Txnssql
revfind_source_mergeimporttosource__temporary_fix_for_dupe_txns.sql✨ Removes duplicate transaction records from the import table based on matching IDs, amounts, and dates.writesCTE· readsImportTransaction - Merge TransactionMergeSqlwrites
SourceTransaction· readsImportTransactionmerge data source type=ThisDatabasetable name=SourceTransactionmerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock - Merge InsuranceMergeSqlwrites
SourceInsurance· readsImportInsurancemerge data source type=ThisDatabasetable name=SourceInsurancemerge mode=InsertUpdateSoftDeletebulk copy timeout seconds=300bulk copy use table lock - Check Deleted Flag on Transactionssql
revfind_source_mergeimporttosource__check_deleted_flag_on_transactions.sql✨ Resets the deletion flag for transactions that have been updated or re-imported after being previously marked as deleted.· readsSourceTransaction↳ feeds from Merge Transaction
▶RevFind-Process-Load-ChargeTypesub-job · 5↗
- Load ChargeTypesql
revfind_process_load_chargetype__load_chargetype.sql✨ Builds the charge type table by categorizing procedure codes into professional or facility bill types based on encounter data.writesChargeType· readsProcedureCodeSourceChargeSourceEncounter - Load ChargeType Dimensionsql
revfind_process_load_chargetype__load_chargetype_dimension.sql✨ Loads charge type members into the dimension table to support hierarchical reporting and data categorization.writesDimensionMember· readsChargeType↳ feeds from Load ChargeType - Load ProcedureCode Dimensionsql
revfind_process_load_chargetype__load_procedurecode_dimension.sql✨ Loads procedure code dimension members by mapping charge types to their respective descriptions and CPT categories.writesDimensionMember· readsChargeTypeProcedureCode↳ feeds from Load ChargeType - Load ProcedureCategory Dimensionsql
revfind_process_load_chargetype__load_procedurecategory_dimension.sql✨ Builds the procedure category dimension by extracting parent categories from existing procedure code records.writesDimensionMember· readsDimensionMember↳ feeds from Load ChargeType Dimension, Load ProcedureCode Dimension - Load Modifiers Dimensionsql
revfind_process_load_chargetype__load_modifiers_dimension.sqldisabled✨ Loads unique billing modifiers from transformed charge records into the dimension member table.writesDimensionMember· readsTransformCharge
▶RevFind-Transform-QuickSight-Account-Transformsub-job · 14↗
- Create AccountKeyMapsql
revfind_transform_quicksight_account_transform__create_accountkeymap.sql✨ Builds a mapping table to link source account identifiers with internal unique keys for consistent cross-system tracking.writesAccountKeyMap· readsAccountKeyMapSourceEncounter - Create AccountDeltasql
revfind_transform_quicksight_account_transform__create_accountdelta.sql✨ Identifies and loads account IDs that have changed or been deleted since the last pipeline execution into a delta table.writesAccountDelta· readsAccountKeyMapProcessLogSourceChargeSourceEncounterSourceTransaction↳ feeds from Create AccountKeyMap - Load AccountDeltaLogsql
revfind_transform_quicksight_account_transform__load_accountdeltalog.sql✨ Records the total number of changed accounts processed in the current batch for auditing and monitoring purposes.writesAccountDeltaLog· readsAccountDelta↳ feeds from Create AccountDelta - Load TransformPatientsql
revfind_transform_quicksight_account_transform__load_transformpatient.sql✨ Builds and updates the patient transformation table by merging demographic data and generating unique identifiers from source systems.writesTransformPatient· readsSourcePatient - Load TransformLocationsql
revfind_transform_quicksight_account_transform__load_transformlocation.sql✨ Builds and updates the location transformation table by mapping facility addresses and zip codes to standardized state codes.writesTransformLocation· readsCMSZipMapSourceLocation - Load TransformProvidersql
revfind_transform_quicksight_account_transform__load_transformprovider.sql✨ Builds and updates the provider transformation table by parsing names and mapping National Provider Identifiers from source systems.writesTransformProvider· readsSourceProvider - Load TransformPayersql
revfind_transform_quicksight_account_transform__load_transformpayer.sql✨ Builds and updates the payer transformation table by consolidating insurance carrier names and generating unique payer identifiers.writesTransformPayer· readsSourceInsurance - Load TransformChargesql
revfind_transform_quicksight_account_transform__load_transformcharge.sql✨ Loads charge-level details into the transformation table, calculating service units, amounts, and bill types for updated accounts.writesTransformCharge· readsAccountDeltaAccountKeyMapSourceChargeSourceEncounterSourceLocation↳ feeds from Create AccountDelta, Create AccountKeyMap - Load TransformAccountInsurancesql
revfind_transform_quicksight_account_transform__load_transformaccountinsurance.sql✨ Updates account insurance records by determining the coordination of benefits order and mapping member policy information for active coverage.writesTransformAccountInsurance· readsAccountDeltaAccountKeyMapSourceEncounterSourcePatientEncounterInsuranceSourcePatientEncounterInsuranceExceptions↳ feeds from Create AccountDelta, Create AccountKeyMap - Load TransformTransactionsql
revfind_transform_quicksight_account_transform__load_transformtransaction.sql✨ Loads and categorizes financial transactions into payments, adjustments, or denials based on source codes and coordination of benefits.writesTransformTransaction· readsAccountDeltaAccountKeyMapAdjustmentCodeRollupSourceEncounterSourceTransactionTransformAccountInsurance↳ feeds from Create AccountDelta, Create AccountKeyMap, Load TransformAccountInsurance - Load TransformAccountsql
revfind_transform_quicksight_account_transform__load_transformaccount.sql✨ Builds the account transformation table by aggregating charge balances and linking encounters to providers, facilities, and primary insurance.writesTransformAccount· readsAccountDeltaAccountKeyMapSourceEncounterSourceLocationTransformCharge↳ feeds from Create AccountDelta, Create AccountKeyMap, Load TransformCharge - Merge TransformTransactionMergeSqldisabledwrites
TransformTransaction· readsAdjustmentCodeRollupImportTransactionSourceEncounterTransformAccountInsurancemerge data source type=ThisDatabasetable name=TransformTransactionmerge mode=InsertUpdateHardDeletebulk copy timeout seconds=300bulk copy use table lock - Update Deltasql
revfind_transform_quicksight_account_transform__update_delta.sqldisabled✨ Identifies and records account IDs with recent transaction updates to track incremental changes for downstream processing.writesAccountDelta· readsAccountDeltaProcessLogTransformTransaction - Load TransformChargeTransactionMeasuresql
revfind_transform_quicksight_account_transform__load_transformchargetransactionmeasure.sql✨ Calculates and updates aggregated financial metrics, including payments, adjustments, and denials, for accounts identified in the delta tracking table.writesTransformChargeTransactionMeasure· readsAccountDeltaAccountKeyMapAdjustmentCodeRollupSourceChargeTransformChargeTransformTransaction↳ feeds from Create AccountDelta, Create AccountKeyMap, Load TransformCharge, Load TransformTransaction
▶RevFind-Transform-Load-Entitiessub-job · 3↗
- Load Providersql
revfind_transform_load_entities__load_provider.sql✨ Updates and inserts provider records by merging source data with national provider identifiers and taxonomy classifications.writesProvider· readsNPIDataProviderTaxonomySourceProviderTransformCharge - Load FacilityLoadEntitieswrites
Facility· readsFacilitySourceLocationTransformLocationentity type=Facilitycustom querykey prefix=F - Load InsuranceMapLoadEntitieswrites
InsuranceMap· readsInsuranceMapSourceInsuranceentity type=InsuranceMapcustom querykey prefix=IM
▶RevFind-Process-QuickSight-Accountssub-job · 8↗
- LoadPatientsLoadPatientsUpsert patient records into the patient dimension.· reads
PatientTransformPatientauto patient query - ReprocessChangedConfigurationsReprocessChangedConfigurationsIdentify and queue visits or accounts affected by configuration changes to ensure historical data reflects updated pricing or insurance rules.reprocess accounts
- Augment AccountDeltasql
revfind_process_quicksight_accounts__augment_accountdelta.sql✨ Flags accounts requiring reprocessing by adding them to the change tracking table for downstream updates.writesAccountDelta· readsAccountDeltaAccountsToReprocessTransformAccount - Load ChargeMeasuresql
revfind_process_quicksight_accounts__load_chargemeasure.sql✨ Builds and updates financial metrics for charges, including payments, adjustments, and denials, for accounts with recent activity.writesChargeMeasures· readsAccountDeltaTransformChargeTransformChargeTransactionMeasure↳ feeds from Augment AccountDelta - LoadAccountDataLoadAccountDataImports historical billing data from external systems into core account tables while preserving manual user modifications and application fields.· reads
AccountChargeAccountDeltaChargeMeasuresChargeTypeFacilityInsuranceMapPatientSourceBCBSPrefixMap↳ feeds from Augment AccountDelta, Load ChargeMeasurecustom account charge querycustom account querycustom account remit querycustom account insurance query ▶RevFind-Process-Load-Dimension-Memberssub-job · 17↗
- LoadInsuranceDimensionMembersLoadInsuranceDimensionMembersSynchronize insurance-specific dimension members from the insurance table to update contract and eligibility filtering options across the application.writes
DimensionMemberload insuranceload quote insuranceload eligibility insurance - Load Time Dimensionssql
revfind_process_load_dimension_members__load_time_dimensions.sql✨ Loads time-based dimensions including days, months, and years into the dimension member table using encounter service dates.writesDimensionMember· readsSourceEncounter - Load DenialCategory Dimensionsql
revfind_process_load_dimension_members__load_denialcategory_dimension.sql✨ Loads denial category dimensions from adjustment code rollup data to categorize claim denials.writesDimensionMember· readsAdjustmentCodeRollup - Load Provider Dimensionsql
revfind_process_load_dimension_members__load_provider_dimension.sql✨ Loads and updates provider dimension members using provider IDs, names, and types from the source provider table.writesDimensionMember· readsProvider - Load ProviderType Dimensionsql
revfind_process_load_dimension_members__load_providertype_dimension.sql✨ Builds the provider type dimension by extracting unique parent categories from existing provider dimension records.writesDimensionMember· readsDimensionMember↳ feeds from Load AllowedStatus Dimension, Load DenialCategory Dimension, Load Facility Dimensions, Load FacilityGroup Dimensions, Load FacilityType Dimensions, Load Modifier Dimension, Load PlaceOfService Dimensions, Load Provider Dimension, Load Time Dimensions, LoadInsuranceDimensionMembers - Load Facility Dimensionssql
revfind_process_load_dimension_members__load_facility_dimensions.sql✨ Updates the facility dimension with unique facility identifiers, state-level parent groupings, and descriptive names.writesDimensionMember· readsFacility - Load FacilityGroup Dimensionssql
revfind_process_load_dimension_members__load_facilitygroup_dimensions.sql✨ Builds the facility group dimension by extracting unique parent identifiers from existing facility records.writesDimensionMember· readsDimensionMember↳ feeds from Load AllowedStatus Dimension, Load DenialCategory Dimension, Load Facility Dimensions, Load FacilityType Dimensions, Load Modifier Dimension, Load PlaceOfService Dimensions, Load Provider Dimension, Load ProviderType Dimension, Load Time Dimensions, LoadInsuranceDimensionMembers - Load FacilityType Dimensionssql
revfind_process_load_dimension_members__load_facilitytype_dimensions.sql✨ Loads facility type dimension members from facility records to enable standardized categorization across the revenue cycle pipeline.writesDimensionMember· readsFacility - Load Practice Dimensionssql
revfind_process_load_dimension_members__load_practice_dimensions.sqldisabled✨ Loads practice dimension members from the practice map for entities that have associated encounter records.writesDimensionMember· readsPracticeMapsourceencounter - Load PlaceOfService Dimensionssql
revfind_process_load_dimension_members__load_placeofservice_dimensions.sql✨ Builds and updates the place of service dimension by mapping charge data to standardized codes and descriptive names.writesDimensionMember· readsPOSCodesTransformCharge - Load VisitType Dimensionssql
revfind_process_load_dimension_members__load_visittype_dimensions.sqldisabled✨ Loads and synchronizes visit type dimensions from event mapping data while removing entries no longer present in the source.writesDimensionMember· readsSourceEventTransformEventMap - Load AccountStatus Dimensionsql
revfind_process_load_dimension_members__load_accountstatus_dimension.sqldisabled✨ Loads account status dimension members by mapping billing status values from patient visit records.writesDimensionMember· readsSourcePatientVisits - Load AllowedStatus Dimensionsql
revfind_process_load_dimension_members__load_allowedstatus_dimension.sql✨ Builds allowed status dimensions to categorize how actual payments compare to expected insurance contract rates.writesDimensionMember - Load DiagnosisCode Dimensionsql
revfind_process_load_dimension_members__load_diagnosiscode_dimension.sqldisabled✨ Builds the diagnosis code dimension by mapping codes from visit charges to their respective clinical categories. - Load Modifier Dimensionsql
revfind_process_load_dimension_members__load_modifier_dimension.sql✨ Builds the procedure modifier dimension by identifying unique modifiers used in charge records and applying descriptive labels.writesDimensionMember· readsProcedureModifierTransformCharge - Load Eligibility Service Type Dimensionsql
revfind_process_load_dimension_members__load_eligibility_service_type_dimension.sqldisabled✨ Loads insurance eligibility service types by mapping procedure codes from account charges to standardized service type descriptions.writesDimensionMember· readsAccountChargeCPTServiceTypeMap - Load EligibilityServiceTypesql
revfind_process_load_dimension_members__load_eligibilityservicetype.sqldisabled✨ Updates eligibility service type dimensions by cross-referencing procedure codes with chargemaster data and standardized service mappings.writesDimensionMember· readsCPTServiceTypeMapProcedureCodeSourceChargeMasterSourceProceduresTransformCharge
▶RevFind-Process-Score-Accountssub-job · 2↗
- Delta-Based Account ScoringProcessAccountDataFromQuerydisabled· reads
AccountDeltacustom queryDISABLED - Score all qualified accountsBatchProcessAccountswrites
AccountEstimate· readsQuoteBatchVisitBatchmin service date=2024-01-01incremental
▶RevFind-Process-Score-Accounts-Medicaresub-job · 2↗
- Delta-Based Account ScoringProcessAccountDataFromQuerydisabled· reads
AccountDeltacustom queryDISABLED - Score all qualified accountsBatchProcessAccountswrites
AccountEstimate· readsQuoteBatchVisitBatchmin service date=2024-01-01scenario insurance=Medicareincremental
- Load recovery reportsql
revfind_daily_processing__load_recovery_report.sql✨ Builds a temporary report summarizing recovered amounts, appeal statuses, and payment details for patient accounts and charges.writesTempChargeRecoveryReport· readsAccountAccountAppealStatusAccountAppealSummaryAccountChargeAccountEstimateChargeChargeTypeFacilityPatient - ProcessScenarioModelProcessScenarioModelCalculates account estimates for queued scenario models to perform what-if analysis for contract modeling and pricing.