Legacy Pipeline Atlas worker-1snapshot 2026-07-15

sub-job

OneGI · Flow-ECW-Source-LoadData

called by Flow-ECW-Processing

sub-job OneGI/Flow-ECW-Source-LoadData · db OneGI_ProdMirror · updated 2026-07-14

task tree

What the sub-job runs

Tasks in order, each with what it does — typed stepsfrom the MDClarity task docs (click to open the reference), sub-jobsyou can expand inline (▶) to reveal their own tasks, and SQL steps summarized by Gemini (✨). Every step shows the tables it writes/reads and which earlier step it feeds from(sqlglot lineage).

sub-job

Flow-ECW-Source-LoadData

16 tasks · OneGI
sub-job / Flow action SQL / native step▶ expand a sub-job inline · ↗ open it as its own page · a typed step opens its doc
  1. ▶Flow-Source-Load-ECW-AlertDatasub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWAlertDataSourceECWAlertData
      file path={FilePath}archive directory=D:\ClientData\OneGI\ECW\RevFind\Archivebase table name=ECWAlertDatarow terminator=Newline
  2. ▶Flow-Source-Load-ECW-ClinicalAlertssub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWClinicalAlertsSourceECWClinicalAlerts
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWClinicalAlertsrow terminator=Newline
  3. ▶Flow-Source-Load-ECW-CurrentOfficeVisitNotesub-job · 3↗
    1. ProcessDataFileProcessDataFiledisabled
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWCurrentOfficeVisitNoteSourceECWCurrentOfficeVisitNote
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWCurrentOfficeVisitNoterow terminator=NewlineDISABLED
    2. LoadDataFileLoadDataFile
      Imports delimited data files into database tables using predefined specifications for validation, loading, and optional file archiving.
      writes ImportECWCurrentOfficeVisitNote
      data file spec=ImportECWCurrentOfficeVisitNotefile path={FilePath}import mode=Replacearchive file path={SourceDataArchiveECW}
    3. ExecuteMergeSqlExecuteMergeSql
      Synchronizes data from external sources into target tables by performing batched insert, update, and delete operations based on keys.
      writes SourceECWCurrentOfficeVisitNote · reads ImportECWCurrentOfficeVisitNote
      ↳ feeds from LoadDataFile
      merge data source type=ThisDatabasetable name=SourceECWCurrentOfficeVisitNotemerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock
  4. ▶Flow-Source-Load-ECW-Encountersub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWEncounterSourceECWEncounter
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWEncounterrow terminator=Newline
  5. ▶Flow-Source-Load-ECW-FlowPatientInsurancesub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWFlowPatientInsuranceSourceECWFlowPatientInsurance
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWFlowPatientInsurancerow terminator=Newline
  6. ▶Flow-Source-Load-ECW-FlowPatientssub-job · 2↗
    1. ExecuteProcessExecuteProcess
      command=powershell.exe
    2. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWFlowPatientsSourceECWFlowPatients
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWFlowPatientsrow terminator=Newline
  7. ▶Flow-Source-Load-ECW-HistoryPresentIllnesssub-job · 3↗
    1. ProcessDataFileProcessDataFiledisabled
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWHistoryPresentIllnessSourceECWHistoryPresentIllness
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWHistoryPresentIllnessrow terminator=NewlineDISABLED
    2. LoadDataFileLoadDataFile
      Imports delimited data files into database tables using predefined specifications for validation, loading, and optional file archiving.
      writes ImportECWHistoryPresentIllness
      data file spec=ImportECWHistoryPresentIllnessfile path={FilePath}import mode=Replacearchive file path={SourceDataArchiveECW}
    3. ExecuteMergeSqlExecuteMergeSql
      Synchronizes data from external sources into target tables by performing batched insert, update, and delete operations based on keys.
      writes SourceECWHistoryPresentIllness · reads ImportECWHistoryPresentIllness
      ↳ feeds from LoadDataFile
      merge data source type=ThisDatabasetable name=SourceECWHistoryPresentIllnessmerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock
  8. ▶Flow-Source-Load-ECW-LastOfficeVisitNotesub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWLastOfficeVisitNoteSourceECWLastOfficeVisitNote
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWLastOfficeVisitNoterow terminator=Newline
  9. ▶Flow-Source-Load-ECW-Orderssub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWOrdersSourceECWOrders
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWOrdersrow terminator=Newline
  10. ▶Flow-Source-Load-ECW-PriorDiagnosissub-job · 4↗
    1. ProcessDataFileProcessDataFiledisabled
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWPriorDiagnosisSourceECWPriorDiagnosis
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWPriorDiagnosisrow terminator=NewlineDISABLED
    2. ExecuteProcessExecuteProcess
    3. LoadDataFileLoadDataFile
      Imports delimited data files into database tables using predefined specifications for validation, loading, and optional file archiving.
      writes ImportECWPriorDiagnosis
      data file spec=ImportECWPriorDiagnosisfile path={FilePath}import mode=Replacearchive file path={SourceDataArchiveECW}
    4. ExecuteMergeSqlExecuteMergeSql
      Synchronizes data from external sources into target tables by performing batched insert, update, and delete operations based on keys.
      writes SourceECWPriorDiagnosis · reads ImportECWPriorDiagnosis
      ↳ feeds from LoadDataFile
      merge data source type=ThisDatabasetable name=SourceECWPriorDiagnosismerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock
  11. ▶Flow-Source-Load-ECW-ProblemListsub-job · 4↗
    1. ProcessDataFileProcessDataFiledisabled
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWProblemListSourceECWProblemList
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWProblemListrow terminator=NewlineDISABLED
    2. ExecuteProcessExecuteProcess
    3. LoadDataFileLoadDataFile
      Imports delimited data files into database tables using predefined specifications for validation, loading, and optional file archiving.
      writes ImportECWProblemList
      data file spec=ImportECWProblemListfile path={FilePath}import mode=Replacearchive file path={SourceDataArchiveECW}
    4. ExecuteMergeSqlExecuteMergeSql
      Synchronizes data from external sources into target tables by performing batched insert, update, and delete operations based on keys.
      writes SourceECWProblemList · reads ImportECWProblemList
      ↳ feeds from LoadDataFile
      merge data source type=ThisDatabasetable name=SourceECWProblemListmerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock
  12. ▶Flow-Source-Load-ECW-Referralsub-job · 4↗
    1. ProcessDataFileProcessDataFiledisabled
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWReferralSourceECWReferral
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWReferralrow terminator=NewlineDISABLED
    2. ExecuteProcessExecuteProcess
    3. LoadDataFileLoadDataFile
      Imports delimited data files into database tables using predefined specifications for validation, loading, and optional file archiving.
      writes ImportECWReferral
      data file spec=ImportECWReferralfile path={FilePath}import mode=Replacearchive file path={SourceDataArchiveECW}
    4. ExecuteMergeSqlExecuteMergeSql
      Synchronizes data from external sources into target tables by performing batched insert, update, and delete operations based on keys.
      writes SourceECWReferral · reads ImportECWReferral
      ↳ feeds from LoadDataFile
      merge data source type=ThisDatabasetable name=SourceECWReferralmerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock
  13. ▶Flow-Source-Load-ECW-VisitCodessub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWVisitCodesSourceECWVisitCodes
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWVisitCodesrow terminator=Newline
  14. ▶Flow-Source-Load-ECW-VisitStatussub-job · 2↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWVisitStatusSourceECWVisitStatus
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWVisitStatusrow terminator=Newline
    2. Remove NUL - ExecuteSqlsqlflow_source_load_ecw_visitstatus__remove_nul_executesql.sql
      ✨ Cleanses visit status data by removing null characters to ensure data integrity and prevent processing errors.
      · reads SourceECWVisitStatus
      ↳ feeds from ProcessDataFile
  15. ▶Flow-Source-Load-ECW-AscProcEncsub-job · 1↗
    1. ProcessDataFileProcessDataFile
      Import delimited files by automatically detecting their structure and merging the data into source tables.
      writes ImportECWAscProcEncSourceECWAscProcEnc
      file path={FilePath}archive directory={SourceDataArchiveECW}base table name=ECWAscProcEncrow terminator=Newline
  16. SourceECWColonoscopyType - ExecuteMergeSqlMergeSql
    writes CombinedDiagnosisCodesConsolidatedNotesEncounterFlagsPatientProblemsSourceECWColonoscopyType · reads SourceECWCurrentOfficeVisitNoteSourceECWEncounterSourceECWHistoryPresentIllnessSourceECWLastOfficeVisitNoteSourceECWProblemListSourceECWReferral
    merge data source type=ThisDatabasetable name=SourceECWColonoscopyTypemerge mode=InsertUpdatebulk copy timeout seconds=300bulk copy use table lock