Saturday, November 21, 2020

Get Error log for child pipelines in an execute pipeline activity in ADF

Scenario: We have a pipeline in Azure Data Factory, where we have a few lookup activities and then sequentially execute another  child pipeline using the execute pipeline activity. see the flow below,

The screenshot below is the parent pipeline, where you have two lookups and a sequential execute pipeline step which calls another child pipeline.




See below the child pipeline where you have lookup and a ForEach1 loop,



The problem here is ,when we execute the parent pipeline and something fails in say the ForEach loop in the child pipeline, ADF doesn't call out the actual error from child pipeline in the parent pipeline and only gives out the following error messages, which is very generic.







Now if you see the error above it is very generic and says something failed in the For Each loop in the child pipeline. 

Solution: The solution is simple, just to go the execute pipeline error step, click on output as shown below.



When you click on output, the below popup box appears, click on  pipelineRunid which takes you to the child pipelin.e




Once inside the child pipeline, click anywhere in the workspace and go to output tab, you should be able to see the failure logs as below.




Hope this helped you.