Wednesday, July 13, 2022

Azure Data Factory - Copy files which are not empty and last modified is today to ADLS container

 Azure Data Factory - Copy files which are not empty and last modified is today to ADLS container


This approach helps us to get only files which are not empty


Expression is @greaterOrEquals(activity('Lookup1').output.count,2)

This checks file along with the header, Sometimes files can also contain header but it could be an empty file with no further rows. The above expressions evaluates row count from 2 or more and that will be treated as non empty file.


For True conditions, 

Take a copy Activity,

Source is same dataset given to lookup activity

FileName as parameter, should have @item().name as value

Sink as dataset which points a folder 

and copy behavior as Preserve hierarchy

For False conditions,

Just create a Wait activity




0 comments:

Post a Comment

Global Certifications: