so here is what I am trying to do, I have a file with a name of BOB_Minion_20250318_load.txt in an sftp location. tommorrow it will be BOB_Minion_20250319_load.txt. I want to create a pipeline that will run daily and pull the file to a blob storage. The problem is in the SFTP task I have to specify the name, which changes. How do I create a variable for the pipeline, that takes the file name pattern and calculates the proper date portion and pass it to a pipeline task? I would like the variable to be usable by multiple pipelines as this is a pretty normal task in my industry.
1 Like
Hi @tom.ballinger, thanks for the question and a very warm welcome to the community!
You can derive today’s date and construct a dynamic filename using ‘Query result to Scalar’ or python. Initiate a variable with that value and configure a Data Transfer component.
If you try that or if you have tried that and are still not getting the results you want, please share the filename format and we can take a further look.
Many thanks,
Claire