# I have a GCP bucket with hundreds of csv files in the format \[filename\]-YYYYMMDD-\[suffix\].csv I want to fetch just the last 3 days' files each day to merge into BigQuery

**URL:** <https://matillioncommunity.discourse.group/t/i-have-a-gcp-bucket-with-hundreds-of-csv-files-in-the-format-filename-yyyymmdd-suffix-csv-i-want-to-fetch-just-the-last-3-days-files-each-day-to-merge-into-bigquery/2232>\
**Category:** Matillion ETL\
**Created:** [June 5, 2023, 7:35pm UTC](https://matillioncommunity.discourse.group/t/i-have-a-gcp-bucket-with-hundreds-of-csv-files-in-the-format-filename-yyyymmdd-suffix-csv-i-want-to-fetch-just-the-last-3-days-files-each-day-to-merge-into-bigquery/2232 "2023-06-05T19:35:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![PN702555](https://avatars.discourse-cdn.com/v4/letter/p/f07891/32.png) [@PN702555](https://matillioncommunity.discourse.group/u/PN702555)\
**Post date:** [June 5, 2023, 7:35pm UTC](https://matillioncommunity.discourse.group/t/i-have-a-gcp-bucket-with-hundreds-of-csv-files-in-the-format-filename-yyyymmdd-suffix-csv-i-want-to-fetch-just-the-last-3-days-files-each-day-to-merge-into-bigquery/2232/1 "2023-06-05T19:35:05Z")

</div>

Wildcards and variables in the URI for the Google Cloud Loader don't seem to appear valid. How would you approach this?

&nbsp;

We are running Matillion on GCP, target is BigQuery.

---

<div class="post-metadata">

**Author:** ![AnudeepK](https://avatars.discourse-cdn.com/v4/letter/a/e36b37/32.png) [@AnudeepK](https://matillioncommunity.discourse.group/u/AnudeepK)\
**Post date:** [June 6, 2023, 7:27am UTC](https://matillioncommunity.discourse.group/t/i-have-a-gcp-bucket-with-hundreds-of-csv-files-in-the-format-filename-yyyymmdd-suffix-csv-i-want-to-fetch-just-the-last-3-days-files-each-day-to-merge-into-bigquery/2232/2 "2023-06-06T07:27:36Z")

</div>

Not sure about GCP, but we tried a similar thing on AWS. Unfortunately, File Iterator doesnt have enough filter options to limit the list of files from a directory other than pattern. You can try using a python script and filter the list of files you require (based on your Last Modified date) and then submit to your File Iterator to fetch them.

---

<div class="post-metadata">

**Author:** ![PN702555](https://avatars.discourse-cdn.com/v4/letter/p/f07891/32.png) [@PN702555](https://matillioncommunity.discourse.group/u/PN702555)\
**Post date:** [June 6, 2023, 12:38pm UTC](https://matillioncommunity.discourse.group/t/i-have-a-gcp-bucket-with-hundreds-of-csv-files-in-the-format-filename-yyyymmdd-suffix-csv-i-want-to-fetch-just-the-last-3-days-files-each-day-to-merge-into-bigquery/2232/3 "2023-06-06T12:38:58Z")

</div>

That's disappointing. The pattern doesn't seem to be recognized anyway, and I don't have the python knowledge to write such a script.