Dynamic DataSet Naming

Manoj Bhosle
Manoj Bhosle Member
edited December 2023 in General

Hi,

We are trying to use Aperture V2 as part of the ETL workflow. We need to pick the data set from a landing zone. The challenge is that the incoming file name follows a pattern i.e. static text followed by Date/Time stamp.

Aperture workflows pick up Dataset with fixed name only. It allows to change the file name while running in manual mode. Does anyone know how to overcome this issue.

Is there a way, I can name a dataset to accept wild characters.

Answers

  • Manoj Bhosle
    Manoj Bhosle Member

    Hi @Clinton Jones - Thanks for quick response.

    We are using API to trigger the workflow from Azure ADF. The files arrive on Azure Storage.

  • Manoj Bhosle
    Manoj Bhosle Member

    Hi @Clinton Jones

    We are using Aperture 2.4.0.8. Azure ADF calls the Aperture Workflow using an API call. We are able to run the workflow and the workflow is able to pick up the file stored on the Azure Blob Storage.

    Our issue then is the Workflow always expects a static file name. we need a way where each time the workflow is executed, it is able to pick up a file name with different date suffix.

    Hope I am able to explain the situation.

    Many Thanks.

  • Manoj Bhosle
    Manoj Bhosle Member

    Azure ADF ( Data factory) calls the API and triggers the Aperture workflow. ADF has inbuilt capacity to call any APIs. This part is working correctly.

    We have a fixed source name. we have not tried with prompt option.

    I will go through this document reference - Data Quality user documentation | Auto-refresh Sources (experianaperture.io) and see if that resolves our problem.

    Many Thanks.

Categories