Use a value in your data as the export file name

Sueann See
Sueann See Experian Super Contributor
edited April 18 in Tips and tricks

If you are looking to use a value in your data as the export file name, you may have looked at the Export step and realized that it does not allow you to directly select a value from your data as a filename component. However, you can actually use a workflow parameter as a filename component to achieve this.

You will have to set up your workfows in such a way to populate the workflow parameter value based on custom event data that can be based on your data (available as of v2.7.4). Here is an example of the set up:

  • Create a custom event with a custom parameter for the export file name.
image.png


  • Create a workflow containing the Export step. You will need to create a workflow parameter for the export file name that is Configurable on runtime, then assign this as a filename component at the Export settings. As part of this example, we will also set a default value for the export file name as MyExportFile to demonstrate that this will eventually be replaced by a value from your data.
image.png


image.png


  • Create another workflow that will act as the Main workflow to fire the custom event in order to run the Export workflow. In the Main workflow, you can determine how you will be deriving the export file name from your data. In reality, you will also be doing all the pre-processing of your data in the Main workflow, for example any standardization, validation, duplicate detection. Assuming for this example, we just want to use the Source file name as the export file name, you can turn on the Source Metadata: Filename, use a Transform step with a Split function to remove the .csv extension from the filename, then use the transformed value as the Export File Name parameter value. Note that the purpose of removing the .csv extension is because the Export step will later on append the file extension again based on the type of exporter you use. Note also that the custom event parameter will only use the value from the first row of your data for the selected Metadata: Filename column.
image.png


  • Create a notification based on the custom event. The notification should run the workflow containing the export step, supplying the workflow parameter value using the event data.
image.png
image.png


  • Now, when you run the Main workflow, it should fire the Export event, and trigger the notification to run the Export workflow. The resulting output file name is Customer V1.csv based on the source filename rather than the default parameter value of MyExportFile.
image.png


image.png


You may have observed that we introduced the ability to use your data as parameters for the Fire Event step and not directly for the Export step. The reason is to allow for more possible use cases. Another useful example worth mentioning is where you could generate some statistics on your data in your workflow, use them as custom event parameter values and finally incorporate them into an email notification.

Tagged:

Comments

  • George_Stephen
    George_Stephen Member

    This is so complex to get the filename. You need an additional workflow, an Event and Notification for each workflow. Is there no other way to override the parameter value from within the workflow? It is often requested that the output export file contain the input file name (may be with a suffix).

    Has anybody found a better way?

  • Henry Simms
    Henry Simms Administrator

    Hi @George_Stephen

    I don’t think there’s a different way to do this, and I do agree that it can feel a bit clunky. Ideally it would be possible to set and then use the parameter in the same workflow rather than needing the Fire Event to pass the parameter to a second workflow which carries out the export.

    What you should hopefully be able to is to make your solution simpler by re-using the same Custom Event), same Notification/Automation, and the same Export Workflow across all your processing workflows. To explain:

    1. You have multiple workflows processing data from different sources.
    2. Say that for each workflow, you want to export a CSV file to the export directory (for example), with the export filename based on the input filename (using the Filename metadata from the Source step)
    3. What you could do is use the same Event to receive the FILENAME parameter from each of your processing workflows…
    4. And create a single Notification (known as an “Automation” from 2.8.10 onwards) to use the Event data to pass to a single 'export' Workflow…
    5. The 'export' workflow takes the FILENAME param and uses it in the export filepath eg “<Parameter:source file name>_output.csv”

    If you have several different file types to use for export, or different systems to export to, you would need to use more than one Event and Notification/Automation, but this approach should still avoid the need for a separate Event & Notification/Automation pair for every processing workflow.

  • George_Stephen
    George_Stephen Member

    Hello @Henry Simms ,

    Thanks for the response.

    Prior to export, we need to go thru a series of work flow steps, which will be different for each workflow. So each export workflow will be unique. So not sure if this solution will work.

    One partial work-around I am thinking of is exporting another file which contains just the input file name. This is possible, but this is not exactly what the customer wants.

    Thanks!

  • Emilia Fuks
    Emilia Fuks Member

    Hello,

    I'm trying to follow this example to set a custom name for an export file based on workflow parameters but I don't seems to have the option to choose a workflow parameter as the file name in the export step as shown on the 2nd photo (Export Settings). I'm on 2.13.0.

    I believe it's now required to use "@Parameter: " in the filename to tap into that.

    this is just FYI for anyone who comes across this post.

Categories