Migration of workflows containing links to Functions

Nigel Light
Nigel Light Contributor
edited December 2023 in General

Hi

I have created several Functions and included these in a Workflow which has been tested in our DEV environment.

However, after migrating (using Export/Synchronise) both the Workflow and associated Functions to our PRD environment i have had to re-add all the functions into the workflow again (and subsequently reassign any parameters).

Is there a way of automating this process?

Thanks

Best Answers

  • Josh Boxer
    Josh Boxer Administrator
    Answer ✓

    It could do, as could the order in which objects in the destination environment are updated.

    We have an improvement coming that might help make this easier so I will let you know once it is ready for you to try.

  • Josh Boxer
    Josh Boxer Administrator
    Answer ✓

    Not sure what you mean by 'the names'?

    Snapshots are just Datasets that have been created by a Workflow. A Dataset is the schema and settings, including a name, relating to data (the actual data itself is a batch(es) within the Dataset), so if you include Dataset(snapshot) objects within the export then they will exist in the destination environment without needing to be recreated.

Answers

  • Josh Boxer
    Josh Boxer Administrator

    Hi Nigel, had you published the Workflow then made changes, so the export contains the older published version without the latest changes?

    https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/get-started/manage-versions/

    https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/monitor-data-and-troubleshoot/migrate-metadata/

  • Nigel Light
    Nigel Light Contributor

    Hi @Josh Boxer

    I initially migrated the Workflow (from DEV or PRD) and then subsequently migrated the Functions (also from DEV to PRD). I had to add the Functions (to the Transform step where they were used). My assumption being that it was my fault for not migrating them together.
    However, having 'tweaked' the PRD version of the Workflow (adding a Transform to suppress some of the output fields), I did a reverse migration (from PRD back to DEV again) to keep both versions in-line. The Transform containing the Functions was unchanged but I found that I still had to re-add the Functions in the DEV environment - hence the question.

  • Nigel Light
    Nigel Light Contributor

    @Josh Boxer One thing that might affect this is that we create our Functions in a separate shared space. Workflows are created in a separate space where access is more restricted. Would this impact things?

  • Nigel Light
    Nigel Light Contributor

    Thanks @Josh Boxer
    One further question… will this improvement also allow the names of any snapshots to be retained in the migration too?
    (I am finding they currently have to be re-entered following the migration)

  • Nigel Light
    Nigel Light Contributor

    Thanks @Josh Boxer got it working now

Categories