Migration of workflows containing links to Functions

Nigel Light

Hi

I have created several Functions and included these in a Workflow which has been tested in our DEV environment.

However, after migrating (using Export/Synchronise) both the Workflow and associated Functions to our PRD environment i have had to re-add all the functions into the workflow again (and subsequently reassign any parameters).

Is there a way of automating this process?

Thanks

Josh Boxer

It could do, as could the order in which objects in the destination environment are updated.

We have an improvement coming that might help make this easier so I will let you know once it is ready for you to try.

Josh Boxer

Not sure what you mean by 'the names'?

Snapshots are just Datasets that have been created by a Workflow. A Dataset is the schema and settings, including a name, relating to data (the actual data itself is a batch(es) within the Dataset), so if you include Dataset(snapshot) objects within the export then they will exist in the destination environment without needing to be recreated.

Josh Boxer

Hi Nigel, had you published the Workflow then made changes, so the export contains the older published version without the latest changes?

https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/get-started/manage-versions/

https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/monitor-data-and-troubleshoot/migrate-metadata/

Nigel Light

Hi @Josh Boxer

I initially migrated the Workflow (from DEV or PRD) and then subsequently migrated the Functions (also from DEV to PRD). I had to add the Functions (to the Transform step where they were used). My assumption being that it was my fault for not migrating them together.
However, having 'tweaked' the PRD version of the Workflow (adding a Transform to suppress some of the output fields), I did a reverse migration (from PRD back to DEV again) to keep both versions in-line. The Transform containing the Functions was unchanged but I found that I still had to re-add the Functions in the DEV environment - hence the question.

Nigel Light

@Josh Boxer One thing that might affect this is that we create our Functions in a separate shared space. Workflows are created in a separate space where access is more restricted. Would this impact things?

Nigel Light

Thanks @Josh Boxer
One further question… will this improvement also allow the names of any snapshots to be retained in the migration too?
(I am finding they currently have to be re-entered following the migration)

Nigel Light

Thanks @Josh Boxer got it working now