Best practices designing complex Workflows

Josh Boxer

A Workflow is a sequence of steps that defines a process to transform and manipulate your data. When a single Workflow tries to tackle too many actions it can become difficult to read, understand and manage:

If you are designing a Workflow alone this might be fine, but if you are collaborating or planning to have others execute the Workflow in the future, then it’s good practice to keep it manageable by breaking it down into multiple Workflows, each performing part of the overall process. This also allows each part to be tested independently of the main parent. Amending a child Workflow can be done without affecting the parent until it is ready to be published. 

Notes on Re-usable Workflows

• A parent Workflow contains or embeds a re-usable child Workflow within it, because of this re-usable Workflows are sometimes called embedded Workflows
• A parent Workflow can contain many child Workflows
• A re-usable Workflow can contain a re-usable Workflow and so on, many levels deep if required
• The performance of re-usable Workflows is identical to having a single sprawling Workflow
• Workflow parameters in a child Workflow can accept values from the parent Workflow (as long as the parameter in the child has Configurable on runtime enabled)

For a Workflow to be re-usable it should start with a Source step with ‘Can supply source when executed’ enabled (setting isn’t required if the source is fixed or not coming from another Workflow) and end with an Output step: 

Making an existing set of steps into a re-useable Workflow:

1. Open an existing Workflow and decide which steps would make a good sub-process. (This works especially well if the same steps are being used in multiple places in this Workflow or in multiple Workflows)
2. Create a new Workflow (with the name of the sub-process)
3. Select the steps from the existing Workflow, copy & paste them into the new Workflow
4. Add a Source step and select a source in the new Workflow
   1. If the source is static or the same as the parent Workflow then ‘Can supply source when executed’ does not need to be enabled
   2. If the data has changed in the parent Workflow between the Source and chosen steps then ‘Can supply source when executed’ should be enabled
   3. If the data schema (columns) has changed in the parent Workflow between the Source step and the chosen steps then create a View that matches this schema. To do this, in the parent Workflow Show step results for the step preceding the chosen steps, then Save as View. In the new re-usable Workflow select this new View as the source and enable Can supply source when executed
5. Add an Output step to the new Workflow
6. Save the new Workflow and edit the properties enabling ‘Can be used within other Workflows’
7. Delete the steps from the existing Workflow
8. Select the new re-usable Workflow from the step list and connect it to the Workflow
9. (Optional) set any Parameter values being passed to the child Workflow
10. (Optional) give the step a custom name
11. (Optional) give the step a description describing what the new Workflow does, which will appear as a note on the step

Improvements to Re-usable Workflows in Version 2.8

A re-usable Workflow appears as a single Workflow step in a parent Workflow, which is great for simplifying the Workflow for the user but can make it appear as a bit of a black box with hidden processes manipulating the data.

The ability to click Open Workflow has been added which will allow a User to open or drill into the child Workflow

Once in the child Workflow, the breadcrumb trail will indicate where the User is with the name of each Workflow and allow them to return to the parent Workflow.

Note that for security reasons, a re-usable Workflow that is shared from another Space that the current User does not have access to will not show the Open Workflow option.