Workflow - Export to Dataset, then Refresh
We have a workflow that is compiling some results from several steps (into a union), and then exporting to a JDBC external system with a defined dataset.
As a last step, we have linked the export to the source of the defined dataset assuming that once it has exported the data, it would refresh the dataset, and we could see the latest data and use it for charts, reports, etc.
However we noticed in a recent run of our workflow, it seems that the refresh of the dataset may be running at the same time as the export. We assumed that the workflow would wait for the export to complete before refreshing the dataset, however it seems it isn't. Are we doing something wrong here?
Best Answer
-
@Ian Hayden , we ended up going with just a notification that uses the system event "Workflow completed" with a filter for the workflow name. This saves us from having to specify a custom event and adding the fire event step at the end of our primary workflow. We have confirmed it works the same either way. Is there a reason to choose the fire event step with custom event, over the way we did it?
2
Answers
-
Hi @George Brown,
I can't access the images you've attached so I can't quite grasp how you've linked the export to the source of the defined dataset. Would you mind explaining how you've done this?
Thanks,
Ian
0 -
Unfortunately URL:https://ibb.co is blocked by Experian, if you can paste or attached the images to a comment please
0 -
If I try to paste, I get an error that says "Permission Problem: You need the Garden.Uploads.Add permission to do that."
I don't see any option for attaching either; just the two buttons for emoji picker or insert media.
How do I successfully paste or attach?
0 -
Hi @George Brown I have now fixed this permissions issue for you. You may need to logout out and in again for the fix to complete.0
-
1
-
Hi @George Brown,
This is actually a really creative way of using the product that we had not anticipated! I think what will happen here is one of two things; either the source step's refresh will not run (as it could be optimised out) or it will run at the same time as the other steps. It sounds like you're encountering the latter.
A better solution (or rather, a solution we have anticipated) would be to add a fire event step (with the fire after workflow option checked) after the export (and instead of the source step), setup a notification on this event that runs another workflow that contains the source step. This ensures that the workflow finishes before the ADS_Duplicate_NK_Check data is refreshed.
I hope this makes sense - please let me know if not or if you have any other questions.
Regards,
Ian
3 -
@Ian Hayden , thanks! I'll work on applying the changes to our workflow and will reach out if I run into any issues or have any questions.
0 -
Agree, very interesting and not something I have seen, but will test myself.
Another approach that might work is a Take snapshot to write the data to a new or existing Dataset.
Rather than linking the step to the export, I would expect to see it forked from the previous map to target step, so that the step is writing to both outputs.
-Josh
1 -
Your way is better actually, and accomplishes the same thing.
The dataset refresh should be reliable now.
Regards,
Ian
0
