Automation of datsource as a dataset
We have created a datasource that will update on periodic basis. To set this up to a workflow, l took the external data and set this up as a dataset. I have successful automated the dataset when updated to kick off the workflow.
However, when we loaded a new set of data in in the datasource, this did not update the dataset.
I then went into the dataset, clicked the refresh botton, this updated the dataset and automated workflow.
- Not sure if process is to create a dataset off a datasource, couldnt see how l attach a datasource to a workflow otherwise. Have l done this process right?
- if l have, how do l automate the dataset getting an automatic update from datasource. I have other files, that get automatically updated to dropzone and this updates dataset and triggers workflow. Want to be able to do the same thing but with a datasource.
Appreciate any advise on this issue
Hey! You will be signed out in 60 seconds due to inactivity. Click here to continue using the site.
Hi @Carolyn ; If I understand your intention correctly, you're looking to fully automate the process from data loading to workflow execution and results generation? You've defined a JDBC source applied credentials, hooked this source up to the workflow and now you're executing the workflow either explicitly or by way of a schedule or the workflow execution API? Is that correct?
If this is the case, there is an update that is scheduled to be released this month which will offer more in depth options for data refresh from JDBC connections. This is likely going to address the challenge you've identified with needing to explicitly update the cached data from a JDBC source.
Thank you for your reply. This is correct. we are on version 2.0.12 and the automation of workflow is done via notifications.
Just so l understand, at the moment, there is no way to have the dataset update when the datasource sends new data to the datasource without manually refreshing the dataset?
Will this be in an upgrade when its released this month?
@Carolyn you'll be able to do an upgrade in place and will need to switch a flag on against the datasource. Our expectation is that this will be available in v2.1 which is currently being planned for mid July
Thank you Clinton, l have raised this with our support team and l am in progress of finding out what is going to be in the new release.