Datasets not refreshing automatically

Marco_13112001 · August 2022

Hi,

We built an integrated set of schedules and workflows that allows data from existent datasets to be refreshed on a daily basis. However, the refresh only happens if it is done manually per dataset. It was noticed when the dataset A (OBJALG as per screenshot) was refreshed, the number of records should have increased from 8611 to 8620, but that did not happen, it kept the original number (8611).

The auto refresh option is turned on in both dataset and workflow (see screenshots below)

workflow A -

dataset A -

The process is as follows -

Schedule A runs daily @ 6AM which triggers the workflow A, and the expectation is that the data within dataset A will be refreshed. The external source system is a SQL database and the connection seems to be working fine.

Are there any other steps required to make this process work?

Many thanks in anticipation

Best regards,

Henry Simms · August 2022

Hi @Marco_13112001

In addition to the Source and Dataset, there's also a "refresh sources" option on the Schedule itself:

You would need this to be checked to ensure that the workflow refreshes data from your SQL database when it's invoked via this schedule.

Josh Boxer · August 2022

Hi Marco

For testing you could manually delete the batch of data from one of the Datasets before running the Schedule.

Have you Published this Workflow in the past and made changes since that have not been Published?

https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/get-started/manage-versions/

The Schedule will use the latest Published version.

Datasets not refreshing automatically

Answers

Categories