Delta Data Loads - Data Studio version 2.0
New to this forum so hope this makes sense
We have just gone live with Aperture Data Studio Version 2.0.
I have a scenario whereby on building a workflow, we will do full customer data extraction (one off at beginning) from source and load as the dataset and build a workflow.
Then every load of data after this will be delta files. Any new customers and any changes to existing data.
The fields wont change or be supplied unless a change if an existing active customer.
For eg - Full extraction load
ID 1 Sue Smith 23 smith street Camberwell 3030 is part of full file
ID 2 Brian Smith 25 smith street Camberwell 3030 is also part of full file
Sues data changes due to getting married and changing address
Brian remains as is
John is a new customer
File sends deltas
ID 1 Sue Johnson 23 Betty street Mont Albert 3031 is part of full file
ID 3 John Black 27 smith street Camberwell 3030 (New customer)
I can send full line or just the fields that changed
I don't want Sue added again, l want it to override the original data loaded in full file and add John.
I tried Multi batch but this just keeps adding any load to the last load, we dont want this. We want to keep a full active database as per source each day without sending full files and override
Can anyone offer suggestions please