How do you work out where the row comes from ?

When you're combining data from multiple data sources that potentially have duplicated data, you may want to work out which row you want to win.

This is partiulcarly relevant when you use harmonization. You can base decisions on the source, or which records to prioritize against in grouping.

You could consider this as a particularly lightweight kind of lineage.

The functions you might use to do this are focused on using the concatenate, constant and currentrow functions.

The two transforms before the union surface a derived column which i have called LINEAGE


Within the LINEAGE column my transform uses concatenate, a text constant and currentrow to create a calculated ID

A more transparent way of achieving the same result is as follows


Tagged:
Sign In or Register to comment.