Exact Match with List functions
There are a number of List functions that may be useful when you are trying to de-duplicate a list of values within a single column.
Let's take a look at this dataset. How can we de-duplicate the list of fruits for each day?
Connecting the dataset to a Transform step with List Frequency and List De-duplicate Functions will provide a count of each fruit per day along with the list of unique fruits per day. Records with frequency > 1 indicate that they are duplicates.
List Frequency provides the count for each value within the Fruits column.
List Deduplicate provides the deduplicated list of Fruits.