Union source information

Wojciech Glowacki
Wojciech Glowacki Member
in General

Hi,

I am performing a union step of 5 sources. I hope its not a dumb question but I cant seem to find a way to add a column that tells me from which source a particular row is coming from in the union step. For example I want to know this row is form source 1, and this row is form source 2 when I look at the Union output. I would be grateful for your help. Thanks !

Answers

  • Ivan Ng
    Ivan Ng Administrator
    edited February 4

    Hi Wojciech,

    Thanks for your question. You can achieve what you need by adding source metadata before the Union. Each source allows you to include metadata fields (for example, “Origin Dataset”) that help identify the row’s source once everything is combined.

    To do this:

    1. Select More options on your source steps
    2. Choose the metadata fields you want to include under Source metadata
    3. Repeat for each source step.

    Once added, those fields will carry through to the Union step, making it easy to see which rows came from which source.

    Let me know if you have further questions or need further help. Thanks.

  • Josh Boxer
    Josh Boxer Administrator

    More info on metadata: https://docs.experianaperture.io/data-quality/aperture-data-studio/manage-and-audit/access-lineage-metadata/

    image.png

    Another option would be to add a Transform step to each of the (5) branches to add a column with a 'Constant value'

Categories