Importing DataSets - Filtering Data

Rachael
Rachael Member
edited December 2023 in General

Hi,

I was wondering if there is currently anyway of limiting/filtering out data from an underlying table before importing it as a dataset? I have a number of transactional tables that I want to bring in to Aperture, however it would be great if I could limit the date range of the transactions before importing the table itself is upwards of 6 billion rows.

If there is no way of currently doing this, is it something that can be considered for a future release?

Thanks!

Rachael

Answers

  • Rachael
    Rachael Member

    Hi Clinton,


    Thanks - that's great news this is something which is being looked in to. The underlying tables have approximately 15-20 million records and it's connecting to a SQL database


    Rachael

  • Simms, Henry
    Simms, Henry Administrator
    Hi @Rachael , just letting you know that you can now specify a SQL query when loading data from a Database: https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/get-started/configure-external-systems/#loaddatausingasqlquery

    This will allow you to filter on load, as well as pushing additional processing like grouping or joining down to the data source system.

    Available from V2 5.6 onwards: https://community.experianaperture.io/discussion/760/

Categories