Pushdown processing on Experian ADS

abhish
abhish Member
edited December 2023 in General

Hello,

We are trying to improve our workflow using "Pushdown processing" so, please someone can give me more detail on when "Pushdown processing" will be available on Experian ADS?

Answers

  • Josh Boxer
    Josh Boxer Administrator

    Hi Abhish

    We have plans on our roadmap to deliver push down functionality, but no exact ETA at this time.

    What is the database you are trying to connect to? Broadly what is the Workflow doing and where are you running into issues today with your workflow(s) that you feel need improvement?

    Let us know and we can see if our engineers think it will be a good candidate for any beta functionality as it becomes available

  • abhish
    abhish Member

    Hello, Josh,

    We are using MySQL Server with a JBDC connection. Our workflow has many steps for data processing such as Source(Database) → View → Source (workflow) followed by steps such as filters, group, union, etc. And is taking such a long time on data refresh. That is what we are looking to improve. So, we need solid answers on how long push-down processing will support.

  • Josh Boxer
    Josh Boxer Administrator

    Thanks Abhish.

    1. what is the data you are downloading, i.e. daily transactional data, customer records?
    2. how many rows (roughly) are in the data tables?
    3. roughly how long is a refresh taking? (We might be able to comment on anything abnormal)
    4. if it is the size of data that is the cause then is it possible you could identify only the changed records and refresh only those?
      1. example1 pull in only yesterday's data and add it to the rest of the data you downloaded previously
      2. example2 query only records that have amended and process these records in your Workflow https://docs.experianaperture.io/data-quality/aperture-data-studio-v2/get-started/configure-external-systems/#loaddatausingasqlquery~jdbc-databases

Categories