Tips to improve workflow performance

Comments

  • Josh Boxer
    Josh Boxer Administrator

    Thanks Katya some really practical tips here that will be very useful on the workflows set up to run frequently.

  • Clinton Jones
    Clinton Jones Experian Elite

    Great tip! @Katya Jermolina - does it make any difference if I am using snapshots, files or database connections for the sources, or mixing them up in the workflows?

  • Katya Jermolina
    Katya Jermolina Experian Super Contributor

    @Clinton Jones , it shouldn't make a difference in principle. However, we occasionally saw examples where workflow performance had been slow due to a dependency on the source or target databases (either slow to import or to export data). It is always best to check that the database is accessible from the Data Studio server and there is no network latency between the database server and Data Studio. If you suspect this is the case, we usually recommend to try running the workflow with a CSV file as an data source instead of the database connection and export data into CSV instead of the database. This is simply to eliminate any dependencies on the external databases, as files and snapshots are stored locally on the Data Studio server.

    If the workflow executes much faster using CSV files, it is probably due to database connectivity and is something to look into. If the performance does not improve, then there are usually other ways to improve the workflow performance, such as mentioned in the tips in this thread.

Categories