Jobs with large lookup files
At St. James's Place, prior to highlighting whether client details need to be quality checked, we need to establish whether the client has a current fund holding (as this is where we realise maximum business benefit in correcting the data)
To do this we are faced with loading very large table(s) ie >100Million rows which contain fund holding details and then discarding the majority of the data as we are only interested in the current position ie those for the maximum snapshot date (<1Million rows).
We then only use these as lookup tables and do not use any of the data held.
We may also face licencing issues re:large dataset sizes without a pre-load filter option.
To i) read large table(s) into Aperture and ii) is quite an overhead and we are considering that it is a better solution to run a SQL view to establish the required set of clients, save this as a file and load it into Aperture (or read the view directly in Aperture). This does introduce an overhead by running it ourside of Aperture but, faced with the above issues, is potentially the best solution.
Is this need common amongst the Aperture community and could this type of functionality be considered for future releases?