External system drop zone stopped working

James T Sidebotham
James T Sidebotham Contributor
in General

Hi,

The drop zones that I have set up for several subfolders for many files has stopped working, and even after a restart continues to not work. The log contains a lot (a lot!) of these messages:

INFO c.e.d.d.ExternalDirectoryWatcher [ExternalDirectoryWatcher-e3649cf8-de9f-42e2-8eec-edee6e98c157 1369857878] Error fetching file for external system: SAFE-DQ (SFTP) [Environment=Default] [externalSystemUuid=e3649cf8-de9f-42e2-8eec-edee6e98c157] - Specified file path is invalid.

The external system is accessible (I can export files to there). If the reasons why "Specified file path is invalid" is generated, that could help with troubleshooting.

Any ideas?

Thanks

Comments

  • Henry Simms
    Henry Simms Administrator
    edited October 1

    Did anything change on the Data Studio side (eg version upgrade, change in external dropzone config or the external system / credential themselves)? If not, it could be a change in the behaviour or folder structure of the 3rd party SFTP server although difficult to say what.

    One thing to note is that the Dropzone will be monitoring for files in a location based on your External System's Root Directory while files exported to the External System from Data Studio may be using a different Upload Directory if one has been specified. I have a test setup like this:

    image.png

    When I remove the "files" folder from my SFTP server itself, my Data Studio exports still work but my file watchers error with something different to yours:

    INFO c.e.d.d.ExternalDirectoryWatcher [ExternalDirectoryWatcher-fe68fb99-9772-472f-8903-52ab25738f1c 1794115766] Error fetching file for external system: rebex (SFTP) [Environment=Demo] [externalSystemUuid=fe68fb99-9772-472f-8903-52ab25738f1c] - File not found.

    I wasn't able to recreate the Specified file path is invalid error

  • James T Sidebotham
    James T Sidebotham Contributor

    Nothing has changed at all with the configuration or versions, so it's all the more baffling. No folders have changed on the SFTP server. I am wondering if some Firewall rule change could generate this error - I would have to check with the network group to see if something changed and now has impacted this. I am not seeing any 'time out' messages though, but maybe they just don't get logged.

    Is the ExternalDirectoryWatcher an off-the-shelf Java module, or written in-house by Experian?

  • Henry Simms
    Henry Simms Administrator

    It's using the JSCH library for this , which is where the error is coming from.

  • James T Sidebotham
    James T Sidebotham Contributor

    "com.experian.datastudio.datasource.ExternalDirectoryWatcherException: 19001: External system not found"

    This could support the Firewall theory? At least for certain request types (whatever its doing in the watcher) - I can test connection successfully in Aperture and send files there without issue.

  • James T Sidebotham
    James T Sidebotham Contributor

    @Henry Simms When I set up the external system, I had the Root Directory set to / and then in the dataset definitions, I set the Folder to use as dropzone set to FromUsers and this has always worked fine for over a year.

    Obviously I am having errors now, but if I create a new external system definition and set the Root Directory to /FromUsers and the dataset drop zone is set to blank (it shows prefix of /FromUsers) then the dropzone works (for new files I am using for testing).

Categories