Can't send to SFTP destinations that don't support FSETSTAT

[reidsunderland]

Sending to SFTP destinations that don't support FSETSTAT causes a failure:

[ERROR] sarracenia.flow send could not send /source/filename to inflight=None sftp://[email protected]/ /destination/filename: FSETSTAT unsupported

Looking at the Paramiko code revealed that FSETSTAT is used for chmod, chown, utime and truncate.

This happens even with timeCopy and permCopy set to False, so we shouldn't be calling chmod, chown or utime.

We do call truncate in 2 places:

1. in readlocal_write of the Transfer class:

   sarracenia/sarracenia/transfer/__init__.py

   Line 401 in f7563bc

   dst.truncate(rw_length)

   • I think this is the case we're encountering, because length is set to 0. This case is for overwriting files that already exist. We want to ensure that remnants of the old file aren't left behind when the new file is smaller than the old one.
2. in the SFTP transfer class:

   sarracenia/sarracenia/transfer/sftp.py

   Line 486 in f7563bc

   if length != 0: rfp.truncate(self.fpos)

In the SFTP code, there's already logic to determine if a file already exists on the destination, so we could possibly do something with that to not call truncate in the case where the file doesn't already exist.

We could additionally add a try/except around the truncate call(s), but need to be careful to make sure we're not ignoring failures to truncate in cases where truncation really was needed (when a new file overwrites a larger old file).

The workaround is to use binary acceleration for all file transfers to these destinations.

[petersilva]

I think we need to wrap all the calls with try/except and emit a warning instead of having the transfer fail:

try:
truncate/chmod/utime... whatever
except Exception as ex:
logger.warning( "it failed : " ex )

we could also have a credentials setting "noFSETSTAT" as a flag on sftp to not even try...

It's a little project.

[petersilva]

should we have timeout(0) implement a disabling of timeouts as a side-quest on this one?

[reidsunderland]

should we have timeout(0) implement a disabling of timeouts as a side-quest on this one?

I vote yes for this, it should be an easy change to have alarm_set to only set the alarm when timeout > 0

[petersilva]

So I have a branch that is running in testing... and I implemented a the warning thing, and the timeout disabling thing. now with just those changes, the sender will work, but it will spit out a warning for every file sent "warning truncate failed"...

So I implemented a nofsetattr as a keyword in the credentials file, and so when you declare your sftp server credentials you add that keyword, and you will no longer get any warnings.

I'm now having second thoughts... should it be a credentials keyword or an option in the sr3_options... if people use ~/.ssh/config to store their sftp auth setup (as they should) then there is no need for an entry in the credentials.config... except this new flag... ugh... that's why I'm wondering if a normal keyword option is the better way to go.

[petersilva]

another option might be something like:

set sarracenia.transfer.sftp.Sftp.nofsetstat on

declare it as an option in the sftp transfer protocol... hmm..
not sure what is preferable.

[petersilva]

I changed it to an option in the config file. Seems more straight-forward.

[reidsunderland]

I prefer the option in the config file, I think it's easier to find and avoids creating credentials.conf entries that we wouldn't need otherwise.

I like set sarracenia.transfer.sftp.Sftp.nofsetstat on. It's a little bit cumbersome but it's an option we shouldn't need very often and makes it clear that it only applies to SFTP.

[petersilva]

need a new branch for a second try.