-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rosette Summary file name #204
Comments
Yes please. We should have that kind of fallback in all read functions. |
It's a bit more complicated to fix the issue for the ctd.read.from_edf() function, the only other function to call ctd.read._basename(). The real issue is we're mixing passing a file path and an io.StringIO object to each of the reader functions, but then later we just make the assumption a file path was passed in and then call _basename() to get the path, name and extension. I've fixed my issue and included a fixstation_hl_02.ros file with a unit test using the solution I described above, but maybe a better solution would be to deal with the issue ctd.read._basename() function causes. currently the function looks like: def _basename(fname):
"""Return file name without path."""
if not isinstance(fname, Path):
fname = Path(fname)
path, name, ext = fname.parent, fname.stem, fname.suffix
return path, name, ext And when an io.StringIO object is passed in the We can solve this in a couple of ways.
def _basename(fname):
"""Return file name without path."""
if isinstance(fname, io.StringIO):
return 'unknown', 'unknown', 'unknown'
if not isinstance(fname, Path):
fname = Path(fname)
path, name, ext = fname.parent, fname.stem, fname.suffix
return path, name, ext
|
Ah yes. I forgot about that! Regarding a more permanent solution, we should trow a hash or some sort of unique ID instead of unknown, for the filename. The extension and path can be unknown IMO. What do you think? |
So something like: def _basename(fname):
"""Return file name without path."""
if isinstance(fname, io.StringIO):
return 'unknown', str(hash(fname)), 'unknown'
if not isinstance(fname, Path):
fname = Path(fname)
path, name, ext = fname.parent, fname.stem, fname.suffix
return path, name, ext |
Looks good. I would restrict the hash to few characters though. Maybe something like: import hashlib
hashlib.shake_256(fname).hexdigest(5) |
BTW, you don't have to tackle that here. If you are OK with the solution for your case we can open an issue for this and solve it in another PR. |
That would be preferable. 😄 |
added unit tests and fix for issue #204
Let's leave this issue open though b/c we want to solve that at some point. I merged your PR and will mint a release soon. Thanks! |
I've come across an issue with yet again how DFO does things. I'm processing BTL/ROS files from what we call a 'fix station' and the files have next to no metadata. The BTL file processes fine, but when I try to use ctd.read.from_summary() I get an error on line 451
name = _basename(fname)[1]
The files don't have a
* FileName = xxxx
line so there is no file name.The ctd.read.from_btl() function solves this on lines 278-280 with:
I'd like to do the same thing for the ctd.read.from_cnv() function, called by the ctd.read.from_summary() function if it's not an issue.
The text was updated successfully, but these errors were encountered: