Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError in JpegParser when calling EXIFR::TIFF #157

Closed
fabioperrella opened this issue Aug 17, 2020 · 5 comments · Fixed by #166
Closed

TypeError in JpegParser when calling EXIFR::TIFF #157

fabioperrella opened this issue Aug 17, 2020 · 5 comments · Fixed by #166
Assignees
@fabioperrella
Labels
bugfix

Comments

@fabioperrella
Copy link
Contributor

I found some files which raise an error like the following:

$ exe/format_parser_inspect file.jpg
Traceback (most recent call last):
	30: from exe/format_parser_inspect:22:in `<main>'
	29: from exe/format_parser_inspect:22:in `map'
	28: from exe/format_parser_inspect:24:in `block in <main>'
	27: from exe/format_parser_inspect:24:in `public_send'
	26: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:104:in `parse_file_at'
	25: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:104:in `open'
	24: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:105:in `block in parse_file_at'
	23: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:173:in `parse'
	22: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:173:in `to_a'
	21: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:173:in `each'
	20: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:173:in `each'
	19: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:173:in `each'
	18: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:173:in `each'
	17: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:169:in `block in parse'
	16: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:201:in `execute_parser_and_capture_expected_exceptions'
	15: from /Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/measurometer-1.1.1/lib/measurometer.rb:48:in `instrument'
	14: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:202:in `block in execute_parser_and_capture_expected_exceptions'
	13: from /Users/fabioperrella/projects/format_parser/lib/parsers/jpeg_parser.rb:21:in `call'
	12: from /Users/fabioperrella/projects/format_parser/lib/parsers/jpeg_parser.rb:29:in `call'
	11: from /Users/fabioperrella/projects/format_parser/lib/parsers/jpeg_parser.rb:65:in `scan'
	10: from /Users/fabioperrella/projects/format_parser/lib/parsers/jpeg_parser.rb:159:in `scan_app1_frame'
	 9: from /Users/fabioperrella/projects/format_parser/lib/parsers/exif_parser.rb:171:in `exif_from_tiff_io'
	 8: from /Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/measurometer-1.1.1/lib/measurometer.rb:48:in `instrument'
	 7: from /Users/fabioperrella/projects/format_parser/lib/parsers/exif_parser.rb:172:in `block in exif_from_tiff_io'
	 6: from /Users/fabioperrella/projects/format_parser/lib/parsers/exif_parser.rb:172:in `new'
	 5: from /Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/exifr-1.3.6/lib/exifr/tiff.rb:377:in `initialize'
	 4: from /Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/exifr-1.3.6/lib/exifr/tiff.rb:647:in `open'
	 3: from /Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/exifr-1.3.6/lib/exifr/tiff.rb:384:in `block in initialize'
	 2: from /Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/exifr-1.3.6/lib/exifr/tiff.rb:384:in `map'
	 1: from /Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/exifr-1.3.6/lib/exifr/tiff.rb:387:in `block (2 levels) in initialize'
/Users/fabioperrella/.rbenv/versions/2.6.5/lib/ruby/gems/2.6.0/gems/exifr-1.3.6/lib/exifr/tiff.rb:387:in `+': no implicit conversion of Integer into String (TypeError)

The same file is considered valid for the linux command file:

$ file file.jpg
file.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=9, manufacturer=Olympus, model=DP21, orientation=upper-left, xresolution=8, yresolution=16, resolutionunit=2, datetime=2020:03:11 15:11:47], baseline, precision 8, 1600x1200, components 3

If I try to parse direct using the gem exifr, it also raises an error:

pry> EXIFR::TIFF.new('file.jpg')
EXIFR::MalformedTIFF: no byte order information found

So, I think there is some problem inside the gem exifr to parse it.

The file that I tested is not mine and I need permission to use it as a fixture, so I'm waiting for it because I can't create a similar file to reproduce it.
@fabioperrella fabioperrella self-assigned this Aug 17, 2020
@fabioperrella
Copy link
Contributor Author

I did the same approach that I did in #155 (comment) and I could reproduce the same error as the original file!

Now the converted file has only 3.4KB and can't be peviewed because I removed the content of it!

But even this way it returns all the metadata when I run the file command:

$ file file.jpg
file.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=9, manufacturer=Olympus, model=DP21, orientation=upper-left, xresolution=8, yresolution=16, resolutionunit=2, datetime=2020:03:11 15:11:47]

And also returns the same error when running format_parser!

I attached the file so you can see:

file

@julik @martijnvermaat wdyt using this file as a fixture?

@fabioperrella
Copy link
Contributor Author

@linkyndy now that you are back, could you give your feedback about it pls ☝️

@julik
Copy link
Contributor

julik commented Aug 20, 2020

If the file no longer has image data it should be OK, but if it is 3.4KB then it might still include a thumbnail in the EXIF tags. It is also possible that this is why parsing fails (parsing this thumbnail) I did notice this happening with a JPEG from an electronic microscope for example

@fabioperrella
Copy link
Contributor Author

ok I will try to remove these tags!

Actually, I'm almost sure this is a picture from a microscope

@fabioperrella
Copy link
Contributor Author

In the end, I think something will need to be fixed in exifr gem, so I opened an issue there remvee/exifr#65

I still haven't been able to create a fake file or remove all the content to use as a fixture yet. I'm trying..

I talked with the support team trying to get permission from the owner of the file to use it as a fixture.

fabioperrella added a commit that referenced this issue Sep 11, 2020
@fabioperrella
Update gem exifr to 1.3.7 (fix #157)
ce0818c
@fabioperrella fabioperrella mentioned this issue Sep 11, 2020
Merged
julik pushed a commit that referenced this issue Sep 11, 2020
@fabioperrella
Update gem exifr to 1.3.7 (fix #157) (#166)
7485a99 
Closes #157 

related to remvee/exifr#66

Before this update:
```bash
$ exe/format_parser_inspect broken_thumbnails.jpg
Traceback (most recent call last):
	30: from exe/format_parser_inspect:22:in `<main>'
	29: from exe/format_parser_inspect:22:in `map'
	28: from exe/format_parser_inspect:24:in `block in <main>'
	27: from exe/format_parser_inspect:24:in `public_send'
	26: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:105:in `parse_file_at'
	25: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:105:in `open'
	24: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:106:in `block in parse_file_at'
	23: from /Users/fabioperrella/projects/format_parser/lib/format_parser.rb:174:in `parse'
...
```

After this update:
```bash
$ exe/format_parser_inspect broken_thumbnails.jpg
[
  {
    "source_path_or_url": "broken_thumbnails.jpg",
    "options": {
      "results": "first"
    },
    "result": {
      "nature": "image",
      "format": "jpg",
      "width_px": 1600,
      "height_px": 1200,
      "display_width_px": 1600,
      "display_height_px": 1200,
      "has_multiple_frames": null,
      "orientation": "top_left",
      "has_transparency": null,
      "color_mode": null,
...
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fabioperrella fabioperrella

Labels
bugfix
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update gem exifr to 1.3.7 (fix #157) WeTransfer/format_parser
2 participants
@julik @fabioperrella