1.3.3 struggling with zero byte files

[ross-spencer]

Stack trace:

  goatslayer@goatslayer-acer-linux:~/git/droid-sqlite-analysis$ fido empty-file.empty 
  FIDO v1.3.3 (formats-v84.xml, container-signature-20160121.xml, format_extensions.xml)
  FIDO: Zero byte file (empty): Path is: empty-file.empty
  Traceback (most recent call last):
    File "/usr/local/bin/fido", line 9, in <module>
      load_entry_point('opf-fido==1.3.3', 'console_scripts', 'fido')()
    File "/usr/local/lib/python2.7/dist-packages/fido/fido.py", line 855, in main
      fido.identify_file(file)
    File "/usr/local/lib/python2.7/dist-packages/fido/fido.py", line 375, in identify_file
      bofbuffer, eofbuffer = self.get_buffers(f, size, seekable=True)
  ValueError: too many values to unpack

Empty file listing below:

  goatslayer@goatslayer-acer-linux:~/git/droid-sqlite-analysis$ ls -l empty-file.empty 
  -rw-rw-r-- 1 goatslayer goatslayer 0 May 15 12:48 empty-file.empty
  goatslayer@goatslayer-acer-linux:~/git/droid-sqlite-analysis$ 

Distro stats:

Python 2.7.6
No LSB modules are available.
Distributor ID: Ubuntu 
Description:    Ubuntu 14.04.4 LTS
Release:    14.04
Codename:   trusty

[mistydemeo]

Hm, I can't repro on OS X:

$ touch /tmp/empty
$ fido /tmp/zero
FIDO v1.3.1 (formats-v84.xml, container-signature-20151217.xml, format_extensions.xml)
FIDO: Zero byte file (empty): Path is: /tmp/zero
bad repeat interval
bad repeat interval
KO,201,,,,0,"/tmp/zero",,"fail"
FIDO: Processed      1 files in 283.72 msec,  4 files/sec

(Note that despite FIDO listing itself as 1.3.1 I am using the latest tarball release.)

[ross-spencer]

Hi Misty,

I installed from PIP and so I removed that and then installed from the downloadable releases to try again also:

Zip:

  goatslayer@goatslayer-acer-linux:~/Desktop/fido/fido-1.3.3-84$ fido tmp/empty-file 
  FIDO v1.3.3 (formats-v84.xml, container-signature-20160121.xml, format_extensions.xml)
  FIDO: Zero byte file (empty): Path is: tmp/empty-file
  Traceback (most recent call last):
    File "/usr/local/bin/fido", line 9, in <module>
      load_entry_point('opf-fido==1.3.3', 'console_scripts', 'fido')()
    File "/usr/local/lib/python2.7/dist-packages/opf_fido-1.3.3-py2.7.egg/fido/fido.py", line 855, in main
      fido.identify_file(file)
    File "/usr/local/lib/python2.7/dist-packages/opf_fido-1.3.3-py2.7.egg/fido/fido.py", line 375, in identify_file
      bofbuffer, eofbuffer = self.get_buffers(f, size, seekable=True)
  ValueError: too many values to unpack
  goatslayer@goatslayer-acer-linux:~/Desktop/fido/fido-1.3.3-84$ 

Tarball:

  goatslayer@goatslayer-acer-linux:~/Desktop/fido/fido-1.3.3-84-tar$ touch tmp/empty-file
  goatslayer@goatslayer-acer-linux:~/Desktop/fido/fido-1.3.3-84-tar$ fido tmp/empty-file 
  FIDO v1.3.3 (formats-v84.xml, container-signature-20160121.xml, format_extensions.xml)
  FIDO: Zero byte file (empty): Path is: tmp/empty-file
  Traceback (most recent call last):
    File "/usr/local/bin/fido", line 9, in <module>
      load_entry_point('opf-fido==1.3.3', 'console_scripts', 'fido')()
    File "/usr/local/lib/python2.7/dist-packages/opf_fido-1.3.3-py2.7.egg/fido/fido.py", line 855, in main
      fido.identify_file(file)
    File "/usr/local/lib/python2.7/dist-packages/opf_fido-1.3.3-py2.7.egg/fido/fido.py", line 375, in identify_file
      bofbuffer, eofbuffer = self.get_buffers(f, size, seekable=True)
  ValueError: too many values to unpack

Differences in Python version?

[anjackson]

Hm, also cannot reproduce. I note fido.py can return three values when the length of the BOF buffer is zero, which seems strange as it usually returns two values. Perhaps this check has moved further up the call chain and this is now dead code that shouldn't get called?

@ross-spencer Any chance you've got an old version of Fido installed somewhere else? (under the old package name?!)

UPDATE Hah, I've managed to break my installation and reproduce this error by using python setup.py install to install fido 1.1.1 from source and then removing it and reinstalling via pip. Not sure what's gone wrong! Something somewhere must be cached. Maybe a pyc file?!

UPDATE 2 @mistydemeo or perhaps you have the old version cached in site-package?

[ross-spencer]

Thanks @anjackson - I haven't been using FIDO on this machine up until today so I wasn't expecting it to be something legacy on my system - from your update do you still think it might be a conflict between an old and a new version?

[anjackson]

If it is, then AFAICT it's the other way around - ie Fido was was only working for me (and is only working for @mistydemeo) because we had/have old Fido installs on the PYTHONPATH.

[ross-spencer]

Cool. Thanks Andy. If it turns out that it is my install then I can nuke whatever I need to nuke and then try again too. I would have looked at the code today as well but had a few other priorities to work on first.

[mistydemeo]

It looks like it was in fact old FIDO installs for me; after nuking my virtualenv I'm now seeing this same behaviour.

[mistydemeo]

I believe this was introduced by 6c7f798. get_buffers used to branch in a couple of places on length == None, which was changed to check length's falsiness. However, if length is zero - as it is, for example, in a zero-byte file - then this proceeds down a branch that previously shouldn't have been possible.

[sevein]

Thanks @mistydemeo, I definitely should pay more attention to changes like that in the future. @jhsimpson will merge #74 soon.