Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore comment lines in ENVI headers, and accept blank values. #7

Merged
merged 1 commit into from
May 8, 2014

Conversation

donm
Copy link
Contributor

@donm donm commented May 5, 2014

Fixes #6.

@tboggs tboggs added the bug label May 6, 2014
@donm
Copy link
Contributor Author

donm commented May 6, 2014

I confirmed that ENVI will load a header file with empty values. If the header says description = then the description field is blank, and if wavelength units = then wavelength units are set to "Unknown". If the header says custom_field =, and if you read that file and apply the header to another file, then the new header that is written will not have the custom_field entry. But it also won't have the field if the origin header said custom_field = 100; that is, it skips custom entries either way.

This is probably bad behavior on ENVI's part, since you're allowed to add whatever keys you want to the header file, but the important thing is that loading (and saving) headers with empty values won't cause incompatibility issues.

The reason I made the change was because some data I was working with had an empty wavelength units = field straight from the sensor.

About the semicolon in the middle of a multi-line value question: I did interpret the docs (http://www.exelisvis.com/docs/ENVIHeaderFiles.html) as saying it could be in the middle of an array, since they explicitly say "anywhere within a header file". But I just tested to see what happens (I have ENVI 5.0) and it looks like the code doesn't match the docs. A line starting with a semicolon in a text field (e.g., description) will be included in the value, and a line starting with a semicolon in a numeric field (wavelength) will cause the first number on that line to be read as 0.

So I'll report this to VIS tomorrow. I'm sure they'll jump right on it, and either the docs or the code will be fixed for the 2019 release or something.

In the meantime, what are your thoughts? As is, the patch follows the docs. If you want identical parsing behavior in both programs then one of us could comment out line #98.

@tboggs
Copy link
Member

tboggs commented May 6, 2014

If you're going to report it to them tomorrow, let's wait a day or so to see if you get a response. If it takes longer than that, I'm content with the patch the way it is now. I can't think of a realistic case where an array-valued parameter would have a value at the beginning of a line where the value actually starts with a semicolon. So commenting out such lines shouldn't be a problem.

If you get a response from Exelis later on, we can always update the code to match what they specify as the correct behavior.

@donm
Copy link
Contributor Author

donm commented May 6, 2014

The tech support rep I emailed is going to report the issue. But she seemed to think that the documentation should be updated to indicate that comments are not allowed in the middle of key/value pair, rather than update the code to match the docs.

@tboggs
Copy link
Member

tboggs commented May 7, 2014

I'm not too concerned about a semicolon inside a <key> = <value> statement. If it is a parameter that is expected to be a numeric value, then it will fail anyway. Did she have any feedback regarding a comment line in the middle of a multi-line array definition?

@donm
Copy link
Contributor Author

donm commented May 7, 2014

She didn't. I'll try to keep an eye on the incident in their tracking system (I already have a few other bugs reported to them that I'm watching) and will let you know what the outcome is. But I'm sure this has, like, negative priority for them.

@tboggs
Copy link
Member

tboggs commented May 8, 2014

I'll go ahead and merge this since either way, it is better than the current behavior. If we get an authoritative response from Exelis later, we can always update it to be consistent.

tboggs added a commit that referenced this pull request May 8, 2014
Ignore comment lines in ENVI headers, and accept blank values. [fixes #6] [fixes #7]
@tboggs tboggs merged commit fd9e826 into spectralpython:master May 8, 2014
@donm donm deleted the dm/iss6 branch May 14, 2014 04:41
@tboggs tboggs added this to the v0.15.0 milestone Jun 5, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ENVI header parsing issues
2 participants