Crash when encountering comments / column numbers change

[CrustyAuklet]

I often encounter CSV files that use comments (lines that start with a specific character, usually ; in my experience) to encode metadata about the file in the first few lines.

Reading these files causes a crash, I suspect because it looks like a single column CSV file at first and then the data starts with many columns.

example:

;Instrument ABCDEF
;Collected 14JUN2018 field site Alpha
time,temperature,humidity,pressure
12312351,23.3,120,234
12312352,24.0,122,233
...
...

I know comments are not mentioned in RFC4180, but even if this library does not handle them it should ignore/throw gracefully.

[vincentlaucsb]

Hi, I haven't been able to work on this library in a while but if you're still interested then read on.

I agree that comments should be ignored gracefully, and in fact one of the test CSV files includes multiple comment lines. I think this crash was due to memory usage issues in the CSV parsing. It got so bad that Valgrind gave up trying to count all of them, but they've been fixed in the latest release.

Marking as closed for now.