Cannot get records out of CSV file (records bleeding into each other?)

[kmauzoul]

Hello -

I have a CSV (attached) -
po_export_20220415001350.csv

For some reason, I cannot figure out why the attached has problems being read - I had to turn all my properties to strings when mapped just to see what was getting placed into my classes properties:

public class EMaintPoCsvModel
    {
        [Name("purno"), Index(0)]
        public string PurNo { get; set; }

        [Name("supplierid"), Index(1)]
        public string SupplierId { get; set; }

        [Name("supplier"), Index(2)]
        public string Supplier { get; set; }

        [Name("purdate"), Index(3)]
        public string PurDate { get; set; }

        [Name("dlvrydate"), Index(4)]
        public string DlvryDate { get; set; }

        [Name("postat"), Index(5)]
        public string PoStat { get; set; }

        [Name("shipvia"), Index(6)]
        public string ShipVia { get; set; }

        [Name("freight"), Index(7)]
        public string Freight { get; set; }

        [Name("terms"), Index(8)]
        public string Terms { get; set; }

        [Name("fob"), Index(9)]
        public string FOB { get; set; }

        [Name("lineid"), Index(10)]
        public string LineId { get; set; }

        [Name("lineno"), Index(11)]
        public string LineNo { get; set; }

        [Name("reqno"), Index(12)]
        public string ReqNo { get; set; }

        [Name("item"), Index(13)]
        public string Item { get; set; }

        [Name("descrip"), Index(14)]
        public string Descrip { get; set; }

        [Name("reqdate"), Index(15)]
        public string ReqDate { get; set; }

        [Name("qtyord"), Index(16)]
        public string QtyOrd { get; set; }

        [Name("umeasur"), Index(17)]
        public string UMeasur { get; set; }

        [Name("cost"), Index(18)]
        public string Cost { get; set; }

        [Name("extcost"), Index(19)]
        public string ExtCost { get; set; }

        [Name("invloc"), Index(20)]
        public string InvLoc { get; set; }

        [Name("postat2"), Index(21)]
        public string PoStat2 { get; set; }

        [Name("wo"), Index(22)]
        public string WO { get; set; }

        [Name("acct_no"), Index(23)]
        public string AcctNo { get; set; }
}

I'm doing the following to read the csv:

CsvConfiguration config = new CsvConfiguration(CultureInfo.InvariantCulture);
            config.Delimiter = ",";
            config.Mode = CsvMode.Escape;

using (StreamReader reader = new StreamReader(stream))
                        {
                            using (CsvReader csv = new CsvReader(reader, config))
                            {
                                Program.Log.Info("Building Csv contents");
                                
                                ret.AddRange(csv.GetRecords<T>());
                            } //clean up csv reader
                        } //clean up stream reader

The stream is coming from an sFTP (I verified the file is downloading correctly) -

If I don't set MissingFieldFound as null I see the following:

CsvHelper.MissingFieldException: Field at index '14' does not exist. You can ignore missing fields by setting MissingFieldFound to null.
IReader state:
   ColumnCount: 0
   CurrentIndex: 14
   HeaderRecord:
["purno","supplierid","supplier","purdate","dlvrydate","postat","shipvia","freight","terms","fob","lineid","lineno","reqno","item","descrip","reqdate","qtyord","umeasur","cost","extcost","invloc","postat2","wo","acct_no"]
IParser state:
   ByteCount: 0
   CharCount: 344
   Row: 2
   RawRow: 2
   Count: 14
   RawRecord:
"00003","V2002","ACR INC.",03/07/2022,03/07/2022,"O","","","","","_6A40KREI0",1,"","0000001","#4 BL HYDRAULIC CAP",03/14/2022,2.000,"EA",100.00000,200.00,"MAIN","O",0,""


   at CsvHelper.Configuration.ConfigurationFunctions.MissingFieldFound(MissingFieldFoundArgs args)
   at CsvHelper.CsvReader.GetField(Int32 index)
   at CsvHelper.CsvReader.get_Item(Int32 index)
   at lambda_method(Closure )
   at CsvHelper.Expressions.RecordCreator.Create[T]()
   at CsvHelper.Expressions.RecordManager.Create[T]()
   at CsvHelper.CsvReader.<GetRecords>d__87`1.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at System.Collections.Generic.List`1.AddRange(IEnumerable`1 collection)
   at Synergy.eMaintVisualConnector.CsvWorker.Import[T](List`1 FileNames) in D:\Work\Clients\McWilliams Forge\eMaint Visual Interface\eMaintVisualConnector\BusinessFacade\CsvWorker.cs:line 62
   at Synergy.eMaintVisualConnector.Program.Main(String[] args) in D:\Work\Clients\McWilliams Forge\eMaint Visual Interface\eMaintVisualConnector\Program.cs:line 130

Which is throwing me off as field 14 does have data in it when looking at the data (??)

When I set MissingFieldFound as null and inspect what my properties contain, i see weirdness such as:

As if the commas are being ignored -

I'm stuck and any help would be greatly appreciated.

Thanks,

• Karl

[kmauzoul]

Sorry - this can be ignored - after further inspecting the file and reading the RFC4180 specification again, I found problems with the files, particularly with how fields are quoted, and how certain fields are quoted and not escaped properly