Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes ArrayIndexOutOfBoundsException when reading CSV file #658 #659

Merged
merged 2 commits into from
Sep 6, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions core/src/main/java/tech/tablesaw/io/FileReader.java
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ private String[] getHeaderNames(
String[] headerNames = parser.parseNext();
// work around issue where Univocity returns null if a column has no header.
for (int i = 0; i < headerNames.length; i++) {
headerNames[i] = headerNames[i].trim();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this could cause an NPE since the next line checks if headerNames[i] == null

Copy link
Collaborator Author

@lwhite1 lwhite1 Sep 6, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll move the call

@lwhite1 Travis is failing

It's the log-length thing again, which I've "fixed" several times by suppressing various kinds of output. I guess we're limited to somewhere 3000-3500 lines of output now or some such. This time the extra output is from issues related to documentation.

With the sub-projects it seems we will run into this a lot. I'd like to suppress all logging except errors. Is that possible?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed a fix

if (headerNames[i] == null) {
headerNames[i] = "C" + i;
}
Expand Down
15 changes: 15 additions & 0 deletions core/src/test/java/tech/tablesaw/io/FileReaderTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package tech.tablesaw.io;

import org.junit.jupiter.api.Test;
import tech.tablesaw.api.Table;

import static org.junit.jupiter.api.Assertions.*;

class FileReaderTest {

@Test
void parseWhenHeadersHaveWhitespace() throws Exception {
Table t = Table.read().csv("../data/2017_Climate_Investment_Funds.csv");
assertNotNull(t);
}
}
25 changes: 25 additions & 0 deletions data/2017_Climate_Investment_Funds.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Reporting year,Project Title,Country,Region,Public/ Private,Technology,Grid connection,SREP Funding(US$m),MDB,Lifetime,Installed Capacity (MW),Target Annual Electricity Output (MWh) ,Women - Target improved energy access,Men - Target improved energy access,People - Target improved energy access,Businesses - Target improved energy access,Community Services - Target improved energy access,Annual Target GHG emissions reduced/avoided (tons of CO2 equivalent),Lifetime Target GHG emissions reduced/avoided (tons of CO2 equivalent),Target co-financing (US$m),Target co-financing Government (US$m),Target co-financing Private sector (US$m),Target co-financing MDBs (US$m),Target co-financing bilaterals (US$m),Target co-financing others (US$m),"Actual Annual Electricity Output (Latest Year, MWh) ","Women- Actual improved energy access, Latest Year","Men- Actual improved energy access, Latest Year","People - Actual improved energy access, Latest Year","Businesses - Actual improved energy access, Latest Year","Community Services - Actual improved energy access, Latest Year","Actual GHG emissions reduced/avoided (Latest Year, tons of CO2 equivalent)","Actual co-financing (Latest Year, US$m)","Actual co-financing Government (Latest Year, US$m)","Actual co-financing Private sector (Latest Year, US$m)","Actual co-financing MDBs (Latest Year, US$m)","Actual co-financing bilaterals (Latest Year, US$m)","Actual co-financing others (Latest Year, US$m)",Cumulative Annual Electricity Output (MWh) ,Women- Cumulative improved energy access ,Men- Cumulative improved energy access ,People - Cumulative improved energy access ,Businesses - Cumulative improved energy access ,Community Services - Cumulative improved energy access ,Cumulative GHG emissions reduced/avoided (tons of CO2 equivalent),Cumulative co-financing (US$m),Cumulative co-financing Government (US$m),Cumulative co-financing Private sector (US$m),Cumulative co-financing MDBs (US$m),Cumulative co-financing bilaterals (US$m),Cumulative co-financing others (US$m)
2017,Rural Electrification Expansion Project,Tanzania,AFR,Public,Mixed RE,off-grid,9,IBRD,20,33,142000,155000,155000,310000,,,112000,2220000,155,28,31,35,56,0,0,0,0,0,,,0,0,0,0,0,0,0,0,0,0,0,,,0,0,0,0,0,0,0
2017,Solar Power Development Project,Solomon Islands,ASIA,Public,Solar,off-grid,6.6,ADB,20,2,3100,2922,3078,6000,,,840,16800,9,6.76,0,2.24,0,0,0,0,0,0,,,0,0,0,0,0,0,0,0,0,0,0,,,0,0,0,0,0,0,0
2017,Renewable Energy for Electrification in North and Center Liberia Project – Mini-grids,Liberia,AFR,Public,Hydro,mini-grid,25,IBRD,20,,4000,74400,75600,150000,,,3174,63496,4.5,0,0,2.5,1.5,0.5,0,0,0,0,0,0,0,0,0,0,0,0,0,0,,,0,0,0,0,0,0,0,0,0,0
2017,Nicaragua Geothermal Exploration and Transmission Improvement under the PINIC,Nicaragua,LAC,Public,Geothermal,on-grid,7.5,IDB,,18,315360,,,,,,87139,2614170,12.8,2.81,0,9.99,0,0,0,,,,,,0,0,0,0,0,0,0,0,,,,,,0,0,0,0,0,0,0
2017,South Asia Subregional Economic Cooperation Power System Expansion Project- Additional Co-financing,Nepal,ASIA,Public,Solar,on-grid,20,ADB,20,25,32850,137505,129495,267000,,,26280,525600,70,0,70,0,0,0,0,0,0,0,,,0,0,0,0,0,0,0,0,0,0,0,,,0,0,0,0,0,0,0
2017,TA-Strengthening Renewable Energy Regulations,Mongolia,ASIA,Public,Mixed RE,capacity building,1.2,IBRD,4,,,,,,,,,,0.1,0.1,0,0,0,0,,,,,,,,0,0,0,0,0,0,,,,,,,,0,0,0,0,0,0
2016,Geothermal Exploratory Drilling Project,Armenia,ECA,Public,Geothermal,on-grid,8.7,IBRD,30,28.5,224694,,,,,,166000,5256000,108.6,8.6,100,0,0,0,0,,,,,,0,0,0,0,0,0,0,0,,,,,,0,0,0,0,0,0,0
2016,Self-Supply RE Guarantee Program,Honduras,LAC,Private,Mixed RE,on-grid,5.5,IDB,20,20,45000,,,,,,40000,800000,40,0,20,20,0,0,1186,,,,,,747,0,0,0,0,0,0,1462,,,,,,921,,0,,1.5,0,0
2016,Honduras Renewable Energy financing facility,Honduras ,LAC,Private,Mixed RE,on-grid/off-grid,21.25,IDB,13,153,427000,,,,22,,285000,5700000,390,0,40,4,0,346,0,,,,0,,0,0,0,0,0,0,0,0,,,,0,,0,4,0,0,4,0,0
2016,Electricity Modernization Project,Kenya,AFR,Public,Mixed RE,mini-grid,7.5,IBRD,20,,1242,10125,10125,20250,,,986,19711,13.2,0,10.7,2.5,0,0,0,0,0,0,,,0,8,0,0,8,0,0,0,0,0,0,,,0,8,0,0,8,0,0
2016,Sustainable Energy Industry Development Project,Pacific Region,ASIA,Public,Mixed RE,capacity building,1.92,IBRD,5,,,,,,,,,,3.7,0,0,0,0,3.7,,,,,,,,0.2,0,0,0,0,0.2,,,,,,,,3.9,0,0,0,0,3.9
2016,Tanzania Mini-grids project,Tanzania,AFR,Private,Mixed RE,mini-grid,4.95,IFC,,10.5,88000,55000,55000,110000,,,200000,4000000,0.5,0,0.5,0,0,0,0,0,0,0,,,0,0,0,0,0,0,0,0,0,0,0,,,0,0,0,0,0,0,0
2015,Preparing Outer Islands for Sustainable Energy Development Program(POISED),Maldives,ASIA,Public,Solar,mini-grid,12.7,ADB,25,21,27600,15410,15410,30820,,,40000,1000000,112,14,0,38,60,0,0,0,0,0,,,0,44.5,0,0,33,0,11.5,0,0,0,0,,,0,44.5,0,0,33,0,11.5
2015,Promoting the Scaling Up of Renewable Energy in Mali,Mali,AFR,Public,Mixed RE,capacity building,1.5,AFDB,3,,,,,,,,,,1.08,0.37,0,0,0,0.71,,,,,,,,0,0,0,0,0,0,,,,,,,,0.43,0.2,0,0.23,0,0
2015,South Asia Subregional Economic Cooperation Power System Expansion Project (formerly Rural Electrification Through Renewable Energy),Nepal,ASIA,Public,Mixed RE,mini-grid,11.78,ADB,20,4.8,25228,75689,67661,143350,,,18000,360000,16.72,3.25,0,5,0,8.47,0,0,0,0,,,0,0,0,0,0,0,0,0,2409,4191,6600,,,0,0,0,0,0,0,0
2015,Extended Biogas Program,Nepal,ASIA,Public,Waste to Heat,off-grid,7.9,IBRD,20,,20400,,,,400,,16970,339404,27.6,27.6,0,0,0,0,0,,,,0,,0,0,0,0,0,0,0,0,,,,0,,0,0,0,0,0,0,0
2015,Lighting Ethiopia,Ethiopia,AFR,Private,Solar,capacity building,2,IFC,2,,,,,,,,,,0.65,0,0.65,0,0,0,,,,,,,,1.04,0,0.06,0,0,0.98,,,,,,,,1.32,0,0.12,0,0,1.2
2014,Strengthening the RE Policy and Regulatory Framework(FOMPIER),Honduras,LAC,Public,Mixed RE,capacity building,0.85,IDB,3,,,,,,,,,,0.09,0.09,0,0,0,0,,,,,,,,0,0,0,0,0,0,,,,,,,,0.03,0.03,0,0,0,0
2014,Menengai Geothermal Project,Kenya,AFR,Public,Geothermal,on-grid,25,AFDB,6,150,1182000,1250000,1250000,2500000,300000,,734650,18327067,477.7,245.2,0,124.1,108.7,0,0,0,0,0,0,,0,11.01,0,0,11.01,0,0,0,0,0,0,0,,0,398.46,0,0,101.91,0,0
2014,Accelerating Sustainable Private Investments in RE Program(ASPIRE),Maldives,ASIA,Public,Solar,on-grid,12.60245896,IBRD,20,20,32610,19303,19303,38605,,,25883,517667,58,0,42,16,0,0,0,0,0,0,,,0,0,0,0,0,0,0,0,,,0,0,0,0,0.2,0,0.2,0,0,0
2014,Rural Electrification Hybrid Systems,Mali,AFR,Public,Solar,mini-grid,15.4,IBRD,20,4.8,8653,343224,337776,681000,,,6868,137365,40.73,8.9,1.83,25,0,5,0,0,0,0,,,0,1.16,0,0,1.16,0,0,0,0,0,0,,,0,3.59,0,0,2.77,0,0
2014,Geothermal Sector Strategy and Regulations,Ethiopia,AFR,Private,Geothermal,on-grid,1.5,IFC,2,,,,,,,,,,0.5,0.5,0,0,0,0,,,,,,,,0,0,0,0,0,0,,,,,,,,0.63,0.46,0,0,0.17,0
2014,Sustainable Rural Energization(ERUS)-Part I & III: Promoting Sustainable Business Models for Clean Cookstoves Dissemination,Honduras,LAC,Private,Cookstoves,off-grid,2.947,IDB,5,,,187500,187500,375000,300,,74532,199093,3.03,0.84,0,0,2.19,0,,1976,2024,4000,0,,7790,0.212,0,0.212,0,0,0,,2369,2427,4796,0,,7867,2.922,0,0.73,2.19,0,0
2014,Geothermal Sector Development Project,Ethiopia,AFR,Public,Geothermal,on-grid,24.5,IBRD,20,70,552000,550000,550000,1100000,,,438122,8762440,304,12,0,178.5,0,113.5,0,0,0,0,,,0,0,0,0,0,0,0,0,0,0,0,,,0,8,0,0,8,0,0