Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interpret all columns as TEXT data type #42

Closed
tsundara opened this issue Nov 30, 2018 · 4 comments
Closed

Interpret all columns as TEXT data type #42

tsundara opened this issue Nov 30, 2018 · 4 comments

Comments

@tsundara
Copy link

tsundara commented Nov 30, 2018

Is it possible to interpret all columns as TEXT datatype (through a flag maybe?)
I think the columns values are sampled and then column datatype is guessed. If there is an incompatible value in some row, then that row seems to be skipped. So instead, is it possible I load everything as TEXT data type?
I just need the data in some format(for data comparison purposes), But all csv data must go into the table without skipping rows.

Thanks
Thyag

@ericlentz
Copy link

Similarly, I have item numbers in the format, 00645155909931 and it reads it as an integer datatype which converts it to 643169607033.

@tsibley
Copy link

tsibley commented Feb 27, 2019

I also have this issue. I thought specifying --shape "csv_col:sql_col(text),…" for each column would work, but the text type isn't passed into pd.read_csv's dtype parameter.

@dannguyen
Copy link
Contributor

It'd be great if there were an option akin to csvkit's csvsql --no-inference to force every column to be just TEXT, for imports in which there are just too many columns to easily specify the --shape flag:

https://csvkit.readthedocs.io/en/1.0.2/scripts/csvsql.html

@simonw
Copy link
Owner

simonw commented Aug 9, 2020

This feature has now been released in csvs-to-sqlite 1.1

@simonw simonw closed this as completed Aug 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants