Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Shiny new feature #59973

Closed
wants to merge 2 commits into from
Closed

Shiny new feature #59973

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9c3a2fb
Added a helper function to help the date column dtype be customized
wshaoul Oct 5, 2024
2ea84ee
Moved dtype setting function to finalize output function
wshaoul Oct 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions pandas/io/parsers/arrow_parser_wrapper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,9 @@ def _finalize_pandas_output(self, frame: DataFrame) -> DataFrame:
except TypeError as err:
# GH#44901 reraise to keep api consistent
raise ValueError(str(err)) from err

self._set_date_column_dtype(frame, date_columns=["date_column1", "date_column2"], dtype="timestamp[ns][pyarrow]")

return frame

def _validate_usecols(self, usecols) -> None:
Expand Down Expand Up @@ -306,4 +309,23 @@ def read(self) -> DataFrame:

else:
frame = table.to_pandas()

return self._finalize_pandas_output(frame)


def _set_date_column_dtype(self, frame: DataFrame, date_columns: list, dtype: str):
"""
Sets the dtype for specified date columns in the DataFrame.

Parameters
----------
frame : DataFrame
The DataFrame to modify.
date_columns : list
List of column names that are date columns.
dtype : str
The dtype to apply to these columns, e.g., 'datetime64[ns]'.
"""
for col in date_columns:
if col in frame.columns:
frame[col] = frame[col].astype(dtype)
Loading