BUG: combine_first does not retain dtypes with Timestamp DataFrames

[arw2019]

• closes combine_first returns unexpected results for timestamp dataframes #28481
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

Picking up #35514

[jbrockmendel]

Why does combin_first use expressions.where instead of mgr.where? Casting should be handled correctly there

[arw2019]

Why does combin_first use expressions.where instead of mgr.where? Casting should be handled correctly there

That line hasn't been touched since #17744, will look

[jbrockmendel]

Now that i have an actual keyboard: this would likely also require fixing Block.where #38073

[arw2019]

Now that i have an actual keyboard: this would likely also require fixing Block.where #38073

Ok - in this case this would sit on top of that

I'm struggling a little with the pattern. Atm I have:

    def combine_first(self, other: DataFrame) -> DataFrame:
        def combiner(x, y):
            mask = extract_array(isna(x))

            x_values = extract_array(x, extract_numpy=True)
            y_values = extract_array(y, extract_numpy=True)

            # If the column y in other DataFrame is not in first DataFrame,
            # just return y_values.
            if y.name not in self.columns:
                return y_values

            _where = self._mgr.where(y_values, mask, align=True, errors="raise", try_cast=True, axis=1)
            # extract array from _where

        return self.combine(other, combiner, overwrite=False)

Is this in the right ballpark?

[jreback]

Now that i have an actual keyboard: this would likely also require fixing Block.where #38073

@jbrockmendel is the current patch not good for 1.2? (agreed could use some refactoring).

[jreback]

can add an Interval example here as well

[arw2019]

Done

[jbrockmendel]

is the current patch not good for 1.2? (agreed could use some refactoring).

This looks good for now

[jreback]

@arw2019 ok small test comment. ping on green.

[jreback]

thanks @arw2019

[jbrockmendel]

@arw2019 do you have time to try to make this go through self._mgr.where? (no hurry)

[arw2019]

@arw2019 do you have time to try to make this go through self._mgr.where? (no hurry)

yes, in the works