-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Regulatory calculations do not work with latest pandas version #153
Comments
I forgot to mention. A quick fix until this is resolved is to downgrade your pandas version. I tested it and it works well. So while your conda environment is active type:
|
Seems related to this stuff pandas-dev/pandas#47079. That is, it seems that the previous behavior of retaining Maybe good fix would be to add a new temporary local time column which can be safely dropped after using, like suggested here - df_rolling_max = df_rolling.groupby("siteid").resample("D", on="time_local").max(min_count=18).reset_index(drop=True).dropna()
+ df_rolling["key"] = df["time_local"]
+ df_rolling_max = df_rolling.groupby("siteid").resample("D", on="key").max(min_count=18).reset_index(drop=True).dropna() That way |
The fix for this has been merged to develop, I will close this once we merge develop to main. |
This fix is now in the main branch. |
The regulatory calculations error with the latest pandas version. The error is
KeyError: 'time_local'
in this code:
File /scratch2/BMC/rcm1/rhs/miniconda3/envs/mm_dev_all/lib/python3.9/site-packages/melodies_monet/plots/surfplots.py:74, in calc_8hr_rolling_max_v1(df, col, window)
72 df_rolling_max = df_rolling.groupby("siteid").resample("D", on="time_local").max(min_count=18).reset_index(drop=True).dropna()
73 df = df.reset_index(drop=True)
---> 74 return df.merge(df_rolling_max, on=["siteid", "time_local"])
@mlirenzhenmayi I think this looks similar to the issue you brought up awhile ago and maybe we just didn't notice the problem because Hera only just allowed the latest version of Pandas on it and maybe you were testing this on a different system like Meiyu? @jianheACM, do you have time next week to help fix this problem, since you know this regulatory code the best? Meng's suggestion awhile back may work well, but we should probably test that it doesn't change the answer and works for all options. Thanks!
The text was updated successfully, but these errors were encountered: