Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding PVGIS horizon data retrieval method #1395

Merged
merged 102 commits into from
May 25, 2023
Merged

Conversation

bgpierc
Copy link
Contributor

@bgpierc bgpierc commented Jan 26, 2022

This PR adds a function to fetch a horizon profile from PVGIS.

In this request, I've added two main functions: one in a new module horizon.py that, given an SRTM DEM, calculates a horizon profile in terms of (azimuth, elevation) manually, and an addition in iotools that fetches the same from pvgis (#1295 ). I also incorporated a few functions from abandoned #758. I've tested that they're compatible, but have not verified the results.

Along with #1295, this gives pvlib 3 potential ways of retrieving the horizon profile. I'm going to be testing them against each other and collected field data like @mikofski suggested in #1295 . I'll add more functionality as I go along but I wanted to check in with @cwhanse and @jsstein with what we have now. I'm currently working on retrieving raw SRTM data automatically, as well.

@cwhanse
Copy link
Member

cwhanse commented Jan 28, 2022

@bgpierc before we start polishing in earnest, could you clear up the stickler items? Makes it much easier to review. My first reactions are 1) that many of the utility functions in horizon.py would be better as private, and 2) several added dependencies should be optional.

@bgpierc
Copy link
Contributor Author

bgpierc commented Jan 31, 2022

@cwhanse fixed stickler errors, and added underscores for private functions. How should I specify optional dependencies?

@kandersolar
Copy link
Member

Thanks @bgpierc!

How should I specify optional dependencies?

Add an entry for each new dependency here: https://github.com/pvlib/pvlib-python/blob/master/setup.py#L56-L58

And then, rather than importing the optional dependencies at the top of the module like normal (which, when the new functions are eventually added to an __init__.py somewhere, will cause import pvlib to fail if the optional deps are not available), import them inside the functions that use them like this: https://github.com/pvlib/pvlib-python/blob/master/pvlib/bifacial.py#L95

@bgpierc
Copy link
Contributor Author

bgpierc commented Feb 1, 2022

Perfect, I added new dependencies to the optional section. I also changed the horizon_map function to accept an array for the DEM rather then a path, as there's a few different file types (GeoTiff, ascii, .hgt) that vary depending on where you get the data from. I'm currently working on an automated download function for DEMs, so I was meaning to change that anyway.

@kandersolar kandersolar mentioned this pull request Feb 2, 2022
4 tasks
pvlib/iotools/pvgis.py Outdated Show resolved Hide resolved
Copy link
Member

@cwhanse cwhanse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bgpierc mostly docstring stuff as I am getting familiar with the new code.

I support adding the DEM functions to a new module horizon.py. Perhaps more of the DEM functions could become private, unless you think that a user will need them to work with DEM files in their code.

I think the three public functions from JPalakapillyKWH's PR should go to shading.py.

The rest of pvlib users latitude and longitude so it would be better use these terms here, required for public functions but not for private functions.

pvlib/horizon.py Outdated Show resolved Hide resolved
pvlib/horizon.py Outdated
Comment on lines 23 to 24
Tuple of np.array of latitude,longitude
corresponding to the pixels in the GeoTiff
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add the unit and sign convention? I assume latitude, longitude are degrees.decimal degrees and negative longitude is west of the prime meridian. This info may be better in a Notes section than in the description of the output.

Suggested change
Tuple of np.array of latitude,longitude
corresponding to the pixels in the GeoTiff
Tuple (latitude, longitude) corresponding to the pixels in
the GeoTiff. Each of latitude and longitude is a numpy.array.
Notes
------
Latitude and longitude are...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, your assumption is correct, I added it to be explicit.

pvlib/horizon.py Outdated Show resolved Hide resolved
pvlib/horizon.py Outdated Show resolved Hide resolved
pvlib/horizon.py Outdated Show resolved Hide resolved
pvlib/horizon.py Outdated Show resolved Hide resolved
pvlib/horizon.py Outdated Show resolved Hide resolved
pvlib/horizon.py Outdated

"""
from skimage.draw import line
azimuth = np.linspace(0, 360, az_len)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, azimuth will include both 0 and 360. I haven't looked closely below to see if the duplicate is removed. If it isn't, I'd drop the last value 360 here before moving forward.

Suggested change
azimuth = np.linspace(0, 360, az_len)
azimuth = np.linspace(0, 360, num=az_len)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, off-by-one. Good catch. Changed to be in range [0,360)

pvlib/horizon.py Outdated Show resolved Hide resolved
pvlib/horizon.py Outdated Show resolved Hide resolved
bgpierc and others added 15 commits February 7, 2022 09:27
Co-authored-by: Karel De Brabandere <kdebrab@users.noreply.github.com>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
Co-authored-by: Cliff Hansen <cwhanse@sandia.gov>
@AdamRJensen AdamRJensen added the remote-data triggers --remote-data pytests label May 9, 2023
@AdamRJensen AdamRJensen added remote-data triggers --remote-data pytests and removed remote-data triggers --remote-data pytests labels May 9, 2023
pvlib/iotools/pvgis.py Outdated Show resolved Hide resolved
pvlib/iotools/pvgis.py Outdated Show resolved Hide resolved
@kandersolar kandersolar modified the milestones: 0.9.6, 0.10.0 May 16, 2023
@AdamRJensen
Copy link
Member

Given that this is an iotools function, it could be considered following the convention of returning a tuple of (data, meta). For this function, the meta object would just be an empty dictionary.

If we can imagine that other horizon fetching functions could return metadata then I would vote for doing this.

@kandersolar
Copy link
Member

For this function, the meta object would just be an empty dictionary.

Well, the PVGIS API does actually return metadata, we just aren't doing anything with it right now. I'm in favor of having this function follow the data, metadata convention and passing through the metadata returned by the API. Good idea!

@bgpierc
Copy link
Contributor Author

bgpierc commented May 23, 2023

Ok, I had it return the meta data from PVGIS as well. I'm not sure if we want to make the function return a Series rather the a DataFrame as the other pvgis functions all return a DataFrame. Also, I think having the horizon_azimuth be explicit as the index column is helpful for clarity.

@AdamRJensen AdamRJensen changed the title Adding horizon data retrieval methods Adding PVGIS horizon data retrieval method May 24, 2023
@kandersolar kandersolar added remote-data triggers --remote-data pytests and removed remote-data triggers --remote-data pytests labels May 25, 2023
Copy link
Member

@kandersolar kandersolar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me! Thanks @bgpierc, @AdamRJensen, and the long list of other contributors here :)

CI failure is an unrelated codecov upload issue and fine to ignore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement io remote-data triggers --remote-data pytests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants