Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding PVGIS horizon data retrieval method #1395

Merged
merged 102 commits into from
May 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
539bc5c
added horizon model first draft
Jan 26, 2022
85754b5
added get horizon function
Jan 26, 2022
4985e9c
fixing stickler items & updates
Jan 31, 2022
132ab4e
stickler whitespace
Jan 31, 2022
64bbc97
finishing up stickler
Jan 31, 2022
19e1073
finishing up stickler, 2
Jan 31, 2022
d563a81
added optional deps, changed argument of horizon_map from path to array
Feb 1, 2022
e9856da
stickler
Feb 1, 2022
c392e29
Update pvlib/iotools/pvgis.py
bgpierc Feb 7, 2022
e8d6100
Update pvlib/horizon.py
bgpierc Feb 7, 2022
3dca8bb
Update pvlib/horizon.py
bgpierc Feb 7, 2022
93d914d
Update pvlib/horizon.py
bgpierc Feb 7, 2022
5c1bc2d
Update pvlib/horizon.py
bgpierc Feb 7, 2022
eb8f8c5
Update pvlib/horizon.py
bgpierc Feb 7, 2022
5a3c9cc
Update pvlib/horizon.py
bgpierc Feb 7, 2022
515ed85
Update pvlib/horizon.py
bgpierc Feb 7, 2022
1c3a398
Update pvlib/horizon.py
bgpierc Feb 7, 2022
a191d2d
Update pvlib/horizon.py
bgpierc Feb 7, 2022
82ba2a5
Update pvlib/horizon.py
bgpierc Feb 7, 2022
ca2eec3
Update pvlib/horizon.py
bgpierc Feb 7, 2022
996b3f9
Update pvlib/horizon.py
bgpierc Feb 7, 2022
3717186
Update pvlib/horizon.py
bgpierc Feb 7, 2022
f9a2193
Update pvlib/horizon.py
bgpierc Feb 7, 2022
989b700
updates
Feb 7, 2022
02082ee
cleaning up stickler
Feb 7, 2022
206f979
cleaning up stickler 2
Feb 7, 2022
156e964
cleaning up stickler 3
Feb 7, 2022
c674bb5
updated math for angle calculation
Feb 7, 2022
3a2cab9
removed old comment
Feb 7, 2022
78f6c1d
updated angle maximization method
Feb 8, 2022
641bf53
one of these days I'll remember stickler before pushing
Feb 8, 2022
3d310c6
changed variable names to make clear that we're working in pixels
Feb 8, 2022
c7b65b3
added module to download SRTM DEM data
Feb 9, 2022
288390b
spelling/spacing
Feb 9, 2022
e8d8795
stickler
Feb 9, 2022
a2496e5
stickler3
Feb 9, 2022
d36c508
fixed f string formatting
bgpierc Mar 4, 2022
60ba9cb
added horizon retrival, functions, dem retrival
Oct 13, 2022
7b9d528
Merge branch 'master' into add_horizon
bgpierc Oct 13, 2022
709493e
stickler
Oct 13, 2022
fa14c67
stickler2
Oct 13, 2022
97ee479
stickler3
Oct 13, 2022
43951a2
merge
Oct 13, 2022
6f9ad5e
merge2
Oct 13, 2022
8e01b57
Revert "merge2"
Oct 13, 2022
7b99b8d
fix formatting
Oct 13, 2022
9ea61e2
fix formatting, 2
Oct 13, 2022
e23d58c
removing horizon.py
Oct 31, 2022
801f4d3
tests added
Oct 31, 2022
860f484
Merge branch 'pvlib:main' into add_horizon
bgpierc Apr 24, 2023
bcca59b
updated test
Apr 24, 2023
6cfb0c4
fixed stickler
Apr 24, 2023
d5cd1af
minor formatting/stickler
Apr 24, 2023
53d9e6c
Update v0.9.6.rst
bgpierc Apr 24, 2023
c831107
Update iotools.rst
bgpierc Apr 24, 2023
17e4f92
Apply suggestions from code review
bgpierc Apr 25, 2023
c176d9b
Change assert to use np builtin
bgpierc Apr 25, 2023
0d2a061
correct docstring for hidden function
bgpierc Apr 25, 2023
6980a4d
Chnage proxy settings to passing through kwargs
bgpierc Apr 25, 2023
cf21974
Changed proxy settings to pass kwargs
bgpierc Apr 25, 2023
0fb9e05
Change assertion to use assert_series_equal from conftest
bgpierc Apr 25, 2023
7736e35
Added horizon profile contributions
bgpierc Apr 25, 2023
7f84531
Update v0.9.6.rst
bgpierc May 1, 2023
cb7eb41
Delete cgiar.py
bgpierc May 1, 2023
25baf2b
Delete test_cgiar.py
bgpierc May 1, 2023
b9edf25
Update shading.py
bgpierc May 1, 2023
d15c7ff
Update setup.py
bgpierc May 1, 2023
2bf5cb2
Update __init__.py
bgpierc May 1, 2023
89e53d9
Update iotools.rst
bgpierc May 1, 2023
e13e322
Update test_pvgis.py
bgpierc May 1, 2023
50e79cd
return only two columns and makde bytes->utf-8 implicit
bgpierc May 1, 2023
75a9407
Use JSON to streamline processing
bgpierc May 1, 2023
d860570
Update pvgis.py
bgpierc May 1, 2023
8aedbe6
added url default info
bgpierc May 1, 2023
62237e2
Merge branch 'main' into add_horizon
bgpierc May 1, 2023
bef7c81
remove np dependency
bgpierc May 1, 2023
d9689d3
Apply suggestions from code review
bgpierc May 3, 2023
302abb0
Moved ref for consistency
bgpierc May 3, 2023
4ad491b
Updated name of horizon data variable
bgpierc May 3, 2023
7901797
Add CSV for horizon tests
bgpierc May 3, 2023
af8988f
Test whole dataframe
bgpierc May 3, 2023
f58f8d7
Change coordinate system to pvlib
bgpierc May 3, 2023
a305ab6
fix test to read in csv properly
bgpierc May 3, 2023
15e5c9c
Apply suggestions from code review
bgpierc May 4, 2023
482bc23
Changed names for consistency
bgpierc May 4, 2023
94fee4c
Match column names in test dataframe
bgpierc May 4, 2023
a8c7693
Merge branch 'main' into add_horizon
bgpierc May 9, 2023
2eea384
stickler
bgpierc May 9, 2023
bf34256
Set horizon_azimuth to the index
bgpierc May 9, 2023
01893d8
Added invalid coords test
bgpierc May 9, 2023
d978b2c
update test to reflect change to index
bgpierc May 9, 2023
b693813
forgot xfail flag
bgpierc May 9, 2023
8de2506
Update test_read_pvgis_horzion_invalid_coords
AdamRJensen May 11, 2023
6425884
Merge branch 'main' into add_horizon
bgpierc May 23, 2023
568c9cd
return metadata object
bgpierc May 23, 2023
fb3d571
Update pvgis.py
bgpierc May 23, 2023
d3bc17b
fix tests for metadata update
bgpierc May 23, 2023
ec52f59
Merge branch 'main' into add_horizon
AdamRJensen May 24, 2023
a696faf
Remove index column from test file
AdamRJensen May 25, 2023
b65d58b
Convert output to pd.Series
AdamRJensen May 25, 2023
d008dab
Update tests to assert pd.Series
AdamRJensen May 25, 2023
22e0988
Remove duplicate north value (0, 360)
AdamRJensen May 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/sphinx/source/reference/iotools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ of sources and file formats relevant to solar energy modeling.
iotools.read_pvgis_tmy
iotools.get_pvgis_hourly
iotools.read_pvgis_hourly
iotools.get_pvgis_horizon
iotools.get_bsrn
iotools.read_bsrn
iotools.parse_bsrn
Expand Down
13 changes: 11 additions & 2 deletions docs/sphinx/source/whatsnew/v0.9.6.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ Deprecations

Enhancements
~~~~~~~~~~~~
* Added function to retrieve horizon data from PVGIS
:py:func:`pvlib.iotools.get_pvgis_horizon`. (:issue:`1290`, :pull:`1395`)
* Added ``map_variables`` argument to the :py:func:`pvlib.iotools.read_tmy3` in
order to offer the option of mapping column names to standard pvlib names.
(:issue:`1517`, :pull:`1623`)
Expand All @@ -25,7 +27,6 @@ Enhancements
* :py:func:`pvlib.iotools.get_psm3` now uses the new NSRDB 3.2.2 endpoint for
hourly and half-hourly single-year datasets. (:issue:`1591`, :pull:`1736`)


Bug fixes
~~~~~~~~~
* `data` can no longer be left unspecified in
Expand Down Expand Up @@ -53,9 +54,17 @@ Contributors
~~~~~~~~~~~~
* Lakshya Garg (:ghuser:`Lakshyadevelops`)
* Adam R. Jensen (:ghuser:`adamrjensen`)
* Ben Pierce (:ghuser:`bgpierc`)
* Joseph Palakapilly (:ghuser:`JPalakapillyKWH`)
* Cliff Hansen (:ghuser:`cwhanse`)
* Anton Driesse (:ghuser:`adriesse`)
* Will Holmgren (:ghuser:`wholmgren`)
* Mark Mikofski (:ghuser:`mikofski`)
* Karel De Brabandere (:ghuser:`kdebrab`)
* Josh Stein (:ghuser:`jsstein`)
* Kevin Anderson (:ghuser:`kandersolar`)
* Siddharth Kaul (:ghuser:`k10blogger`)
* Kshitiz Gupta (:ghuser:`kshitiz305`)
* Stefan de Lange (:ghuser:`langestefan`)
* :ghuser:`ooprathamm`
* Kevin Anderson (:ghuser:`kandersolar`)

49 changes: 49 additions & 0 deletions pvlib/data/test_read_pvgis_horizon.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
horizon_azimuth,horizon_elevation
0,9.9
7.5,13
15,14.5
22.5,15.7
30,14.9
37.5,15.3
45,15.7
52.5,15.7
60,13
67.5,11.5
75,11.1
82.5,11.5
90,10.3
97.5,11.5
105,10.3
112.5,9.5
120,10.7
127.5,11.8
135,11.8
142.5,8.8
150,8.4
157.5,7.3
165,5.7
172.5,5.7
180,4.6
187.5,3.4
195,0.8
202.5,0
210,0
217.5,0
225,0
232.5,0
240,0
247.5,0
255,0
262.5,0
270,0
277.5,0
285,0
292.5,0
300,0
307.5,0
315,1.1
322.5,1.9
330,3.8
337.5,5
345,6.5
352.5,9.2
1 change: 1 addition & 0 deletions pvlib/iotools/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
from pvlib.iotools.pvgis import get_pvgis_tmy, read_pvgis_tmy # noqa: F401
from pvlib.iotools.pvgis import read_pvgis_hourly # noqa: F401
from pvlib.iotools.pvgis import get_pvgis_hourly # noqa: F401
from pvlib.iotools.pvgis import get_pvgis_horizon # noqa: F401
from pvlib.iotools.bsrn import get_bsrn # noqa: F401
from pvlib.iotools.bsrn import read_bsrn # noqa: F401
from pvlib.iotools.bsrn import parse_bsrn # noqa: F401
Expand Down
54 changes: 54 additions & 0 deletions pvlib/iotools/pvgis.py
Original file line number Diff line number Diff line change
Expand Up @@ -665,3 +665,57 @@ def read_pvgis_tmy(filename, pvgis_format=None, map_variables=None):
data = data.rename(columns=VARIABLE_MAP)

return data, months_selected, inputs, meta


def get_pvgis_horizon(latitude, longitude, url=URL, **kwargs):
"""Get horizon data from PVGIS.

Parameters
bgpierc marked this conversation as resolved.
Show resolved Hide resolved
----------
latitude : float
Latitude in degrees north
longitude : float
Longitude in degrees east
url: str, default: :const:`pvlib.iotools.pvgis.URL`
Base URL for PVGIS
kwargs:
Passed to requests.get

Returns
-------
data : pd.Series
Pandas Series of the retrived horizon elevation angles. Index is the
corresponding horizon azimuth angles.
metadata : dict
Metadata returned by PVGIS.

Notes
-----
The horizon azimuths are specified clockwise from north, e.g., south=180.
This is the standard pvlib convention, although the PVGIS website specifies
south=0.

References
----------
.. [1] `PVGIS horizon profile tool
<https://ec.europa.eu/jrc/en/PVGIS/tools/horizon>`_
"""
params = {'lat': latitude, 'lon': longitude, 'outputformat': 'json'}
res = requests.get(url + 'printhorizon', params=params, **kwargs)
if not res.ok:
try:
err_msg = res.json()
kandersolar marked this conversation as resolved.
Show resolved Hide resolved
except Exception:
res.raise_for_status()
else:
raise requests.HTTPError(err_msg['message'])
json_output = res.json()
metadata = json_output['meta']
data = pd.DataFrame(json_output['outputs']['horizon_profile'])
data.columns = ['horizon_azimuth', 'horizon_elevation']
# Convert azimuth to pvlib convention (north=0, south=180)
data['horizon_azimuth'] += 180
data.set_index('horizon_azimuth', inplace=True)
data = data['horizon_elevation'] # convert to pd.Series
data = data[data.index < 360] # remove duplicate north point (0 and 360)
return data, metadata
20 changes: 19 additions & 1 deletion pvlib/tests/iotools/test_pvgis.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@
import requests
from pvlib.iotools import get_pvgis_tmy, read_pvgis_tmy
from pvlib.iotools import get_pvgis_hourly, read_pvgis_hourly
from pvlib.iotools import get_pvgis_horizon
from ..conftest import (DATA_DIR, RERUNS, RERUNS_DELAY, assert_frame_equal,
fail_on_pvlib_version)
fail_on_pvlib_version, assert_series_equal)
from pvlib._deprecation import pvlibDeprecationWarning


Expand Down Expand Up @@ -509,6 +510,23 @@ def test_get_pvgis_map_variables(pvgis_tmy_mapped_columns):
assert all([c in pvgis_tmy_mapped_columns for c in actual.columns])


@pytest.mark.remote_data
@pytest.mark.flaky(reruns=RERUNS, reruns_delay=RERUNS_DELAY)
def test_read_pvgis_horizon():
pvgis_data, _ = get_pvgis_horizon(35.171051, -106.465158)
horizon_data = pd.read_csv(DATA_DIR / 'test_read_pvgis_horizon.csv',
index_col=0)
horizon_data = horizon_data['horizon_elevation']
assert_series_equal(pvgis_data, horizon_data)


@pytest.mark.remote_data
@pytest.mark.flaky(reruns=RERUNS, reruns_delay=RERUNS_DELAY)
def test_read_pvgis_horizon_invalid_coords():
with pytest.raises(requests.HTTPError, match='lat: Incorrect value'):
_, _ = get_pvgis_horizon(100, 50) # unfeasible latitude


def test_read_pvgis_tmy_map_variables(pvgis_tmy_mapped_columns):
fn = DATA_DIR / 'tmy_45.000_8.000_2005_2016.json'
actual, _, _, _ = read_pvgis_tmy(fn, map_variables=True)
Expand Down