UA production parser down

[tmrow-bot]

Description

This is an automatic error report generated for Ukraine (UA).

Issues:

• No recent data found for consumption parser
• No recent data found for consumptionForecast parser
• No recent data found for generationForecast parser
• No recent data found for price parser
• No recent data found for production parser
• No recent data found for productionPerModeForecast parser

Suggestions

• Try running the parser locally using the command poetry run test_parser UA production
• Explore the runtime logs

You can see an overview of all parser issues here.

[jarek]

I'm getting an "Access denied Error code 1020" error from Cloudflare attempting to load https://ua.energy/ from Canada, I guess it got geolocked?

[VIKTORVAV99]

Hmm, it's loading just fine for me in Sweden.
Perhaps we could use a proxy in Sweden (or another EU/Europe country?) to get it back up?

[ajoga]

I'm getting an "Access denied Error code 1020" error from Cloudflare attempting to load https://ua.energy/ from Canada, I guess it got geolocked?

Working well in France too

[madsnedergaard]

Seems like it's consumption data that is failing (in the ENTSO-E parser):

[Darkvyl]

Hey guys, it looks like we can fix a UA.py parser

if we use http.client instead of requests and mock User-Agent to "PostmanRuntime/7.32.3". I haven't got time to set up my environment and create proper PR to fix this bug, but I'll get to it today in the evening (probably). But, if someone wants to fix it earlier, it looks like this

    conn = http.client.HTTPSConnection("ua.energy")
    payload = f"action=get_data_oes&report_date={today}&type=day"
    headers = {
        "Content-Type": "application/x-www-form-urlencoded",
        "User-Agent": "PostmanRuntime/7.32.3",
    }
    conn.request("POST", "/wp-admin/admin-ajax.php", payload, headers)
    res = conn.getresponse()
    response = json.loads(res.read().decode("utf-8"))

    for serie in response:

instead of

    url = "https://ua.energy/wp-admin/admin-ajax.php"
    postdata = {"action": "get_data_oes", "report_date": today, "type": "day"}

    response = r.post(
        url, postdata, headers={"User-Agent": "electricitymap-parser/1.0"}
    )

    for serie in response.json():

BTW, this is my first experience with open source, if I do something wrong, please let me know.

[Darkvyl]

Hi all, I see in logs that parser is still crashing. From where electricityMaps server is trying to connect? Maybe we should set up some proxy for that parser?

[VIKTORVAV99]

Hi all, I see in logs that parser is still crashing. From where electricityMaps server is trying to connect? Maybe we should set up some proxy for that parser?

We are already on it ;)

[madsnedergaard]

I've been looking a bit into this yesterday, but our server is calling from "europe-west1". So the EU proxy we could use wouldn't change anything.

When I ssh into the server and curl, I hit the CloudFlare protection page:

[shuuji3]

I just tried showing the response (the same URL as above) on the browser:

tmp.html.zip

One option might be to ask the website owner to whitelist the static IP from the backend server, though I'm not sure that will be allowed to access wp-admin. 😅

[NeoUKR]

In connection with a targeted attack on the energy infrastructure of the Russian Federation. All data was closed until the end of the war.

https://translate.google.com/translate?sl=uk&tl=en&hl=en&u=https://www.nerc.gov.ua/acts/pro-vnesennya-zmin-do-postanovi-nacionalnoyi-komisiyi-shcho-zdijsnyuye-derzhavne-regulyuvannya-u-sferah-energetiki-ta-komunalnih-poslug-vid-26-bereznya-2022-roku-349&client=webapp