Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bad error for gateway time out #339

Closed
araiho opened this issue Jan 19, 2024 · 3 comments
Closed

bad error for gateway time out #339

araiho opened this issue Jan 19, 2024 · 3 comments

Comments

@araiho
Copy link

araiho commented Jan 19, 2024 •
edited
Loading

For fetchSoilGrids(), I'm getting the following error when the URL https://rest.isric.org/soilgrids/v2.0/properties/ can't be accessed which could be replaced with a better error message about the URL not working.

Error: lexical error: invalid char in json text.
<title>504 Gatewa
(right here) ------^

Is there a backup for when this website is down?
@brownag
Copy link
Member

brownag commented Jan 19, 2024

For fetchSoilGrids(), I'm getting the following error when the URL https://rest.isric.org/soilgrids/v2.0/properties/ can't be accessed which could be replaced with a better error message about the URL not working.

Error: lexical error: invalid char in json text. <title>504 Gatewa (right here) ------^

Thanks for reporting this, I can adjust the function so that when this type of non-JSON HTML error response is received the error is properly trapped rather than passed on for parsing.

Unfortunately its really common for total failures of APIs like this to regress to a non-JSON response, hence the lexical error. Normal error messages issued by a functioning API are already handled.

Is there a backup for when this website is down?

Unfortunately there is no complete alternative to the REST API endpoint. Please contact ISRIC as they are the ones responsible for the endpoint (and the response when it fails)

From the SoilGrids FAQ https://www.isric.org/explore/soilgrids/faq-soilgrids

NOTE: We are closing down an old domain "soilgrids.isric.org". Please make sure that you use the current domain "soilgrids.org"

It is not clear at this time whether the above notice is indicating that the REST API has moved (or will move). A couple years ago the API was accessible via soilgrids.org and they changed to the current rest.isric.org URL.

There have been significant periods of downtime in past years, not much anyone but ISRIC can do about it. Note also that there is a "Fair Use" limit of ~5 requests per minute... but this 504 error is unrelated to that. The entire rest.isric.org site appears to return the same 504 error.

I will provide an option in fetchSoilGrids() to specify an alternate endpoint if someone was interested in mirroring the REST service elsewhere; but providing an alternate source is out of scope for this R package, in general.

In the meantime you could consider using the grid=TRUE argument to make requests directly to the cloud optimized geotiffs found here https://files.isric.org/soilgrids/latest/data/. I recognize this is not a direct replacement for the point-based queries to the REST API, as it returns grids covering an area, rather than a point. Also even for small areas it will take some time to get the information for all properties analogous the point query. To use this approach you would need to buffer an area around your point(s) of interest, download the grids, then extract the values at target point locations.

brownag added a commit that referenced this issue Jan 19, 2024
@brownag
for #339
0a7cc90
@brownag
Copy link
Member

brownag commented Jan 19, 2024

Thanks again for reporting, after 0a7cc90 I get:

library(soilDB)
your.points <- data.frame(id  = c("A", "B"), 
                          lat = c(37.9, 38.1), 
                          lon = c(-120.3, -121.5), 
                          stringsAsFactors = FALSE)
x <- fetchSoilGrids(your.points, verbose=TRUE)
#> Error : lexical error: invalid char in json text.
#>                                        <html>  <head><title>504 Gatewa
#>                      (right here) ------^
#> 
#> Error : lexical error: invalid char in json text.
#>                                        <html>  <head><title>504 Gatewa
#>                      (right here) ------^
#> 
#> Error : SoilGrids API is not accessible
inherits(x, 'try-error')
#> [1] TRUE

@brownag brownag closed this as completed Jan 19, 2024
@araiho
Copy link
Author

araiho commented Jan 22, 2024

Cool! Thanks for adding that and for the detailed response!

brownag added a commit that referenced this issue Jan 23, 2024
@brownag
for #339 (2)
ce73459
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@araiho @brownag