-
Notifications
You must be signed in to change notification settings - Fork 370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a simple local file cache for img_tiles. #795
Closed
Closed
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -34,6 +34,7 @@ | |
import numpy as np | ||
import six | ||
import warnings | ||
import pickle | ||
|
||
import cartopy.crs as ccrs | ||
|
||
|
@@ -47,7 +48,8 @@ class GoogleTiles(object): | |
""" | ||
def __init__(self, desired_tile_form='RGB', style="street", | ||
url=('https://mts0.google.com/vt/lyrs={style}' | ||
'@177000000&hl=en&src=api&x={x}&y={y}&z={z}&s=G')): | ||
'@177000000&hl=en&src=api&x={x}&y={y}&z={z}&s=G'), | ||
cache=None): | ||
""" | ||
:param desired_tile_form: | ||
:param style: The style for the Google Maps tiles. One of 'street', | ||
|
@@ -57,6 +59,7 @@ def __init__(self, desired_tile_form='RGB', style="street", | |
{y}, and {z}. Such as: | ||
('https://server.arcgisonline.com/ArcGIS/rest/services/' | ||
'World_Shaded_Relief/MapServer/tile/{z}/{y}/{x}.jpg') | ||
:param cache: Implementation of TileCache to use as a local cache. | ||
""" | ||
# Only streets are partly transparent tiles that can be overlayed over | ||
# the satellite map to create the known hybrid style from google. | ||
|
@@ -80,6 +83,7 @@ def __init__(self, desired_tile_form='RGB', style="street", | |
self.imgs = [] | ||
self.crs = ccrs.Mercator.GOOGLE | ||
self.desired_tile_form = desired_tile_form | ||
self.cache = cache | ||
|
||
def image_for_domain(self, target_domain, target_z): | ||
tiles = [] | ||
|
@@ -188,7 +192,7 @@ def _image_url(self, tile): | |
z=tile[2], Z=tile[2]) | ||
return url | ||
|
||
def get_image(self, tile): | ||
def _get_image(self, tile): | ||
if six.PY3: | ||
from urllib.request import urlopen | ||
else: | ||
|
@@ -205,6 +209,17 @@ def get_image(self, tile): | |
|
||
return img, self.tileextent(tile), 'lower' | ||
|
||
def get_image(self, tile): | ||
if self.cache is None: | ||
return self._get_image(tile) | ||
else: | ||
url = self._image_url(tile) | ||
img = self.cache.get(url) | ||
if img is None: | ||
img, _, _ = self._get_image(tile) | ||
self.cache.put(url, img) | ||
return img, self.tileextent(tile), 'lower' | ||
|
||
|
||
class MapQuestOSM(GoogleTiles): | ||
# http://developer.mapquest.com/web/products/open/map for terms of use | ||
|
@@ -393,6 +408,61 @@ def find_images(self, target_domain, target_z, start_tile=None): | |
yield self.tms_to_quadkey(tile, google=True) | ||
|
||
|
||
class TileCache(object): | ||
"""Simple caching API""" | ||
|
||
def put(self, url, img): | ||
""" | ||
:param url: key used in cache | ||
:param img: value used in cache | ||
""" | ||
pass | ||
|
||
def get(self, url): | ||
""" | ||
:param url: key to lookup in cache | ||
""" | ||
pass | ||
|
||
def close(self): | ||
pass | ||
|
||
|
||
class ShelveTileCache(TileCache): | ||
def __init__(self, cache_name=None): | ||
""" | ||
Cache implementation using shelve. | ||
|
||
If appdirs is available, the proper user_cache_dir for the system will | ||
be used, otherwise the current working directory will be used. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not sure that's the best place; there's a |
||
|
||
:param cache_name: file name for cache. Defaults to 'img_tiles' | ||
""" | ||
import os | ||
import shelve | ||
try: | ||
import appdirs | ||
cachedir = appdirs.user_cache_dir("cartopy", "SciTools") | ||
if not os.path.exists(cachedir): | ||
os.makedirs(cachedir) | ||
except: | ||
cachedir = '.' | ||
self.cache = shelve.open(os.path.join(cachedir, 'img_tiles'), | ||
protocol=pickle.HIGHEST_PROTOCOL) | ||
|
||
def put(self, url, img): | ||
self.cache[url] = img | ||
|
||
def get(self, url): | ||
if url in self.cache: | ||
return self.cache[url] | ||
else: | ||
return None | ||
|
||
def close(self): | ||
self.cache.close() | ||
|
||
|
||
def _merge_tiles(tiles): | ||
"""Return a single image, merging the given images.""" | ||
if not tiles: | ||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given how easy duck typing is in Python, I would eliminate the
TileCache
base classI would also say we should just implement
__getitem__
and__setitem__
to make the interface more Pythonic. Or at least that's my inclination.