Dev merge to master, PR (#3146)

* Adding plugin support (#2679)

* Adding plugin support

* Adding an empty __init__.py

* Moving the base task to the project root (#2702)

* Moving the base task to the project root

* Moving the base class more

* Changing the import again

* Adding a heartbeat to the analytics (#2709)

* Adding a heartbeat to the analytics

* Heartbeat every 30 seconds, not every 5

* Don't double track clients

* Fix 'local variable 'bot' referenced before assignment'

* Providing an error if tasks don't work for the given api (#2732)

* Fix for utf8 encoding when catching lured pokemon (#2720)

* Fixing lure pokestop encoding

* fixing lure encoding

* Fix For catchable not being displayed on the web (#2719)

* Fix For catchable not being displayed on the web

* Update catch_visible_pokemon.py

* Added encrypt.so compilation process to Dockerfile (#2695)

* OS Detection for encrypt lib (#2768)

Fix 32bit check, darwin and linux use the same file

Make it a function

Check if file exists, if not show error

Define file_name first

Fix return

Check if file exists, if not show error

Print info about paths

Fix for 32/64bit detection

* Fix Typo in unexpected_response_retry (#2531)

fixes #2525 #2523

* Revert "changing license from MIT to GPLv3"

This reverts commit 69fb64f.

* When the google analytics domain is blocked the bot crashed. (#2764)

With a simple try / except this can be solved.

Fix dirty catch all

* Fixes #2698 - Prevents "Possibly searching too often" error after re-login. (#2771)

* Fixes #2698
- Added api.activate_signature call to prevent issue after re-login.
- Also replaced deprecated log call with event_manager emit to prevent exception being thrown.

* Modified to use OS detected library path as per PR #2768

* Support loading plugins from .zip files (#2766)

* Keep track of how many pokemon released (#2884)

* Setting Library path to work with encrypt.so (#2899)

Setting LD_LIBRARY_PATH on Dockerfile

* ✨ Added login and username to available stats (#2494)

Added a player_data property in PokemonGoBot to access player data from outside
Added unit tests for login and username stats
Added tests for call args when updating the window title
Added a platform-specific test for window title updating on win32 platform

* [dev] small fixes (#2912)

* Fixed emit_event typo

* Update CONTRIBUTORS.md

* Changed initialization location for "bot"

We use bot in main exception on 128

* Update pokecli.py

* Rename load_path to load_plugin (#2947)

* Adding some logic for pulling plugins from github (#2967)

* flush after title update (#2977)

* correctly re-raise exception to keep backtrace (#2944)

* Update MoveToMapPokemon to use events instead of logger. (#2913)

* Config/encrypt.so (#2964)

* Add config option for libencrypt.so

* Correctly set the config value and check for the file in said dir

* Fixed mispelling for "formatted" variable (#2984)

* Loading plugins from Github (#2992)

* Checking github plugin file existence

* Loading plugins from github

* Fixed #3000 (#3003)

Fixed syntax error on "move_to_map_pokemon.py" that makes the client crash when using this feature.

* Added MaxPotion inventory count to summary. (#3015)

Short Description: 
The Max Potion count was missing from the inventory summary.

Was #2456

* Added cleanup of download and files for encrypt.so after they are no longer needed (#3011)

* Fix bot not returning back after telepoting (#3014)

* Fix typo: last_long -> last_lon

* Whitespace cleanup

* Fix bug introduced by #3037: bot not returning back

* Fix Dockerfile installation (#3057)

* Fix for #3045 (#3055)

* Added request to check configuration (#3089)

* Fixed Dockerfile - missing \ on command lines (#3096)

* Fixed mispelling for "formatted" variable

* Docker commands missing trailing \

* Fix for FileIO slowing bot performance.This puts the map writing into a thread and makes sure it only executes  once. (#3100)

* Change word usage: "fled" to "escaped" (#3118)

"fled" is confusing to lot of people and is easily confused with pokemon vanishing. "escaped" is a better term.

* Update the example config file (#3120)

* Add config option for libencrypt.so

* Add config option for libencrypt.so

* Add config option for libencrypt.so

* Add config option for libencrypt.so

* Rename path.example.json to path.json.example

* typo: logrmation -> information (#2601)

Fix a typo. 

I assume that it was "information" initially, but became "logrmation" when someone used replace all functionality to replace all infos with logs. But I might be totally wrong at this point, idk. Just didn't like the word and wanted to fix that typo.

* Change fled to escaped (#3129)

Fix an issue after PR #3118

* When JSON parsing fails, give a rough indication of why (#3137)

* When JSON parsing fails, give a rough indication of why

* Use the official package instead of SHA1 commit

* Handle Github Download Zip Format (#3108)

* Checking github plugin file existence

* Loading plugins from github

* Starting install code for github plugins

* Updating GithubPlugin to support extracting folders

* Handling github zip formats by extracting to the correct location

.github/ISSUE_TEMPLATE.md

@@ -1,3 +1,5 @@
+Please check configuration at http://jsonlint.com/ before posting an issue.
+
 ### Expected Behavior
 
 

.gitignore

@@ -35,6 +35,9 @@ var/
 .pydevproject
 .settings/
 
+# Cloud9 Users
+.c9/
+
 # Installer logs
 pip-log.txt
 pip-delete-this-directory.txt
@@ -86,6 +89,8 @@ celerybeat-schedule
 # virtualenv
 venv/
 ENV/
+local/
+share/
 
 # Spyder project settings
 .spyderproject

CONTRIBUTORS.md

@@ -53,3 +53,4 @@
  * lucasfevi
  * Moonlight-Angel
  * mjmadsen
+ * nikofil

Dockerfile

@@ -11,7 +11,9 @@ RUN cd /tmp && wget "http://pgoapi.com/pgoencrypt.tar.gz" \
     && tar zxvf pgoencrypt.tar.gz \
     && cd pgoencrypt/src \
     && make \
-    && cp libencrypt.so /usr/src/app/encrypt.so
+    && cp libencrypt.so /usr/src/app/encrypt.so \
+    && cd /tmp \
+    && rm -rf /tmp/pgoencrypt*
 
 VOLUME ["/usr/src/app/web"]
 

configs/config.json.cluster.example

@@ -4,6 +4,7 @@
     "password": "YOUR_PASSWORD",
     "location": "SOME_LOCATION",
     "gmapkey": "GOOGLE_MAPS_API_KEY",
+    "libencrypt_location": "",
     "tasks": [
       {
         "type": "HandleSoftBan"

configs/config.json.example

@@ -4,6 +4,7 @@
     "password": "YOUR_PASSWORD",
     "location": "SOME_LOCATION",
     "gmapkey": "GOOGLE_MAPS_API_KEY",
+    "libencrypt_location": "",
     "tasks": [
       {
         "type": "HandleSoftBan"

configs/config.json.map.example

@@ -4,6 +4,7 @@
     "password": "YOUR_PASSWORD",
     "location": "SOME_LOCATION",
     "gmapkey": "GOOGLE_MAPS_API_KEY",
+    "libencrypt_location": "",
     "tasks": [
       {
         "type": "HandleSoftBan"

configs/config.json.path.example

@@ -4,6 +4,7 @@
     "password": "YOUR_PASSWORD",
     "location": "SOME_LOCATION",
     "gmapkey": "GOOGLE_MAPS_API_KEY",
+    "libencrypt_location": "",
     "tasks": [
       {
         "type": "HandleSoftBan"

configs/config.json.pokemon.example

@@ -4,6 +4,7 @@
     "password": "YOUR_PASSWORD",
     "location": "SOME_LOCATION",
     "gmapkey": "GOOGLE_MAPS_API_KEY",
+    "libencrypt_location": "",
     "tasks": [
       {
         "type": "HandleSoftBan"

pokecli.py

@@ -42,6 +42,12 @@
 from pokemongo_bot.health_record import BotEvent
 from pokemongo_bot.plugin_loader import PluginLoader
 
+try:
+    from demjson import jsonlint
+except ImportError:
+    # Run `pip install -r requirements.txt` to fix this
+    jsonlint = None
+
 if sys.version_info >= (2, 7, 9):
     ssl._create_default_https_context = ssl._create_unverified_context
 
@@ -53,7 +59,7 @@
 
 def main():
     bot = False
-    
+
     try:
         logger.info('PokemonGO Bot v1.0')
         sys.stdout = codecs.getwriter('utf8')(sys.stdout)
@@ -104,7 +110,7 @@ def main():
                     'api_error',
                     sender=bot,
                     level='info',
-                    formmated='Log logged in, reconnecting in {:s}'.format(wait_time)
+                    formatted='Log logged in, reconnecting in {:d}'.format(wait_time)
                 )
                 time.sleep(wait_time)
             except ServerBusyOrOfflineException:
@@ -130,7 +136,7 @@ def main():
         if bot:
             report_summary(bot)
 
-        raise e
+        raise
 
 def report_summary(bot):
     if bot.metrics.start_time is None:
@@ -162,16 +168,28 @@ def init_config():
     # If config file exists, load variables from json
     load = {}
 
+    def _json_loader(filename):
+        try:
+            with open(filename, 'rb') as data:
+                load.update(json.load(data))
+        except ValueError:
+            if jsonlint:
+                with open(filename, 'rb') as data:
+                    lint = jsonlint()
+                    rc = lint.main(['-v', filename])
+
+            logger.critical('Error with configuration file')
+            sys.exit(-1)
+
     # Select a config file code
     parser.add_argument("-cf", "--config", help="Config File to use")
     config_arg = parser.parse_known_args() and parser.parse_known_args()[0].config or None
+
     if config_arg and os.path.isfile(config_arg):
-        with open(config_arg) as data:
-            load.update(json.load(data))
+        _json_loader(config_arg)
     elif os.path.isfile(config_file):
         logger.info('No config argument specified, checking for /configs/config.json')
-        with open(config_file) as data:
-            load.update(json.load(data))
+        _json_loader(config_file)
     else:
         logger.info('Error: No /configs/config.json or specified config')
 
@@ -384,6 +402,7 @@ def init_config():
     if not config.password and 'password' not in load:
         config.password = getpass("Password: ")
 
+    config.encrypt_location = load.get('encrypt_location','')
     config.catch = load.get('catch', {})
     config.release = load.get('release', {})
     config.action_wait_max = load.get('action_wait_max', 4)
@@ -446,7 +465,7 @@ def task_configuration_error(flag_name):
 
     plugin_loader = PluginLoader()
     for plugin in config.plugins:
-        plugin_loader.load_path(plugin)
+        plugin_loader.load_plugin(plugin)
 
     # create web dir if not exists
     try:

pokemongo_bot/__init__.py

@@ -9,6 +9,8 @@
 import re
 import sys
 import time
+import Queue
+import threading
 
 from geopy.geocoders import GoogleV3
 from pgoapi import PGoApi
@@ -69,6 +71,11 @@ def __init__(self, config):
         # Make our own copy of the workers for this instance
         self.workers = []
 
+        # Theading setup for file writing
+        self.web_update_queue = Queue.Queue(maxsize=1)
+        self.web_update_thread = threading.Thread(target=self.update_web_location_worker)
+        self.web_update_thread.start()
+
     def start(self):
         self._setup_event_system()
         self._setup_logging()
@@ -249,7 +256,7 @@ def _register_events(self):
             )
         )
         self.event_manager.register_event(
-            'pokemon_fled',
+            'pokemon_escaped',
             parameters=('pokemon',)
         )
         self.event_manager.register_event(
@@ -393,6 +400,35 @@ def _register_events(self):
         )
         self.event_manager.register_event('unset_pokemon_nickname')
 
+        # Move To map pokemon
+        self.event_manager.register_event(
+            'move_to_map_pokemon_fail',
+            parameters=('message',)
+        )
+        self.event_manager.register_event(
+            'move_to_map_pokemon_updated_map',
+            parameters=('lat', 'lon')
+        )
+        self.event_manager.register_event(
+            'move_to_map_pokemon_teleport_to',
+            parameters=('poke_name', 'poke_dist', 'poke_lat', 'poke_lon',
+                        'disappears_in')
+        )
+        self.event_manager.register_event(
+            'move_to_map_pokemon_encounter',
+            parameters=('poke_name', 'poke_dist', 'poke_lat', 'poke_lon',
+                        'disappears_in')
+        )
+        self.event_manager.register_event(
+            'move_to_map_pokemon_move_towards',
+            parameters=('poke_name', 'poke_dist', 'poke_lat', 'poke_lon',
+                        'disappears_in')
+        )
+        self.event_manager.register_event(
+            'move_to_map_pokemon_teleport_back',
+            parameters=('last_lat', 'last_lon')
+        )
+
     def tick(self):
         self.health_record.heartbeat()
         self.cell = self.get_meta_cell()
@@ -607,7 +643,6 @@ def login(self):
         )
 
     def get_encryption_lib(self):
-        file_name = ''
         if _platform == "linux" or _platform == "linux2" or _platform == "darwin":
             file_name = 'encrypt.so'
         elif _platform == "Windows" or _platform == "win32":
@@ -617,15 +652,18 @@ def get_encryption_lib(self):
             else:
                 file_name = 'encrypt.dll'
 
-        path = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir))
-        full_path = path + '/'+ file_name
+        if self.config.encrypt_location == '':
+            path = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir))
+        else:
+            path = self.config.encrypt_location
 
+        full_path = path + '/'+ file_name
         if not os.path.isfile(full_path):
-            self.logger.error(file_name + ' is not found! Please place it in the bots root directory.')
-            self.logger.info('Platform: '+ _platform + ' Bot root directory: '+ path)
+            self.logger.error(file_name + ' is not found! Please place it in the bots root directory or set libencrypt_location in config.')
+            self.logger.info('Platform: '+ _platform + ' Encrypt.so directory: '+ path)
             sys.exit(1)
         else:
-            self.logger.info('Found '+ file_name +'! Platform: ' + _platform + ' Bot root directory: ' + path)
+            self.logger.info('Found '+ file_name +'! Platform: ' + _platform + ' Encrypt.so directory: ' + path)
 
         return full_path
 
@@ -716,7 +754,8 @@ def _print_character_info(self):
         self.logger.info(
             'Potion: ' + str(items_stock[101]) +
             ' | SuperPotion: ' + str(items_stock[102]) +
-            ' | HyperPotion: ' + str(items_stock[103]))
+            ' | HyperPotion: ' + str(items_stock[103]) +
+            ' | MaxPotion: ' + str(items_stock[104]))
 
         self.logger.info(
             'Incense: ' + str(items_stock[401]) +
@@ -944,7 +983,15 @@ def heartbeat(self):
         request.get_player()
         request.check_awarded_badges()
         request.call()
-        self.update_web_location()  # updates every tick
+        try:
+            self.web_update_queue.put_nowait(True)  # do this outside of thread every tick
+        except Queue.Full:
+            pass
+
+    def update_web_location_worker(self):
+        while True:
+            self.web_update_queue.get()
+            self.update_web_location()
 
     def get_inventory_count(self, what):
         response_dict = self.get_inventory()