Merge pull request #39 from weareferal/develop

Large backups & volume backend improvements

CHANGELOG.md

@@ -72,3 +72,19 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/) and this p
 ### Changed
 
 - Updated core library version
+
+## 1.4.0 - 2020-12-08
+
+### Added
+
+- Added support for remote volumes
+- Added TTR to queue jobs (issue #38)
+- Added time and duration to console command output
+
+### Changed
+
+- Bumped version number for parity between sync & backup plugins
+- Updated readme to call-out cron requirement
+- Fixed filename regex (issue #26 on craft-remote-sync)
+- Moved shared utilities JS and CSS to core module
+- Updated the formatting for file table (issue #10 on craft-remote-backup)

README.md

@@ -2,7 +2,7 @@
 
 <img src="src/icon.svg" width="125px">
 
-Sync your database and assets across Craft environments using a remote destinations (AWS, Digital Ocean, Google Drive, Backblaze, Dropbox)
+Sync your database and assets across Craft environments via a remote destinations (AWS, Digital Ocean, Google Drive, Backblaze, Dropbox).
 
 ## Overview
 
@@ -44,21 +44,21 @@ To install the plugin, follow these instructions.
 
 ## Configuration
 
-Remote Sync supports a number of destinations to push files to. Each provider has slightly different configuration so please follow the links below to get a short guide for configuring your preferred provider.
+Remote Sync supports a number of cloud providers. Each provider has slightly different configuration so please follow the links below to get a short guide for configuring your preferred provider.
 
-Bear in mind that you will need to additionally install the relevent SDK for your provider.
+Bear in mind that you will need to additionally install the relevant SDK for your provider.
 
 - [Amazon S3](https://github.com/weareferal/craft-remote-backup/wiki/Amazon-S3)
 - [Backblaze B2](https://github.com/weareferal/craft-remote-backup/wiki/Backblaze-B2)
 - [Dropbox](https://github.com/weareferal/craft-remote-backup/wiki/Dropbox)
 - [Google Drive](https://github.com/weareferal/craft-remote-backup/wiki/Google-Drive)
 - [Digital Ocean Spaces](https://github.com/weareferal/craft-remote-backup/wiki/Digital-Ocean-Spaces)
 
-In each case you will be required to configure the plugin via the Contorl Panel settings page and optional (but recommended) environment variables.
+In each case you will be required to configure the plugin via the Control Panel settings page and optional (but recommended) environment variables.
 
 ## Usage
 
-### Control Panel Utilties
+### Control Panel Utilities
 
 ![Craft Remote Sync Utilities Screenshot](resources/img/utilities-screenshot.png)
 
@@ -96,17 +96,17 @@ There are also console commands available for creating, pushing and pulling back
 
 You can optionally use Craft's built-in queue to sync files. This is useful when they are large and you don't want to have to wait on the Control Panel interface every time you backup. Instead, the files will be added to the queue and completed in the background.
 
-You can enable this via the "Use Queue" lightswitch in the settings or via the `useQueue` setting in your config.
+You can enable this via the "Use Queue" light-switch in the settings or via the `useQueue` setting in your config.
 
 #### ⚠️ CLI commands
 
-The CLI commands ignore the queue setting. In other words, they will always run synchronously. This is by design as it's likely you will want to see the results of these operations if they are part of your crontab or deployment script.
+The CLI commands ignore the queue setting. In other words, they will always run synchronously. This is by design as it's likely you will want to see the results of these operations if they are part of your crontab or deployment script. 
 
 ### Emergency Backup
 
 When enabled, a local copy of both current database or volumes will be created and saved *before* pull & restoring a remote version.
 
-This is a last resort in case you accidently pull and restore something you didn't mean to. Bear in mind that only 1 emergency backup is kept:
+This is a last resort in case you accidentally pull and restore something you didn't mean to. Bear in mind that only 1 emergency backup is kept:
 
 - `storage/sync/emergency-backup.sql` (database)
 - `storage/sync/emergency-backup.zip` (volumes)
@@ -176,6 +176,26 @@ If you are getting errors, the first thing to check is the Craft logs at `storag
 
 When are syncing volumes, it's possible that your PHP memory limit will cause the process to crash. Make sure your memory limit is > than the volume folder you are trying to backup.
 
+### Large Files
+
+If you have a lot of volume files or a large database to back up, there are a number of potential issues you may face.
+
+#### Storage Issues
+
+All data from your volumes is copied to the local server before being sent to the remote destination. It is then zipped and sent to the remote destination. This means that for every 1Gb you want to backup, you need 2Gb extra space.
+
+#### Queue Operations Failing
+
+If you are backing up via the web interface and using the queue, you might notice backup operations failing (sometimes silently)
+
+##### TTR (Time to Reserve)
+
+TTR ("Time To Reserve") is the max time (in seconds) that a queue operation can take. This is a Craft setting and not a PHP setting, so it's not affected by the `max_execution` time.
+
+Remote Backup offers an easy way to adjust this value via the settings page, so if you are noticing issues backing-up then increase this value from the default 300 (5 minutes).
+
+![TTR setting](resources/img/ttr.png)
+
 ## Support
 
 <img src="resources/img/feral-logo.svg" width="250px">

composer.json

@@ -2,6 +2,7 @@
     "name": "weareferal/remote-sync",
     "description": "Sync your database and assets across Craft environments",
     "type": "craft-plugin",
+    "version": "1.4.0",
     "keywords": [
         "craft",
         "cms",
@@ -22,7 +23,7 @@
     ],
     "require": {
         "craftcms/cms": "^3.0.0-RC1",
-        "weareferal/remote-core": "^1.1.5"
+        "weareferal/remote-core": "1.4.0"
     },
     "autoload": {
         "psr-4": {

src/assets/RemoteSyncUtility/RemoteSyncUtilityAsset.php

@@ -20,10 +20,6 @@ public function init()
             'js/RemoteSyncUtility.js'
         ];
 
-        $this->css = [
-            'css/RemoteSyncUtility.css',
-        ];
-
         parent::init();
     }
 }