Merge branch 'devel'

nexcess · Jun 10, 2015 · a6e3f0c · a6e3f0c
2 parents 0a4d267 + 8b4bee2
commit a6e3f0c
Show file tree

Hide file tree

Showing 16 changed files with 165 additions and 35 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -372,3 +372,17 @@ Magento CE 1.8+ or EE 1.13+, see [these instructions](https://github.com/nexcess
   utility (@jeroenvermeulen)
   * [#536] Allow `//` (double slash) comments in VCL files and strip them when
   trimming whitespace (@eth8505)
+
+### RELEASE-0.6.2
+
+  * [#635] Fixed typos (@pborelli)  
+  * [#668] Replace short open tag (@XnS)
+  * [#721] Fix license according to SPDX (@ihor-sviziev)
+  * [#740] Do not return pipe for OPTIONS straight away (@ashsmith)
+  * [#598] Ensure POST request gets normalized data (@melvyn-sopacua)
+  * [#739] Ban CMS Page after a CMS Revision has been updated (@ashsmith)
+  * Added fix for community poll problems
+  * [#516] Fixes multiple sessions generated on first page visit (@jharrisonau)
+  * [#796] Ignore additional GET Parameters in Varnish Cache via Configuration (@thampe)
+  * [#626] Fix for dummy blocks not working (@jeroenvermeulen)
+  * [#719) Replace connect20/nexcessnet_turpentine package in composer.json (@ihor-sviziev)
diff --git a/README.md b/README.md
@@ -90,7 +90,7 @@ the page and may differ between different visitors/clients.
   requests are not blocked, and per-block TTLs in ESI policies are not honored
   (all blocks use the default TTL)
   * The core parts of Turpentine (caching and ESI/AJAX injection) work under Magento CE 1.5, but a significant
-  portion of the auxillary functionality doesn't work due to changes to event names. That
+  portion of the auxiliary functionality doesn't work due to changes to event names. That
   said, it would be possible to use Turpentine with Magento CE 1.5 with an understanding
   that it is not supported and what actions need to be taken manually. Both
   *cache flushing* (both automatic an manual) and *cache warming* (due to

diff --git a/TECHNICAL_NOTES.md b/TECHNICAL_NOTES.md
@@ -31,7 +31,7 @@ Turpentine and Magento-Varnish differ most significantly on the final step. To r
 
 ## I got 99 problems...
 
-As mentioned, Magento isn't really designed to easily facilitate the rendering of a single block, or really work with the proxy cache/ESI model which means a number of work arounds are required to make it work.
+As mentioned, Magento isn't really designed to easily facilitate the rendering of a single block, or really work with the proxy cache/ESI model which means a number of workarounds are required to make it work.
 
 ### Generating Session Cookies without PHP
 
@@ -41,7 +41,7 @@ It sounds simple enough, but Varnish doesn't actually have any easy way to gener
 
 ### Serializing Registry Data
 
-To include the registry data required for ESI block rendering in the ESI URL, Turpentine simply takes a list of the needed registry keys (provided in the layout file) for the block and serializes it using PHP's native [`serialize`](http://us3.php.net/serialize) function which turns stores the data in a string which we can then include in the URL as a simple GET parameter. However, a registry key can be associated with any data type, including whole objects. While `serialize` will typically work on objects there are some edge cases (such as XML documents) that cannot be serialized. In order to accomodate these documents, Turpentine considers some objects to be "[complex registry data](https://github.com/nexcess/magento-turpentine/blob/master/app/code/community/Nexcessnet/Turpentine/Model/Observer/Esi.php#L441)" and handles those separately from the standard PHP types like string, int, simple classes, etc. Complex registry data is considered to be Magento "models" that correspond to database records and have `getId` methods (typically things like products, categories, etc). Then Turpentine can simply serialize the model's class and ID, then load the model with the ID instead of serializing the entire object. This neatly sidesteps the whole issue of finding a way to serialize objects that are unserializable.
+To include the registry data required for ESI block rendering in the ESI URL, Turpentine simply takes a list of the needed registry keys (provided in the layout file) for the block and serializes it using PHP's native [`serialize`](http://us3.php.net/serialize) function which turns stores the data in a string which we can then include in the URL as a simple GET parameter. However, a registry key can be associated with any data type, including whole objects. While `serialize` will typically work on objects there are some edge cases (such as XML documents) that cannot be serialized. In order to accommodate these documents, Turpentine considers some objects to be "[complex registry data](https://github.com/nexcess/magento-turpentine/blob/master/app/code/community/Nexcessnet/Turpentine/Model/Observer/Esi.php#L441)" and handles those separately from the standard PHP types like string, int, simple classes, etc. Complex registry data is considered to be Magento "models" that correspond to database records and have `getId` methods (typically things like products, categories, etc). Then Turpentine can simply serialize the model's class and ID, then load the model with the ID instead of serializing the entire object. This neatly sidesteps the whole issue of finding a way to serialize objects that are unserializable.
 
 ### Runtime Events Registration and Class Rewrites
 

diff --git a/app/code/community/Nexcessnet/Turpentine/Block/Poll/ActivePoll.php b/app/code/community/Nexcessnet/Turpentine/Block/Poll/ActivePoll.php
@@ -0,0 +1,34 @@
+<?php
+
+/**
+ * Nexcess.net Turpentine Extension for Magento
+ * Copyright (C) 2012  Nexcess.net L.L.C.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+ */
+
+class Nexcessnet_Turpentine_Block_Poll_Activepoll extends Mage_Poll_Block_ActivePoll {
+
+    public function setTemplate($template)
+    {
+    	$debugHelper = Mage::helper( 'turpentine/debug' );
+    	$debugHelper->logInfo('Set poll template');
+
+        $this->_template = $template;
+        $this->setPollTemplate('turpentine/ajax.phtml', 'poll' );
+        $this->setPollTemplate('turpentine/ajax.phtml', 'results' );         
+        return $this;
+    }
+}
diff --git a/app/code/community/Nexcessnet/Turpentine/Model/Observer/Ban.php b/app/code/community/Nexcessnet/Turpentine/Model/Observer/Ban.php
@@ -217,6 +217,34 @@ public function banCmsPageCache( $eventObject ) {
         }
     }
 
+    /**
+     * Ban a specific CMS page revision from cache after edit (enterprise edition only)
+     * Events:
+     *     enterprise_cms_revision_save_commit_after
+     *
+     * @param Varien_Object $eventObject
+     * @return null
+     */
+    public function banCmsPageRevisionCache($eventObject) {
+        if ( Mage::helper( 'turpentine/varnish' )->getVarnishEnabled() ) {
+            $pageId = $eventObject->getDataObject()->getPageId();
+            $page = Mage::getModel( 'cms/page' )->load( $pageId );
+
+            // Don't do anything if the page isn't found.
+            if( !$page ) {
+                return;
+            }
+            $pageIdentifier = $page->getIdentifier();
+            $result = $this->_getVarnishAdmin()->flushUrl( $pageIdentifier . '(?:\.html?)?$' );
+            Mage::dispatchEvent( 'turpentine_ban_cms_page_cache', $result );
+            $cronHelper = Mage::helper( 'turpentine/cron' );
+            if( $this->_checkResult( $result ) &&
+                $cronHelper->getCrawlerEnabled() ) {
+                $cronHelper->addCmsPageToCrawlerQueue( $pageIdentifier );
+            }
+        }
+    }
+
     /**
      * Do a full cache flush, corresponds to "Flush Magento Cache" and
      * "Flush Cache Storage" buttons in admin > cache management

diff --git a/app/code/community/Nexcessnet/Turpentine/Model/Observer/Esi.php b/app/code/community/Nexcessnet/Turpentine/Model/Observer/Esi.php
@@ -307,7 +307,11 @@ protected function _getEsiData( $blockObject, $esiOptions ) {
             $esiData->setParentUrl( Mage::app()->getRequest()->getRequestString() );
         }
         if( is_array( $esiOptions['dummy_blocks'] ) ) {
-            $esiData->setDummyBlocks( $esiOptions['dummy_blocks'] );
+            $dummyBlocks = array();
+            foreach( $esiOptions['dummy_blocks'] as $key => $value ) {
+                $dummyBlocks[] = ( empty($value) && !is_numeric($key) ) ? $key : $value;
+            }
+            $esiData->setDummyBlocks( $dummyBlocks );
         } else {
             Mage::helper( 'turpentine/debug' )->logWarn(
                 'Invalid dummy_blocks for block: %s',

diff --git a/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php b/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php
@@ -27,7 +27,6 @@ class Nexcessnet_Turpentine_Model_Varnish_Admin {
     /**
      * Flush all Magento URLs in Varnish cache
      *
-     * @param  Nexcessnet_Turpentine_Model_Varnish_Configurator_Abstract $cfgr
      * @return bool
      */
     public function flushAll() {
@@ -37,8 +36,7 @@ public function flushAll() {
     /**
      * Flush all Magento URLs matching the given (relative) regex
      *
-     * @param  Nexcessnet_Turpentine_Model_Varnish_Configurator_Abstract $cfgr
-     * @param  string $pattern regex to match against URLs
+     * @param  string $subPattern regex to match against URLs
      * @return bool
      */
     public function flushUrl( $subPattern ) {
@@ -94,7 +92,6 @@ public function flushContentType( $contentType ) {
     /**
      * Generate and apply the config to the Varnish instances
      *
-     * @param  Nexcessnet_Turpentine_Model_Varnish_Configurator_Abstract $cfgr
      * @return bool
      */
     public function applyConfig() {

diff --git a/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Configurator/Abstract.php b/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Configurator/Abstract.php
@@ -322,6 +322,14 @@ protected function _getGetParamExcludes() {
             Mage::getStoreConfig( 'turpentine_vcl/params/get_params' ) ) );
     }
 
+    protected function _getIgnoreGetParameters()
+    {
+        /** @var Nexcessnet_Turpentine_Helper_Data $helper */
+        $helper = Mage::helper('turpentine');
+        $ignoredParameters = $helper->cleanExplode(',', Mage::getStoreConfig( 'turpentine_vcl/params/ignore_get_params'));
+        return implode( '|',  $ignoredParameters);
+    }
+
     /**
      * Get the Force Static Caching option
      *
@@ -491,9 +499,10 @@ protected function _cleanVclHelper( $line ) {
     /**
      * Format a VCL backend declaration
      *
-     * @param  string $name name of the backend
-     * @param  string $host backend host
-     * @param  string $port backend port
+     * @param  string $name    name of the backend
+     * @param  string $host    backend host
+     * @param  string $port    backend port
+     * @param  array  $options options
      * @return string
      */
     protected function _vcl_backend( $name, $host, $port, $options=array() ) {
@@ -581,7 +590,7 @@ protected function _vcl_sub_normalize_encoding() {
         } else if (req.http.Accept-Encoding ~ "deflate") {
             set req.http.Accept-Encoding = "deflate";
         } else {
-            # unkown algorithm
+            # unknown algorithm
             unset req.http.Accept-Encoding;
         }
     }
@@ -618,8 +627,10 @@ protected function _getTemplateVars() {
             'url_base_regex'    => $this->getBaseUrlPathRegex(),
             'url_excludes'  => $this->_getUrlExcludes(),
             'get_param_excludes'    => $this->_getGetParamExcludes(),
+            'get_param_ignored' => $this->_getIgnoreGetParameters(),
             'default_ttl'   => $this->_getDefaultTtl(),
             'enable_get_excludes'   => ($this->_getGetParamExcludes() ? 'true' : 'false'),
+            'enable_get_ignored' => ($this->_getIgnoreGetParameters()) ? 'true' : 'false',
             'debug_headers' => $this->_getEnableDebugHeaders(),
             'grace_period'  => $this->_getGracePeriod(),
             'force_cache_static'    => $this->_getForceCacheStatic(),

diff --git a/app/code/community/Nexcessnet/Turpentine/controllers/EsiController.php b/app/code/community/Nexcessnet/Turpentine/controllers/EsiController.php
@@ -66,7 +66,7 @@ public function getBlockAction() {
             $esiDataHmac = $req->getParam( $esiHelper->getEsiHmacParam() );
             $esiDataParamValue = $req->getParam( $esiHelper->getEsiDataParam() );
             if( $esiDataHmac !== ( $hmac = $dataHelper->getHmac( $esiDataParamValue ) ) ) {
-                $debugHelper->logWarn( 'ESI data HMAC mismatch, expected (%s) but recieved (%s)',
+                $debugHelper->logWarn( 'ESI data HMAC mismatch, expected (%s) but received (%s)',
                     $hmac, $esiDataHmac );
                 $resp->setHttpResponseCode( 500 );
                 $resp->setBody( 'ESI data is not valid' );

diff --git a/app/code/community/Nexcessnet/Turpentine/etc/config.xml b/app/code/community/Nexcessnet/Turpentine/etc/config.xml
@@ -69,6 +69,7 @@
             </urls>
             <params>
                 <get_params>__SID,XDEBUG_PROFILE</get_params>
+                <ignore_get_params>utm_source,utm_medium,utm_campaign,utm_content,utm_term,gclid,cx,ie,cof,siteurl</ignore_get_params>
             </params>
             <static>
                 <force_static>1</force_static>
@@ -116,6 +117,15 @@
                 </rewrite>
             </core>
             -->
+            <!--
+            Rewrite the poll ActivePoll block, as it uses its own custom 
+            template setting methods.
+            -->
+            <poll>
+                <rewrite>
+                    <activePoll>Nexcessnet_Turpentine_Block_Poll_ActivePoll</activePoll>
+                </rewrite>
+            </poll>
         </blocks>
         <helpers>
             <turpentine>
@@ -311,6 +321,15 @@
                     </turpentine_varnish_cms_page_save_commit_after>
                 </observers>
             </cms_page_save_commit_after>
+            <enterprise_cms_revision_save_commit_after>
+                <observers>
+                    <turpentine_varnish_enterprise_cms_revision_save_commit_after>
+                        <class>turpentine/observer_ban</class>
+                        <method>banCmsPageRevisionCache</method>
+                        <type>singleton</type>
+                    </turpentine_varnish_enterprise_cms_revision_save_commit_after>
+                </observers>
+            </enterprise_cms_revision_save_commit_after>
             <adminhtml_cache_flush_system>
                 <observers>
                     <turpentine_varnish_adminhtml_cache_flush_system>

diff --git a/app/code/community/Nexcessnet/Turpentine/etc/system.xml b/app/code/community/Nexcessnet/Turpentine/etc/system.xml
@@ -400,8 +400,17 @@
                             <sort_order>10</sort_order>
                             <show_in_default>1</show_in_default>
                             <show_in_website>0</show_in_website>
-                            <show_in_store>0</show_in_store>
+                            <show_in_store>0</show_in_store>                        
                         </get_params>
+                        <ignore_get_params translate="label" module="turpentine">
+                            <label>Ignore GET Parameters</label>
+                            <comment>Comma-separated list of GET variables that will be ignored for caching</comment>
+                            <frontend_type>textarea</frontend_type>
+                            <sort_order>20</sort_order>
+                            <show_in_default>1</show_in_default>
+                            <show_in_website>0</show_in_website>
+                            <show_in_store>0</show_in_store>
+                        </ignore_get_params>
                     </fields>
                 </params>
                 <static translate="label" module="turpentine">

diff --git a/app/code/community/Nexcessnet/Turpentine/misc/version-2.vcl b/app/code/community/Nexcessnet/Turpentine/misc/version-2.vcl
@@ -105,22 +105,26 @@ sub vcl_recv {
     set req.http.X-Opt-Force-Static-Caching = "{{force_cache_static}}";
     set req.http.X-Opt-Enable-Get-Excludes = "{{enable_get_excludes}}";
 
+    # Normalize request data before potentially sending things off to the
+    # backend. This ensures all request types get the same information, most
+    # notably POST requests getting a normalized user agent string to empower
+    # adaptive designs.
+    {{normalize_encoding}}
+    {{normalize_user_agent}}
+    {{normalize_host}}
+
     # We only deal with GET and HEAD by default
     # we test this here instead of inside the url base regex section
     # so we can disable caching for the entire site if needed
     if (req.http.X-Opt-Enable-Caching != "true" || req.http.Authorization ||
-            !(req.request ~ "^(GET|HEAD)$") ||
+            !(req.request ~ "^(GET|HEAD|OPTIONS)$") ||
             req.http.Cookie ~ "varnish_bypass={{secret_handshake}}") {
         return (pipe);
     }
 
     # remove double slashes from the URL, for higher cache hit rate
     set req.url = regsuball(req.url, "(.*)//+(.*)", "\1/\2");
 
-    {{normalize_encoding}}
-    {{normalize_user_agent}}
-    {{normalize_host}}
-
     # check if the request is for part of magento
     if (req.url ~ "{{url_base_regex}}") {
         # set this so Turpentine can see the request passed through Varnish
@@ -188,11 +192,12 @@ sub vcl_recv {
                 req.url ~ "(?:[?&](?:{{get_param_excludes}})(?=[&=]|$))") {
             return (pass);
         }
-        if (req.url ~ "[?&](utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=") {
-            # Strip out Google related parameters
-            set req.url = regsuball(req.url, "(?:(\?)?|&)(?:utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=[^&]+", "\1");
+        if ({{enable_get_ignored}} && req.url ~ "[?&]({{get_param_ignored}})=") {
+            # Strip out ignored GET related parameters
+            set req.url = regsuball(req.url, "(?:(\?)?|&)(?:{{get_param_ignored}})=[^&]+", "\1");
             set req.url = regsuball(req.url, "(?:(\?)&|\?$)", "\1");
         }
+
 
         return (lookup);
     }

diff --git a/app/code/community/Nexcessnet/Turpentine/misc/version-3.vcl b/app/code/community/Nexcessnet/Turpentine/misc/version-3.vcl
@@ -106,22 +106,26 @@ sub vcl_recv {
         }
     }
 
+    # Normalize request data before potentially sending things off to the
+    # backend. This ensures all request types get the same information, most
+    # notably POST requests getting a normalized user agent string to empower
+    # adaptive designs.
+    {{normalize_encoding}}
+    {{normalize_user_agent}}
+    {{normalize_host}}
+
     # We only deal with GET and HEAD by default
     # we test this here instead of inside the url base regex section
     # so we can disable caching for the entire site if needed
     if (!{{enable_caching}} || req.http.Authorization ||
-        req.request !~ "^(GET|HEAD)$" ||
+        req.request !~ "^(GET|HEAD|OPTIONS)$" ||
         req.http.Cookie ~ "varnish_bypass={{secret_handshake}}") {
         return (pipe);
     }
 
     # remove double slashes from the URL, for higher cache hit rate
     set req.url = regsuball(req.url, "(.*)//+(.*)", "\1/\2");
 
-    {{normalize_encoding}}
-    {{normalize_user_agent}}
-    {{normalize_host}}
-
     # check if the request is for part of magento
     if (req.url ~ "{{url_base_regex}}") {
         # set this so Turpentine can see the request passed through Varnish
@@ -153,8 +157,8 @@ sub vcl_recv {
                 error 403 "External ESI requests are not allowed";
             }
         }
-        # no frontend cookie was sent to us
-        if (req.http.Cookie !~ "frontend=") {
+        # no frontend cookie was sent to us AND this is not an ESI or AJAX call
+        if (req.http.Cookie !~ "frontend=" && !req.http.X-Varnish-Esi-Method) {
             if (client.ip ~ crawler_acl ||
                     req.http.User-Agent ~ "^(?:{{crawler_user_agent_regex}})$") {
                 # it's a crawler, give it a fake cookie
@@ -186,12 +190,13 @@ sub vcl_recv {
             # TODO: should this be pass or pipe?
             return (pass);
         }
-        if (req.url ~ "[?&](utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=") {
-            # Strip out Google related parameters
-            set req.url = regsuball(req.url, "(?:(\?)?|&)(?:utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=[^&]+", "\1");
+        if ({{enable_get_ignored}} && req.url ~ "[?&]({{get_param_ignored}})=") {
+            # Strip out Ignored GET parameters
+            set req.url = regsuball(req.url, "(?:(\?)?|&)(?:{{get_param_ignored}})=[^&]+", "\1");
             set req.url = regsuball(req.url, "(?:(\?)&|\?$)", "\1");
         }
 
+
         # everything else checks out, try and pull from the cache
         return (lookup);
     }

diff --git a/app/design/frontend/base/default/layout/turpentine_esi.xml b/app/design/frontend/base/default/layout/turpentine_esi.xml
@@ -96,6 +96,7 @@
             <action method="setEsiOptions">
                 <params>
                     <access>private</access>
+                    <method>ajax</method>
                     <flush_events>
                         <poll_vote_add/>
                     </flush_events>