strange experience with very large URL parameters

[donnie-darko]

Hi,
In MailChimp campaign newsletters the URL parameters are very long, like this one:
?utm_source=THIS+TEST+NEWSLETTER&utm_campaign=256b7777e7a-newsletter201313&utm_medium=email&utm_term=0_80b5c23cde-256b711e7a-%5BLIST_EMAIL_ID%5D&gooal=eyJjaWQiOiIyNTZiNzExZTdhIiwidGFnIjoibmV3c2xldHRlcjIwMTMwNiIsInVpZCI6IjQ1MTRlYjQxNDg5NGExODczNmMzMDQ2YWQifQ%3D%3D%7CY3Npa29zLmFrb3NAcHJldHR5bGl0dGxlL777&mc_cid=256b711777&mc_eid=%5BUNIQID%5D
I recently updated turpentine in my live system to 0.5.4 so I haven't got enough time to test it fully so I run into these errors in the landing page with the above URL parameters today morning after the newsletter was sent to every subscriber:
Invalid URI supplied /srv/www/www.example.com/magento/lib/Zend/Uri.php(143): Zend_Uri_Http->__construct('http', '//www.example
So I stop varnish and bring nginx to front (port 80) to disable turpentine.
Then I tried this URL parameters on my development environment and in the first few times I can reproduce the error. I have found out that the gooal parameter cause the problem. URLs without the very long gooal parameter works, with gooal parameters doesn't. But after a few attempts the URLs with gooal parameters suddenly start working and I don't know why.
Can somebody confirm this weird issue?

[aheadley]

I run into these errors in the landing page with the above URL parameters today morning after the newsletter was sent to every subscriber:
Invalid URI supplied
/srv/www/www.example.com/magento/lib/Zend/Uri.php(143): Zend_Uri_Http->__construct('http', '//www.example

What is the full error message? It looks like Varnish may be truncating the URL passed to the backend.

[donnie-darko]

I think I found it:
MailChimp track the clicks with its own system
(like: http://XXXXX.us6.list-manage.com/track/click?u=XXXXXX )
and redirecting to the real URL of my site with analytics parameters.
MC put my newsletter name into utm_source analytics parameter. And in my newsletter name there are some hungarian characters like íéáőú. But MC haven't URL encode it at all after the redirect, so this is where Varnish truncate the URL and pass it wrong to the backend. If I turn Varnish off Nginx properly encode the URL and serve the page without errors.
When I copy+paste the URL parameters from the browser URL bar to the browser tab where my development system opened it has already URL encoded so this is why I can't reproduce the error.
So this is a Varnish bug?

[aheadley]

So this is a Varnish bug?

It may be, can you copy & paste the full error message from the log?

[donnie-darko]

yep:

a:5:{i:0;s:20:"Invalid URI supplied";i:1;s:1668:"#0 /srv/www/www.example.com/magento/lib/Zend/Uri.php(143): Zend_Uri_Http->__construct('http', '//www.example...')
#1 /srv/www/www.example.com/magento/lib/Zend/Controller/Request/Http.php(113): Zend_Uri::factory('http://www.example...')
#2 /srv/www/www.example.com/magento/.modman/turpentine/app/code/community/Nexcessnet/Turpentine/Model/Dummy/Request.php(43): Zend_Controller_Request_Http->__construct('http://www.example...')
#3 /srv/www/www.example.com/magento/.modman/turpentine/app/code/community/Nexcessnet/Turpentine/Helper/Esi.php(175): Nexcessnet_Turpentine_Model_Dummy_Request->__construct('http://www.example...')
#4 /srv/www/www.example.com/magento/.modman/turpentine/app/code/community/Nexcessnet/Turpentine/controllers/EsiController.php(74): Nexcessnet_Turpentine_Helper_Esi->getDummyRequest('http://www.example...')
#5 /srv/www/www.example.com/magento/app/code/core/Mage/Core/Controller/Varien/Action.php(419): Nexcessnet_Turpentine_EsiController->getBlockAction()
#6 /srv/www/www.example.com/magento/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(250): Mage_Core_Controller_Varien_Action->dispatch('getBlock')
#7 /srv/www/www.example.com/magento/app/code/core/Mage/Core/Controller/Varien/Front.php(176): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#8 /srv/www/www.example.com/magento/app/code/core/Mage/Core/Model/App.php(354): Mage_Core_Controller_Varien_Front->dispatch()
#9 /srv/www/www.example.com/magento/app/Mage.php(683): Mage_Core_Model_App->run(Array)
#10 /srv/www/www.example.com/magento/index.php(87): Mage::run('default', 'store')
#11 {main}";s:3:"url";s:500:"/turpentine/esi/getBlock/method/esi/access/private/ttl/3600/hmac/4d4181c44859a41d9454044d92eea04327d32ffa1b905d8d8d62dec156d55503/data/6MU0WiumIZtFCP8588HCq1Oa4FlnYZpgYo67VAv4TpyXhD2j2sXWEs5NJ-xlAye0.O5c9YwdYgD3AaVcVZy3VoHkou7To0Sy7PCISuyplvksahYP7CLxBMomBfQqJFJlwvnnEzy8C0KTJNt9F-ICYnl.19KZNmpzE3TnS3LnA5C4tAYQDkBa2xMo3kxT5cKdnCKQeOPXqhKBxPmknLZilI2Hq-GnYRmOqqLx0.IX0e9zJQuyR2P0ibceAstk-.b2yk1nbKIlGdUNneD6Y3Nspk1U6wpVKXytGkIG5doKButUitc0LUhZWPK2ebdaqzCLlTtf9zS8E0QdQP1SEGbTK3k7SUx0eUg2hoky5dkKcCc=/";s:11:"script_name";s:10:"/index.php";s:4:"skin";s:7:"default";}

[aheadley]

Ok, it seems there's two parts to this:

• One is a bug in Turpentine where it doesn't use the right URL for the dummy request during ESI requests. I'm working on an update for this, and should have a patch you can try soon.
• The other is apparently a bug in Zend that doesn't handle those characters in URLs. Unfortunately there's not really anything I can do about this but fixing the first issue should make it not matter.

[tegansnyder]

@aheadley I'm also seeing this bug. Can you recommend any quick patches? All in all I appreciate the timet that went into this extension. Thank you so much.

[donnie-darko]

Hi,
Is there any news about this issue?
Thx

[mounish]

Same issue. Did any one find any solution ?

[studio2f]

This appears to be happening to us to. Was there ever a fix?

Error:

a:5:{i:0;s:20:"Invalid URI supplied";i:1;s:1494:"#0 /home//public_html/lib/Zend/Uri.php(143): Zend_Uri_Http->__construct('http', '//www....')
#1 /home/MYSITE/public_html/lib/Zend/Controller/Request/Http.php(113): Zend_Uri::factory('http://www.joan...')
#2 /home/MYSITE/public_html/app/code/community/Nexcessnet/Turpentine/Model/Dummy/Request.php(43): Zend_Controller_Request_Http->__construct('http://www.joan...')
#3 /home/MYSITE/public_html/app/code/community/Nexcessnet/Turpentine/Helper/Esi.php(175): Nexcessnet_Turpentine_Model_Dummy_Request->__construct('http://www.joan...')
#4 /home/MYSITE/public_html/app/code/community/Nexcessnet/Turpentine/controllers/EsiController.php(74): Nexcessnet_Turpentine_Helper_Esi->getDummyRequest('http://www.joan...')
#5 /home/MYSITE/public_html/app/code/core/Mage/Core/Controller/Varien/Action.php(419): Nexcessnet_Turpentine_EsiController->getBlockAction()
#6 /home/MYSITE/public_html/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(250): Mage_Core_Controller_Varien_Action->dispatch('getBlock')
#7 /home/MYSITE/public_html/app/code/core/Mage/Core/Controller/Varien/Front.php(176): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#8 /home/MYSITE/public_html/app/code/core/Mage/Core/Model/App.php(354): Mage_Core_Controller_Varien_Front->dispatch()
#9 /home/MYSITE/public_html/app/Mage.php(683): Mage_Core_Model_App->run(Array)
#10 /home/MYSITE/public_html/index.php(88): Mage::run('', 'store')
#11 {main}";s:3:"url";s:503:"/turpentine/esi/getBlock/method/esi/access/private/ttl/7776000/hmac/3316b5d49911ac95c2d9df9a6d955000ce766b956418bf3f6ee7279ee0f5103f/data/u9kMpPTS-Dtj45xUrdM4MkQshszz9Me0PGDNxFJEc9UtHeSQ6XdwKZnckTaGqatB00PooXgiPnW0UmpgWJrdIGR25qK.d28pqr4gTdFcIIs7R6WjDqxre-l9PCSFTv2Zaofm3QA1dOFGwIeOF1.IQCsbG6pAlqfEvebv2bjl.6E8UZrJ5jHNXlUq5g0Scs5bnq8j75Rzxl3aQRvt8SLrHyCorQBLOhx1hdpGa3WrRQoA1ONxn4mWNn7-oPJzIe270Q42QvXEfnHeWnU-RFUekvprGtvNoufxkRCIUDd9eOde5rzDq4Q6eI454LZjtcqgFoHjXiZOvpiQvQEJ0LbCiatQ08Sk9RSQtrLBZXx5YJs=/";s:11:"script_name";s:10:"/index.php";s:4:"skin";s:7:"default";}

[mounish]

Hi studio2f :

We didnt find any solution for this, but surely found the reason why this error was getting generated. We closely checked, and found that the url parameters which were getting generated had spaces. For eg

www.example.com?color=Light Blue.

Generally chrome and mozilla appends %20 to the urls, but IE browser didnt appended it. Once we manually started passing the %20 value to spaces for IE browser, we stopped getting the above error.

New Url : www.example.com?color=Light%20Blue

Though we still are getting few such errors in log reports every day, but our main problem area was solved with this little fix.

Hope this might help.

[luigifab]

For us, I have if (!empty($uri)) $uri = str_replace('|', '%7C', $uri); to fix the same problem:

magento-turpentine/app/code/community/Nexcessnet/Turpentine/Model/Dummy/Request.php

Lines 40 to 46 in c670a49

	public function __construct($uri = null) {
	$this->_initFakeSuperGlobals();
	$this->_fixupFakeSuperGlobals($uri);
	try {
	parent::__construct($uri);
	} catch (Exception $e) {
	Mage::helper('turpentine/debug')