Invalid Form Key - Magento 1.9.2.1 #1023

vithefiddler · 2015-11-24T19:51:47Z

We are having an intermittent error on our site (2-10 times per day) where someone is unable to add an item to their cart in magento. I haven't been able to replicate the behavior myself, but one of our staff members has seen the behavior on his tablet, and I'm getting the error messages in the log, and our office staff is getting phone calls from people who are trying to place orders and can't. The general process and behavior is:

Add an item to the cart.
The cart is empty and the below message is logged.

The current configuration is:
Magento 1.9.2.1
Varnish: 3.0.X
Turpentine: 0.6.7

This bug report for turpentine is the closest thing I've found to a discussion of the issue:
#345

Obviously inability to add a product to cart is a big problem on an ecommerce site. I really appreciate your help in solving this issue.

This is what's currently in my URL Blacklist:
cron.php
cron.sh
robots.txt
sitemap.xml
sitemap.xml.gz
catalogsearch
js/amasty/*
checkout/*
cart
customer
account
ajax*

The below error is what shows up in the log.

URL: http://www.americanlegacyfishing.com/checkout/cart/ajaxDelete/id/226243/uenc/aHR0cDovL3d3dy5hbWVyaWNhbmxlZ2FjeWZpc2hpbmcuY29tL2Zpc2hpbmcvZmlzaGluZy1yb2RzL3Nob3BieS9nX2xvb21pcy5odG1sP3V0bV9zb3VyY2U9TmV3cytCbGFzdCtOb3ZlbWJlcisyNCUyQysyMDE1LTImdXRtX2NhbXBhaWduPU5vdmVtYmVyKzI0cmQmdXRtX21lZGl1bT1lbWFpbA,,/
IP Address: 166.137.139.76
Time: 2015-11-24 19:05:43 GMT
Error:
Invalid form key

Trace:
#0 /home/newagcom/public_html/app/code/core/Mage/Checkout/controllers/CartController.php(604): Mage::throwException('Invalid form ke...')
#1 /home/newagcom/public_html/app/code/core/Mage/Core/Controller/Varien/Action.php(418): Mage_Checkout_CartController->ajaxDeleteAction()
#2 /home/newagcom/public_html/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(254): Mage_Core_Controller_Varien_Action->dispatch('ajaxDelete')
#3 /home/newagcom/public_html/app/code/core/Mage/Core/Controller/Varien/Front.php(172): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#4 /home/newagcom/public_html/app/code/core/Mage/Core/Model/App.php(365): Mage_Core_Controller_Varien_Front->dispatch()
#5 /home/newagcom/public_html/app/Mage.php(684): Mage_Core_Model_App->run(Array)
#6 /home/newagcom/public_html/index.php(91): Mage::run('', 'store')
#7 {main}

This is what's in the main varnish vcl:
C{

include <stdlib.h>

include <stdio.h>

include <time.h>

include <pthread.h>

static pthread_mutex_t lrand_mutex = PTHREAD_MUTEX_INITIALIZER;
void generate_uuid(char* buf) {
pthread_mutex_lock(&lrand_mutex);
long a = lrand48();
long b = lrand48();
long c = lrand48();
long d = lrand48();
pthread_mutex_unlock(&lrand_mutex);
sprintf(buf, "frontend=%08lx%04lx%04lx%04lx%04lx%08lx",
a,
b & 0xffff,
(b & ((long)0x0fff0000) >> 16) | 0x4000,
(c & 0x0fff) | 0x8000,
(c & (long)0xffff0000) >> 16,
d
);
return;
}
}C
import std;
sub vcl_recv {
if(req.http.User-Agent ~ "iP(?:hone|ad|od)|BlackBerry|Palm|Googlebot-Mobile|Mobile|mobile|mobi|Windows Mobile|Safari Mobile|Android|Opera (?:Mini|Mobi)") {
return(pipe);
}
if(req.http.Host ~ "test.americanlegacyfishing.com") {
return(pipe);
}
}
backend default {
.host = "198.91.28.69";
.port = "8080";
.first_byte_timeout = 300s;
.between_bytes_timeout = 300s;
}
backend admin {
.host = "198.91.28.69";
.port = "8080";
.first_byte_timeout = 21600s;
.between_bytes_timeout = 21600s;
}
acl crawler_acl {
"127.0.0.1";
}
acl debug_acl {
"101.99.23.40";
"113.190.242.147";
}
/* -- REMOVED
sub generate_session {
if (req.url ~ ".[&?]SID=([^&]+).") {
set req.http.X-Varnish-Faked-Session = regsub(
req.url, ".[&?]SID=([^&]+).", "frontend=\1");
} else {
C{
char uuid_buf [50];
generate_uuid(uuid_buf);
VRT_SetHdr(sp, HDR_REQ,
"\030X-Varnish-Faked-Session:",
uuid_buf,
vrt_magic_string_end
);
}C
}
if (req.http.Cookie) {
std.collect(req.http.Cookie);
set req.http.Cookie = req.http.X-Varnish-Faked-Session +
"; " + req.http.Cookie;
} else {
set req.http.Cookie = req.http.X-Varnish-Faked-Session;
}
}
sub generate_session_expires {
C{
time_t now = time(NULL);
struct tm now_tm = gmtime(&now);
now_tm.tm_sec += 21600;
mktime(&now_tm);
char date_buf [50];
strftime(date_buf, sizeof(date_buf)-1, "%a, %d-%b-%Y %H:%M:%S %Z", &now_tm);
VRT_SetHdr(sp, HDR_RESP,
"\031X-Varnish-Cookie-Expires:",
date_buf,
vrt_magic_string_end
);
}C
}
-- */
sub vcl_recv {
if (req.restarts == 0) {
if (req.http.X-Forwarded-For) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
if(false) {
set req.http.X-Varnish-Origin-Url = req.url;
}
if (req.http.Accept-Encoding) {
if (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} else if (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
unset req.http.Accept-Encoding;
}
}
if (req.http.User-Agent ~ "iP(?:hone|ad|od)|BlackBerry|Palm|Googlebot-Mobile|Mobile|mobile|mobi|Windows Mobile|Safari Mobile|Android|Opera (?:Mini|Mobi)") {
set req.http.X-Normalized-User-Agent = "mobile";
} else if (req.http.User-Agent ~ "MSIE") {
set req.http.X-Normalized-User-Agent = "msie";
} else if (req.http.User-Agent ~ "Firefox") {
set req.http.X-Normalized-User-Agent = "firefox";
} else if (req.http.User-Agent ~ "Chrome") {
set req.http.X-Normalized-User-Agent = "chrome";
} else if (req.http.User-Agent ~ "Safari") {
set req.http.X-Normalized-User-Agent = "safari";
} else if (req.http.User-Agent ~ "Opera") {
set req.http.X-Normalized-User-Agent = "opera";
} else {
set req.http.X-Normalized-User-Agent = "other";
}
if (!true || req.http.Authorization ||
req.request !~ "^(GET|HEAD|OPTIONS)$" ||
req.http.Cookie ~ "varnish_bypass=1") {
if (req.url ~ "^(/media/|/skin/|/js/|/)(?:(?:index|litespeed).php/)?alfc-admin") {
set req.backend = admin;
}
return (pipe);
}
set req.url = regsuball(req.url, "([^:])//+", "\1/");
if (req.url ~ "^(/media/|/skin/|/js/|/)(?:(?:index|litespeed).php/)?") {
set req.http.X-Turpentine-Secret-Handshake = "1";
if (req.url ~ "^(/media/|/skin/|/js/|/)(?:(?:index|litespeed).php/)?alfc-admin") {
set req.backend = admin;
return (pipe);
}
if (req.http.Cookie ~ "\bcurrency=") {
set req.http.X-Varnish-Currency = regsub(
req.http.Cookie, ".\bcurrency=([^;]).", "\1");
}
if (req.http.Cookie ~ "\bstore=") {
set req.http.X-Varnish-Store = regsub(
req.http.Cookie, ".\bstore=([^;]).", "\1");
}
if (req.url ~ "/turpentine/esi/get(?:Block|FormKey)/") {
set req.http.X-Varnish-Esi-Method = regsub(
req.url, "./method/(\w+)/.", "\1");
set req.http.X-Varnish-Esi-Access = regsub(
req.url, "./access/(\w+)/.", "\1");
if (req.http.X-Varnish-Esi-Method == "esi" && req.esi_level == 0 &&
!(false || client.ip ~ debug_acl)) {
error 403 "External ESI requests are not allowed";
}
}
if (req.http.Cookie !~ "frontend=" && !req.http.X-Varnish-Esi-Method) {
if (client.ip ~ crawler_acl ||
req.http.User-Agent ~ "^(?:ApacheBench/.|.Googlebot.|JoeDog/.Siege.|magespeedtest.com|Nexcessnet_Turpentine/.|Baidu/.|Bing/.|Yandex/.|Yahoo/.)$") {
set req.http.Cookie = "frontend=crawler-session";
} else {
return (pipe);
}
}
if (true &&
req.url ~ "..(?:css|js|jpe?g|png|gif|ico|swf)(?=?|&|$)") {
unset req.http.Cookie;
unset req.http.X-Varnish-Faked-Session;
return (lookup);
}
if (req.url ~ "^(/media/|/skin/|/js/|/)(?:(?:index|litespeed).php/)?(?:alfc-admin|api|cron.php|cron.sh|robots.txt|sitemap.xml|sitemap.xml.gz|catalogsearch|js/amasty/|checkout/|cart|customer|account|ajax_)" ||
req.url ~ "?.from_store=") {
return (pipe);
}
if (true &&
req.url ~ "(?:?&(?=[&=]|$))") {
return (pass);
}
if (true && req.url ~ "?&=") {
set req.url = regsuball(req.url, "(?:(?)?|&)(?:utm_source|utm_medium|utm_campaign|utm_content|utm_term|gclid|cx|ie|cof|siteurl)=[^&]+", "\1");
set req.url = regsuball(req.url, "(?:(?)&|?$)", "\1");
}
if(false) {
set req.http.X-Varnish-Cache-Url = req.url;
set req.url = req.http.X-Varnish-Origin-Url;
unset req.http.X-Varnish-Origin-Url;
}
return (lookup);
}
}
sub vcl_pipe {
unset bereq.http.X-Turpentine-Secret-Handshake;
set bereq.http.Connection = "close";
}
sub vcl_hash {
if(false && req.http.X-Varnish-Cache-Url) {
hash_data(req.http.X-Varnish-Cache-Url);
} else {
hash_data(req.url);
}
if (req.http.Host) {
hash_data(req.http.Host);
} else {
hash_data(server.ip);
}
hash_data(req.http.Ssl-Offloaded);
if (req.http.X-Normalized-User-Agent) {
hash_data(req.http.X-Normalized-User-Agent);
}
if (req.http.Accept-Encoding) {
hash_data(req.http.Accept-Encoding);
}
if (req.http.X-Varnish-Store || req.http.X-Varnish-Currency) {
hash_data("s=" + req.http.X-Varnish-Store + "&c=" + req.http.X-Varnish-Currency);
}
if (req.http.X-Varnish-Esi-Access == "private" &&
req.http.Cookie ~ "frontend=") {
hash_data(regsub(req.http.Cookie, "^.?frontend=([^;]);.$", "\1"));
}
if (req.http.X-Varnish-Esi-Access == "customer_group" &&
req.http.Cookie ~ "customer_group=") {
hash_data(regsub(req.http.Cookie, "^.?customer_group=([^;]);.$", "\1"));
}
return (hash);
}
sub vcl_hit {
}
sub vcl_fetch {
set req.grace = 15s;
set beresp.http.X-Varnish-Host = req.http.host;
set beresp.http.X-Varnish-URL = req.url;
if (req.url ~ "^(/media/|/skin/|/js/|/)(?:(?:index|litespeed).php/)?") {
unset beresp.http.Vary;
set beresp.do_gzip = true;
if (beresp.status != 200 && beresp.status != 404) {
set beresp.ttl = 15s;
return (hit_for_pass);
} else {
if (beresp.http.Set-Cookie) {
set beresp.http.X-Varnish-Set-Cookie = beresp.http.Set-Cookie;
unset beresp.http.Set-Cookie;
}
unset beresp.http.Cache-Control;
unset beresp.http.Expires;
unset beresp.http.Pragma;
unset beresp.http.Cache;
unset beresp.http.Age;
if (beresp.http.X-Turpentine-Esi == "1") {
set beresp.do_esi = true;
}
if (beresp.http.X-Turpentine-Cache == "0") {
set beresp.ttl = 15s;
return (hit_for_pass);
} else {
if (true &&
bereq.url ~ "..(?:css|js|jpe?g|png|gif|ico|swf)(?=?|&|$)") {
set beresp.ttl = 28800s;
set beresp.http.Cache-Control = "max-age=28800";
} elseif (req.http.X-Varnish-Esi-Method) {
if (req.http.X-Varnish-Esi-Access == "private" &&
req.http.Cookie ~ "frontend=") {
set beresp.http.X-Varnish-Session = regsub(req.http.Cookie,
"^.?frontend=([^;]);.$", "\1");
}
if (req.http.X-Varnish-Esi-Method == "ajax" &&
req.http.X-Varnish-Esi-Access == "public") {
set beresp.http.Cache-Control = "max-age=" + regsub(
req.url, "./ttl/(\d+)/.", "\1");
}
set beresp.ttl = std.duration(
regsub(
req.url, "./ttl/(\d+)/.", "\1s"),
300s);
if (beresp.ttl == 0s) {
set beresp.ttl = 15s;
return (hit_for_pass);
}
} else {
set beresp.ttl = 3600s;
}
}
}
return (deliver);
}
}
sub vcl_deliver {
if (req.http.X-Varnish-Faked-Session) {
set resp.http.Set-Cookie = req.http.X-Varnish-Faked-Session +
"; expires=" + resp.http.X-Varnish-Cookie-Expires + "; path=/";
if (req.http.Host) {
if (req.http.User-Agent ~ "^(?:ApacheBench/.|.Googlebot.|JoeDog/.Siege.|magespeedtest.com|Nexcessnet_Turpentine/.|Baidu/.|Bing/.|Yandex/._|Yahoo/.*)$") {
set resp.http.Set-Cookie = resp.http.Set-Cookie +
"; domain=" + regsub(req.http.Host, ":\d+$", "");
} else {
if(req.http.Host ~ "") {
set resp.http.Set-Cookie = resp.http.Set-Cookie +
"; domain=";
} else {
set resp.http.Set-Cookie = resp.http.Set-Cookie +
"; domain=" + regsub(req.http.Host, ":\d+$", "");
}
}
}
set resp.http.Set-Cookie = resp.http.Set-Cookie + "; httponly";
unset resp.http.X-Varnish-Cookie-Expires;
}
if (req.http.X-Varnish-Esi-Method == "ajax" && req.http.X-Varnish-Esi-Access == "private") {
set resp.http.Cache-Control = "no-cache";
}
if (false || client.ip ~ debug_acl) {
set resp.http.X-Varnish-Hits = obj.hits;
set resp.http.X-Varnish-Esi-Method = req.http.X-Varnish-Esi-Method;
set resp.http.X-Varnish-Esi-Access = req.http.X-Varnish-Esi-Access;
set resp.http.X-Varnish-Currency = req.http.X-Varnish-Currency;
set resp.http.X-Varnish-Store = req.http.X-Varnish-Store;
} else {
unset resp.http.X-Varnish;
unset resp.http.Via;
unset resp.http.X-Powered-By;
unset resp.http.Server;
unset resp.http.X-Turpentine-Cache;
unset resp.http.X-Turpentine-Esi;
unset resp.http.X-Turpentine-Flush-Events;
unset resp.http.X-Turpentine-Block;
unset resp.http.X-Varnish-Session;
unset resp.http.X-Varnish-Host;
unset resp.http.X-Varnish-URL;
unset resp.http.X-Varnish-Set-Cookie;
}
}

This is what's in the varnish_custom vcl

sub vcl_recv {
if(req.http.User-Agent ~ "iP(?:hone|ad|od)|BlackBerry|Palm|Googlebot-Mobile|Mobile|mobile|mobi|Windows Mobile|Safari Mobile|Android|Opera (?:Mini|Mobi)") {
return(pipe);
}

if(req.http.Host ~ "test\.americanlegacyfishing\.com") {
    return(pipe);
}

}

The text was updated successfully, but these errors were encountered:

miguelbalparda · 2015-11-24T19:55:23Z

I checked your log and I noted 2 things:

You are using an ajax cart.
You are getting an error with a delete action and not an add action, not sure what this means.
What I suggest is to test with a Magento without ajax and then start debugging to see where the error is happening. Also check this repository (issues, PR, documentation) for known ajax issues.

vithefiddler · 2015-11-24T20:15:20Z

I am using the default add to cart method that is used by the magento 1.9 default theme, RWD. I am not using any 3rd party ajax cart. It does use the minicart which uses an ajax remove from cart method. I've gone through the admin panel and don't see any option for disabling ajax add to cart.

I also agree that it's strange that the error I'm getting is on a delete action but the error on the customer's end is when they add an item to the cart.

Since the invalid form key is on a URL that's within the checkout top level folder, I figured that it shouldn't be cached. Do you concur?

Thanks for your help!
Vi

thampe · 2015-11-26T20:37:18Z

Hi,

I've had a similar issue in my shops. When a customer reaches your shop for the first time on cached product page the form_keyis not generated by the shop and is different (the shop generates also a different form_key on a subsequent request), because the response comes from your varnish server.

Possible Solutions

There is a setting in the backend Varnish Options > General Options > Use VCL fix, set it to yes the first request is delivered to your backend server, on the downside every first request will be slow, see also First Request is never cached #957
Disable from-key validation (Observer) for add-to-cart-requests, less secure but still fast

vithefiddler · 2015-11-27T15:33:44Z

I do have Use VCL fix enabled. I'm hesitant to disable form-key validation for add-to-cart requests. Beyond a brute force attack of add-to-carts what else are the risks of disabling form key validation on the add-to-cart?

miguelbalparda · 2015-12-07T16:00:54Z

@vithefiddler the disable form key validation is not something we created but the official solution Magento gives to integrate Varnish.

vithefiddler · 2015-12-16T20:07:49Z

Ok. That's unfortunate, but I guess there are worse things than lacking form validation.

What is the big risk of turning off form validation?

aricwatson · 2015-12-16T22:42:20Z

The form key validation is there to protect against CSRF attacks (Cross-site request forgery) - there's a good explanation here.

vithefiddler · 2015-12-16T23:54:52Z

I understand the generality, and that it would allow someone to potentially add an item to the cart from another site, or maybe post a comment or review. I see that this could potentially be used for DOS attacks. Are there other risks that I'm missing?

addison74 · 2015-12-22T18:39:23Z

Why don't you disable the encoding part only for checkout/cart/ajaxDelete URL? Here are the files you need to edit then test.

/app/code/core/Mage/Checkout/Block/Cart/Item/Renderer.php (original)
/app/code/local/Mage/Checkout/Block/Cart/Item/Renderer.php (copied)

see the comment part bellow:

    /**
     * Get item ajax delete url
     *
     * @return string
     */
    public function getAjaxDeleteUrl()
    {
        return $this->getUrl(
            'checkout/cart/ajaxDelete',
            array(
                'id'=>$this->getItem()->getId()/**,
                Mage_Core_Controller_Front_Action::PARAM_NAME_URL_ENCODED => $this->helper('core/url')->getEncodedUrl(),
                '_secure' => $this->_getApp()->getStore()->isCurrentlySecure(),**/
            )
        );
    }

For removing, deleting actions there are no issues in my tests. But I do not recommend doing the same thing for an action like add, it won't work.

See this post for other issues related to form_keys and encoded params in URLs: #1058.

miguelbalparda · 2016-02-03T14:44:59Z

Closing for lack of feedback.

eugenbg · 2018-06-13T11:45:44Z

I have been able to reproduce it by logging in to a customer's account who reported the same issue (can't delete an item from cart)

i found that this method
app/code/community/Nexcessnet/Turpentine/Model/Core/Session.php::getFormKey()

returns the form_key with a preceding space character
when i leave just return parent::getFormKey();
everything works fine

miguelbalparda added the question label Nov 24, 2015

miguelbalparda self-assigned this Nov 24, 2015

miguelbalparda closed this as completed Feb 3, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Invalid Form Key - Magento 1.9.2.1 #1023

Invalid Form Key - Magento 1.9.2.1 #1023

vithefiddler commented Nov 24, 2015

miguelbalparda commented Nov 24, 2015

vithefiddler commented Nov 24, 2015

thampe commented Nov 26, 2015

vithefiddler commented Nov 27, 2015

miguelbalparda commented Dec 7, 2015

vithefiddler commented Dec 16, 2015

aricwatson commented Dec 16, 2015

vithefiddler commented Dec 16, 2015

addison74 commented Dec 22, 2015

miguelbalparda commented Feb 3, 2016

eugenbg commented Jun 13, 2018

Invalid Form Key - Magento 1.9.2.1 #1023

Invalid Form Key - Magento 1.9.2.1 #1023

Comments

vithefiddler commented Nov 24, 2015

include <stdlib.h>

include <stdio.h>

include <time.h>

include <pthread.h>

miguelbalparda commented Nov 24, 2015

vithefiddler commented Nov 24, 2015

thampe commented Nov 26, 2015

Possible Solutions

vithefiddler commented Nov 27, 2015

miguelbalparda commented Dec 7, 2015

vithefiddler commented Dec 16, 2015

aricwatson commented Dec 16, 2015

vithefiddler commented Dec 16, 2015

addison74 commented Dec 22, 2015

miguelbalparda commented Feb 3, 2016

eugenbg commented Jun 13, 2018