Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors in admin area #5

Closed
Springerton opened this issue Nov 18, 2012 · 31 comments
Closed

Errors in admin area #5

Springerton opened this issue Nov 18, 2012 · 31 comments
Labels

Comments

@Springerton
Copy link

I have some bugs in admin area.
In System > Varnish Management don't work functions. No one.

After click on any button, I'm redirecting to Magento error page:

"There has been an error processing your request
Exception printing is disabled by default for security reasons.

Error log record number: 837656914520"

Inf file:

a:5:{i:0;s:106:"Got unexpected response code from Varnish: 107
<removed>

Authentication required.
";i:1;s:2565:"#0 /magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(406): Mage::throwException('Got unexpected ...')
#1 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(299): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_command('auth', 200, '9629ca4046e3c5c...')
#2 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(329): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_connect()
#3 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(440): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_write('help')
#4 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(226): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_determineVersion()
#5 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php(135): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->getVersion()
#6 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php(44): Nexcessnet_Turpentine_Model_Varnish_Admin->getConfigurator()
#7 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php(33): Nexcessnet_Turpentine_Model_Varnish_Admin->flushUrl('.*')
#8 PATHTOMAGENTO/magento/app/code/community/Nexcessnet/Turpentine/controllers/Varnish/ManagementController.php(48): Nexcessnet_Turpentine_Model_Varnish_Admin->flushAll()
#9 PATHTOMAGENTO/magento/app/code/core/Mage/Core/Controller/Varien/Action.php(419): Nexcessnet_Turpentine_Varnish_ManagementController->flushAllAction()
#10 PATHTOMAGENTO/magento/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(250): Mage_Core_Controller_Varien_Action->dispatch('flushAll')
#11 PATHTOMAGENTO/magento/app/code/core/Mage/Core/Controller/Varien/Front.php(176): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#12 PATHTOMAGENTO/magento/app/code/core/Mage/Core/Model/App.php(354): Mage_Core_Controller_Varien_Front->dispatch()
#13 PATHTOMAGENTO/magento/app/Mage.php(683): Mage_Core_Model_App->run(Array)
#14 PATHTOMAGENTO/magento/index.php(87): Mage::run('', 'store')
#15 {main}";s:3:"url";s:90:"/magento/index.php/admin/varnish_management/flushAll/key/<removed>/";s:11:"script_name";s:18:"/magento/index.php";s:4:"skin";s:5:"admin";}
@aheadley
Copy link
Contributor

Your Varnish secret key is incorrect, it needs to match what Varnish uses exactly. If you are using a CentOS package then that would be the contents of /etc/varnish/secret. Usually this is caused by leaving off the newline at the end. Try adding \n to the end of the Varnish Options > Servers > Varnish Authentication Key option.

@Springerton
Copy link
Author

Hi Alex!

Thanks for the reply. I'm using Debian and have the same directories for
Varnish. Secret key I've copy directly to admin area with no changes.
May be something wrong with Varnish?
How can I check it?

In attachment all config files and Magento Admin Area Screenshot.

Best regards,
Alexey Maltsev

19.11.2012 19:24, Alex Headley пишет:

Your Varnish secret key is incorrect, it needs to match what Varnish
uses exactly. If you are using a CentOS package then that would be the
contents of |/etc/varnish/secret|. Usually this is caused by leaving
off the newline at the end. Try adding |\n| to the end of it.


Reply to this email directly or view it on GitHub
#5 (comment).

This is a basic VCL configuration file for PageCache powered by Varnish for Magento module.

default backend definition. Set this to point to your content server.

backend default {
.host = "127.0.0.1";
.port = "8080";
}

admin backend with longer timeout values. Set this to the same IP & port as your default server.

backend admin {
.host = "127.0.0.1";
.port = "8080";
.first_byte_timeout = 18000s;
.between_bytes_timeout = 18000s;
}

add your Magento server IP to allow purges from the backend

acl purge {
"localhost";
"127.0.0.1";
}

sub vcl_recv {
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For ", " client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}

if (req.request != "GET" &&
  req.request != "HEAD" &&
  req.request != "PUT" &&
  req.request != "POST" &&
  req.request != "TRACE" &&
  req.request != "OPTIONS" &&
  req.request != "DELETE" &&
  req.request != "PURGE") {
    /* Non-RFC2616 or CONNECT which is weird. */
    return (pipe);
}

# purge request
if (req.request == "PURGE") {
    if (!client.ip ~ purge) {
        error 405 "Not allowed.";
    }
    purge("obj.http.X-Purge-Host ~ " req.http.X-Purge-Host " && obj.http.X-Purge-URL ~ " req.http.X-Purge-Regex " && obj.http.Content-Type ~ " req.http.X-Purge-Content-Type);
    error 200 "Purged.";
}

# switch to admin backend configuration
if (req.http.cookie ~ "adminhtml=") {
    set req.backend = admin;
}

# we only deal with GET and HEAD by default    
if (req.request != "GET" && req.request != "HEAD") {
    return (pass);
}

# normalize url in case of leading HTTP scheme and domain
set req.url = regsub(req.url, "^http[s]?://[^/]+", "");

# static files are always cacheable. remove SSL flag and cookie
if (req.url ~ "^/(media|js|skin)/.*\.(png|jpg|jpeg|gif|css|js|swf|ico)$") {
    unset req.http.Https;
    unset req.http.Cookie;
}

# not cacheable by default
if (req.http.Authorization || req.http.Https) {
    return (pass);
}

# do not cache any page from
# - index files
# - ...
if (req.url ~ "^/(index)") {
    return (pass);
}

# as soon as we have a NO_CACHE cookie pass request
if (req.http.cookie ~ "NO_CACHE=") {
    return (pass);
}

# normalize Aceept-Encoding header
# http://varnish.projects.linpro.no/wiki/FAQ/Compression
if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$") {
        # No point in compressing these
        remove req.http.Accept-Encoding;
    } elsif (req.http.Accept-Encoding ~ "gzip") {
        set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") {
        set req.http.Accept-Encoding = "deflate";
    } else {
        # unkown algorithm
        remove req.http.Accept-Encoding;
    }
}

# remove Google gclid parameters
set req.url = regsuball(req.url,"\?gclid=[^&]+$",""); # strips when QS = "?gclid=AAA"
set req.url = regsuball(req.url,"\?gclid=[^&]+&","?"); # strips when QS = "?gclid=AAA&foo=bar"
set req.url = regsuball(req.url,"&gclid=[^&]+",""); # strips when QS = "?foo=bar&gclid=AAA" or QS = "?foo=bar&gclid=AAA&bar=baz"

return (lookup);

}

sub vcl_pipe {

# Note that only the first request to the backend will have

# X-Forwarded-For set. If you use X-Forwarded-For and want to

# have it set for all requests, make sure to have:

# set bereq.http.connection = "close";

# here. It is not set by default as it might break some broken web

# applications, like IIS with NTLM authentication.

return (pipe);

}

sub vcl_pass {

return (pass);

}

sub vcl_hash {
set req.hash += req.url;
if (req.http.host) {
set req.hash += req.http.host;
} else {
set req.hash += server.ip;
}
if (!(req.url ~ "^/(media|js|skin)/.*.(png|jpg|jpeg|gif|css|js|swf|ico)$")) {
call design_exception;
}
return (hash);
}

sub vcl_hit {

if (!obj.cacheable) {

return (pass);

}

return (deliver);

}

sub vcl_miss {

return (fetch);

}

sub vcl_fetch {
if (beresp.status == 500) {
set beresp.saintmode = 10s;
restart;
}
set beresp.grace = 5m;

# add ban-lurker tags to object
set beresp.http.X-Purge-URL = req.url;
set beresp.http.X-Purge-Host = req.http.host;

if (beresp.status == 200 || beresp.status == 301 || beresp.status == 404) {
    if (beresp.http.Content-Type ~ "text/html" || beresp.http.Content-Type ~ "text/xml") {
        if ((beresp.http.Set-Cookie ~ "NO_CACHE=") || (beresp.ttl < 1s)) {
            set beresp.ttl = 0s;
            return (pass);
        }

        # marker for vcl_deliver to reset Age:
        set beresp.http.magicmarker = "1";

        # Don't cache cookies
        unset beresp.http.set-cookie;
    } else {
        # set default TTL value for static content
        set beresp.ttl = 4h;
    }
    return (deliver);
}

return (pass);

}

sub vcl_deliver {
# debug info
if (resp.http.X-Cache-Debug) {
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
set resp.http.X-Cache-Hits = obj.hits;
} else {
set resp.http.X-Cache = "MISS";
}
set resp.http.X-Cache-Expires = resp.http.Expires;
} else {
# remove Varnish/proxy header
remove resp.http.X-Varnish;
remove resp.http.Via;
remove resp.http.Age;
remove resp.http.X-Purge-URL;
remove resp.http.X-Purge-Host;
}

if (resp.http.magicmarker) {
    # Remove the magic marker
    unset resp.http.magicmarker;

    set resp.http.Cache-Control = "no-store, no-cache, must-revalidate, post-check=0, pre-check=0";
    set resp.http.Pragma = "no-cache";
    set resp.http.Expires = "Mon, 31 Mar 2008 10:00:00 GMT";
    set resp.http.Age = "0";
}

}

sub vcl_error {

set obj.http.Content-Type = "text/html; charset=utf-8";

synthetic {"

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<title>"} obj.status " " obj.response {"</title>

Error "} obj.status " " obj.response {"

"} obj.response {"

Guru Meditation:

XID: "} req.xid {"


Varnish cache server

"};

return (deliver);

}

sub design_exception {
}

Configuration file for varnish

/etc/init.d/varnish expects the variables $DAEMON_OPTS, $NFILES and $MEMLOCK

to be set from this shell script fragment.

Should we start varnishd at boot? Set to "yes" to enable.

START=yes

Maximum number of open files (for ulimit -n)

NFILES=131072

Maximum locked memory size (for ulimit -l)

Used for locking the shared memory log in memory. If you increase log size,

you need to increase this number as well

MEMLOCK=82000

Default varnish instance name is the local nodename. Can be overridden with

the -n switch, to have more instances on a single server.

INSTANCE=$(uname -n)

This file contains 4 alternatives, please use only one.

Alternative 1, Minimal configuration, no VCL

Listen on port 6081, administration on localhost:6082, and forward to

content server on localhost:8080. Use a 1GB fixed-size cache file.

DAEMON_OPTS="-a :6081
-T localhost:6082
-b localhost:8080
-u varnish -g varnish
-S /etc/varnish/secret
-s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"

Alternative 2, Configuration with VCL

Listen on port 6081, administration on localhost:6082, and forward to

one content server selected by the vcl file, based on the request. Use a 1GB

fixed-size cache file.

DAEMON_OPTS="-a :6081
-T localhost:6082
-f /etc/varnish/default.vcl
-S /etc/varnish/secret
-s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"

Alternative 3, Advanced configuration

See varnishd(1) for more information.

# Main configuration file. You probably want to change it :)

VARNISH_VCL_CONF=/etc/varnish/default.vcl

# Default address and port to bind to

# Blank address means all IPv4 and IPv6 interfaces, otherwise specify

# a host name, an IPv4 dotted quad, or an IPv6 address in brackets.

VARNISH_LISTEN_ADDRESS=

VARNISH_LISTEN_PORT=6081

# Telnet admin interface listen address and port

VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1

VARNISH_ADMIN_LISTEN_PORT=6082

# The minimum number of worker threads to start

VARNISH_MIN_THREADS=1

# The Maximum number of worker threads to start

VARNISH_MAX_THREADS=1000

# Idle timeout for worker threads

VARNISH_THREAD_TIMEOUT=120

# Cache file location

VARNISH_STORAGE_FILE=/var/lib/varnish/$INSTANCE/varnish_storage.bin

# Cache file size: in bytes, optionally using k / M / G / T suffix,

# or in percentage of available disk space using the % suffix.

VARNISH_STORAGE_SIZE=1G

# File containing administration secret

VARNISH_SECRET_FILE=/etc/varnish/secret

# Backend storage specification

VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"

# Default TTL used when the backend does not specify one

VARNISH_TTL=120

# DAEMON_OPTS is used by the init script. If you add or remove options, make

# sure you update this section, too.

DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \

-f ${VARNISH_VCL_CONF} \

-T ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \

-t ${VARNISH_TTL} \

-w ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \

-S ${VARNISH_SECRET_FILE} \

-s ${VARNISH_STORAGE}"

Alternative 4, Do It Yourself

DAEMON_OPTS=""

@aheadley
Copy link
Contributor

Alexey,
Did you try adding the \n to the end of the Varnish Options > Servers > Varnish Authentication Key option? Based on your config, it looks like it is indeed reading from /etc/varnish/secret so that option needs to match the contents of that file including non-printable characters like the newline. Turpentine will replace a literal \n in the Varnish Authentication Key option with a real newline character.

For example, on my dev machine:

[root@host ~]# cat /etc/varnish/secret 
754bca59-a8ed-4057-95fa-6d4c73d09e62
[root@host ~]# 

It's not obvious, but there is a new line at the end of the file, which is why the next prompt is on a line by itself. If it didn't have a newline at the end it would look like this:

[root@host ~]# cat /etc/varnish/secret 
754bca59-a8ed-4057-95fa-6d4c73d09e62[root@host ~]# 

Since it does have the newline, my Varnish Authentication Key option looks like this: 754bca59-a8ed-4057-95fa-6d4c73d09e62\n

@Springerton
Copy link
Author

Alex,

I added \n to the Varnish key in admin area, but now I see other
error. Look at attachment please.
May be I have incorrect IP:PORT or somethig else?

Best regards,
Alexey

19.11.2012 21:15, Alex Headley пишет:

Alexey,
Did you try adding the |\n| to the end of the |Varnish Options >
Servers > Varnish Authentication Key| option? Based on your config, it
looks like it is indeed reading from |/etc/varnish/secret| so that
option needs to match the contents of that file including
non-printable characters like the newline. Turpentine will replace a
literal |\n| in the Varnish Authentication Key option with a real
newline character.

For example, on my dev machine:

|[root@host ~]# cat /etc/varnish/secret
754bca59-a8ed-4057-95fa-6d4c73d09e62
[root@host ~]#
|

It's not obvious, but there is a new line at the end of the file,
which is why the next prompt is on a line by itself. If it didn't have
a newline at the end it would look like this:

|[root@host ~]# cat /etc/varnish/secret
754bca59-a8ed-4057-95fa-6d4c73d09e62[root@host ~]#
|

Since it does have the newline, my Varnish Authentication Key option
looks like this: |754bca59-a8ed-4057-95fa-6d4c73d09e62\n|


Reply to this email directly or view it on GitHub
#5 (comment).

a:5:{i:0;s:28:"Varnish admin socket timeout";i:1;s:2409:"#0 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(356): Mage::throwException('Varnish admin s...')
#1 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(294): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_read()
#2 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(329): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_connect()
#3 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(440): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_write('help')
#4 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin/Socket.php(226): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->_determineVersion()
#5 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php(135): Nexcessnet_Turpentine_Model_Varnish_Admin_Socket->getVersion()
#6 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php(44): Nexcessnet_Turpentine_Model_Varnish_Admin->getConfigurator()
#7 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/Model/Varnish/Admin.php(33): Nexcessnet_Turpentine_Model_Varnish_Admin->flushUrl('.*')
#8 /var/www/devaccess/data/www/dev.com/app/code/community/Nexcessnet/Turpentine/controllers/Varnish/ManagementController.php(48): Nexcessnet_Turpentine_Model_Varnish_Admin->flushAll()
#9 /var/www/devaccess/data/www/dev.com/app/code/core/Mage/Core/Controller/Varien/Action.php(419): Nexcessnet_Turpentine_Varnish_ManagementController->flushAllAction()
#10 /var/www/devaccess/data/www/dev.com/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(250): Mage_Core_Controller_Varien_Action->dispatch('flushAll')
#11 /var/www/devaccess/data/www/dev.com/app/code/core/Mage/Core/Controller/Varien/Front.php(176): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#12 /var/www/devaccess/data/www/dev.com/app/code/core/Mage/Core/Model/App.php(354): Mage_Core_Controller_Varien_Front->dispatch()
#13 /var/www/devaccess/data/www/dev.com/app/Mage.php(683): Mage_Core_Model_App->run(Array)
#14 /var/www/devaccess/data/www/dev.com/index.php(87): Mage::run('', 'store')
#15 {main}";s:3:"url";s:82:"/index.php/admin/varnish_management/flushAll/key/0f7a50c3c2253770356a20ecba3475f9/";s:11:"script_name";s:10:"/index.php";s:4:"skin";s:5:"admin";}

@aheadley
Copy link
Contributor

Alexey,
I'm not sure attachments come through GitHub's issue system (I don't see the image you said you attached earlier).

This error is odd, can you send me the output of varnishd -V and the contents of the Varnish Options > Servers > Server List option?

@Springerton
Copy link
Author

Alex,
below Turpentine Varnish Options:


Backend Host: 127.0.0.1
Backend Port: 80
Frontend Timeout: 300

Admin Timeout: 21600

Servers
Varnish Version: Auto
Server List: 127.0.0.1:8080
Varnish Authentication Key: a8e43039-a676-4c26-9f3b-9e3ebf64053a\n

Config File Location: {{root_dir}}/var/default.vcl


And the output of varnishd -v

root@dev ~ # varnishd -V
varnishd (varnish-2.1.3 SVN 5049:5055)
Copyright (c) 2006-2009 Linpro AS / Verdens Gang AS

19.11.2012 22:54, Alex Headley пишет:

Alexey,
I'm not sure attachments come through GitHub's issue system (I don't
see the image you said you attached earlier).

This error is odd, can you send me the output of |varnishd -V| and the
contents of the |Varnish Options > Servers > Server List| option?


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
The Server List option should contain the Varnish management sockets, not the listening sockets (this is probably not real clear in the documentation). The management socket is used to control Varnish, while the listening socket is where it listens for requests. Based on the config file you sent in earlier try changing the 127.0.0.1:8080 to 127.0.0.1:6082

@Springerton
Copy link
Author

Alex,

thank you very much. At last it works :)
I had 2 problems with errors:

  1. Not correct *Management *socket in extension settings.
  2. Extension doesn't correct detect Varnish version. When I chanched
    from AUTO to 2.1.x it start to work. Do I need update Varnish to
    3.0+? or this is not important?

Your extension is perfect! But need only fix bug with Varnish version
detection and add info in manual about possible errors with "socket
timout" and "Authentication required".
Also, can you tell me some recommendations with Magento load speed? May
be need other solutions? Update HTML templates, split JS and CSS files
or something else?

20.11.2012 0:15, Alex Headley пишет:

Alexey,
The Server List option should contain the Varnish /management
sockets/, not the /listening sockets/ (this is probably not real clear
in the documentation). The management socket is used to control
Varnish, while the listening socket is where it listens for requests.
Based on the config file you sent in earlier try changing the
|127.0.0.1:8080| to |127.0.0.1:6082|


Reply to this email directly or view it on GitHub
#5 (comment).

@Springerton
Copy link
Author

Alex,

I'm testing site and don't see any changes. Looks like the same page
load time: 4-30 sec per page.

20.11.2012 0:37, Alexey Maltsev пишет:

Alex,

thank you very much. At last it works :)
I had 2 problems with errors:

  1. Not correct *Management *socket in extension settings.
  2. Extension doesn't correct detect Varnish version. When I
    chanched from AUTO to 2.1.x it start to work. Do I need update
    Varnish to 3.0+? or this is not important?

Your extension is perfect! But need only fix bug with Varnish version
detection and add info in manual about possible errors with "socket
timout" and "Authentication required".
Also, can you tell me some recommendations with Magento load speed?
May be need other solutions? Update HTML templates, split JS and CSS
files or something else?

20.11.2012 0:15, Alex Headley пишет:

Alexey,
The Server List option should contain the Varnish /management
sockets/, not the /listening sockets/ (this is probably not real
clear in the documentation). The management socket is used to control
Varnish, while the listening socket is where it listens for requests.
Based on the config file you sent in earlier try changing the
|127.0.0.1:8080| to |127.0.0.1:6082|


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,

  1. Do you mean it should be more clear that the Server List setting refers to the management sockets? If so, I agree and am planning to significantly improve the documentation for the next release since it is currently pretty lackluster (to put it mildly).

  2. I will open a separate bug for this, I originally tested that feature with Varnish 2.1.5 so it may not work with the 2.1.3 version you are using.

I'm testing site and don't see any changes. Looks like the same page load time: 4-30 sec per page.

Turpentine does not serve any cached pages to you if you have the adminhtml cookie set (have logged into the admin panel), I usually recommend testing with a separate browser. I don't think I actually documented this anywhere and this "feature" is going away in the next release since it causes more problems than it solves.

@Springerton
Copy link
Author

Alex,

  1. Really I don't know how to better call this setting. Maybe management
    sockets.
  2. Ok.
  3. I've tested at Chrome and FireFox on other computer, but don't see
    changes :(
    Sometimes I think Magento never works fast. Too slow script.

I found a site with Varnish, which looks like fast, but have errors when
click on some products.
http://varnish.demo.aoemedia.de/

You can see author's blog, there are have extension with all settings.

20.11.2012 0:56, Alex Headley пишет:

Alexey,

  1. Do you mean it should be more clear that the |Server List| setting
    refers to the management sockets? If so, I agree and am planning to
    significantly improve the documentation for the next release since it
    is currently pretty lackluster (to put it mildly).

  2. I will open a separate bug for this, I originally tested that
    feature with Varnish 2.1.5 so it may not work with the 2.1.3 version
    you are using.

I'm testing site and don't see any changes. Looks like the same
page load time: 4-30 sec per page.

Turpentine does not serve any cached pages to you if you have the
adminhtml cookie set (have logged into the admin panel), I usually
recommend testing with a separate browser. I don't think I actually
documented this anywhere and this "feature" is going away in the next
release since it causes more problems than it solves.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

  1. Can you give me more details on how you're testing? There could be a few reasons you're not seeing any improvement.

I found a site with Varnish, which looks like fast, but have errors when click on some products.
http://varnish.demo.aoemedia.de/
You can see author's blog, there are have extension with all settings.

It looks like they're doing a sort of hacky AJAX hole-punching, where Varnish caches the page, then the session-specific parts of the page are replaced via AJAX requests. There is another extension that does something similar on Magento Connect I believe. The major feature of the next release of Turpentine (hopefully by the end of this week) will be ESI support so Varnish will do the fetching/replacing of the session-specific parts of the page. It will also fix some things in the current version that do not work real well (like caching being disabled after a customer logs in or adds something to their cart).

@Springerton
Copy link
Author

I'm testing in Google Chrome on 2 computers and look speed at tools. For
example results from Chrome:

Product Page
60 requests ❘ 60.32KB transferred ❘ 3.75s (onload: 3.93s,
DOMContentLoaded: 3.52s)

Category
112 requests ❘ 286.92KB transferred ❘ 18.42s (onload: 17.19s,
DOMContentLoaded: 16.38s)

Add to Cart (mega slow)
30-50 seconds every time

May be you also can use this AJAX method to Turpentine?

20.11.2012 1:34, Alex Headley пишет:

  1. Can you give me more details on how you're testing? There could be
    a few reasons you're not seeing any improvement.
I found a site with Varnish, which looks like fast, but have
errors when click on some products.
http://varnish.demo.aoemedia.de/
You can see author's blog, there are have extension with all settings.

It looks like they're doing a sort of hacky AJAX hole-punching, where
Varnish caches the page, then the session-specific parts of the page
are replaced via AJAX requests. There is another extension that does
something similar on Magento Connect I believe. The major feature of
the next release of Turpentine (hopefully by the end of this week)
will be ESI support so Varnish will do the fetching/replacing of the
session-specific parts of the page. It will also fix some things in
the current version that do not work real well (like caching being
disabled after a customer logs in or adds something to their cart).


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
Turpentine won't improve the speed for things like adding products to the cart since that doesn't interact with the caching at all (it doesn't make sense to cache an "action" like that since the point is to cause a change).

Regarding the tests with Google Chrome, I'm not familiar with Chrome's performance testing tools that you seem to be using, do you know if those tests support cookies or not? If they don't, then Turpentine will never serve a cached page because it always forces a request without a cookie to go to the backend so that the visitor gets the frontend cookie set. If a client doesn't support cookies they will never get a cached page (this is another thing that is sort of fixed in the next release).

May be you also can use this AJAX method to Turpentine?
I will be using ESI rather than AJAX for the hole-punching as it is faster, but the idea is the same.

@Springerton
Copy link
Author

Hi Alex!

Yes, I understood about "dynamical" parts of Magento. Really, if some
type of content will works higher, It also will be perfect. But now I
can't understand why page load speed didn't change.

I've tested my site at GTMetrix (http://gtmetrix.com) and have following
results:

Page Speed Grade:
(42%)
76%

  D

YSlow Grade:
(64%)
75%
_Page load time:_4.04s
_Total page size:_684KB
_Total number of requests:_49

This results are independent from my cookies or browsers.

If ESI is faster - that's really good. I will be waiting your next version.

20.11.2012 20:43, Alex Headley пишет:

Alexey,
Turpentine won't improve the speed for things like adding products to
the cart since that doesn't interact with the caching at all (it
doesn't make sense to cache an "action" like that since the point is
to cause a change).

Regarding the tests with Google Chrome, I'm not familiar with Chrome's
performance testing tools that you seem to be using, do you know if
those tests support cookies or not? If they don't, then Turpentine
will never serve a cached page because it always forces a request
without a cookie to go to the backend so that the visitor gets the
|frontend| cookie set. If a client doesn't support cookies they will
never get a cached page (this is another thing that is sort of fixed
in the next release).

May be you also can use this AJAX method to Turpentine?
I will be using ESI rather than AJAX for the hole-punching as it
is faster, but the idea is the same.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
The new release is out (v0.1.0), which should fix the issues you're seeing. Note that depending on your testing tools you may need to add the IP address you're using the tools from to the new Crawler IP Addresses setting.

@Springerton
Copy link
Author

Hey Alex!

Thanks for the new version. I've installed it, but something wrong with
my settings. When I click on buttons at Varnish management section I see
"Error flushing Varnish cache on: 127.0.0.1:6082"
|
My settings:

Varnish Version: 2.1.x
Server List: 127.0.0.1:6082
Varnish Authentication Key: a8e43039-a676-4c26-9f3b-9e3ebf64055a\n

Config File Location:{{root_dir}}/var/default.vcl

Backend Host: 127.0.0.1
Backend Port: 8080
Crawler IP Addresses: 176.9.45.196

Is it correct?
|

27.11.2012 3:11, Alex Headley пишет:

Alexey,
The new release is out (v0.1.0), which should fix the issues you're
seeing. Note that depending on your testing tools you may need to add
the IP address you're using the tools from to the new |Crawler IP
Addresses| setting.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
Sorry about this, I forgot you were using Varnish 2.1 . The new release dropped support for Varnish 2.1 (for now at least) as ESI support in Varnish 2.1 is... lackluster to say the least. I'm planning on re-adding Varnish 2.1 support at some point in the future but it requires some significant changes to the 2.1 VCL template.

Unfortunately because of that I don't really have a solution for you right now, sorry.

@Springerton
Copy link
Author

Alex, if you can tell me how yo update Varnish from 2.1 to 3.0.3 for
Debian, I'll do it. I mean SSH commands.

27.11.2012 21:45, Alex Headley пишет:

Alexey,
Sorry about this, I forgot you were using Varnish 2.1 . The new
release dropped support for Varnish 2.1 (for now at least) as ESI
support in Varnish 2.1 is... lackluster to say the least. I'm planning
on re-adding Varnish 2.1 support at some point in the future but it
requires some significant changes to the 2.1 VCL template.

Unfortunately because of that I don't really have a solution for you
right now, sorry.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
I am not very familiar with Debian but it looks like the are some instructions on the Varnish site. Note that you will want to run those commands as root.

@Springerton
Copy link
Author

Alex, I've tried to installed it early, but had error:

Configuration file `/etc/default/varnish'
  ==> Modified (by you or by a script) since installation.
  ==> Package distributor has shipped an updated version.
    What would you like to do about it ?  Your options are:
     Y or I  : install the package maintainer's version
     N or O  : keep your currently-installed version
       D     : show the differences between the versions
       Z     : start a shell to examine the situation
  The default action is to keep your current version.
*** varnish (Y/I/N/O/D/Z) [default=N] ? N
Starting HTTP accelerator: varnishd failed!
Message from VCC-compiler:
Expected ';' got '", "'
(program line 174), at
('input' Line 29 Pos 38)
             req.http.X-Forwarded-For ", " client.ip;
-------------------------------------####-----------

Running VCC-compiler failed, exit 1

VCL compilation failed
invoke-rc.d: initscript varnish, action "start" failed.
dpkg: error processing varnish (--configure):
  subprocess installed post-installation script returned error exit
status 1
configured to not write apport reports
                                       Errors were encountered while
processing:
  varnish
E: Sub-process /usr/bin/dpkg returned an error code (1)

I'm not server admin, so update something on server is difficult for me.
If you can make support Varnish 2.1, it will be great!

28.11.2012 0:18, Alex Headley пишет:

Alexey,
I am not very familiar with Debian but it looks like the are some
instructions on the Varnish site
https://www.varnish-cache.org/installation/debian. Note that you
will want to run those commands as root.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
I do plan to re-add support 2.1 again but it is not currently a high priority.

Your update failed because the Varnish default.vcl (probably located at /etc/varnish/default.vcl) was still the 2.1 version which is not compatible with 3.0 (there were some syntax changes). You could try renaming the file with mv /etc/varnish/default.vcl{,.bak} and try running the upgrade again.

@Springerton
Copy link
Author

Alex, really I can't install Varnish 3.0+ to the server. Every time I
see errors:

apt-get install varnish

Reading package lists... Done
Building dependency tree
Reading state information... Done
varnish is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 90 not upgraded.
root@dev ~ # /etc/init.d/varnish restart
-bash: /etc/init.d/varnish: No such file or directory

or such errors (after apt-get update)

...
Hit http://backports.debian.org squeeze-backports/main amd64
Packages/DiffIndex
Hit http://backports.debian.org squeeze-backports/main amd64 Packages
Fetched 13.8 kB in 0s (16.1 kB/s)
W: GPG error: http://old-releases.ubuntu.com jaunty Release: The
following signatures couldn't be verified because the public key is not
available: NO_PUBKEY 40976EAF437D05B5
W: GPG error: http://repo.varnish-cache.org squeeze Release: The
following signatures couldn't be verified because the public key is not
available: NO_PUBKEY 60E7C096C4DEFFEB
W: Failed to fetch
http://old-releases.ubuntu.com/ubuntu/dists/jaunty/universedeb/binary-amd64/Packages.gz
404 Not Found

W: Failed to fetch
http://old-releases.ubuntu.com/ubuntu/dists/jaunty/http://repo.varnish-cache.org/debian//binary-amd64/Packages.gz
404 Not Found

W: Failed to fetch
http://old-releases.ubuntu.com/ubuntu/dists/jaunty/squeeze/binary-amd64/Packages.gz
404 Not Found

W: Failed to fetch
http://old-releases.ubuntu.com/ubuntu/dists/jaunty/varnish-3.0/binary-amd64/Packages.gz
404 Not Found

E: Some index files failed to download, they have been ignored, or old
ones used instead.

After removing and installing Varnish 3.0, I have no many files on
server, include |/etc/varnish/default.vcl :(|

28.11.2012 3:16, Alex Headley пишет:

Alexey,
I do plan to re-add support 2.1 again but it is not currently a high
priority.

Your update failed because the Varnish default.vcl (probably located
at |/etc/varnish/default.vcl|) was still the 2.1 version which is not
compatible with 3.0 (there were some syntax changes). You could try
renaming the file with |mv /etc/varnish/default.vcl{,.bak}| and try
running the upgrade again.


Reply to this email directly or view it on GitHub
#5 (comment).

@Springerton
Copy link
Author

Hi Alex!

Today I've talked with my friend, he is server admin. And he told me,
that Varnish doesn't help if on server installed Apache+Nginx.
He think need remove Apache and in this time Varnish+Nginx can give results.

Also, do you know something about Magento's Zend FullPage Cache option
in settings and Memcached?

Best Regards, Alexey.

28.11.2012 3:16, Alex Headley пишет:

Alexey,
I do plan to re-add support 2.1 again but it is not currently a high
priority.

Your update failed because the Varnish default.vcl (probably located
at |/etc/varnish/default.vcl|) was still the 2.1 version which is not
compatible with 3.0 (there were some syntax changes). You could try
renaming the file with |mv /etc/varnish/default.vcl{,.bak}| and try
running the upgrade again.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
The backend webserver doesn't matter to Varnish so it should work fine using Nginx w/Apache for PHP (what I assume you meant by Apache+Nginx) or just plain Nginx.

Also, do you know something about Magento's Zend FullPage Cache option in settings and Memcached?

Not really, I've not used it with Memcached. I do know that it is not likely to work real well with Turpentine due to FPC cached pages not sending the Set-Cookie header.

@Springerton
Copy link
Author

Alex, I'll ask my friend to install completely new software on server
without Apache. only Nginx. After this he will optimize MySQL, add APC
and we test Magento. He think it will work fast like
http://varnish.demo.aoemedia.de/ - hundreds ms. per page. I don't know
that's real or unreal... I'll let you know when we finish with this tests.

What average page load time you have on server? And how many products
you have?

29.11.2012 21:16, Alex Headley пишет:

Alexey,
The backend webserver doesn't matter to Varnish so it should work fine
using Nginx w/Apache for PHP (what I assume you meant by Apache+Nginx)
or just plain Nginx.

Also, do you know something about Magento's Zend FullPage Cache
option in settings and Memcached?

Not really, I've not used it with Memcached. I do know that it is not
likely to work real well with Turpentine due to FPC cached pages not
sending the Set-Cookie header.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

aheadley commented Dec 2, 2012

What average page load time you have on server? And how many products you have?

I don't have any numbers to give you that would be relevant to a "real" site, I don't run a Magento store.

@Springerton
Copy link
Author

Alex,

Today I'm and admin have test different server and Magento solutions. We
reinstall all software and now: CentOS 6.3, Nginx (with no Apache), APC
and Varnish 2.1
.

Before Varnish we have following results:

Main Page
37 requests ❘ 17.63KB transferred ❘ 2.38s (onload: 2.16s,
DOMContentLoaded: 1.09s)
37 requests ❘ 17.40KB transferred ❘ 2.41s (onload: 2.26s,
DOMContentLoaded: 1.21s)

Cartegories
44 requests ❘ 16.60KB transferred ❘ 2.37s (onload: 2.37s,
DOMContentLoaded: 1.33s)
44 requests ❘ 16.46KB transferred ❘ 2.29s (onload: 2.29s,
DOMContentLoaded: 1.32s)
41 requests ❘ 15.62KB transferred ❘ 2.20s (onload: 2.06s,
DOMContentLoaded: 1.05s)

Product Page
47 requests ❘ 17.57KB transferred ❘ 2.48s (onload: 2.48s,
DOMContentLoaded: 1.44s)
48 requests ❘ 17.84KB transferred ❘ 2.25s (onload: 2.25s,
DOMContentLoaded: 1.34s)

Add to Cart
36 requests ❘ 348.98KB transferred ❘ 3.35s (onload: 3.06s,
DOMContentLoaded: 2.05s)
36 requests ❘ 29.18KB transferred ❘ 2.59s (onload: 2.29s,
DOMContentLoaded: 1.26s)

After Varnish was enabled:

Main Page
37 requests ❘ 28.14KB transferred ❘ 2.60s (onload: 2.45s,
DOMContentLoaded: 1.33s)
37 requests ❘ 28.14KB transferred ❘ 2.61s (onload: 2.39s,
DOMContentLoaded: 1.13s)
37 requests ❘ 28.14KB transferred ❘ 2.72s (onload: 2.49s,
DOMContentLoaded: 1.18s)

Cartegories
41 requests ❘ 26.36KB transferred ❘ 2.32s (onload: 2.15s,
DOMContentLoaded: 1.17s)
44 requests ❘ 27.20KB transferred ❘ 2.38s (onload: 2.38s,
DOMContentLoaded: 1.52s)
44 requests ❘ 27.35KB transferred ❘ 2.56s (onload: 2.57s,
DOMContentLoaded: 1.56s)

Product Page
48 requests ❘ 28.67KB transferred ❘ 2.83s (onload: 2.83s,
DOMContentLoaded: 1.57s)
48 requests ❘ 28.64KB transferred ❘ 2.54s (onload: 2.55s,
DOMContentLoaded: 1.36s)
47 requests ❘ 28.30KB transferred ❘ 2.57s (onload: 2.57s,
DOMContentLoaded: 1.38s)

Add to Cart
36 requests ❘ 40.51KB transferred ❘ 2.89s (onload: 2.66s,
DOMContentLoaded: 1.54s)
36 requests ❘ 40.67KB transferred ❘ 2.83s (onload: 2.49s,
DOMContentLoaded: 1.40s)
36 requests ❘ 40.79KB transferred ❘ 1.7min (onload: 2.48s,
DOMContentLoaded: 1.34s)

So really I have no difference :(
Server admin enabled JS/CSS compression and enable Magento compilation.

We can't get the same results as on this site:
http://varnish.demo.aoemedia.de/
May be developer have something else on server...

Best Regards,
Alexey

02.12.2012 10:34, Alex Headley пишет:

What average page load time you have on server? And how many
products you have?

I don't have any numbers to give you that would be relevant to a
"real" site, I don't run a Magento store.


Reply to this email directly or view it on GitHub
#5 (comment).

@aheadley
Copy link
Contributor

Alexey,
I've just released version 0.2.0 which re-adds support for Varnish 2.1 so you may wish to update and try out the ESI support.

@Springerton
Copy link
Author

Hi Alex!

Ok, thanks, I'll try it today.

13.12.2012 0:18, Alex Headley пишет:

Alexey,
I've just released version 0.2.0 which re-adds support for Varnish 2.1
so you may wish to update and try out the ESI support.


Reply to this email directly or view it on GitHub
#5 (comment).

@ifnull
Copy link

ifnull commented Apr 7, 2014

For anyone else who came to this page and realized that you were missing \n but found that you still had the same problem, make sure you refresh your cache. The Varnish Authentication Key may be cached.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants