JS not blocked/unblocked properly in incognito mode (was "Permissions Not Correctly Applied")

[GuardianMajor]

When you come to a site and you click on HTTPsb and make your choices, some permanent maybe, or maybe not, doesn't matter, it reloads the page but the scripting is not always turned on. This happens in nearly 2/3 of the situations as you can see here:

As you can see the scripting is still disabled despite being allowed and the page reloading. Manually reloading the page has no effect, and even force reloading the page has no effect. In fact, the tab on which we are on is completely useless after this point, we have no choice but to put the link in another tab and everything works fine and the permissions are applied. This is a huge bug and flaw and considerably affects usability.

[gorhill]

I've never come across this problem (except for issue #35, which is addressed by forcing a reload of the page.)

Things that would help me investigate further:

• Do you have a web page I can use which exhibit the problem?
• Can you check in "Content Settings" the block/allow status of javascript on the hostname of the page?
• What are other extensions you have installed (I see two other icons),I wonder if there another one which also tries to control javascript block/allow list in Chromium?
• What version of Chromium/Chrome are you using?

[gorhill]

Also, I appreciate you took the time to provide a snapshot, it would be great if you could provide a snapshot of HTTPSB matrix for a page which exhibit the problem. However, one of the only way to snapshot an extension popup menu is to check "Developer mode" on the "Extensions" page, which will give you access to the option "Inspect pop-up" when right-clicking on HTTPSB icon. This allows to make the pop-up menu stay opened while you snapshot the screen.

[GuardianMajor]

I encounter it quite a bit. I am sorry I didn't think to include the matrix of the HTTPSB because everything was allowed minus the google-analytics.com but you are right, that might have helped, my bad, I will include that next time.

Don't worry about snapping a shot of the matrix, I am well adept at it :) I run in developer mode all the time since I develop, test and manage my own extensions as well. I like your concept here and that's why I am trying to help. I have been a contributor to NoScript and so its only natural for me to help you, your solution so far is the one I like the best, the rest are a joke. I'll get you that snap as soon as I encounter the issue with another site.

In the meantime, we have an icon for removing all temporary permissions, where can we see all the ones we have padlocked? So we can remove one and/or all of them (sort of if you want to start over kind of thing). On NS for example we have access to the whitelist for that and the blacklist can be dumped in a single line. It would be nice to have an interface to access the padlocked items, red or green. Come to think of it, this probably should be its own enhancement request. You can split and move this paragraph if you see fit.

[GuardianMajor]

Okie dokie, lucky for you I came across it while I was doing some coding and voila, here is a shot of the whole kit and kaboodle, let me know what else you might need.

[gorhill]

Thank you very much for your time.

I reproduced as close as I could what I see in your screenshot, and I just can't get the
same problem.

I found that the entry cpngackimfmo... was an extension to capture screen, so I installed it.

Something I find puzzling is that in my matrix there is an entry for gstatic.com and facebook.com which
I don't see in your matrix.

Since I can't reproduce the problem, the next step is to investigate if there is another
extension interfering, creating its own "block" rule in "Settings" -> "Privacy" -> "Content settings" -> "Javascript" -> "Manage exceptions...".

At this point I don't see any other explanation than this, so looking at the list of rules has to lead to the answer.

Also, I was wondering if you get something useful at HTTPSB console.

[GuardianMajor]

Thank you for taking a look. I appreciate that. Yes that extension you see is the Google screengrabber, something light that gets injected into the page, very small footprint.

I have had the problem occur several times, in fact I was here just now to give you a new screenshot and matrix when I saw your message, I am surprised the system didn't notify me of your response so I can comment.

Here is what happens at StackExchange:

This happens alot to me and what puzzles me the most is that it makes the tab itself worthless no amount of refreshing salvages the tab, you have to open in a new tab, that just blows my mind there. what you see on my matrix is what I see, don't know what to tell you, sometimes when things are "reloaded" properly - meaning this issue we are talking about doesn't happen, new items will show up, but what I gave you is all I had on my matrix.

I can send you a copy of my exceptions, there is nothing in there that is created by anything else. If I disable or delete HTTPsb, it results in just a very empty list, always has been. As for the console, Nope, absolutely nothing there, In fact for the example I just posted, there was a single entry in your console about its favicon.ico and that's IT. Nothing else.

So tell me what else I can do and its yours.

[gorhill]

Just to be sure, in the above case, you saw this entry in the list:

stackexchange.com block [icon which indicates that this is from an extension]

?

[gorhill]

Thinking about something... I enable/disable the script block/allow rule only when there is an explicit webRequest for the top page (the one which might contain inline scripting), thus if ever for some reasons the refresh of the top page doesn't generate a webRequest event, this means the rule to unblock won't be added and javascript will still be blocked.

Now can this happen? Can the page be pulled by chromium from the cache which maybe won't trigger a webRequest event? Now I wonder.

[gorhill]

I used to have code to block/allow outside webRequest, when a URL was being navigated to, but I removed it in order to minimize as much as possible required permissions, I think I will put that back, and if you want, you could install the developer version from github (I could make a rolling version for people who want to test the latest dev version.)

[gorhill]

One way to confirm if this could be the problem, is to force-clear the cache when the problem happens. If clearing the cache and forcing a reload fix the problem, then this is it.

[gorhill]

Argh... I forgot there was this:

https://github.com/gorhill/httpswitchboard/blob/master/js/tab.js#L397

which should actually block/allow regardless wthether the top page goes through webRequest().

So I am really at a lost to understand why the problem. Only thing left I can think of is that another extension is setting the block rule.

[gorhill]

Found something: I can reproduce exactly the behavior you are reporting from within an incognito mode window. Was it your case?

1. Open incognito mode window
2. Go to http://www.isjavascriptenabled.com/
3. From HTTPSB matrix, enable JS for www.isjavascriptenabled.com
4. Force refresh

Result: javascript is still disabled.

Fix: apply javascript setting to incognito scope as well.

Documentation says incognito mode inherit exceptions from regular mode. Since this is not happening, I opened a bug for chromium: https://code.google.com/p/chromium/issues/detail?id=319400

[GuardianMajor]

Ok my friend, a few comments and I need to reply to each one so give me a second to try and organize it so you can follow my responses, I with Github had the ability to reply to specific comments so it could be better threaded and followed. Anyway.

1. You say to look in my entry list for stackexchange.com block, but there was nothing blocked in there once the permissions were given the "blocks" disappear from the console. I didn't see anything.
2. Its not pulling from cache for a couple of reasons, one I have it configured not to do that, second I run incognito ALL the time, so when a tab is gone, so is the goods attached to it, barring persistent cookies but no cache. So not sure that's our issue but I am open to testing it, tell me what you need.
3. Yes, I really think that putting back the external requests would be a great idea. You can try to minimize the impact on the novice users by offering a switch near the same options for strict blocking, cache dumping and background blocking and disable it by default to help them but leave it for the advanced users to enable it, just a thought. But I really thought it was covered by another part of your code, maybe I misread it.
4. I will do the force cache dump in case it helps. It will be a chance to confirm.
5. I thought I had seen it. If you are at a loss, imagine how I feel ;) but seriously I am trying to figure it out, I have attached debuggers and trying to figure it out, so far I did hit one incident that may look sort of promising but looking into the code, it shouldn't be causing an issue. I will share with you just in case two heads can figure this out faster. I noticed that with Adblock Plus installed, occasionally HTTPsb will cripple a request and redirect it back to the extension, this could cause the request to technically get stuck and not be resolved. It shows up in chrome as an extension error. Still investigating but I don't think its our issue to be honest.
6. BINGO that's what's happening, oh I am so relieved you found it, I was beginning to think I am losing my mind here and glad you caught the bug and reported to them, hopefully it gets fixed. In the meantime anything we can dirty patch to work around it until they fix it and then we can deprecate it, or just take it as they come and live with it for now? Thoughts?

Thanks for looking into it though, I am glad I stuck with it and you did too. Let me know if anything I can do to help further.

[GuardianMajor]

Thank you, gonna give this baby a run and let you know if anything. Looked at the code, good clean effort, crossing fingers 👍 Waiting for the 5.5 to push, I have forced developer update but nothing yet, last time it pushed quicker, will give it another try in the morning, gotta call it a night anyway, take it easy.

Off-topic: Sind Sie Deutsch? (my german is rusty)

[gorhill]

Nein, Québec

I greatly appreciate you reported/helped with this one, this was a very serious bug given the purpose of the extension, it's why I labelled it "top priority".

[GuardianMajor]

AH! My dumb mistake, I should have looked. I must admit I confused you with another developer friend who does TamperMonkey. I had just sent him a message when I came here to respond to you and something in my brain got scrambled. I am embarrassed. parler français ?

It was my pleasure to help and I will continue to help in any way that I can, you are doing something good and I respect that. I wish you all the luck and if anything i can do gets you there, then I consider it an honor to contribute. I am still waiting for the 0.5.5 to push out since force update hasn't updated it yet, still says 0.5.4 since last night. I KNOW you rolled the version number, so it should reflect on the extension unless you forgot which I doubt ;)

[GuardianMajor]

My dear friend, a quick question for clarity please. In accordance with your Issue #35 suggestion that by default JS be disabled, doing so results in EVERY page whether permissions are given or not, for the icon that we encountered in THIS issue to appear indicating that the JS is disabled. It seems that that option is overriding your permissions/exceptions if you will to enable scripting for that page. Is that an expected and desired effect or is it another unpredictable behavior by Chromium code again?

See this as an example, the site is permitted as seen on the matrix, however the icon indicating JS is disabled is shown, so what happened here? Expected behavior?

Also note, this happens on EVERY TAB, EVERY SITE, as opposed to how #53 issue affected us. I thought maybe we need to take a look at this and if necessary update/change the instructions given on the wiki about #35 issue recommendation. Please advise, I wasn't sure if I should open a new ticket for this, figured I minimize the hassle for now by putting it here, create an official issue for it later if necessary.

[gorhill]

This chromium issue will often cause this:

• If javascript is enabled by default in chromium settings, a site with blacklisted script might have its inline script run the first time the page is loaded -- and no icon showing javascript is blocked (very bad security wise).

or

• if javascript is disabled by default in chromium settings, a site with whitelisted script might have its inline script blocked the first time the page is loaded -- and an icon showing javascript is blocked (better security wise despite bug).

In both case, a force refresh resolves the issue.

[GuardianMajor]

The reason I wrote you is that force refresh DIDN'T fix the problem. I had assumed what you explained above in my approach to it, that's why I was unsure why it was happening. Also, even though the web store has listed the latest 0.5.5 version, the installed version does NOT update to the 0.5.5 even with forced update trigger. Do you know what's going on? One last addition, it seems that it is actually MISSING resources, items that don't show up on the matrix and the permission for which are NOT processed despite having previously shown up, been allowed and have worked but then suddenly it won't show up no matter how much you reload. But if you disable HTTPsb it shows up just fine. An example being:

The resource (stackauth) is missing and therefore with it goes the XHR that would authenticate across the multiple sites on the same network. What's happening? Nothing shows up in the console either which means somehow it never dealt with the resource in any way.

Also found out why the update doesn't work, its throwing an error due to a null value for the parentnode.

[gorhill]

Concerning the missing sub_frame from stackauth.com: I got it the first time I went to stackoverflow.com. Once cookies from the site are dropped on my machine, it appears the site won't try to create the sub_frame from stackauth.com. So this is a site behavior. You can see yourself by manually deleting cookies from stackoverflow.com, then refresh the page: stackauth.com will show up.

Regardless, clicking "log in" at the top lead to a page which sees stackauth.com.

[gorhill]

Concerning the strange state of your browser (javascript not enabling, update not happening, etc.), I suspect sometimes chrome can end up in a bad state for whatever reason. Yesterday I experienced a strange behavior on a laptop: I deleted all extensions, I cleared the cache, I restarted chromium multiple times. No matter what, there were "rules created by an extension" stuck in the javascript exception list, despite no extension being left installed. I finally resolved this issue by deleting my chromium user account.

Looking at the system file in chromium config folder, I suspect quitting chromium and one by one deleting the contents of...

• "Extension Rules"
• "Application Cache"
• "Extension State"

... might help.

See if after each deletion to see if it helps. If nothing good comes from this, will see what else we can think of. I suspect if I had deleted "Extension Rules" instead of deleting my account would have resolved my problem yesterday.

(currently working on import/export of HTTPSB rules, so soon it will matter less to remove completely the extension to force a complete reinstalled).

[gorhill]

Re. "throwing an error due to a null value for the parentnode"

Where is this happening, in the console of which page did you see this?

[GuardianMajor]

For the stackauth issue, I know if the cookies are dropped it won't show but the cookies shouldn't be dropped as I don't have it configured to do that, the site is allowed on the matrix, so it shouldn't affect it, so why is it dropping it? I know login will get you there, but my issue is that I open multiple sites within StackExchange, SO being one of them, so the stackauth is very useful to me and very important to make sure I don't spend half hour clicking login buttons and then the subsequent SE button to have it kick in. Site behavior aside, it works fine without HTTPsb and given that I have not configured anything that would affect my cookies and permissions are already set and there, it shouldn't happen with it active either. So hence my question.

[GuardianMajor]

As for the update, it happens on the extensions page when I click on [Update Extensions] in developer mode. Here is the console output:

Uncaught TypeError: Object function BrowserOptions() {
    OptionsPage.call(this, 'settings', loadTimeData.getString('settingsTitle'),
                     'settings');
  } has no method 'removeMessageCenterOption'
(anonymous function)
Uncaught TypeError: Cannot read property 'parentNode' of null options_bundle.js:7096
ContentSettings.setOTRExceptions options_bundle.js:7096
(anonymous function)

[I wish there was a prettier way to post code here, yuck]

Investigating if that's an internal issue or not but so far can't narrow down the culprit. Since the last updated to 0.5.4 pushed within minutes of the update when I force updated, having two days elapsed, not working and issuing this error seems a bit unlikely to be a core issue. Ideas?

[gorhill]

"the cookies shouldn't be dropped"

When I said "dropped on your machine" I meant "written to your machine"... stackauth.com doesn't appear if cookies from stackoverflow.com are present. The cookies from stackoverflow.com are not removed (isn't it what you want?), hence stackauth.com won't appear.

"it works fine without HTTPsb"

Ah ok, you were originally raising the issue stackauth.com does not appear which I explain this is normal given how the site is designed. So what exactly doesn't not work? What behavior do you expect that is not happening (or the reverse)? I need exact steps (all details matter no matter how trivial they may seem) of what I should try to reproduce what you consider broken. This will allow me to validate, and if validated, to investigate, then fix.

[gorhill]

Questions:

• Is "Process behind-the-scene" option checked in "Settings" of HTTPSB? If so, turn it off.
• Is option "Delete cookies" checked in "Settings" of HTTPSB? If so, turn it off.
• Is option "Delete local storage" checked in "Settings" of HTTPSB? If so, turn it off.

[GuardianMajor]

Sorry, I misinterpreted the terminology, I am used to dump meaning to the system and dropped meaning poof :)
Ok, let me explain my issue, I think there might be smidgen of confusion, possibly due to my explanation, so let me be clear as mud if I can ;)

I ALWAYS go to StackExchange.com as my primary start, in fact this link to be exact (https://stackexchange.com/filters/99916/my-filter) and if I am not logged in, I log in and then proceed to load the other sites that I need to check in on, as follows:

this triggers the above in tabs that then log in automatically with stackauth and show a refresh on the top of the page to see the page logged in. This is the normal behavior and has worked like this with no issue for ever. But with HTTPsb running, no matter what is allowed already, this behavior is crippled, no stackauth login. As you can see, since the Site#28 is opened in the furthest right most tab, it is always the last to load completely and last to log in. But currently all the tabs load but NOTHING logs in like before. This is the behavior I am not understanding. Disabling/Removing HTTPsb resolves the issue and everything works as expected so narrows the suspect. Did I explain better this time?

[GuardianMajor]

None of the three options you have listed are selected at the moment because I am using this program on my semi-production environment so I can put it through the rigors of normal use so I can debug it for you better but those options make that extremely difficult on my normal function :)

I removed all the instructions to make this shot more friendly. Maybe we can make of the laborious explanations into collapsible divs? I can do that for you if you'd like. Shouldn't take much, just saying.

As you can see, this is how I have it, simple as that right now. the large matrix is because I have a large monitor and can afford a better look at the matrix, especially when the long items get cropped off this helps keep them inbound :)

[gorhill]

Ok re. stackexchange.com, I think I understand what might be the problem, and that would be issue #35. Do you run with Javascript disable by default? If so, that means that when a page load for the first time since you launched chromium, there is a good chance inline scripting wont be executed. To validate whether this is the problem, set javascript on by default in chromium "Settings" and see if that makes your scenario go through normally. If so, then the problem is bug #35, and for which I entered a bug in chromium today.

[gorhill]

Re. the update not working, I have no clue -- clearly something is failing in chromium. My wife's machine updated fine (she didn't use "force" though). Maybe removing the content of some of the folders used for transient data in chrome config directory would help?

[GuardianMajor]

No, I changed the behavior back to the enabled by default because I didn't like the unexpected side effects of #35 recommendations, sorry.

[GuardianMajor]

What version is she running? 29.x still? The reason I ask is because I updated to 30 and you are right, it might be an issue with that, I am investigating but it would help to know if she got it successfully on 29 or 30 so it would narrow it down for me a bit. On 29 it worked fine with v0.5.4 but with 30 and 0.5.5 I had to do it manually, didn't work automatic update or forced update. Debugging and rebuilding the chromium code is a pain in the BLIP

[gorhill]

She is using 30 (LinuxMint 15).

Re. stackex, I guess the last thing I can think of is to look at the requests blocked in the "Stats"page and see if anything was blocked which might have prevented the log ins to properly work. Every request is recorded, so there had to be something in there showing as blocked which is required by stackex.

Sorry for all the time you have to spend on this. Early kinks are usually the worst, so hopefully the extension can come out of this phase sooner than later. Thanks to hang in there.

[GuardianMajor]

Well that dashed my hope that this was a Chrome issue :(

That's the problem I have and I mentioned before, NOTHING about it shows up on the stat page either it is simply acting like it doesn't exist. That's why I am saying it is not being processed at all as if it is not even there.

I am sure you will come out of this my friend and anything worth doing is worth sticking with, so I don't mind the the effort and I am glad that in the the short time I have been able to report and we have been able to fix at least two issues and if more come up, we'll tackle them too. Luckily I am not a novice, so I am hopeful to be able to help more as time goes on.

[ghost]

@ GuardianMajor: Re the update issue: Did you check if you have "Process behind-the-scene HTTP requests." enabled in HTTP Switchboard? I had it and my add-ons wouldn't update anymore. After disabling it I got v. 0.5.5. (and some other updates).

[GuardianMajor]

Hello tlu, from NS Forum? If so, yes my friend, as i posted my settings earlier in full if you check a few posts up, I did not check the background, in fact, I didn't enable any of the further restrictions for now due to the reasons I mentioned to Ray because I am debugging for him on a semi-production machine and it would slow me down too much to deal with it like that, but later I will test those extensively as well.

[ghost]

Hi GuardianMajor, yes it's me ;-) Glad to see you here to support Raymond. HTTPSB is definitely the best "Noscript-like" Chrome add-on so far. It's not yet bug-free but has a lot of potential.

[GuardianMajor]

Yeah we are still in early alpha on this, hence pre v1 phase ;) but it has active development, dedicated tester/bug hunter, and quick/responsive bug fix/code updates, so we should be able to iron out stuff pretty quick. In fact, we are dealing with chromium on parallel issues affecting things as well, so it should prove generally useful to everyone too in the long run. Anywho, great to see you and hope you will be a tester too, at least you can find one of us in more than one place ;)

[GuardianMajor]

Small update my friend, the fix you implemented to deal with this has a small unpredictablity as seen here, even with the permissions done, the workaround applied, it will not show the JS missing icon (as seen in the green circle) but none the less the page shows there is no JS (as seen in the red circle). Sorry but I had to censor this a bit because its where I teach.

Reload DID fix the problem and applied correctly but as you know, reload is not always an option as there could be duplicated form submission and stuff like that but in this case not an issue. Just a heads up.

[gorhill]

Sounds like exactly issue #35. I still experience it regularly (many times today). I filed a bug with chromium about this, but I am puzzled by the response, as I thought this was a long time known issue (other blockers have same issue). I have a nice step by step which makes it easily occur (using acidtest), but since it has the chance to work just fine, I fear I will be dismissed again. I think I will try again but forcing chromium to use only one core (I don't know chromium enough to know whether it is an async or a race condition problem).

[GuardianMajor]

Hmm, seems to be no relief from the way Chrome handles things huh? I guess too much to ask for it work intuitively huh? Yeah people over at Chromium are downplaying it claiming that they can't reproduce it, which even further adds to Chrome's unpredictable behavior which should support our request but seems to make it harder, since they are not very willing to make their code better. Sad.