PhantomJS client died while processing

[konklone]

This YAML file was last worked on 15 hours ago, has had multiple successful passes, but also some errors with the message PhantomJS client died while processing.

http://efforg.github.io/congress-forms-test/?bioguide_id=S001189

It seems like this error is a result of server load, not YAML correct-ness. So, it seems like errors of this type shouldn't affect the success % for a given YAML file.

[konklone]

Reference - could be intermittent font issue - teampoltergeist/poltergeist#44

This hack makes Rack 403 on TTF files: https://gist.github.com/goosetav/3996516

Maybe also related: teampoltergeist/poltergeist#232

[konklone]

I don't see TTF files loaded on the referenced form, but I do see JS errors, which might make the 3rd link above (to a still-open issue) relevant.

[jaredhirsch]

Do we know what version of phantomJS is running on the server currently?

[jaredhirsch]

yeah, so, you can actually test this manually to find the JS error.

first, have phantom installed (I've got version 1.9.7 locally).

then, put this stuff inside a js file, say, foo.js:

var url = 'https://austinscott.house.gov/email-me/';
var page = require('webpage').create();
page.open(url, function(status) {
  if (status != 'success') {
    console.log('hmm, something went wrong');
  } else {
    console.log('loaded ok');
  }
  phantom.exit();
});

Then you can run the file at the command line: phantomjs foo.js

For me, the output looks like this:

TypeError: 'undefined' is not an object (evaluating '= $.propHooks')

  https://austinscott.house.gov/js/jquery.scrollTo.js:26
  https://austinscott.house.gov/js/jquery.scrollTo.js:5
  https://austinscott.house.gov/js/jquery.scrollTo.js:261
loaded ok

When I load the page in Chrome, I see a different but related error in the console (along with a crapton of mixed-content warnings):

Uncaught TypeError: Cannot set property 'scrollLeft' of undefined jquery.scrollTo.js:12
(anonymous function)

The line that throws an error tries to assign to $.propHooks, which is undefined on the page. Basically, it looks like phantom is barfing because the page itself throws TypeErrors. To me, this seems an awful lot like teampoltergeist/poltergeist#232

[jaredhirsch]

Here's another one that always fails:

http://efforg.github.io/congress-forms-test/index.html?bioguide_id=L000480

Again, if you load it up in a browser, you can see that the page itself is broken and throws a TypeError: Uncaught TypeError: Cannot read property 'getElementsByTagName' of null

And trying to open it from the command line with phantom gives me the same error, just worded a bit differently:

TypeError: 'null' is not an object (evaluating 'centerbox.getElementsByTagName')

  https://lowey.house.gov/common/js/customForm.js:10
loaded ok

So, I suspect that busted pages are the source of this error.

I'm not sure how best to fix it, though.

[jaredhirsch]

hmmmm. This page doesn't throw any JS errors, but it has the same phantomJS failure:

http://efforg.github.io/congress-forms-test/index.html?bioguide_id=L000566

So, maybe JS errors aren't the cause.

[Hainish]

It would be great to get to the bottom of this. Most of the remaining forms are erroring out on the PhantomJS layer (unitedstates/contact-congress#285, unitedstates/contact-congress#296, unitedstates/contact-congress#30, unitedstates/contact-congress#519)
Filling out these forms with PhantomJS is preferable, but there is another possible solution to this problem. These few remaining congress members we're getting errors on could be completed with Watir (selenium) if need be. Watir support is already built in to congress-forms. If we decide to do this, we'd have to specify somewhere that these members should be using the Watir driver, possibly in the YAML for contact-congress, though there is possibly a more appropriate place, since contact-congress is trying to be implementation-agnostic.

[j-ro]

Could we have Watir as a failover option? If a PhantomJS error happens, try Watir before reporting failure?

[konklone]

Failover sounds like a good idea.

[Hainish]

@j-ro the problem with automatic failover is it's hard to determine if the error is caused by PhantomJS or not just by the error message. Another problem is that we were seeing a lot more PhantomJS errors with high load. If we fail over to Watir (which is heavier) when we start seeing PhantomJS errors, we could be making the problem worse. Ideally I'd like to mark certain congress forms as we see them encountering errors with PhantomJS.

Strangely, when I run some of these on my local, no such PhantomJS errors occur - seems to be some problem with the production env.

[j-ro]

We've had others report the same -- that it works locally but not in our production environment.

[jaredhirsch]

Do you guys have docs around what's running in your production environment? I could try spinning up a VM locally with the same distro/packages, and try to repro the errors that way.

[scrozier]

Can confirm that W000815 has succeeded on each of the 30+ tests (with curl) I've run on my local development machine (while still failing intermittently on the production tester). I'm running the same phantomjs (1.9.7) as in production.

[Hainish]

I've created a vagrant file to bootstrap the congress-forms environment. In both the 32 and 64 bit vagrant envs, L000480 passes. This means its definitely something to do with our production env, maybe due to memory limitations, still checking on this.

[drinks]

Cynically, this could also be IP-dependent ;)

[Hainish]

I just spun up another AWS instance, and I got one success but the rest are PhantomJS errors for L000480.

[Hainish]

The four members that were encountering PhantomJS problems (unitedstates/contact-congress#285, unitedstates/contact-congress#296, unitedstates/contact-congress#30, unitedstates/contact-congress#519) are now being filled out by capybara-webkit, successfully. Any remaining issues with PhantomJS/Poltergeist have to be fixed within those respective projects.

So PhantomJS (Poltergeist) is the preferred filling method, and Capybara-Webkit is the backup plan when Poltergeist fails. This seems to be okay, since Capybara-Webkit is still pretty fast. The one thing we might encounter ist that Capybara-Webkit doesn't do well at figuring out captcha locations. So if we encounter an instance where PhantomJS is failing and there is a captcha, we may have to look for another solution.