Some examples of running in a background job environment?

[cheeyeo]

Hi

I have been trying to use webshot in a multi threaded / concurrent environment but phantomjs is producing unexpected results. Sometimes it works but on occassions if multiple background jobs are using webshot to grab the url screenshots, it crashes phantomjs with a fatal error

Is phantomjs not meant to be run in a multi threaded environment or if so can you please provide some form of setup instructions?

thanks

[vitalie]

It's not recommended to start multiple PhantomJS concurrently. You should serialize the access, treat it as unreliable and monitor it with daemontools, god, monit, etc. My current setup:

   S3 <-- CloudFront + 404 handler <-- User Request
    ^
    |
Worker <--> Screenshot Service API (Heroku)
    |
    ^
    |
Queue < -- App

Screenshots are cached on S3 and served through CloudFront CDN. Heroku restarts the service automatically when it crashes and CloudFront serves a default image with a low TTL if the screenshot is not yet ready. Take a look at this gems too:

unicorn-worker-killer
unicorn-soft-timeout