Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

avoid deadlock in AMQPUrlReceiver hopefully #102

Merged
merged 2 commits into from
Nov 3, 2014

Conversation

nlevitt
Copy link
Contributor

@nlevitt nlevitt commented Oct 17, 2014

Use interruptible locking in the one spot where it's needed to avoid this deadlock.

"AMQPUrlReceiver-starter-restarter" prio=10 tid=0x00007f62bd0dc000 nid=0x27b5 waiting for monitor entry [0x00007f62ec319000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.archive.crawler.frontier.AMQPUrlReceiver$StarterRestarter.run(AMQPUrlReceiver.java:119)
        - waiting to lock <0x00000000cbcecb98> (a org.archive.crawler.frontier.AMQPUrlReceiver)

"org.archive.crawler.frontier.BdbFrontier@4c2d0c64.managerThread" prio=10 tid=0x00007f62bd471000 nid=0x27ae in Object.wait() [0x00007f62adbda000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000cc491e00> (a org.archive.crawler.frontier.AMQPUrlReceiver$StarterRestarter)
        at java.lang.Thread.join(Thread.java:1281)
        - locked <0x00000000cc491e00> (a org.archive.crawler.frontier.AMQPUrlReceiver$StarterRestarter)
        at java.lang.Thread.join(Thread.java:1355)
        at org.archive.crawler.frontier.AMQPUrlReceiver.stop(AMQPUrlReceiver.java:164)
        - locked <0x00000000cbcecb98> (a org.archive.crawler.frontier.AMQPUrlReceiver)
        at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:236)
        at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:213)
        at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:213)
        at org.springframework.context.support.DefaultLifecycleProcessor.access$2(DefaultLifecycleProcessor.java:206)
        at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:352)
        at org.springframework.context.support.DefaultLifecycleProcessor.stopBeans(DefaultLifecycleProcessor.java:195)
        at org.springframework.context.support.DefaultLifecycleProcessor.stop(DefaultLifecycleProcessor.java:103)
        at org.springframework.context.support.AbstractApplicationContext.stop(AbstractApplicationContext.java:1241)
        at org.archive.crawler.framework.CrawlController.completeStop(CrawlController.java:392)
        at org.archive.crawler.framework.CrawlController.noteFrontierState(CrawlController.java:663)
        at org.archive.crawler.frontier.AbstractFrontier.reachedState(AbstractFrontier.java:442)
        at org.archive.crawler.frontier.AbstractFrontier.managementTasks(AbstractFrontier.java:399)
        at org.archive.crawler.frontier.AbstractFrontier$1.run(AbstractFrontier.java:316)

…to be deadlocked, found that the thread trying to shut down was being starved of the lock by another thread (the StarterRestarter) locking and unlocking it repeatedly, which can apparently happen because the lock was "unfair"
vonrosen pushed a commit that referenced this pull request Nov 3, 2014
avoid deadlock in AMQPUrlReceiver hopefully
@vonrosen vonrosen merged commit f5cf30e into internetarchive:master Nov 3, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants