Skip to content

Commit

Permalink
Disable test_joing_running_workers for TSAN. (pytorch#46966)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: pytorch#46966

These tests had false positives in TSAN for modifying thread local
variables:

```
WARNING: ThreadSanitizer: data race (pid=5364)
  Write of size 8 at 0x7b2c0004ff70 by thread T2:
    #0 free <null> (libtools_build_sanitizers_tsan-py.so+0xde6ad)
    #1 __GI__dl_deallocate_tls

  Previous write of size 1 at 0x7b2c0004ff71 by thread T3:
    #0 at::GradMode::set_enabled(bool) caffe2/aten/src/ATen/core/grad_mode.cpp:20 (libcaffe2_ATen-core.so+0x40e013)
    #1 torch::autograd::set_grad_enabled(_object*, _object*) caffe2/torch/csrc/autograd/init.cpp:143 (libcaffe2__C_impl_cuda.so+0x115ef0e)
    #2 _PyMethodDef_RawFastCallKeywords

  Thread T3 (tid=5385, finished) created by main thread at:
    #0 pthread_create <null> (libtools_build_sanitizers_tsan-py.so+0xc5a86)
    #1 PyThread_start_new_thread
```
ghstack-source-id: 115330433

Test Plan: waitforbuildbot

Reviewed By: mrshenli

Differential Revision: D24584411

fbshipit-source-id: e35f704dfcb7b161a13a4902beaf8b1e41ccd596
  • Loading branch information
pritamdamania authored and facebook-github-bot committed Oct 29, 2020
1 parent 9fefb40 commit ad260ae
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions test/distributed/_pipeline/sync/test_worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from torch.distributed._pipeline.sync.microbatch import Batch
from torch.distributed._pipeline.sync.stream import CPUStream
from torch.distributed._pipeline.sync.worker import Task, spawn_workers
from torch.testing._internal.common_utils import TEST_WITH_TSAN


class fake_device:
Expand All @@ -24,6 +25,7 @@ class fake_device:
index = None


@pytest.mark.skipif(TEST_WITH_TSAN, reason="False positive in TSAN")
def test_join_running_workers():
count = 0

Expand All @@ -47,6 +49,7 @@ def call_in_worker(i, f):
assert count == 10


@pytest.mark.skipif(TEST_WITH_TSAN, reason="False positive in TSAN")
def test_join_running_workers_with_exception():
class ExpectedException(Exception):
pass
Expand Down

0 comments on commit ad260ae

Please sign in to comment.