Segmentation fault (core dumped) when running nlp_data.py

[xiulinyang]

Hi I was trying to run leamr but the segment fault kept happening no matter if I ran it on my local machine or a server. Could you please help me on this issue? I put the error log below. Thank you!

(rel) xiulyang@jones-5:/local/xiulyang/leamr$ python nlp_data.py pred_pc.txt
<frozen importlib._bootstrap>:219: RuntimeWarning: spacy.morphology.Morphology size changed, may
indicate binary incompatibility. Expected 104 from C header, got 112 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: spacy.vocab.Vocab size changed, may indicate
binary incompatibility. Expected 96 from C header, got 112 from PyObject
<frozen importlib._bootstrap>:219: RuntimeWarning: spacy.tokens.span.Span size changed, may
indicate binary incompatibility. Expected 72 from C header, got 80 from PyObject
2023-10-26 15:49:46 INFO: Loading these models for language: en (English):
=========================
| Processor | Package |
-------------------------
| tokenize | combined |
| pos | combined |
| lemma | combined |
| ner | ontonotes |
=========================
/local/xiulyang/anaconda3/envs/rel/lib/python3.8/site-packages/torch/cuda/__init__.py:138: UserWarning:
CUDA initialization: The NVIDIA driver on your system is too old (found version 11070). Please update
your GPU driver by downloading and installing a new version from the URL:
http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch
version that has been compiled with your version of the CUDA driver. (Triggered internally at
../c10/cuda/CUDAFunctions.cpp:108.)
return torch._C._cuda_getDeviceCount() > 0
2023-10-26 15:49:46 INFO: Use device: cpu
2023-10-26 15:49:46 INFO: Loading: tokenize
2023-10-26 15:49:46 INFO: Loading: pos
2023-10-26 15:49:47 INFO: Loading: lemma
2023-10-26 15:49:47 INFO: Loading: ner
2023-10-26 15:49:48 INFO: Done loading processors!
[amr] Loading AMRs from file: pred_pc.txt
[<amr_utils.amr.AMR object at 0x7fa7e97c9a60>, <amr_utils.amr.AMR object at 0x7fa7e97c9a90>,
<amr_utils.amr.AMR object at 0x7fa7e97c9a30>, <amr_utils.amr.AMR object at 0x7fa7e97c9640>,
<amr_utils.amr.AMR object at 0x7fa7e97c9760>, <amr_utils.amr.AMR object at 0x7fa7e97c9a00>,
<amr_utils.amr.AMR object at 0x7fa7e97c9b80>, <amr_utils.amr.AMR object at 0x7fa7e97c9be0>,
<amr_utils.amr.AMR object at 0x7fa7e97c9bb0>, <amr_utils.amr.AMR object at 0x7fa7e97c9580>,
<amr_utils.amr.AMR object at 0x7fa7e97c9af0>, <amr_utils.amr.AMR object at 0x7fa7e97c9b20>,
<amr_utils.amr.AMR object at 0x7fa7e97c9c70>, <amr_utils.amr.AMR object at 0x7fa7e97c9ca0>,
<amr_utils.amr.AMR object at 0x7fa7e97c9c10>, <amr_utils.amr.AMR object at 0x7fa7e97c9d00>,
<amr_utils.amr.AMR object at 0x7fa7e97c9d90>, <amr_utils.amr.AMR object at 0x7fa7e97c9b50>,
<amr_utils.amr.AMR object at 0x7fa7e97c9c40>, <amr_utils.amr.AMR object at 0x7fa7e97c9cd0>,
<amr_utils.amr.AMR object at 0x7fa7e97c9dc0>, <amr_utils.amr.AMR object at 0x7fa7e97c9eb0>,
<amr_utils.amr.AMR object at 0x7fa7e97c9e80>, <amr_utils.amr.AMR object at 0x7fa7e97c9df0>,
<amr_utils.amr.AMR object at 0x7fa7e97c9e20>, <amr_utils.amr.AMR object at 0x7fa7e97c9f10>,
<amr_utils.amr.AMR object at 0x7fa7e97c9f70>, <amr_utils.amr.AMR object at 0x7fa7e97c9fd0>,
<amr_utils.amr.AMR object at 0x7fa7e97c9ee0>, <amr_utils.amr.AMR object at 0x7fa7e97c9f40>,
<amr_utils.amr.AMR object at 0x7fa7e97c9fa0>, <amr_utils.amr.AMR object at 0x7fa7e97c9e50>,
<amr_utils.amr.AMR object at 0x7fa8d8ca30a0>, <amr_utils.amr.AMR object at 0x7fa8d8ca3070>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3130>, <amr_utils.amr.AMR object at 0x7fa8d8ca30d0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3160>, <amr_utils.amr.AMR object at 0x7fa8d8ca31f0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3100>, <amr_utils.amr.AMR object at 0x7fa8d8ca31c0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3040>, <amr_utils.amr.AMR object at 0x7fa8d8ca3250>,
<amr_utils.amr.AMR object at 0x7fa8d8ca32b0>, <amr_utils.amr.AMR object at 0x7fa8d8ca3280>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3370>, <amr_utils.amr.AMR object at 0x7fa8d8ca33a0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3190>, <amr_utils.amr.AMR object at 0x7fa8d8ca32e0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3400>, <amr_utils.amr.AMR object at 0x7fa8d8ca3310>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3220>, <amr_utils.amr.AMR object at 0x7fa8d8ca3460>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3430>, <amr_utils.amr.AMR object at 0x7fa8d8ca33d0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3340>, <amr_utils.amr.AMR object at 0x7fa8d8ca3520>,
<amr_utils.amr.AMR object at 0x7fa8d8ca34f0>, <amr_utils.amr.AMR object at 0x7fa8d8ca34c0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3580>, <amr_utils.amr.AMR object at 0x7fa8d8ca35b0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3610>, <amr_utils.amr.AMR object at 0x7fa8d8ca3550>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3490>, <amr_utils.amr.AMR object at 0x7fa8d8ca3670>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3760>, <amr_utils.amr.AMR object at 0x7fa8d8ca3730>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3640>, <amr_utils.amr.AMR object at 0x7fa8d8ca35e0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3820>, <amr_utils.amr.AMR object at 0x7fa8d8ca3790>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3700>, <amr_utils.amr.AMR object at 0x7fa8d8ca36a0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3850>, <amr_utils.amr.AMR object at 0x7fa8d8ca37c0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca38e0>, <amr_utils.amr.AMR object at 0x7fa8d8ca3880>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3940>, <amr_utils.amr.AMR object at 0x7fa8d8ca3910>,
<amr_utils.amr.AMR object at 0x7fa8d8ca38b0>, <amr_utils.amr.AMR object at 0x7fa8d8ca39d0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca3a60>, <amr_utils.amr.AMR object at 0x7fa8d8ca39a0>,
<amr_utils.amr.AMR object at 0x7fa8d8ca37f0>, <amr_utils.amr.AMR object at 0x7fa8d8ca3a00>]
0%| | 0/84 [00:00<?, ?
it/s]Segmentation fault (core dumped)

[ablodge]

Hi Xiulin,

This might be a compatibility issue with neuralcoref and spacy (See huggingface/neuralcoref#164). Can you please list which versions you're using of python, spacy, neuralcoref, and stanza?

[xiulinyang]

Hi Dr. Blodgett,

Thank you for your prompt reply!
I use Python 3.8.18 for my virtual environment. The package information is listed below. Thanks!

spacy                    2.3.5
neuralcoref              4.0
stanza                   1.2

[xiulinyang]

Hi Dr. Blodgett,

Just a quick update: I upgraded spacy to 3.5.3 but the problem remains.

[ablodge]

You should be using spacy 2.3.5.

[ablodge]

Can you please try the following? (instructions from huggingface/neuralcoref#222 (comment))

pip uninstall neuralcoref
git clone https://github.com/huggingface/neuralcoref.git
cd neuralcoref
pip install -r requirements.txt
pip install -e .
pip uninstall spacy
pip install spacy
python -m spacy download en_core_web_sm

[xiulinyang]

Hi, thanks for the information. It works!
I made some changes, though, because errors occurred I ran pip install -e .. In case people come across the same issue, I post my solution here:

#step1
vim /path/to/python3.8/site-packages/spacy/vocab.pxd
#step 2 replace cpdef with cdef
#step 3
pip install -e .
#step 4
vim neuralcoref/neuralcoref/neuralcoref.pyx
# step 5 copy the information about NEURALCOREF_MODEL_PATH and replace the relevant lines with from file_utils import NEURALCOREF_MODEL_PATH in  neuralcoref/neuralcoref/neuralcoref.pyx

I also got some log information by using the aligner.
It says the preprocessing coverage for reentrancy is only 6% - how should I interpret the number? Is it something that I should worry about if I care about reentrancies? Thank you!

100%|████████████████████████████████████████████████████████████████████████████████████████████| 557/557 [00:45<00:00, 12.20it/s]
Writing subgraph alignments to: pred_amrlib/output_sc_amrlib.subgraph_alignments.json
Loading model: ldc+little_prince.relation_params.pkl
Preprocessing coverage: 85.91%
100%|███████████████████████████████████████████████████████████████████████████████████████████| 557/557 [00:00<00:00, 599.03it/s]
Writing relation alignments to: pred_amrlib/output_sc_amrlib.relation_alignments.json
Loading model: ldc+little_prince.reentrancy_params.pkl
Preprocessing coverage: 6.19%
100%|███████████████████████████████████████████████████████████████████████████████████████████| 557/557 [00:03<00:00, 183.64it/s]
Writing reentrancy alignments to: pred_amrlib/output_sc_amrlib.reentrancy_alignments.json

[xiulinyang]

Hi sorry, I have some follow-up questions. I noticed that in the annotation guideline, reentrancies are classified into different types. I was wondering, which group should we classify the reentrancies caused by relative clauses into? Will the reentrancy relation be aligned with the relative pronoun, the token that is relativized on, or some specific token in the relative clause? Many thanks in advance for your clarification.

[ablodge]

No worries! Feel free to contact me again if you have more questions.

To your first question, the script displays the "preprocessing coverage", which is the coverage of alignments that could be made just using rule-based preprocessing. If you look over the data, you should see that after the LEAMR probabilistic aligner has run, you should have 100% coverage. Let me know if that is not the case.

To your second question, relative clauses fall under either Adjunct Control (and should be aligned to a relative pronoun or "that") or Unmarked Adjunct Control (and are aligned to the predicate inside the relative clause).