Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GCG evaluation error #40

Open
ftfrz opened this issue Aug 26, 2024 · 5 comments
Open

GCG evaluation error #40

ftfrz opened this issue Aug 26, 2024 · 5 comments

Comments

@ftfrz
Copy link

ftfrz commented Aug 26, 2024

When running the following command:

NPROC_PER_NODE=4 xtuner gcg_omg_seg_llava \
  ${PATH_TO_CONFIG} \
  ${PATH_TO_PTH} \
  --output-name gcg_pred

I encountered this error:

Traceback (most recent call last):
  File "/root/OMG-Seg/omg_llava/omg_llava/tools/gcg_omg_seg_llava.py", line 484, in <module>
    main()
  File "/root/OMG-Seg/omg_llava/omg_llava/tools/gcg_omg_seg_llava.py", line 295, in main
    predict, seg_hidden_states = forward_model(
  File "/root/OMG-Seg/omg_llava/omg_llava/tools/gcg_omg_seg_llava.py", line 370, in forward_model
    generate_output = llm.generate(
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/transformers/generation/utils.py", line 1718, in generate
    return self.greedy_search(
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/transformers/generation/utils.py", line 2579, in greedy_search
    outputs = self(
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/accelerate/hooks.py", line 169, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/root/.cache/huggingface/modules/transformers_modules/internlm2-chat-7b/modeling_internlm2.py", line 1060, in forward
    logits = self.output(hidden_states)
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/accelerate/hooks.py", line 169, in new_forward
    output = module._old_forward(*args, **kwargs)
  File "/root/miniconda3/envs/omg-llava/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
RuntimeError: expected mat1 and mat2 to have the same dtype, but got: float != c10::Half

My conda environment:

# packages in environment at /root/miniconda3/envs/omg-llava:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
_openmp_mutex             5.1                       1_gnu    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
accelerate                0.33.0                   pypi_0    pypi
addict                    2.4.0                    pypi_0    pypi
aiofiles                  23.2.1                   pypi_0    pypi
aiohappyeyeballs          2.4.0                    pypi_0    pypi
aiohttp                   3.10.5                   pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
altair                    5.4.0                    pypi_0    pypi
annotated-types           0.7.0                    pypi_0    pypi
anyio                     4.4.0                    pypi_0    pypi
argon2-cffi               23.1.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.3.0                    pypi_0    pypi
arxiv                     2.1.3                    pypi_0    pypi
asttokens                 2.4.1                    pypi_0    pypi
async-lru                 2.0.4                    pypi_0    pypi
async-timeout             4.0.3                    pypi_0    pypi
attrs                     24.2.0                   pypi_0    pypi
babel                     2.16.0                   pypi_0    pypi
beautifulsoup4            4.12.3                   pypi_0    pypi
bitsandbytes              0.43.3                   pypi_0    pypi
bleach                    6.1.0                    pypi_0    pypi
blinker                   1.8.2                    pypi_0    pypi
brotli                    1.1.0                    pypi_0    pypi
bzip2                     1.0.8                h5eee18b_6    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ca-certificates           2024.7.2             h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
cachetools                5.5.0                    pypi_0    pypi
certifi                   2022.12.7                pypi_0    pypi
cffi                      1.17.0                   pypi_0    pypi
charset-normalizer        2.1.1                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
colorama                  0.4.6                    pypi_0    pypi
comm                      0.2.2                    pypi_0    pypi
contourpy                 1.2.1                    pypi_0    pypi
cycler                    0.12.1                   pypi_0    pypi
datasets                  2.21.0                   pypi_0    pypi
debugpy                   1.8.5                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
deepspeed                 0.15.0                   pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
dill                      0.3.8                    pypi_0    pypi
distro                    1.9.0                    pypi_0    pypi
duckduckgo-search         5.3.1b1                  pypi_0    pypi
einops                    0.8.0                    pypi_0    pypi
et-xmlfile                1.1.0                    pypi_0    pypi
exceptiongroup            1.2.2                    pypi_0    pypi
executing                 2.0.1                    pypi_0    pypi
fastapi                   0.112.1                  pypi_0    pypi
fastjsonschema            2.20.0                   pypi_0    pypi
feedparser                6.0.11                   pypi_0    pypi
ffmpy                     0.4.0                    pypi_0    pypi
filelock                  3.13.1                   pypi_0    pypi
fonttools                 4.53.1                   pypi_0    pypi
fqdn                      1.5.1                    pypi_0    pypi
frozenlist                1.4.1                    pypi_0    pypi
fsspec                    2024.2.0                 pypi_0    pypi
ftfy                      6.2.3                    pypi_0    pypi
func-timeout              4.3.5                    pypi_0    pypi
gitdb                     4.0.11                   pypi_0    pypi
gitpython                 3.1.43                   pypi_0    pypi
gradio                    4.37.2                   pypi_0    pypi
gradio-client             1.0.2                    pypi_0    pypi
gradio-image-prompter     0.1.0                    pypi_0    pypi
griffe                    1.1.1                    pypi_0    pypi
h11                       0.14.0                   pypi_0    pypi
h2                        4.1.0                    pypi_0    pypi
hjson                     3.1.0                    pypi_0    pypi
hpack                     4.0.0                    pypi_0    pypi
httpcore                  1.0.5                    pypi_0    pypi
httpx                     0.27.0                   pypi_0    pypi
huggingface-hub           0.24.6                   pypi_0    pypi
hyperframe                6.0.1                    pypi_0    pypi
idna                      3.4                      pypi_0    pypi
imageio                   2.35.1                   pypi_0    pypi
importlib-metadata        8.4.0                    pypi_0    pypi
importlib-resources       6.4.4                    pypi_0    pypi
ipykernel                 6.29.5                   pypi_0    pypi
ipython                   8.26.0                   pypi_0    pypi
ipywidgets                8.1.5                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
jedi                      0.19.1                   pypi_0    pypi
jinja2                    3.1.3                    pypi_0    pypi
json5                     0.9.25                   pypi_0    pypi
jsonpointer               3.0.0                    pypi_0    pypi
jsonschema                4.23.0                   pypi_0    pypi
jsonschema-specifications 2023.12.1                pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            8.6.2                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-core              5.7.2                    pypi_0    pypi
jupyter-events            0.10.0                   pypi_0    pypi
jupyter-lsp               2.2.5                    pypi_0    pypi
jupyter-server            2.14.2                   pypi_0    pypi
jupyter-server-terminals  0.5.3                    pypi_0    pypi
jupyterlab                4.2.4                    pypi_0    pypi
jupyterlab-pygments       0.3.0                    pypi_0    pypi
jupyterlab-server         2.27.3                   pypi_0    pypi
jupyterlab-widgets        3.0.13                   pypi_0    pypi
kiwisolver                1.4.5                    pypi_0    pypi
kornia                    0.7.3                    pypi_0    pypi
kornia-rs                 0.1.5                    pypi_0    pypi
lagent                    0.2.3                    pypi_0    pypi
lazy-loader               0.4                      pypi_0    pypi
ld_impl_linux-64          2.38                 h1181459_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libffi                    3.4.4                h6a678d5_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgcc-ng                 11.2.0               h1234567_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgomp                   11.2.0               h1234567_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libstdcxx-ng              11.2.0               h1234567_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libuuid                   1.41.5               h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
markdown                  3.7                      pypi_0    pypi
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
mat4py                    0.6.0                    pypi_0    pypi
matplotlib                3.9.2                    pypi_0    pypi
matplotlib-inline         0.1.7                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mistune                   3.0.2                    pypi_0    pypi
mmcv                      2.0.1                    pypi_0    pypi
mmdet                     3.1.0                    pypi_0    pypi
mmengine                  0.10.4                   pypi_0    pypi
mmpretrain                1.0.1                    pypi_0    pypi
mmsegmentation            1.1.1                    pypi_0    pypi
model-index               0.1.11                   pypi_0    pypi
modelindex                0.0.2                    pypi_0    pypi
modelscope                1.17.1                   pypi_0    pypi
mpi4py-mpich              3.1.5                    pypi_0    pypi
mpmath                    1.3.0                    pypi_0    pypi
multidict                 6.0.5                    pypi_0    pypi
multiprocess              0.70.16                  pypi_0    pypi
narwhals                  1.5.3                    pypi_0    pypi
nbclient                  0.10.0                   pypi_0    pypi
nbconvert                 7.16.4                   pypi_0    pypi
nbformat                  5.10.4                   pypi_0    pypi
ncurses                   6.4                  h6a678d5_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
nest-asyncio              1.6.0                    pypi_0    pypi
networkx                  3.2.1                    pypi_0    pypi
ninja                     1.11.1.1                 pypi_0    pypi
notebook                  7.2.1                    pypi_0    pypi
notebook-shim             0.2.4                    pypi_0    pypi
numpy                     1.26.3                   pypi_0    pypi
nvidia-ml-py              12.560.30                pypi_0    pypi
opencv-python             4.10.0.84                pypi_0    pypi
openpyxl                  3.1.5                    pypi_0    pypi
openssl                   3.0.14               h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ordered-set               4.1.0                    pypi_0    pypi
orjson                    3.10.7                   pypi_0    pypi
overrides                 7.7.0                    pypi_0    pypi
packaging                 24.1                     pypi_0    pypi
pandas                    2.2.2                    pypi_0    pypi
pandocfilters             1.5.1                    pypi_0    pypi
parso                     0.8.4                    pypi_0    pypi
peft                      0.12.0                   pypi_0    pypi
pexpect                   4.9.0                    pypi_0    pypi
phx-class-registry        4.1.0                    pypi_0    pypi
pillow                    10.2.0                   pypi_0    pypi
pip                       24.2            py310h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
platformdirs              4.2.2                    pypi_0    pypi
prettytable               3.11.0                   pypi_0    pypi
prometheus-client         0.20.0                   pypi_0    pypi
prompt-toolkit            3.0.47                   pypi_0    pypi
protobuf                  5.27.3                   pypi_0    pypi
psutil                    6.0.0                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.3                    pypi_0    pypi
py-cpuinfo                9.0.0                    pypi_0    pypi
pyarrow                   17.0.0                   pypi_0    pypi
pycocotools               2.0.8                    pypi_0    pypi
pycparser                 2.22                     pypi_0    pypi
pydantic                  2.8.2                    pypi_0    pypi
pydantic-core             2.20.1                   pypi_0    pypi
pydeck                    0.9.1                    pypi_0    pypi
pydub                     0.25.1                   pypi_0    pypi
pygments                  2.18.0                   pypi_0    pypi
pyparsing                 3.1.2                    pypi_0    pypi
python                    3.10.14              h955ad1f_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python-dateutil           2.9.0.post0              pypi_0    pypi
python-json-logger        2.0.7                    pypi_0    pypi
python-multipart          0.0.9                    pypi_0    pypi
pytz                      2024.1                   pypi_0    pypi
pyyaml                    6.0.2                    pypi_0    pypi
pyzmq                     26.2.0                   pypi_0    pypi
qtconsole                 5.5.2                    pypi_0    pypi
qtpy                      2.4.1                    pypi_0    pypi
readline                  8.2                  h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
referencing               0.35.1                   pypi_0    pypi
regex                     2024.7.24                pypi_0    pypi
requests                  2.32.3                   pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rich                      13.7.1                   pypi_0    pypi
rpds-py                   0.20.0                   pypi_0    pypi
ruff                      0.6.2                    pypi_0    pypi
safetensors               0.4.4                    pypi_0    pypi
scikit-image              0.24.0                   pypi_0    pypi
scipy                     1.14.1                   pypi_0    pypi
semantic-version          2.10.0                   pypi_0    pypi
send2trash                1.8.3                    pypi_0    pypi
sentencepiece             0.2.0                    pypi_0    pypi
setuptools                72.1.0          py310h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sgmllib3k                 1.0.0                    pypi_0    pypi
shapely                   2.0.6                    pypi_0    pypi
shellingham               1.5.4                    pypi_0    pypi
six                       1.16.0                   pypi_0    pypi
smmap                     5.0.1                    pypi_0    pypi
sniffio                   1.3.1                    pypi_0    pypi
socksio                   1.0.0                    pypi_0    pypi
soupsieve                 2.6                      pypi_0    pypi
sqlite                    3.45.3               h5eee18b_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
stack-data                0.6.3                    pypi_0    pypi
starlette                 0.38.2                   pypi_0    pypi
streamlit                 1.37.1                   pypi_0    pypi
sympy                     1.12                     pypi_0    pypi
tenacity                  8.5.0                    pypi_0    pypi
termcolor                 2.4.0                    pypi_0    pypi
terminado                 0.18.1                   pypi_0    pypi
terminaltables            3.1.10                   pypi_0    pypi
tifffile                  2024.8.10                pypi_0    pypi
tiktoken                  0.7.0                    pypi_0    pypi
timeout-decorator         0.5.0                    pypi_0    pypi
timm                      1.0.8                    pypi_0    pypi
tinycss2                  1.3.0                    pypi_0    pypi
tk                        8.6.14               h39e8969_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tokenizers                0.15.2                   pypi_0    pypi
toml                      0.10.2                   pypi_0    pypi
tomli                     2.0.1                    pypi_0    pypi
tomlkit                   0.12.0                   pypi_0    pypi
torch                     2.1.2+cu118              pypi_0    pypi
torchaudio                2.1.2+cu118              pypi_0    pypi
torchvision               0.16.2+cu118             pypi_0    pypi
tornado                   6.4.1                    pypi_0    pypi
tqdm                      4.66.5                   pypi_0    pypi
traitlets                 5.14.3                   pypi_0    pypi
transformers              4.36.0                   pypi_0    pypi
transformers-stream-generator 0.0.5                    pypi_0    pypi
triton                    2.1.0                    pypi_0    pypi
typer                     0.12.4                   pypi_0    pypi
types-python-dateutil     2.9.0.20240821           pypi_0    pypi
typing-extensions         4.12.2                   pypi_0    pypi
tzdata                    2024.1                   pypi_0    pypi
uri-template              1.3.0                    pypi_0    pypi
urllib3                   2.2.2                    pypi_0    pypi
uvicorn                   0.30.6                   pypi_0    pypi
watchdog                  4.0.2                    pypi_0    pypi
wcwidth                   0.2.13                   pypi_0    pypi
webcolors                 24.8.0                   pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.8.0                    pypi_0    pypi
websockets                11.0.3                   pypi_0    pypi
wheel                     0.43.0          py310h06a4308_0    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
widgetsnbextension        4.0.13                   pypi_0    pypi
xtuner                    0.1.21                    dev_0    <develop>
xxhash                    3.5.0                    pypi_0    pypi
xz                        5.4.6                h5eee18b_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
yapf                      0.40.2                   pypi_0    pypi
yarl                      1.9.4                    pypi_0    pypi
zipp                      3.20.0                   pypi_0    pypi
zlib                      1.2.13               h5eee18b_1    https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@ftfrz and others