basic_demo/cli_demo.py _issue

[pangye202264690373]

System Info / 系統信息

系统信息

Linux version 5.15.0-112-generic (buildd@lcy02-amd64-051) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38)

环境信息

Name	Version	Build	Channel
_libgcc_mutex	0.1	main	defaults
_openmp_mutex	5.1	1_gnu	defaults
aiofiles	23.2.1	pypi_0	pypi
annotated-types	0.7.0	pypi_0	pypi
anyio	3.7.1	pypi_0	pypi
asyncer	0.0.2	pypi_0	pypi
bidict	0.23.1	pypi_0	pypi
bitsandbytes	0.43.2	pypi_0	pypi
bzip2	1.0.8	h5eee18b_6	defaults
ca-certificates	2024.7.2	h06a4308_0	defaults
certifi	2024.7.4	pypi_0	pypi
chainlit	1.1.306	pypi_0	pypi
charset-normalizer	3.3.2	pypi_0	pypi
chevron	0.14.0	pypi_0	pypi
click	8.1.7	pypi_0	pypi
dataclasses-json	0.5.14	pypi_0	pypi
deprecated	1.2.14	pypi_0	pypi
distro	1.9.0	pypi_0	pypi
einops	0.8.0	pypi_0	pypi
expat	2.6.2	h6a678d5_0	defaults
fastapi	0.110.3	pypi_0	pypi
filelock	3.15.4	pypi_0	pypi
filetype	1.2.0	pypi_0	pypi
fsspec	2024.6.1	pypi_0	pypi
googleapis-common-protos	1.63.2	pypi_0	pypi
grpcio	1.65.1	pypi_0	pypi
h11	0.14.0	pypi_0	pypi
httpcore	1.0.5	pypi_0	pypi
httpx	0.27.0	pypi_0	pypi
huggingface-hub	0.24.1	pypi_0	pypi
idna	3.7	pypi_0	pypi
importlib-metadata	8.0.0	pypi_0	pypi
jinja2	3.1.4	pypi_0	pypi
lazify	0.4.0	pypi_0	pypi
ld_impl_linux-64	2.38	h1181459_1	defaults
libffi	3.4.4	h6a678d5_1	defaults
libgcc-ng	11.2.0	h1234567_1	defaults
libgomp	11.2.0	h1234567_1	defaults
libstdcxx-ng	11.2.0	h1234567_1	defaults
libuuid	1.41.5	h5eee18b_0	defaults
literalai	0.0.607	pypi_0	pypi
loguru	0.7.2	pypi_0	pypi
markupsafe	2.1.5	pypi_0	pypi
marshmallow	3.21.3	pypi_0	pypi
mpmath	1.3.0	pypi_0	pypi
mypy-extensions	1.0.0	pypi_0	pypi
ncurses	6.4	h6a678d5_0	defaults
nest-asyncio	1.6.0	pypi_0	pypi
networkx	3.3	pypi_0	pypi
numpy	1.26.4	pypi_0	pypi
nvidia-cublas-cu12	12.1.3.1	pypi_0	pypi
nvidia-cuda-cupti-cu12	12.1.105	pypi_0	pypi
nvidia-cuda-nvrtc-cu12	12.1.105	pypi_0	pypi
nvidia-cuda-runtime-cu12	12.1.105	pypi_0	pypi
nvidia-cudnn-cu12	8.9.2.26	pypi_0	pypi
nvidia-cufft-cu12	11.0.2.54	pypi_0	pypi
nvidia-curand-cu12	10.3.2.106	pypi_0	pypi
nvidia-cusolver-cu12	11.4.5.107	pypi_0	pypi
nvidia-cusparse-cu12	12.1.0.106	pypi_0	pypi
nvidia-nccl-cu12	2.20.5	pypi_0	pypi
nvidia-nvjitlink-cu12	12.5.82	pypi_0	pypi
nvidia-nvtx-cu12	12.1.105	pypi_0	pypi
openai	1.37.0	pypi_0	pypi
openssl	3.0.14	h5eee18b_0	defaults
opentelemetry-api	1.26.0	pypi_0	pypi
opentelemetry-exporter-otlp	1.26.0	pypi_0	pypi
opentelemetry-exporter-otlp-proto-common	1.26.0	pypi_0	pypi
opentelemetry-exporter-otlp-proto-grpc	1.26.0	pypi_0	pypi
opentelemetry-exporter-otlp-proto-http	1.26.0	pypi_0	pypi
opentelemetry-instrumentation	0.47b0	pypi_0	pypi
opentelemetry-proto	1.26.0	pypi_0	pypi
opentelemetry-sdk	1.26.0	pypi_0	pypi
opentelemetry-semantic-conventions	0.47b0	pypi_0	pypi
packaging	23.2	pypi_0	pypi
pillow	10.4.0	pypi_0	pypi
pip	24.0	py312h06a4308_0	defaults
protobuf	4.25.4	pypi_0	pypi
pydantic	2.8.2	pypi_0	pypi
pydantic-core	2.20.1	pypi_0	pypi
pyjwt	2.8.0	pypi_0	pypi
python	3.12.4	h5148396_1	defaults
python-dotenv	1.0.1	pypi_0	pypi
python-engineio	4.9.1	pypi_0	pypi
python-multipart	0.0.9	pypi_0	pypi
python-socketio	5.11.3	pypi_0	pypi
pyyaml	6.0.1	pypi_0	pypi
readline	8.2	h5eee18b_0	defaults
regex	2024.5.15	pypi_0	pypi
requests	2.32.3	pypi_0	pypi
safetensors	0.4.3	pypi_0	pypi
setuptools	69.5.1	py312h06a4308_0	defaults
simple-websocket	1.0.0	pypi_0	pypi
sniffio	1.3.1	pypi_0	pypi
sqlite	3.45.3	h5eee18b_0	defaults
sse-starlette	2.1.2	pypi_0	pypi
starlette	0.37.2	pypi_0	pypi
sympy	1.13.1	pypi_0	pypi
syncer	2.0.3	pypi_0	pypi
timm	1.0.7	pypi_0	pypi
tk	8.6.14	h39e8969_0	defaults
tokenizers	0.19.1	pypi_0	pypi
tomli	2.0.1	pypi_0	pypi
torch	2.3.1	pypi_0	pypi
torchaudio	2.3.1	pypi_0	pypi
torchvision	0.18.1	pypi_0	pypi
tqdm	4.66.4	pypi_0	pypi
transformers	4.43.1	pypi_0	pypi
triton	3.0.0	pypi_0	pypi
typing-extensions	4.12.2	pypi_0	pypi
typing-inspect	0.9.0	pypi_0	pypi
tzdata	2024a	h04d1e81_0	defaults
uptrace	1.26.0	pypi_0	pypi
urllib3	2.2.2	pypi_0	pypi
uvicorn	0.25.0	pypi_0	pypi
watchfiles	0.20.0	pypi_0	pypi
wheel	0.43.0	py312h06a4308_0	defaults
wrapt	1.16.0	pypi_0	pypi
wsproto	1.2.0	pypi_0	pypi
xformers	0.0.27	pypi_0	pypi
xz	5.4.6	h5eee18b_1	defaults
zipp	3.19.2	pypi_0	pypi
zlib	1.2.13	h5eee18b_1	defaults

Who can help? / 谁可以帮助到您？

@zr

Information / 问题信息

• The official example scripts / 官方的示例脚本
• My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

运行basic_demo/cli__demo.py

Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████| 8/8 [00:06<00:00, 1.17it/s]
image path >>>>> /share/home/wangyufeng/pangye/cogVLM2/CogVLM2/basic_demo/demo.jpg
Human: (NOTHING HERE)

(得到结果）
Traceback (most recent call last):
File "/share/home/wangyufeng/pangye/cogVLM2/CogVLM2/basic_demo/cli_demo.py", line 102, in
outputs = model.generate(**inputs, **gen_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/miniconda3/envs/CogVLM2/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/miniconda3/envs/CogVLM2/lib/python3.12/site-packages/transformers/generation/utils.py", line 1989, in generate
result = self._sample(
^^^^^^^^^^^^^
File "/share/home/wangyufeng/miniconda3/envs/CogVLM2/lib/python3.12/site-packages/transformers/generation/utils.py", line 2932, in _sample
outputs = self(**model_inputs, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/miniconda3/envs/CogVLM2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/miniconda3/envs/CogVLM2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/.cache/huggingface/modules/transformers_modules/cogvlm2-llama3-chat-19B/modeling_cogvlm.py", line 649, in forward
outputs = self.model(
^^^^^^^^^^^
File "/share/home/wangyufeng/miniconda3/envs/CogVLM2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/miniconda3/envs/CogVLM2/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/.cache/huggingface/modules/transformers_modules/cogvlm2-llama3-chat-19B/modeling_cogvlm.py", line 403, in forward
return self.llm_forward(
^^^^^^^^^^^^^^^^^
File "/share/home/wangyufeng/.cache/huggingface/modules/transformers_modules/cogvlm2-llama3-chat-19B/modeling_cogvlm.py", line 452, in llm_forward
past_key_values_length = past_key_values[0][0].shape[2]
^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'shape'

Expected behavior / 期待表现

正常产出结果，输入文本后应当得到正确的结果

[pangye202264690373]

同时也试过更换图片，尝试不同输入文本等方法

[tomerkeren42]

Exact same problem here.
The correction should be either:
past_key_values_length = past_key_values[1][0][0].shape[2]
or just remove the first part of the past_key_values tuple, which is just a string says "past_key_values". This is the error but I dont know where it happens.

[JeffRody]

同样的问题

[JeffRody]

[JeffRody]

transformers 降到4.40.4可以解决问题

[dwyane20]

transformers 降到4.40.4可以解决问题

你好，我遇到了一样的问题，修改transformers后没有报错，但是发送图片后没有响应，只能文字对话，发送图片页面不显示，后端也没有日志输出，能看看你所有的python库吗？

[cageyoko]

same issue.