Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audio autoplay does not work in Notebook renderers #178642

Closed
1 of 2 tasks
TeamDman opened this issue Mar 27, 2023 · 6 comments · Fixed by #179204
Closed
1 of 2 tasks

Audio autoplay does not work in Notebook renderers #178642

TeamDman opened this issue Mar 27, 2023 · 6 comments · Fixed by #179204
Assignees

Comments

@TeamDman
Copy link

TeamDman commented Mar 27, 2023

Applies To

  • Notebooks (.ipynb files)
  • Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?

Audio isn't autoplaying

Code_WWj9qOWSCM.mp4

Possibly related:
microsoft/vscode-jupyter#1012 (comment)

VS Code Version

Version: 1.76.2 (system setup) Commit: ee2b180 Date: 2023-03-14T17:55:54.936Z Electron: 19.1.11 Chromium: 102.0.5005.196 Node.js: 16.14.2 V8: 10.2.154.26-electron.0 OS: Windows_NT x64 10.0.19045 Sandboxed: No

Jupyter Extension Version

Name: Jupyter Id: ms-toolsai.jupyter Description: Jupyter notebook support, interactive programming and computing that supports Intellisense, debugging and more. Version: 2023.2.1200692131 Publisher: Microsoft VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter

Jupyter logs

Visual Studio Code (1.76.2, undefined, desktop)
Jupyter Extension Version: 2023.2.1200692131.
Python Extension Version: 2023.4.1.
Multiple Workspace folders opened d:\Repos\Other\toolformer, d:\Repos\Other\toolformer\backend, d:\Repos\Other\toolformer\frontend, d:\Repos\Other\llama.cpp, d:\Repos\Other\text-generation-webui, d:\Repos\Other\alpaca.cpp, c:\Users\TeamD\OneDrive\Documents\Ideas, d:\Repos\Other\alpaca-lora, d:\Repos\Other\dalai
User belongs to experiment group 'jupyterEnhancedDataViewer'
User belongs to experiment group 'jupyterTest'
info 19:36:20.709: LSP Notebooks experiment is enabled
info 19:36:21.420: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd03582d8edd22589ac626c71cd7485dc1d0beb8fcde46e30e00a75c6de569a9447.d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe.d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe.-m#ipykernel_launcher (Python Path: d:\Repos\Other\text-generation-webui\venv, EnvType: Venv, EnvName: 'venv', Version: 3.10.9 | packaged by Anaconda, Inc. | (main, Mar  1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)]) for 'd:\Repos\Other\toolformer\backend\exploration.ipynb' (disableUI=true)
info 19:36:22.668: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m pip list
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m pip list
info 19:36:22.892: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 19:36:23.075: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\vscode_datascience_helpers\kernel_interrupt_daemon.py --ppid 14984
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\vscode_datascience_helpers\kernel_interrupt_daemon.py --ppid 14984
info 19:36:23.075: Process Execution: cwd: ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\vscode_datascience_helpers
cwd: ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\vscode_datascience_helpers
info 19:36:23.427: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"00e7e2fd-93d0-4249-8a78-a87c8456b289" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984aazbBcf3RKBS.json
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"00e7e2fd-93d0-4249-8a78-a87c8456b289" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984aazbBcf3RKBS.json
info 19:36:23.427: Process Execution: cwd: d:\Repos\Other\toolformer\backend
cwd: d:\Repos\Other\toolformer\backend
info 19:36:24.384: ipykernel version & path 6.22.0, d:\Repos\Other\text-generation-webui\venv\lib\site-packages\ipykernel\__init__.py for d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe
info 19:36:25.931: Started Kernel venv (Python 3.10.9) (pid: 14196)
info 19:36:26.717: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\printJupyterDataDir.py
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\printJupyterDataDir.py
info 20:47:25.447: Dispose Kernel 'd:\Repos\Other\toolformer\backend\exploration.ipynb' associated with 'd:\Repos\Other\toolformer\backend\exploration.ipynb'
info 20:47:25.452: Dispose Kernel process 14196.
info 22:41:53.763: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd03582d8edd22589ac626c71cd7485dc1d0beb8fcde46e30e00a75c6de569a9447.d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe.d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe.-m#ipykernel_launcher (Python Path: d:\Repos\Other\text-generation-webui\venv, EnvType: Venv, EnvName: 'venv', Version: 3.10.9 | packaged by Anaconda, Inc. | (main, Mar  1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)]) for 'd:\Repos\Other\toolformer\backend\exploration.ipynb' (disableUI=true)
info 22:41:54.122: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 22:41:54.132: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m pip list
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m pip list
info 22:41:54.140: Process Execution: > d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"66b2bea6-aa17-4019-8893-f51f1088988a" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-149849CH6IMkKh4Ru.json
> d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"66b2bea6-aa17-4019-8893-f51f1088988a" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-149849CH6IMkKh4Ru.json
info 22:41:54.140: Process Execution: cwd: d:\Repos\Other\toolformer\backend
cwd: d:\Repos\Other\toolformer\backend
info 22:41:54.993: ipykernel version & path 6.22.0, d:\Repos\Other\text-generation-webui\venv\lib\site-packages\ipykernel\__init__.py for d:\Repos\Other\text-generation-webui\venv\Scripts\python.exe
info 22:41:56.886: Started Kernel venv (Python 3.10.9) (pid: 24732)
info 22:42:01.747: Dispose Kernel 'd:\Repos\Other\toolformer\backend\exploration.ipynb' associated with 'd:\Repos\Other\toolformer\backend\exploration.ipynb'
info 22:42:01.751: Dispose Kernel process 24732.
info 22:42:02.187: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd0589f69e76bf646647623828f8eb4c232dce96d54508f09981a82407eb6f2efd0.d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe.d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe.-m#ipykernel_launcher (Python Path: d:\Repos\Other\toolformer\backend\venv, EnvType: Venv, EnvName: 'venv', Version: 3.10.9 | packaged by Anaconda, Inc. | (main, Mar  1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)]) for 'd:\Repos\Other\toolformer\backend\tts.ipynb' (disableUI=true)
info 22:42:02.527: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m pip list
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m pip list
info 22:42:02.623: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 22:42:02.755: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"f26c4d04-08b8-468c-9f27-fd2a535ec187" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984pLxtJ0SRVNpm.json
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"f26c4d04-08b8-468c-9f27-fd2a535ec187" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984pLxtJ0SRVNpm.json
info 22:42:02.755: Process Execution: cwd: d:\Repos\Other\toolformer\backend
cwd: d:\Repos\Other\toolformer\backend
info 22:42:03.247: ipykernel version & path 6.21.3, d:\Repos\Other\toolformer\backend\venv\lib\site-packages\ipykernel\__init__.py for d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe
info 22:42:04.635: Started Kernel venv (Python 3.10.9) (pid: 6564)
info 22:42:05.376: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\printJupyterDataDir.py
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe ~\.vscode\extensions\ms-toolsai.jupyter-2023.2.1200692131\pythonFiles\printJupyterDataDir.py
info 22:42:13.571: Dispose Kernel 'd:\Repos\Other\toolformer\backend\tts.ipynb' associated with 'd:\Repos\Other\toolformer\backend\tts.ipynb'
info 22:42:13.576: Dispose Kernel process 6564.
info 22:43:28.210: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd0589f69e76bf646647623828f8eb4c232dce96d54508f09981a82407eb6f2efd0.d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe.d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe.-m#ipykernel_launcher (Python Path: d:\Repos\Other\toolformer\backend\venv, EnvType: Venv, EnvName: 'venv', Version: 3.10.9 | packaged by Anaconda, Inc. | (main, Mar  1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)]) for 'd:\Repos\Other\toolformer\backend\tts.ipynb' (disableUI=true)
info 22:43:28.235: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 22:43:28.346: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"4f251dba-6a4d-452a-be9b-5a02270096d5" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984gqVPsa2mdH00.json
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"4f251dba-6a4d-452a-be9b-5a02270096d5" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984gqVPsa2mdH00.json
info 22:43:28.346: Process Execution: cwd: d:\Repos\Other\toolformer\backend
cwd: d:\Repos\Other\toolformer\backend
info 22:43:28.995: ipykernel version & path 6.21.3, d:\Repos\Other\toolformer\backend\venv\lib\site-packages\ipykernel\__init__.py for d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe
info 22:43:30.203: Started Kernel venv (Python 3.10.9) (pid: 9040)
info 22:43:33.442: Dispose Kernel 'd:\Repos\Other\toolformer\backend\tts.ipynb' associated with 'd:\Repos\Other\toolformer\backend\tts.ipynb'
info 22:43:33.448: Dispose Kernel process 9040.
info 22:44:02.145: Starting Jupyter Session startUsingPythonInterpreter, .jvsc74a57bd0589f69e76bf646647623828f8eb4c232dce96d54508f09981a82407eb6f2efd0.d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe.d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe.-m#ipykernel_launcher (Python Path: d:\Repos\Other\toolformer\backend\venv, EnvType: Venv, EnvName: 'venv', Version: 3.10.9 | packaged by Anaconda, Inc. | (main, Mar  1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)]) for 'd:\Repos\Other\toolformer\backend\tts.ipynb' (disableUI=true)
info 22:44:02.173: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
info 22:44:02.287: Process Execution: > d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"ad967861-8b6c-4eb8-b2c4-4f4db85c9a76" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984g0fmgYmmlxBQ.json
> d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe -m ipykernel_launcher --ip=127.0.0.1 --stdin=9003 --control=9001 --hb=9000 --Session.signature_scheme="hmac-sha256" --Session.key=b"ad967861-8b6c-4eb8-b2c4-4f4db85c9a76" --shell=9002 --transport="tcp" --iopub=9004 --f=c:\Users\TeamD\AppData\Roaming\jupyter\runtime\kernel-v2-14984g0fmgYmmlxBQ.json
info 22:44:02.287: Process Execution: cwd: d:\Repos\Other\toolformer\backend
cwd: d:\Repos\Other\toolformer\backend
info 22:44:03.137: ipykernel version & path 6.21.3, d:\Repos\Other\toolformer\backend\venv\lib\site-packages\ipykernel\__init__.py for d:\Repos\Other\toolformer\backend\venv\Scripts\python.exe
info 22:44:04.279: Started Kernel venv (Python 3.10.9) (pid: 22444)
info 22:56:51.501: Cancel all remaining cells true || Error || undefined
info 22:56:56.472: Cancel all remaining cells true || Error || undefined
info 23:01:13.367: Loading webview. View is notset
info 23:01:13.368: Loading web view...
info 23:01:13.368: Webview panel created.
info 23:01:13.584: Web view react rendered

Coding Language and Runtime Version

Python 3.10.9

Language Extension Version (if applicable)

Name: Python
Id: ms-python.python
Description: IntelliSense (Pylance), Linting, Debugging (multi-threaded, remote), Jupyter Notebooks, code formatting, refactoring, unit tests, and more.
Version: 2023.4.1
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.python

Anaconda Version (if applicable)

conda 22.11.1

Running Jupyter locally or remotely?

Local

@TeamDman TeamDman added the bug Issue identified by VS Code Team member as probable bug label Mar 27, 2023
@amunger amunger assigned DonJayamanne and unassigned amunger Mar 27, 2023
@DonJayamanne
Copy link
Contributor

@deepak1556 @mjbvz /cc
Not sure why autoplay attribute in the audio element isn't working
based on the docs https://developer.mozilla.org/en-US/docs/Web/Media/Autoplay_guide#autoplay_availability this shoudl work
I can see that we have autoplay turned on in the iframe hosting the output.
audio.ipynb.zip

Above is a sample file that you can just open in VS Code, no need of any extensions.
Ideally the audio should start playing automatically without having to hit the play button, i couldn't see any warnings or error messages in the console window.

Also, the output is plain HTML, nothing specific to jupyter.

@DonJayamanne DonJayamanne changed the title IPython.display.Audio autoplay not working Audio autoplay does not work in Notebook renderers Mar 30, 2023
@DonJayamanne DonJayamanne assigned deepak1556 and mjbvz and unassigned DonJayamanne Mar 30, 2023
@DonJayamanne DonJayamanne transferred this issue from microsoft/vscode-jupyter Mar 30, 2023
@DonJayamanne DonJayamanne added triage-needed and removed bug Issue identified by VS Code Team member as probable bug labels Mar 30, 2023
@TeamDman
Copy link
Author

Untitled.mp4

video of reproduction after disabling extensions in case it helps

@mjbvz
Copy link
Collaborator

mjbvz commented Apr 4, 2023

@DonJayamanne Autoplay typically requires user interaction. We may be able to workaround this though for the notebook iframe: https://developer.mozilla.org/en-US/docs/Web/Media/Autoplay_guide#autoplay_availability

A little wary though of just enabling autoplay by default. Definitely should only be done for trusted notebooks

I also wonder if this PR would fix this: #179204?

@meganrogge
Copy link
Contributor

I haven't tested, but believe #179204 would fix it. I wonder if we should have a setting (off by default) for this?

@DonJayamanne
Copy link
Contributor

wonder if we should have a setting (off by default) for this?

Is it possible to display a warning message in the HTML (above audio element) indicating the fact that the audio hasn't been auto-played and could be enabled via a setting (this way users know what is going on and can easily discovery the setting and change it if they want).

@vscodenpa vscodenpa added the unreleased Patch has not yet been released in VS Code Insiders label Apr 5, 2023
@vscodenpa vscodenpa added this to the April 2023 milestone Apr 5, 2023
@meganrogge meganrogge reopened this Apr 6, 2023
@vscodenpa vscodenpa removed the unreleased Patch has not yet been released in VS Code Insiders label Apr 6, 2023
@vscodenpa vscodenpa added the stale Issues that have not been triaged in an appropriate amount of time label Apr 13, 2023
@deepak1556 deepak1556 removed triage-needed stale Issues that have not been triaged in an appropriate amount of time labels Apr 13, 2023
@deepak1556 deepak1556 removed this from the April 2023 milestone Apr 13, 2023
@joyceerhl joyceerhl removed their assignment Apr 13, 2023
@deepak1556
Copy link
Collaborator

It is tricky to resolve this, requires a new API from the runtime to override the autoplay policy of a particular frame, isolate the notebooks to a different origin. Also the solution cannot work for the web version, closing as not actionable.

@github-actions github-actions bot locked and limited conversation to collaborators Jan 20, 2024
@aiday-mar aiday-mar added this to the December / January 2024 milestone Feb 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants