Skip to content

Provides patches for widely used http libraries to make them work in Pyodide environments like JupyterLite

License

Notifications You must be signed in to change notification settings

koenvo/pyodide-http

Repository files navigation

Pyodide-HTTP

PyPI Latest Release GHA

Provides patches for widely used http libraries to make them work in Pyodide environments like JupyterLite.

Usage

# 1. Install this package
import micropip
await micropip.install('pyodide-http')

# 2. Patch requests
import pyodide_http
pyodide_http.patch_all()  # Patch all libraries

# 3. Use requests
import requests
response = requests.get('https://raw.githubusercontent.com/statsbomb/open-data/master/data/lineups/15946.json')

How does this work?

This package applies patches to common http libraries. How the patch works depends on the package.

All non-streaming requests are replaced with calls using XMLHttpRequest.

Streaming requests (i.e. calls with stream=True in requests) are replaced by calls to fetch in a separate web-worker if (and only if) you are in a state that can support web-threading correctly, which is that cross-origin isolation is enabled, and you are running pyodide in a web-worker. Otherwise it isn't possible until WebAssembly stack-switching becomes available, and it falls back to an implementation that fetches everything then returns a stream wrapper to a memory buffer.

Enabling Cross-Origin isolation

The implementation of streaming requests makes use of Atomics.wait and SharedArrayBuffer to do the fetch in a separate web worker. For complicated web-security reasons, SharedArrayBuffers cannot be passed to a web-worker unless you have cross-origin isolation enabled. You enable this by serving the page using the following two headers:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Be aware that this will have effects on what you are able to embed on the page - check out https://web.dev/cross-origin-isolation-guide/ for more details.

Supported packages

Currently the following packages can be patched:

Package Patched
requests Session
request
head, get, post, put, patch, delete
urllib urlopen
OpenerDirector

Used by

pyodide-http is used by a some awesome projects:

  • Pyodide - included as a standard package
  • Panel - included since 0.14.1 (can be disabled) when running Panel in the Browser using WASM. Read more

About

Provides patches for widely used http libraries to make them work in Pyodide environments like JupyterLite

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published