Trim Python dependencies #990

Job	Run time
build-vegafusion-python-linux-arm64	16m 3s
build-vegafusion-python-osx-64	19m 52s
build-vegafusion-python-osx-arm64	13m 44s
build-vegafusion-python-linux-64	15m 17s
build-vegafusion-python-win-64	34m 48s
build-vegafusion-server-linux-64	13m 5s
build-vegafusion-server-osx-64	29m 24s
build-vegafusion-server-linux-arm64	14m 33s
build-vegafusion-server-osx-arm64	12m 42s
build-vegafusion-server-win-64	28m 58s
build-vegafusion-wasm	10m 51s
check-rust	7m 38s
check-vegafusion-python	10m 46s
test-rust (ubuntu-20.04)	25m 59s
test-rust (windows-2022)	49m 26s
test-rust (macos-14)	25m 7s
test-vegafusion-python-osx-arm64	24s
test-vegafusion-python-linux-64	1m 28s
test-vegafusion-python-win-64	2m 6s
	5h 32m 11s

Provide feedback