Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerfile/chore: Update Dockerfile #5097

Merged
merged 1 commit into from
Feb 25, 2024
Merged

Conversation

Constantin1489
Copy link
Contributor

@Constantin1489 Constantin1489 commented Jan 22, 2024

Reduce a layer, update pip and apk, and reduce increased image size by optimization.

The previous image size is approximately 247 MB. With this PR, 244 MB

Without the code below

python3 -m pip install --no-cache-dir -U pip  && \

the image will be 10MB smaller.

@mikf
Copy link
Owner

mikf commented Jan 26, 2024

Wait, so you reduced the Docker image size by a few MB and then also increased it by 10 MB with pip install --no-cache-dir -U pip for a net total of -3 MB? Why upgrade pip then?

And wouldn't it be even better to cram all shell commands into a single RUN block?

RUN python3 -m pip install --no-cache-dir --disable-pip-version-check -U gallery-dl yt-dlp && \
    apk update && \
    apk add --no-cache ffmpeg && \
    rm -rf /var/cache/apk/*

edit: added --disable-pip-version-check

@Constantin1489
Copy link
Contributor Author

Constantin1489 commented Jan 28, 2024

  1. Ah, the second one is a good question.
    I'm going to answer it generally not docker-specific nor gallery-dl docker. (docker used to take reportedly huge size for a layer. the recent versions seem doesn't)

Try pip freeze, and create each pip install layer for the package. there will be a huge overhead. Even because of the packages installed in the previous layers as a dependency, pip will still check whether the package exists.

So generally cramming all the dependencies or packages is good.

But joining the apk and pip doesn't have such a benefit. That may require rebuilding without caching if one of them is updated.

So joining the pip package is efficient and syntactically looks good.

Let's install over 100 packages without any CMD, ENTRYPOINT.

example Dockerfile: over 10 minutes. Also if one of them is updated you need to spend a lot of time, again.
From python:alpine
RUN python3 -m pip install -U Brotli
RUN python3 -m pip install -U Jinja2
RUN python3 -m pip install -U MarkupSafe
RUN python3 -m pip install -U PTable
RUN python3 -m pip install -U Pillow
RUN python3 -m pip install -U PyYAML
RUN python3 -m pip install -U Pygments
RUN python3 -m pip install -U QtPy
RUN python3 -m pip install -U SQLAlchemy
RUN python3 -m pip install -U Send2Trash
RUN python3 -m pip install -U aiosignal
RUN python3 -m pip install -U anyio
RUN python3 -m pip install -U argon2-cffi
RUN python3 -m pip install -U argon2-cffi-bindings
RUN python3 -m pip install -U arrow
RUN python3 -m pip install -U asttokens
RUN python3 -m pip install -U async-timeout
RUN python3 -m pip install -U attrs
RUN python3 -m pip install -U backcall
RUN python3 -m pip install -U beautifulsoup4
RUN python3 -m pip install -U bleach
RUN python3 -m pip install -U certifi
RUN python3 -m pip install -U cffi
RUN python3 -m pip install -U charset-normalizer
RUN python3 -m pip install -U comm
RUN python3 -m pip install -U contourpy
RUN python3 -m pip install -U crab
RUN python3 -m pip install -U cycler
RUN python3 -m pip install -U db
RUN python3 -m pip install -U db-sqlite3
RUN python3 -m pip install -U debugpy
RUN python3 -m pip install -U decorator
RUN python3 -m pip install -U defusedxml
RUN python3 -m pip install -U elementpath
RUN python3 -m pip install -U entrypoints
RUN python3 -m pip install -U executing
RUN python3 -m pip install -U fastjsonschema
RUN python3 -m pip install -U flake8
RUN python3 -m pip install -U fonttools
RUN python3 -m pip install -U fqdn
RUN python3 -m pip install -U frozenlist
RUN python3 -m pip install -U gallery-dl
RUN python3 -m pip install -U gevent
RUN python3 -m pip install -U greenlet
RUN python3 -m pip install -U inscriptis
RUN python3 -m pip install -U isoduration
RUN python3 -m pip install -U jedi
RUN python3 -m pip install -U jsonpointer
RUN python3 -m pip install -U jsonschema
RUN python3 -m pip install -U kiwisolver
RUN python3 -m pip install -U loguru
RUN python3 -m pip install -U lxml
RUN python3 -m pip install -U matplotlib
RUN python3 -m pip install -U matplotlib-inline
RUN python3 -m pip install -U mccabe
RUN python3 -m pip install -U mistune
RUN python3 -m pip install -U mutagen
RUN python3 -m pip install -U nest-asyncio
RUN python3 -m pip install -U numpy
RUN python3 -m pip install -U openai
RUN python3 -m pip install -U overrides
RUN python3 -m pip install -U packaging
RUN python3 -m pip install -U pandas
RUN python3 -m pip install -U pandocfilters
RUN python3 -m pip install -U parso
RUN python3 -m pip install -U pexpect
RUN python3 -m pip install -U pickleshare
RUN python3 -m pip install -U platformdirs
RUN python3 -m pip install -U prettytable
RUN python3 -m pip install -U prometheus-client
RUN python3 -m pip install -U prompt-toolkit
RUN python3 -m pip install -U ptyprocess
RUN python3 -m pip install -U pure-eval
RUN python3 -m pip install -U pycodestyle
RUN python3 -m pip install -U pycparser
RUN python3 -m pip install -U pycryptodomex
RUN python3 -m pip install -U pyflakes
RUN python3 -m pip install -U pyparsing
RUN python3 -m pip install -U pyrsistent
RUN python3 -m pip install -U python-dateutil
RUN python3 -m pip install -U python-json-logger
RUN python3 -m pip install -U pytz
RUN python3 -m pip install -U pyvipe
RUN python3 -m pip install -U pyzmq
RUN python3 -m pip install -U requests
RUN python3 -m pip install -U rfc3339-validator
RUN python3 -m pip install -U rfc3986-validator
RUN python3 -m pip install -U six
RUN python3 -m pip install -U sniffio
RUN python3 -m pip install -U soupsieve
RUN python3 -m pip install -U sqlparse
RUN python3 -m pip install -U stack-data
RUN python3 -m pip install -U tabulate
RUN python3 -m pip install -U terminado
RUN python3 -m pip install -U tinycss2
RUN python3 -m pip install -U tomli_w
RUN python3 -m pip install -U tomlkit
RUN python3 -m pip install -U tornado
RUN python3 -m pip install -U tqdm
RUN python3 -m pip install -U traitlets
RUN python3 -m pip install -U trrc
RUN python3 -m pip install -U truth-table-generator
RUN python3 -m pip install -U typing_extensions
RUN python3 -m pip install -U tzdata
RUN python3 -m pip install -U uri-template
RUN python3 -m pip install -U urllib3
RUN python3 -m pip install -U wcwidth
RUN python3 -m pip install -U webcolors
RUN python3 -m pip install -U webencodings
RUN python3 -m pip install -U websocket-client
RUN python3 -m pip install -U websockets
RUN python3 -m pip install -U widgetsnbextension
RUN python3 -m pip install -U yt-dlp
example Dockerfile2 : almost 1.5 minutes. Even one of them is updated still almost 1.5 minutes
From python:alpine
RUN python3 -m pip install -U Brotli Jinja2 MarkupSafe PTable Pillow PyYAML Pygments QtPy SQLAlchemy Send2Trash aiosignal anyio argon2-cffi argon2-cffi-bindings arrow asttokens async-timeout attrs backcall beautifulsoup4 bleach certifi cffi charset-normalizer comm contourpy crab cycler db db-sqlite3 debugpy decorator defusedxml elementpath entrypoints executing fastjsonschema flake8 fonttools fqdn frozenlist gallery-dl gevent greenlet inscriptis isoduration jedi jsonpointer jsonschema kiwisolver loguru lxml matplotlib matplotlib-inline mccabe mistune mutagen nest-asyncio numpy openai overrides packaging pandas pandocfilters parso pexpect pickleshare platformdirs prettytable prometheus-client prompt-toolkit ptyprocess pure-eval pycodestyle pycparser pycryptodomex pyflakes pyparsing pyrsistent python-dateutil python-json-logger pytz pyvipe pyzmq requests rfc3339-validator rfc3986-validator six sniffio soupsieve sqlparse stack-data tabulate terminado tinycss2 tomli_w tomlkit tornado tqdm traitlets trrc truth-table-generator typing_extensions tzdata uri-template urllib3 wcwidth webcolors webencodings websocket-client websockets widgetsnbextension yt-dlp
  1. Updating pip is for security and so on. The first question may connote downsizing ffmpeg. If you want, I can make it install optionally.

@mikf mikf merged commit efccd3d into mikf:master Feb 25, 2024
9 checks passed
@mikf mikf added the docker label Mar 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants