Add description in a few more places about adding airflow to pip inst…

…all (#31448)

It is important to keep airflow as a dependency with pinned version if
you are adding new dependencies. This PR adds it in case of
docker-compose documentation and also improves examples in our
images, by using AIRFLOW_VERSION variable defined in the images. This
way users can write future-proof ways of installind dependencies basing
on our examples.

Follow-up after #30813

(cherry picked from commit d722fba)

docker_tests/test_examples_of_prod_image_building.py

@@ -59,7 +59,6 @@ def test_dockerfile_example(dockerfile):
     new_content = re.sub(
         r"FROM apache/airflow:.*", rf"FROM apache/airflow:{latest_released_version}", content
     )
-    new_content = re.sub(r"apache-airflow==\S*", rf"apache-airflow=={latest_released_version}", new_content)
     try:
         run_command(
             ["docker", "build", ".", "--tag", image_name, "-f", "-"],

docs/apache-airflow/howto/docker-compose/index.rst

@@ -307,18 +307,23 @@ you should do those steps:
    to (use correct image tag):
 
 ```
-#image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.5.1}
+#image: ${AIRFLOW_IMAGE_NAME:-apache/airflow:2.6.1}
 build: .
 ```
 
 2) Create ``Dockerfile`` in the same folder your ``docker-compose.yaml`` file is with content similar to:
 
 ```
-FROM apache/airflow:2.5.1
+FROM apache/airflow:2.6.1
 ADD requirements.txt .
-RUN pip install -r requirements.txt
+RUN pip install apache-airflow==${AIRFLOW_VERSION} -r requirements.txt
 ```
 
+It is the best practice to install apache-airflow in the same version as the one that comes from the
+original image. This way you can be sure that ``pip`` will not try to downgrade or upgrade apache
+airflow while installing other requirements, which might happen in case you try to add a dependency
+that conflicts with the version of apache-airflow that you are using.
+
 3) Place ``requirements.txt`` file in the same directory.
 
 Run ``docker compose build`` to build the image, or add ``--build`` flag to ``docker compose up`` or

docs/apache-airflow/installation/installing-from-pypi.rst

@@ -181,6 +181,15 @@ Airflow versions or other providers. Some providers have minimum-required versio
 versions of providers might have conflicting requirements with Airflow or other dependencies you
 might have installed.
 
+It is the best practice to install apache-airflow in the same version as the one that comes from the
+original image. This way you can be sure that ``pip`` will not try to downgrade or upgrade apache
+airflow while installing other requirements, which might happen in case you try to add a dependency
+that conflicts with the version of apache-airflow that you are using:
+
+.. code-block:: bash
+
+    pip install "apache-airflow==|version|" "apache-airflow-providers-google==8.0.0"
+
 
 Installation and upgrade of Airflow core
 ========================================

docs/docker-stack/docker-examples/extending/add-build-essential-extend/Dockerfile

@@ -24,5 +24,5 @@ RUN apt-get update \
   && apt-get clean \
   && rm -rf /var/lib/apt/lists/*
 USER airflow
-RUN pip install --no-cache-dir mpi4py
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" mpi4py
 # [END Dockerfile]

docs/docker-stack/docker-examples/extending/add-providers/Dockerfile

@@ -25,5 +25,5 @@ RUN apt-get update \
   && rm -rf /var/lib/apt/lists/*
 USER airflow
 ENV JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
-RUN pip install --no-cache-dir apache-airflow-providers-apache-spark==2.1.3 apache-airflow==2.6.0dev0
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" apache-airflow-providers-apache-spark==2.1.3
 # [END Dockerfile]

docs/docker-stack/docker-examples/extending/add-pypi-packages/Dockerfile

@@ -16,5 +16,5 @@
 # This is an example Dockerfile. It is not intended for PRODUCTION use
 # [START Dockerfile]
 FROM apache/airflow:2.6.1
-RUN pip install --no-cache-dir lxml apache-airflow==2.6.0.dev0
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" lxml
 # [END Dockerfile]

docs/docker-stack/docker-examples/extending/add-requirement-packages/Dockerfile

@@ -17,5 +17,5 @@
 # [START Dockerfile]
 FROM apache/airflow:2.6.1
 COPY requirements.txt /
-RUN pip install --no-cache-dir -r /requirements.txt apache-airflow==2.6.0.dev0
+RUN pip install --no-cache-dir "apache-airflow==${AIRFLOW_VERSION}" -r /requirements.txt
 # [END Dockerfile]

docs/docker-stack/docker-examples/extending/custom-providers/Dockerfile

@@ -16,5 +16,5 @@
 # This is an example Dockerfile. It is not intended for PRODUCTION use
 # [START Dockerfile]
 FROM apache/airflow:2.6.1
-RUN pip install --no-cache-dir apache-airflow-providers-docker==2.5.1 apache-airflow==2.6.0.dev0
+RUN pip install "apache-airflow==${AIRFLOW_VERSION}" --no-cache-dir apache-airflow-providers-docker==2.5.1
 # [END Dockerfile]