From d6fa4746b7d0494ceb0a84f6f3f4d238941ccbf5 Mon Sep 17 00:00:00 2001 From: JosepSampe Date: Wed, 27 Mar 2024 16:07:30 +0100 Subject: [PATCH] Update runtime docs and changelog --- CHANGELOG.md | 15 +++++- config/README.md | 3 +- runtime/aliyun_fc/README.md | 61 +++--------------------- runtime/aliyun_fc/requirements.txt | 9 +++- runtime/aws_lambda/README.md | 6 +-- runtime/azure_containers/README.md | 30 ++++++------ runtime/azure_functions/README.md | 58 +++------------------- runtime/azure_functions/requirements.txt | 9 +++- runtime/code_engine/README.md | 28 +++++------ runtime/gcp_cloudrun/README.md | 8 ++-- runtime/gcp_functions/README.md | 10 ++-- runtime/knative/README.md | 28 +++++------ runtime/kubernetes/README.md | 18 +++---- runtime/openwhisk/README.md | 28 +++++------ 14 files changed, 122 insertions(+), 189 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c5fdf451..51bd0f9c8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,18 @@ # Changelog -## [v3.1.3.dev1] +## [v3.2.1.dev0] + +### Added +- + +### Changed +- + +### Fixed +- + + +## [v3.2.0] ### Added - [Lithops] Addded support for Python 3.12 @@ -17,6 +29,7 @@ - [SSH Cli] Fixed minor error with the "err" variable - [Cli] Fixed job status on "lithops job list" for standalone backends - [Standalone] Fixed issue in the "lithops image build" that appears when the vpc is already created +- [Future] Fixed issue with missing 'worker_end_tstamp' variable ## [v3.1.2] diff --git a/config/README.md b/config/README.md index 5351daaa9..638670e94 100644 --- a/config/README.md +++ b/config/README.md @@ -54,7 +54,6 @@ Storage Backends - [Aliyun Functions](../docs/source/compute_config/aliyun_functions.md) - [Oracle Cloud Functions](../docs/source/compute_config/oracle_functions.md) - [OpenWhisk](../docs/source/compute_config/openwhisk.md) -- [IBM Cloud Functions](../docs/source/compute_config/ibm_cf.md) Serverless (CaaS) Backends: - [IBM Code Engine](../docs/source/compute_config/code_engine.md) @@ -113,7 +112,7 @@ if __name__ == '__main__': ``` ### Providing configuration in runtime -Example of providing configuration keys for IBM Cloud Functions and IBM Cloud Object Storage +Example of providing configuration keys for IBM Code Engine and IBM Cloud Object Storage ```python import lithops diff --git a/runtime/aliyun_fc/README.md b/runtime/aliyun_fc/README.md index f70a18442..55fe8b8f9 100644 --- a/runtime/aliyun_fc/README.md +++ b/runtime/aliyun_fc/README.md @@ -1,27 +1,10 @@ # Lithops runtime for Aliyun Functions Compute -The runtime is the place where your functions are executed. +The runtime is the place where your functions are executed. The default runtime is automatically created the first time you execute a function. Lithops automatically detects the Python version of your environment and deploys the default runtime based on it. -The default runtime is created the first time you execute a function. Lithops automatically detects the Python version of your environment and deploys the default runtime based on it. +Currently, Aliyun Functions Compute supports Python 3.6, 3.9 and 3.10. You can find the list of pre-installed modules [here](https://www.alibabacloud.com/help/en/function-compute/latest/python-event-functions). In addition, the Lithops default runtimes are built with the packages included in this [requirements.txt](requirements.txt) file: -Currently, Aliyun Functions Compute supports Python 3.6, and 3.9, and it provides the following default runtimes with some packages already preinstalled: - - -| Runtime name | Python version | Packages included | -| ----| ----| ---- | -| lithops-default-runtime-v36 | 3.6 | [list of packages](https://www.alibabacloud.com/help/en/function-compute/latest/python-event-functions) | -| lithops-default-runtime-v38 | 3.9 | [list of packages](https://www.alibabacloud.com/help/en/function-compute/latest/python-event-functions) | - -Lithops default runtimes are also ship with the following packages: -``` -pika -tblib -cloudpickle -ps-mem -``` - - -To run a function with the default runtime you don't need to specify anything in the code, since everything is managed internally by Lithops: +To run a function with the default runtime you don't need to specify anything in the code, since everything is handled internally by Lithops: ```python import lithops @@ -45,44 +28,14 @@ pw = lithops.FunctionExecutor(runtime_memory=512) **Build your own Lithops runtime for Aliyun Functions Compute** -If you need some Python modules which are not included in the default runtime, it is possible to build your own Lithops runtime with all of them. - -To build your own runtime, you have to collect all necessary modules in a `requirements.txt` file. +If you require additional Python modules not included in the default runtime, you can create your own custom Lithops runtime incorporating them. To create a custom runtime, compile all the necessary modules into a `requirements.txt` file. -For example, we want to add module `matplotlib` to our runtime, since it is not provided in the default runtime. +For instance, if you wish to integrate the `matplotlib` module into your runtime, which isn't part of the default setup, you need to append it to the existing [requirements.txt](requirements.txt) file. Note that this `requirements.txt` contains the mandatory pakcges required by lithops, so you don't have to remove any of them from the list, but just add your packages at the end. -First, we need to extend the default `requirements.txt` file provided with Lithops with all the modules we need. For our example, the `requirements.txt` will contain the following modules: -``` -numpy -scikit-learn -scipy -pandas -google-cloud -google-cloud-storage -google-cloud-pubsub -certifi -chardet -docutils -httplib2 -idna -jmespath -kafka-python -lxml -pika==0.13.0 -python-dateutil -redis -requests -simplejson -six -urllib3 -virtualenv -PyYAML -matplotlib -``` +After updating the file accordingly, you can proceed to build the custom runtime by specifying the modified `requirements.txt` file along with a chosen runtime name: -Then, we will build the runtime, specifying the modified `requirements.txt` file and a runtime name: ``` -$ lithops runtime build -f requirements.txt my_matplotlib_runtime -b aliyun_fc +$ lithops runtime build -b aliyun_fc -f requirements.txt my_matplotlib_runtime ``` This command will built and deploy a runtime called `my_matplotlib_runtime` to the available Aliyun FUnctions Compute runtimes. diff --git a/runtime/aliyun_fc/requirements.txt b/runtime/aliyun_fc/requirements.txt index e4b11fecb..0e9ce45f7 100644 --- a/runtime/aliyun_fc/requirements.txt +++ b/runtime/aliyun_fc/requirements.txt @@ -1,3 +1,6 @@ +# Requirements.txt contains a list of dependencies for the Python Application # + +# Mandatory Lithops packages aliyun-fc2 oss2 pika @@ -11,4 +14,8 @@ numpy cloudpickle ps-mem tblib -psutil \ No newline at end of file +psutil + +# Optional packages (Add your packages below) +#pandas +#matplotlib \ No newline at end of file diff --git a/runtime/aws_lambda/README.md b/runtime/aws_lambda/README.md index 81fa35a8d..471df70e0 100644 --- a/runtime/aws_lambda/README.md +++ b/runtime/aws_lambda/README.md @@ -71,14 +71,14 @@ lithops runtime build -f MyDockerfile -b aws_lambda my-container-runtime-name For example: ``` -lithops runtime build -f MyDockerfile -b aws_lambda lithops-ndvi-v310:01 +lithops runtime build -f MyDockerfile -b aws_lambda lithops-ndvi-v312:01 ``` Finally, we can specify this new runtime in the lithops config: ```yaml aws_lambda: - runtime: lithops-ndvi-v310:01 + runtime: lithops-ndvi-v312:01 ``` or when creating a Lithops Function Executor: @@ -89,7 +89,7 @@ import lithops def test(): return 'hello' -lith = lithops.FunctionExecutor(runtime='lithops-ndvi-v310:01') +lith = lithops.FunctionExecutor(runtime='lithops-ndvi-v312:01') lith.call_async(test, data=()) res = lith.get_result() print(res) # Prints 'hello' diff --git a/runtime/azure_containers/README.md b/runtime/azure_containers/README.md index 089bfa9f7..d677c0bf8 100644 --- a/runtime/azure_containers/README.md +++ b/runtime/azure_containers/README.md @@ -51,27 +51,27 @@ azure_containers: $ docker login Update the Dockerfile that better fits to your requirements with your required system packages and Python modules. - If you need another Python version, for example Python 3.9, you must change the initial line of the Dockefile: + If you need another Python version, for example Python 3.12, you must change the initial line of the Dockefile: - $ lithops runtime build docker_username/runtimename:tag -b azure_containers + $ lithops runtime build -b azure_containers docker_username/runtimename:tag Note that Docker hub image names look like *"docker_username/runtimename:tag"* and must be all lower case, for example: - $ lithops runtime build -b azure_containers myaccount/lithops-ca-custom-v39:01 + $ lithops runtime build -b azure_containers myaccount/lithops-ca-custom-v312:01 By default the Dockerfile should be located in the same folder from where you execute the **lithops runtime** command. If your Dockerfile is located in another folder, or the Dockerfile has another name, you can specify its location with the **-f** parameter, for example: - $ lithops runtime build -b azure_containers -f azure_containers/Dockerfile myaccount/lithops-ca-custom-v39:01 + $ lithops runtime build -b azure_containers -f azure_containers/Dockerfile myaccount/lithops-ca-custom-v312:01 Once you have built your runtime with all of your necessary packages, you can already use it with Lithops. To do so, you have to specify the full docker image name in the configuration or when you create the **FunctionExecutor** instance, or directly in the config file, for example: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='myaccount/lithops-ca-custom-v39:01') + fexec = lithops.FunctionExecutor(runtime='myaccount/lithops-ca-custom-v312:01') ``` - *NOTE: In this previous example shows how to build a Docker image based on Python 3.9, this means that now you also need Python 3.9 in the client machine.* + *NOTE: In this previous example shows how to build a Docker image based on Python 3.12, this means that now you also need Python 3.12 in the client machine.* 2. **Use an already built runtime from a public repository** @@ -80,29 +80,29 @@ azure_containers: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='lithopscloud/ca-conda-v39:01') + fexec = lithops.FunctionExecutor(runtime='lithopscloud/ca-conda-v312:01') ``` Alternatively, you can create a Lithops runtime based on already built Docker image by executing the following command, which will deploy all the necessary information to use the runtime with your Lithops. $ lithops runtime deploy -b azure_containers -s azure_storage docker_username/runtimename:tag - For example, you can use an already created runtime based on Python 3.9 and with the *matplotlib* and *nltk* libraries by running: + For example, you can use an already created runtime based on Python 3.12 and with the *matplotlib* and *nltk* libraries by running: - $ lithops runtime deploy -b azure_containers -s azure_storage lithopscloud/ca-matplotlib-v39:01 + $ lithops runtime deploy -b azure_containers -s azure_storage lithopscloud/ca-matplotlib-v312:01 Once finished, you can use the runtime in your Lithops code: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='lithopscloud/ca-matplotlib:v39:01') + fexec = lithops.FunctionExecutor(runtime='lithopscloud/ca-matplotlib:v312:01') ``` ## Runtime Management 1. **Update an existing runtime** - If you are a developer, and modified the PyWeen source code, you need to deploy the changes before executing Lithops. + If you are a developer, and modified the Lithops source code, you need to deploy the changes before executing Lithops. You can update default runtime by: @@ -112,9 +112,9 @@ azure_containers: $ lithops runtime update docker_username/runtimename:tag -b azure_containers -s azure_storage - For example, you can update an already created runtime based on the Docker image `lithopscloud/ca-matplotlib-v39:01` by: + For example, you can update an already created runtime based on the Docker image `lithopscloud/ca-matplotlib-v312:01` by: - $ lithops runtime update lithopscloud/ca-matplotlib-v39:01 -b azure_containers -s azure_storage + $ lithops runtime update lithopscloud/ca-matplotlib-v312:01 -b azure_containers -s azure_storage Alternatively, you can update all the deployed runtimes at a time by: @@ -132,9 +132,9 @@ azure_containers: $ lithops runtime delete docker_username/runtimename:tag -b azure_containers -s azure_storage - For example, you can delete runtime based on the Docker image `lithopscloud/ca-conda-v39:01` by: + For example, you can delete runtime based on the Docker image `lithopscloud/ca-conda-v312:01` by: - $ lithops runtime delete lithopscloud/ca-conda-v39:01 -b azure_containers -s azure_storage + $ lithops runtime delete lithopscloud/ca-conda-v312:01 -b azure_containers -s azure_storage You can delete all the runtimes at a time by: diff --git a/runtime/azure_functions/README.md b/runtime/azure_functions/README.md index 5de3962d8..4980ccf63 100644 --- a/runtime/azure_functions/README.md +++ b/runtime/azure_functions/README.md @@ -1,37 +1,10 @@ # Lithops runtime for Azure Functions -The runtime is the place where your functions are executed. +The runtime is the place where your functions are executed. The default runtime is automatically created the first time you execute a function. Lithops automatically detects the Python version of your environment and deploys the default runtime based on it. -The default runtime is created the first time you execute a function. Lithops automatically detects the Python version of your environment and deploys the default runtime based on it. +Currently, Azure Functions supports Python 3.6, 3.7, 3.8 and 3.9. You can find the list of pre-installed modules [here](https://github.com/Azure/azure-functions-python-worker/wiki/Preinstalled-Python-Libraries). In addition, the Lithops default runtimes are built with the packages included in this [requirements.txt](requirements.txt) file -Currently, Azure Functions supports Python 3.6, 3.7, 3.8 and 3.9, and it provides the following default runtimes with some packages already preinstalled: - -| Runtime name | Python version | Packages included | -| ----| ----| ---- | -| lithops-runtime-v36 | 3.6 | [list of packages](https://github.com/Azure/azure-functions-python-worker/wiki/Preinstalled-Python-Libraries) | -| lithops-runtime-v38 | 3.7 | [list of packages](https://github.com/Azure/azure-functions-python-worker/wiki/Preinstalled-Python-Libraries) | -| lithops-runtime-v38 | 3.8 | [list of packages](https://github.com/Azure/azure-functions-python-worker/wiki/Preinstalled-Python-Libraries) | -| lithops-runtime-v38 | 3.9 | [list of packages](https://github.com/Azure/azure-functions-python-worker/wiki/Preinstalled-Python-Libraries) | - -Lithops default runtimes are also ship with the following packages: -``` -azure-functions -azure-storage-blob -azure-storage-queue -pika -flask -gevent -redis -requests -PyYAML -kubernetes -numpy -cloudpickle -ps-mem -tblib -``` - -To run a function with the default runtime you don't need to specify anything in the code, since everything is managed internally by Lithops: +To run a function with the default runtime you don't need to specify anything in the code, since everything is handled internally by Lithops: ```python import lithops @@ -50,32 +23,13 @@ result = lithops.get_result() **Build your own Lithops runtime for Azure Functions** -If you need some Python modules which are not included in the default runtime, it is possible to build your own Lithops runtime with all of them. - -To build your own runtime, you have to collect all necessary modules in a `requirements.txt` file. For example, if you want to add the modules `numpy` and `matplotlib` to our runtime, since they are not provided in the default runtime. +If you require additional Python modules not included in the default runtime, you can create your own custom Lithops runtime incorporating them. To create a custom runtime, compile all the necessary modules into a `requirements.txt` file. -First, we need to extend the `requirements.txt` file provided with Lithops with all the modules we need. For our example, the `requirements.txt` should contain the following modules (note that we added `numpy` and `matplotlib` at the end): -``` -azure-functions -azure-storage-blob -azure-storage-queue -pika -flask -gevent -redis -requests -PyYAML -kubernetes -cloudpickle -ps-mem -tblib -numpy -matplotlib -``` +For instance, if you wish to integrate the `matplotlib` module into your runtime, which isn't part of the default setup, you need to append it to the existing [requirements.txt](requirements.txt) file. Note that this `requirements.txt` contains the mandatory pakcges required by lithops, so you don't have to remove any of them from the list, but just add your packages at the end. **IMPORTANT**: Note that the runtime is built using your local machine, and some libraries, like Numpy, compile some *C* code based on the Operating System you are using. Azure functions run on a Linux machine, this mean that if you use **MacOS** or **Windows** for building the runtime, those libraries that compiled *C* code cannot be executed from within the function. In this case, you must use a Linux machine for building the runtime. -Then, we will build the runtime, specifying the modified `requirements.txt` file and a runtime name: +After updating the file accordingly, you can proceed to build the custom runtime by specifying the modified `requirements.txt` file along with a chosen runtime name: ``` $ lithops runtime build -b azure_functions -f requirements.txt matplotlib-runtime ``` diff --git a/runtime/azure_functions/requirements.txt b/runtime/azure_functions/requirements.txt index 6ae1bd23a..dcf6336c2 100644 --- a/runtime/azure_functions/requirements.txt +++ b/runtime/azure_functions/requirements.txt @@ -1,3 +1,6 @@ +# Requirements.txt contains a list of dependencies for the Python Application # + +# Mandatory Lithops packages azure-functions azure-storage-blob azure-storage-queue @@ -12,4 +15,8 @@ kubernetes cloudpickle ps-mem tblib -psutil \ No newline at end of file +psutil + +# Optional packages (Add your packages below) +#pandas +#matplotlib \ No newline at end of file diff --git a/runtime/code_engine/README.md b/runtime/code_engine/README.md index d3e462509..04639a4fa 100644 --- a/runtime/code_engine/README.md +++ b/runtime/code_engine/README.md @@ -52,27 +52,27 @@ code_enigne: $ docker login Update the Dockerfile that better fits to your requirements with your required system packages and Python modules. - If you need another Python version, for example Python 3.9, you must change the initial line of the Dockefile: + If you need another Python version, for example Python 3.12, you must change the initial line of the Dockefile: $ lithops runtime build docker_username/runtimename:tag -b code_engine Note that Docker hub image names look like *"docker_username/runtimename:tag"* and must be all lower case, for example: - $ lithops runtime build -b code_engine myaccount/lithops-ce-custom-v39:01 + $ lithops runtime build -b code_engine myaccount/lithops-ce-custom-v312:01 By default the Dockerfile should be located in the same folder from where you execute the **lithops runtime** command. If your Dockerfile is located in another folder, or the Dockerfile has another name, you can specify its location with the **-f** parameter, for example: - $ lithops runtime build -b code_engine -f code_engine/Dockerfile.conda myaccount/lithops-ce-custom-v39:01 + $ lithops runtime build -b code_engine -f code_engine/Dockerfile.conda myaccount/lithops-ce-custom-v312:01 Once you have built your runtime with all of your necessary packages, you can already use it with Lithops. To do so, you have to specify the full docker image name in the configuration or when you create the **FunctionExecutor** instance, or directly in the config file, for example: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='myaccount/lithops-ce-custom-v39:01') + fexec = lithops.FunctionExecutor(runtime='myaccount/lithops-ce-custom-v312:01') ``` - *NOTE: In this previous example shows how to build a Docker image based on Python 3.9, this means that now you also need Python 3.9 in the client machine.* + *NOTE: In this previous example shows how to build a Docker image based on Python 3.12, this means that now you also need Python 3.12 in the client machine.* 2. **Use an already built runtime from a public repository** @@ -81,29 +81,29 @@ code_enigne: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='lithopscloud/ce-conda-v39:01') + fexec = lithops.FunctionExecutor(runtime='lithopscloud/ce-conda-v312:01') ``` Alternatively, you can create a Lithops runtime based on already built Docker image by executing the following command, which will deploy all the necessary information to use the runtime with your Lithops. $ lithops runtime deploy -b code_engine -s ibm_cos docker_username/runtimename:tag - For example, you can use an already created runtime based on Python 3.9 and with the *matplotlib* and *nltk* libraries by running: + For example, you can use an already created runtime based on Python 3.12 and with the *matplotlib* and *nltk* libraries by running: - $ lithops runtime deploy -b code_engine -s ibm_cos lithopscloud/ce-matplotlib-v39:01 + $ lithops runtime deploy -b code_engine -s ibm_cos lithopscloud/ce-matplotlib-v312:01 Once finished, you can use the runtime in your Lithops code: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='lithopscloud/ce-matplotlib:v39:01') + fexec = lithops.FunctionExecutor(runtime='lithopscloud/ce-matplotlib-v312:01') ``` ## Runtime Management 1. **Update an existing runtime** - If you are a developer, and modified the PyWeen source code, you need to deploy the changes before executing Lithops. + If you are a developer, and modified the Lithops source code, you need to deploy the changes before executing Lithops. You can update default runtime by: @@ -113,9 +113,9 @@ code_enigne: $ lithops runtime update docker_username/runtimename:tag -b code_engine -s ibm_cos - For example, you can update an already created runtime based on the Docker image `lithopscloud/ce-matplotlib-v39:01` by: + For example, you can update an already created runtime based on the Docker image `lithopscloud/ce-matplotlib-v312:01` by: - $ lithops runtime update lithopscloud/ce-matplotlib-v39:01 -b code_engine -s ibm_cos + $ lithops runtime update lithopscloud/ce-matplotlib-v312:01 -b code_engine -s ibm_cos Alternatively, you can update all the deployed runtimes at a time by: @@ -133,9 +133,9 @@ code_enigne: $ lithops runtime delete docker_username/runtimename:tag -b code_engine -s ibm_cos - For example, you can delete runtime based on the Docker image `lithopscloud/ce-conda-v39:01` by: + For example, you can delete runtime based on the Docker image `lithopscloud/ce-conda-v312:01` by: - $ lithops runtime delete lithopscloud/ce-conda-v39:01 -b code_engine -s ibm_cos + $ lithops runtime delete lithopscloud/ce-conda-v312:01 -b code_engine -s ibm_cos You can delete all the runtimes at a time by: diff --git a/runtime/gcp_cloudrun/README.md b/runtime/gcp_cloudrun/README.md index f0a080f50..3c6b7f1af 100644 --- a/runtime/gcp_cloudrun/README.md +++ b/runtime/gcp_cloudrun/README.md @@ -55,11 +55,11 @@ gcp_cloudrun: To build your own runtime, first install the Docker CE version in your client machine. You can find the instructions [here](https://docs.docker.com/get-docker/). If you already have Docker installed omit this step. Update the Dockerfile that better fits to your requirements with your required system packages and Python modules. - If you need another Python version, for example Python 3.8, you must change the initial line of the Dockefile. + If you need another Python version, for example Python 3.12, you must change the initial line of the Dockefile. For example, we will add `PyTorch` to our Lithops runtime. The Dockerfile would look like this: ```dockerfile - FROM python:3.8-slim-buster + FROM python:3.12-slim-bookworm RUN apt-get update && apt-get install -y \ zip \ @@ -119,14 +119,14 @@ gcp_cloudrun: fexec = lithops.FunctionExecutor(runtime='pytorchruntime') fexec.call_async(my_function, 'hello') - print(fexec.get_result()) # Prints + print(fexec.get_result()) # Prints ``` ## Runtime Management 1. **Update an existing runtime** - If you are a developer, and modified the PyWeen source code, you need to deploy the changes before executing Lithops. + If you are a developer, and modified the Lithops source code, you need to deploy the changes before executing Lithops. You can update default runtime by: diff --git a/runtime/gcp_functions/README.md b/runtime/gcp_functions/README.md index 7cea3bbe4..e80016859 100644 --- a/runtime/gcp_functions/README.md +++ b/runtime/gcp_functions/README.md @@ -2,7 +2,7 @@ Google Cloud Functions operate within a runtime environment distinct from other serverless platforms like Google Cloud Run, as they do not rely on containers from the user prespective. Consequently, specifying a container image as the function's runtime isn't feasible. However, you can enhance the default package set by providing a custom `requirements.txt` file, allowing for the inclusion of additional Python modules automatically installable via `pip`. -Google Cloud Functions supports Python >= 3.7. You can find the list of pre-installed modules [here](https://cloud.google.com/functions/docs/writing/specifying-dependencies-python#pre-installed_packages). In addition, the Lithops default runtimes are built with the packages included in this [requirements.txt](requirements.txt): +Currently, Google Cloud Functions supports Python >= 3.7. You can find the list of pre-installed modules [here](https://cloud.google.com/functions/docs/writing/specifying-dependencies-python#pre-installed_packages). In addition, the Lithops default runtimes are built with the packages included in this [requirements.txt](requirements.txt) file: The default runtime is created automatically the first time you execute a function. Lithops automatically detects the Python version of your environment and deploys the default runtime based on it. In this sense, to run a function with the default runtime you don't need to specify anything in the code, since everything is managed internally by Lithops: @@ -32,10 +32,10 @@ If you require additional Python modules not included in the default runtime, yo For instance, if you wish to integrate the `matplotlib` module into your runtime, which isn't part of the default setup, you need to append it to the existing [requirements.txt](requirements.txt) file. Note that this `requirements.txt` contains the mandatory pakcges required by lithops, so you don't have to remove any of them from the list, but just add your packages at the end. -After updating the file accordingly, you can proceed to build the custom runtime by specifying the modified `requirements.txt` file along with a chosen runtime name +After updating the file accordingly, you can proceed to build the custom runtime by specifying the modified `requirements.txt` file along with a chosen runtime name: ``` -$ lithops runtime build -f requirements.txt my_matplotlib_runtime -b gcp_functions +$ lithops runtime build -b gcp_functions -f requirements.txt my_matplotlib_runtime ``` This command will add an extra runtime called `my_matplotlib_runtime` to the available Google Cloud Function runtimes. @@ -52,7 +52,7 @@ def test(): lith = lithops.FunctionExecutor(runtime='my_matplotlib_runtime') lith.call_async(test, data=()) res = lith.get_result() -print(res) # Prints +print(res) # Prints ``` -If we are running Lithops, for example, with Python 3.8, `my_matplotlib_runtime` will be a Python 3.8 runtime with the extra modules specified installed. +If we are running Lithops, for example, with Python 3.12, `my_matplotlib_runtime` will be a Python 3.12 runtime with the extra modules specified installed. diff --git a/runtime/knative/README.md b/runtime/knative/README.md index 3056456ec..8ca56ef12 100644 --- a/runtime/knative/README.md +++ b/runtime/knative/README.md @@ -57,27 +57,27 @@ knative: $ docker login Update the Dockerfile that better fits to your requirements with your required system packages and Python modules. - If you need another Python version, for example Python 3.8, you must change the initial line of the Dockefile: + If you need another Python version, for example Python 3.12, you must change the initial line of the Dockefile: $ lithops runtime build -b knative docker.io/username/runtimename:tag Note that Docker hub image names look like *"docker_username/runtimename:tag"* and must be all lower case, and must not include '.' or '_', for example: - $ lithops runtime build -b knative docker.io/username/lithops-kn-custom-v38:01 + $ lithops runtime build -b knative docker.io/username/lithops-kn-custom-v312:01 By default the Dockerfile should be located in the same folder from where you execute the **lithops runtime** command. If your Dockerfile is located in another folder, or the Dockerfile has another name, you can specify its location with the **-f** parameter, for example: - $ lithops runtime build -b knative -f knative/Dockerfile.conda docker.io/username/lithops-kn-custom-v38:01 + $ lithops runtime build -b knative -f knative/Dockerfile.conda docker.io/username/lithops-kn-custom-v312:01 Once you have built your runtime with all of your necessary packages, you can already use it with Lithops. To do so, you have to specify the full docker image name in the configuration or when you create the **FunctionExecutor** instance, or directly in the config file, for example: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='docker.io/username/lithops-kn-custom-v38:01') + fexec = lithops.FunctionExecutor(runtime='docker.io/username/lithops-kn-custom-v312:01') ``` - *NOTE: In this previous example shows how to build a Docker image based on Python 3.8, this means that now you also need Python 3.8 in the client machine.* + *NOTE: In this previous example shows how to build a Docker image based on Python 3.12, this means that now you also need Python 3.12 in the client machine.* 2. **Use an already built runtime from a public repository** @@ -86,29 +86,29 @@ knative: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='docker.io/username/lithops-kn-conda-v38:01') + fexec = lithops.FunctionExecutor(runtime='docker.io/username/lithops-kn-conda-v312:01') ``` Alternatively, you can create a Lithops runtime based on already built Docker image by executing the following command, which will deploy all the necessary information to use the runtime with your Lithops. $ lithops runtime deploy -b knative docker.io/username/runtimename:tag - For example, you can use an already built runtime based on Python 3.8 and with the *matplotlib* and *nltk* libraries by running: + For example, you can use an already built runtime based on Python 3.12 and with the *matplotlib* and *nltk* libraries by running: - $ lithops runtime deploy -b knative docker.io/username/lithops-kn-matplotlib-v38:01 + $ lithops runtime deploy -b knative docker.io/username/lithops-kn-matplotlib-v312:01 Once finished, you can use the runtime in your Lithops code: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='docker.io/username/lithops-kn-matplotlib:v38:01') + fexec = lithops.FunctionExecutor(runtime='docker.io/username/lithops-kn-matplotlib:v312:01') ``` ## Runtime Management 1. **Update an existing runtime** - If you are a developer, and modified the PyWeen source code, you need to deploy the changes before executing Lithops. + If you are a developer, and modified the Lithops source code, you need to deploy the changes before executing Lithops. You can update default runtime by: @@ -118,9 +118,9 @@ knative: $ lithops runtime update docker.io/username/runtimename:tag -b knative - For example, you can update an already created runtime based on the Docker image `jsampe/lithops-kn-matplotlib-v38:01` by: + For example, you can update an already created runtime based on the Docker image `jsampe/lithops-kn-matplotlib-v312:01` by: - $ lithops runtime update docker.io/username/lithops-kn-matplotlib-v38:01 -b knative + $ lithops runtime update docker.io/username/lithops-kn-matplotlib-v312:01 -b knative Alternatively, you can update all the deployed runtimes at a time by: @@ -138,9 +138,9 @@ knative: $ lithops runtime delete docker.io/username/runtimename:tag -b knative - For example, you can delete runtime based on the Docker image `docker.io/username/lithops-kn-conda-v36:01` by: + For example, you can delete runtime based on the Docker image `docker.io/username/lithops-kn-conda-v312:01` by: - $ lithops runtime delete docker.io/username/lithops-kn-conda-v36:01 -b knative + $ lithops runtime delete docker.io/username/lithops-kn-conda-v312:01 -b knative You can delete all the runtimes at a time by: diff --git a/runtime/kubernetes/README.md b/runtime/kubernetes/README.md index 341092818..58cad0e4a 100644 --- a/runtime/kubernetes/README.md +++ b/runtime/kubernetes/README.md @@ -33,27 +33,27 @@ k8s: $ docker login Update the Dockerfile that better fits to your requirements with your required system packages and Python modules. - If you need another Python version, for example Python 3.9, you must change the initial line of the Dockefile: + If you need another Python version, for example Python 3.12, you must change the initial line of the Dockefile: $ lithops runtime build -b k8s docker_username/runtimename:tag Note that Docker hub image names look like *"docker_username/runtimename:tag"* and must be all lower case, for example: - $ lithops runtime build -b k8s myaccount/lithops-k8s-custom-v39:01 + $ lithops runtime build -b k8s myaccount/lithops-k8s-custom-v312:01 By default the Dockerfile should be located in the same folder from where you execute the **lithops runtime** command. If your Dockerfile is located in another folder, or the Dockerfile has another name, you can specify its location with the **-f** parameter, for example: - $ lithops runtime build -b k8s -f kubernetes/Dockerfile.conda myaccount/lithops-k8s-custom-v39:01 + $ lithops runtime build -b k8s -f kubernetes/Dockerfile.conda myaccount/lithops-k8s-custom-v312:01 Once you have built your runtime with all of your necessary packages, you can already use it with Lithops. To do so, you have to specify the full docker image name in the configuration or when you create the **FunctionExecutor** instance, or directly in the config file, for example: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='myaccount/lithops-k8s-custom-v39:01') + fexec = lithops.FunctionExecutor(runtime='myaccount/lithops-k8s-custom-v312:01') ``` - *NOTE: In this previous example shows how to build a Docker image based on Python 3.9, this means that now you also need Python 3.9 in the client machine.* + *NOTE: In this previous example shows how to build a Docker image based on Python 3.12, this means that now you also need Python 3.12 in the client machine.* 2. **Use an already built runtime from a public repository** @@ -62,7 +62,7 @@ k8s: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='docker.io/lithopscloud/lithops-k8s-conda-v39:01') + fexec = lithops.FunctionExecutor(runtime='docker.io/lithopscloud/lithops-k8s-conda-v312:01') ``` Alternatively, you can create a Lithops runtime based on already built Docker image by executing the following command, which will deploy all the necessary information to use the runtime with your Lithops. @@ -71,15 +71,15 @@ k8s: $ lithops runtime deploy -b k8s docker_username/runtimename:tag ``` - For example, you can use an already buit runtime based on Python 3.9 and with the *matplotlib* and *nltk* libraries by running: + For example, you can use an already buit runtime based on Python 3.12 and with the *matplotlib* and *nltk* libraries by running: ``` - $ lithops runtime deploy -b k8s docker.io/lithopscloud/lithops-k8s-matplotlib-v39:01 + $ lithops runtime deploy -b k8s docker.io/lithopscloud/lithops-k8s-matplotlib-v312:01 ``` ```python import lithops - fexec = lithops.FunctionExecutor(runtime='docker.io/lithopscloud/lithops-k8s-matplotlib:v39:01') + fexec = lithops.FunctionExecutor(runtime='docker.io/lithopscloud/lithops-k8s-matplotlib:v312:01') ``` 3. **Clean everything** diff --git a/runtime/openwhisk/README.md b/runtime/openwhisk/README.md index cbf519e87..0432f4056 100644 --- a/runtime/openwhisk/README.md +++ b/runtime/openwhisk/README.md @@ -46,28 +46,28 @@ fexec = lithops.FunctionExecutor(runtime_memory=512) $ docker login Navigate to [openwhisk/](openwhisk/) and update the Dockerfile that better fits to your requirements with your required system packages and Python modules. - If you need another Python version, for example Python 3.11, you must use the [Dockerfile.python38](openwhisk/Dockerfile.python38) that - points to a source image based on Python 3.11. Finally run the build script: + If you need another Python version, for example Python v3.12, you must use this [Dockerfile](openwhisk/Dockerfile) that + points to a source image based on Python 3.12. Finally run the build script: $ lithops runtime build -b openwhisk docker_username/runtimename:tag Note that Docker hub image names look like *"docker_username/runtimename:tag"* and must be all lower case, for example: - $ lithops runtime build -b openwhisk jsampe/lithops-custom-runtime-3.11:0.1 + $ lithops runtime build -b openwhisk jsampe/lithops-custom-runtime-v312:0.1 By default the Dockerfile should be located in the same folder from where you execute the **lithops runtime** command. If your Dockerfile is located in another folder, or the Dockerfile has another name, you can specify its location with the **-f** parameter, for example: - $ lithops runtime build -b openwhisk -f openwhisk/Dockerfile.conda username/lithops-conda-runtime-3.11:0.1 + $ lithops runtime build -b openwhisk -f openwhisk/Dockerfile.conda username/lithops-conda-runtime-v312:0.1 Once you have built your runtime with all of your necessary packages, you can already use it with Lithops. To do so, you have to specify the full docker image name in the configuration or when you create the **ibm_cf_executor** instance, for example: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='jsampe/lithops-custom-runtime-3.11:0.1') + fexec = lithops.FunctionExecutor(runtime='jsampe/lithops-custom-runtime-v312:0.1') ``` - *NOTE: In this previous example we built a Docker image based on Python 3.11, this means that now we also need Python 3.11 in the client machine.* + *NOTE: In this previous example we built a Docker image based on Python v312, this means that now we also need Python v312 in the client machine.* 2. **Use an already built runtime from a public repository** @@ -76,22 +76,22 @@ fexec = lithops.FunctionExecutor(runtime_memory=512) ```python import lithops - fexec = lithops.FunctionExecutor(runtime='jsampe/lithops-conda-3.11:0.1') + fexec = lithops.FunctionExecutor(runtime='jsampe/lithops-conda-v312:0.1') ``` Alternatively, you can create a Lithops runtime based on already built Docker image by executing the following command, which will deploy all the necessary information to use the runtime with your Lithops. $ lithops runtime deploy docker_username/runtimename:tag -b openwhisk -s minio - For example, you can use an already built runtime based on Python 3.11 and with the *matplotlib* and *nltk* libraries by running: + For example, you can use an already built runtime based on Python v312 and with the *matplotlib* and *nltk* libraries by running: - $ lithops runtime deploy jsampe/lithops-matplotlib-3.11:0.1 -b openwhisk -s minio + $ lithops runtime deploy jsampe/lithops-matplotlib-v312:0.1 -b openwhisk -s minio Once finished, you can use the runtime in your Lithops code: ```python import lithops - fexec = lithops.FunctionExecutor(runtime='jsampe/lithops-matplotlib:3.11:0.1') + fexec = lithops.FunctionExecutor(runtime='jsampe/lithops-matplotlib:v312:0.1') ``` ## Runtime Management @@ -108,9 +108,9 @@ fexec = lithops.FunctionExecutor(runtime_memory=512) $ lithops runtime update docker_username/runtimename:tag -b openwhisk -s minio - For example, you can update an already created runtime based on the Docker image `jsampe/lithops-conda-3.11:0.1` by: + For example, you can update an already created runtime based on the Docker image `jsampe/lithops-conda-v312:0.1` by: - $ lithops runtime update jsampe/lithops-conda-3.11:0.1 -b openwhisk -s minio + $ lithops runtime update jsampe/lithops-conda-v312:0.1 -b openwhisk -s minio Alternatively, you can update all the deployed runtimes at a time by: @@ -128,9 +128,9 @@ fexec = lithops.FunctionExecutor(runtime_memory=512) $ lithops runtime delete docker_username/runtimename:tag -b openwhisk -s minio - For example, you can delete runtime based on the Docker image `jsampe/lithops-conda-3.11:0.1` by: + For example, you can delete runtime based on the Docker image `jsampe/lithops-conda-v312:0.1` by: - $ lithops runtime delete jsampe/lithops-conda-3.11:0.1 -b openwhisk -s minio + $ lithops runtime delete jsampe/lithops-conda-v312:0.1 -b openwhisk -s minio You can delete all the runtimes at a time by: