Merge pull request #651 from BoostryJP/dev-24.6

Release v24.6.0

.dockerignore

@@ -1,2 +1,3 @@
 .github/
 .python-version
+/node_modules/

.github/workflows/pr.yml

@@ -14,11 +14,11 @@ jobs:
     steps:
       - uses: actions/checkout@v4
       - name: run unit test using postgres
-        run: docker-compose run ibet-prime-postgres
+        run: docker compose run ibet-prime-postgres
   migration-test-postgres:
     name: 'Migration tests (PostgreSQL)'
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
       - name: run unit test using postgres
-        run: docker-compose run ibet-prime-postgres bash --login -c "cd /app/ibet-Prime && pytest -vv --test-alembic -m 'alembic'"
+        run: docker compose run ibet-prime-postgres bash --login -c "cd /app/ibet-Prime && pytest -vv --test-alembic -m 'alembic'"

.gitignore

@@ -9,6 +9,9 @@ __pycache__/
 # pyenv
 .python-version
 
+# Node
+node_modules/
+
 # Intellij
 .idea
 /*.iml

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
   - repo: https://github.com/psf/black
-    rev: 24.1.1
+    rev: 24.4.2
     hooks:
       - id: black
-        language_version: python3.11
+        language_version: python3.12

Dockerfile

@@ -52,15 +52,16 @@ RUN echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~apl/.bash_profile \
 # install python
 USER apl
 RUN . ~/.bash_profile \
- && pyenv install 3.11.2 \
- && pyenv global 3.11.2 \
+ && pyenv install 3.12.2 \
+ && pyenv global 3.12.2 \
  && pip install --upgrade pip setuptools
 
 # install poetry
 RUN . ~/.bash_profile \
- && python -m pip install poetry==1.7.1
+ && python -m pip install poetry==1.8.2
 RUN . ~/.bash_profile \
- && poetry config virtualenvs.create false
+ && poetry config virtualenvs.create false \
+ && poetry config installer.max-workers 1
 
 # install python packages
 USER root
@@ -88,10 +89,10 @@ COPY pyproject.toml /app/ibet-Prime/pyproject.toml
 COPY poetry.lock /app/ibet-Prime/poetry.lock
 RUN . ~/.bash_profile \
  && cd /app/ibet-Prime \
- && poetry install --only main --no-root -E ibet-explorer \
+ && poetry install --only main --no-root --all-extras \
  && rm -f /app/ibet-Prime/pyproject.toml \
  && rm -f /app/ibet-Prime/poetry.lock
-ENV PYTHONPATH /app/ibet-Prime
+ENV PYTHONPATH /app/ibet-Prime:/app/ibet-Prime/cmd
 
 # command deploy
 USER apl

Makefile

@@ -1,7 +1,7 @@
-.PHONY: isort black test run
+.PHONY: format isort black doc test test_migrations run
 
 install:
-	poetry install --no-root -E ibet-explorer
+	poetry install --no-root --all-extras
 	poetry run pre-commit install
 
 update:
@@ -15,6 +15,9 @@ isort:
 black:
 	black .
 
+doc:
+	poetry run python docs/generate_openapi_doc.py
+
 test:
 	pytest tests/
 

README.md

@@ -5,7 +5,7 @@
 # ibet-Prime
 
 <p>
-  <img alt="Version" src="https://img.shields.io/badge/version-24.3-blue.svg?cacheSeconds=2592000" />
+  <img alt="Version" src="https://img.shields.io/badge/version-24.6-blue.svg?cacheSeconds=2592000" />
   <img alt="License: Apache--2.0" src="https://img.shields.io/badge/License-Apache--2.0-yellow.svg" />
 </p>
 
@@ -22,11 +22,11 @@ English | [日本語](./README_JA.md)
 
 ## Dependencies
 
-- [Python3](https://www.python.org/downloads/release/python-3811/) - version 3.11
+- [Python3](https://www.python.org/downloads/release/python-3811/) - version 3.12
 - [PostgreSQL](https://www.postgresql.org/) - version 15
 - [GoQuorum](https://github.com/ConsenSys/quorum)
   - We support the official GoQuorum node of [ibet-Network](https://github.com/BoostryJP/ibet-Network).
-  - We use [ganache](https://github.com/trufflesuite/ganache) for local development and unit testing, and we use the latest version.
+  - We use [hardhat network](https://hardhat.org/hardhat-network/) for local development and unit testing, and we use the latest version.
 
 
 ## Supported ibet smart contract version
@@ -52,14 +52,19 @@ English | [日本語](./README_JA.md)
 
 Install python packages with:
 ```bash
-$ poetry install --no-root --only main -E ibet-explorer
+$ poetry install --no-root --only main --all-extras
 ```
 
 ### Install pre-commit hook
 ```bash
 $ poetry run pre-commit install
 ```
 
+### Install hardhat
+```bash
+$ npm install
+```
+
 ### Setting environment variables
 
 The main environment variables are as follows. 

README_JA.md

@@ -5,7 +5,7 @@
 # ibet-Prime
 
 <p>
-  <img alt="Version" src="https://img.shields.io/badge/version-24.3-blue.svg?cacheSeconds=2592000" />
+  <img alt="Version" src="https://img.shields.io/badge/version-24.6-blue.svg?cacheSeconds=2592000" />
   <img alt="License: Apache--2.0" src="https://img.shields.io/badge/License-Apache--2.0-yellow.svg" />
 </p>
 
@@ -23,11 +23,11 @@
 
 ## 依存
 
-- [Python3](https://www.python.org/downloads/release/python-3811/) - バージョン 3.11
+- [Python3](https://www.python.org/downloads/release/python-3811/) - バージョン 3.12
 - [PostgreSQL](https://www.postgresql.org/) - バージョン 15
 - [GoQuorum](https://github.com/ConsenSys/quorum)
   - [ibet-Network](https://github.com/BoostryJP/ibet-Network) の公式の GoQuorum をサポートしています。
-  - 最新の [ganache](https://github.com/trufflesuite/ganache) (ganache-cli) をローカル開発およびユニットテストで利用しています。
+  - 最新の [hardhat network](https://hardhat.org/hardhat-network/) をローカル開発およびユニットテストで利用しています。
 
 ## コントラクトのバージョン
 
@@ -53,14 +53,19 @@
 
 以下のコマンドで Python パッケージをインストールします。
 ```bash
-$ poetry install --no-root --only main -E ibet-explorer
+$ poetry install --no-root --only main --all-extras
 ```
 
 ### pre-commit hookのインストール
 ```bash
 $ poetry run pre-commit install
 ```
 
+### hardhatのインストール
+```bash
+$ npm install
+```
+
 ### 環境変数の設定
 
 主要な環境変数は以下の通りです。

app/exceptions.py

@@ -56,6 +56,15 @@ class Integer64bitLimitExceededError(AppError):
     code = 5
 
 
+class OperationNotSupportedVersionError(AppError):
+    """
+    The token version for which the operation is not supported
+    """
+
+    status_code = status.HTTP_400_BAD_REQUEST
+    code = 6
+
+
 class OperationNotAllowedStateError(AppError):
     """
     Error returned when server-side data is not ready to process the request

app/log.py

@@ -20,7 +20,7 @@
 import logging
 import sys
 import urllib
-from datetime import datetime
+from datetime import UTC, datetime
 
 from fastapi import Request, Response
 
@@ -34,6 +34,9 @@
 ACCESS_LOG = logging.getLogger("issuer_api_access")
 ACCESS_LOG.propagate = False
 
+logging.getLogger("web3.manager.RequestManager").propagate = False
+logging.getLogger("web3.manager.RequestManager").addHandler(logging.NullHandler())
+
 INFO_FORMAT = "[%(asctime)s] {}[%(process)d] [%(levelname)s] %(message)s"
 DEBUG_FORMAT = "[%(asctime)s] {}[%(process)d] [%(levelname)s] %(message)s [in %(pathname)s:%(lineno)d]"
 TIMESTAMP_FORMAT = "%Y-%m-%d %H:%M:%S %z"
@@ -108,7 +111,9 @@ def output_access_log(req: Request, res: Response, request_start_time: datetime)
         method = req.scope.get("method", "")
         http_version = req.scope.get("http_version", "")
         status_code = res.status_code
-        response_time = (datetime.utcnow() - request_start_time).total_seconds()
+        response_time = (
+            datetime.now(UTC).replace(tzinfo=None) - request_start_time
+        ).total_seconds()
         access_msg = ACCESS_FORMAT % (
             method,
             url,

app/main.py

@@ -17,7 +17,7 @@
 SPDX-License-Identifier: Apache-2.0
 """
 
-from datetime import datetime
+from datetime import UTC, datetime
 
 from fastapi import FastAPI, Request
 from fastapi.encoders import jsonable_encoder
@@ -40,6 +40,7 @@
     ledger,
     notification,
     position,
+    settlement,
     share,
     token_holders,
 )
@@ -62,7 +63,7 @@
 app = FastAPI(
     title="ibet Prime",
     description="Security token management system for ibet network",
-    version="24.3.0",
+    version="24.6",
     contact={"email": "dev@boostry.co.jp"},
     license_info={
         "name": "Apache 2.0",
@@ -74,7 +75,7 @@
 
 @app.middleware("http")
 async def api_call_handler(request: Request, call_next):
-    request_start_time = datetime.utcnow()
+    request_start_time = datetime.now(UTC).replace(tzinfo=None)
     response = await call_next(request)
     output_access_log(request, response, request_start_time)
     return response
@@ -105,6 +106,7 @@ async def root():
 app.include_router(token_holders.router)
 app.include_router(bc_explorer.router)
 app.include_router(freeze_log.router)
+app.include_router(settlement.router)
 
 
 ###############################################################
@@ -206,7 +208,7 @@ async def response_limit_exceeded_error_handler(
     )
 
 
-# 400:ResponseLimitExceededError
+# 400:Integer64bitLimitExceededError
 @app.exception_handler(Integer64bitLimitExceededError)
 async def response_limit_exceeded_error_handler(
     request: Request, exc: Integer64bitLimitExceededError
@@ -218,6 +220,18 @@ async def response_limit_exceeded_error_handler(
     )
 
 
+# 400:OperationNotSupportedVersionError
+@app.exception_handler(OperationNotSupportedVersionError)
+async def operation_not_supported_version_error_handler(
+    request: Request, exc: OperationNotSupportedVersionError
+):
+    meta = {"code": 6, "title": "OperationNotSupportedVersionError"}
+    return JSONResponse(
+        status_code=exc.status_code,
+        content=jsonable_encoder({"meta": meta, "detail": exc.args[0]}),
+    )
+
+
 # 400:OperationNotAllowedStateError
 @app.exception_handler(OperationNotAllowedStateError)
 async def operation_not_permitted_error_handler(

app/model/__init__.py

@@ -17,6 +17,7 @@
 SPDX-License-Identifier: Apache-2.0
 """
 
+from datetime import datetime
 from typing import Annotated, Any
 
 from pydantic import WrapValidator
@@ -37,3 +38,28 @@ def ethereum_address_validator(
 
 
 EthereumAddress = Annotated[str, WrapValidator(ethereum_address_validator)]
+
+
+def datetime_string_validator(
+    value: Any, handler: ValidatorFunctionWrapHandler, *args, **kwargs
+):
+    """Validate string datetime format
+
+    - %Y/%m/%d %H:%M:%S
+    """
+    if value is not None:
+        datetime_format = "%Y-%m-%d %H:%M:%S"
+
+        if not isinstance(value, str):
+            raise ValueError(f"value must be of string datetime format")
+
+        try:
+            converted = datetime.strptime(value, datetime_format)
+            if value != converted.strftime(datetime_format):
+                raise ValueError(f"value must be string datetime format")
+        except ValueError:
+            raise ValueError(f"value must be of string datetime format")
+    return value
+
+
+ValidatedDatetimeStr = Annotated[str, WrapValidator(datetime_string_validator)]

app/model/blockchain/__init__.py

@@ -20,7 +20,7 @@
 from .e2e_messaging import E2EMessaging
 from .exchange import IbetExchangeInterface, IbetSecurityTokenEscrow
 from .freeze_log import FreezeLogContract
-from .personal_info import PersonalInfoContract
+from .personal_info import ContractPersonalInfoType, PersonalInfoContract
 from .token import (
     IbetSecurityTokenInterface,
     IbetShareContract,