Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(imagebind): add imagebind #21958

Merged
merged 1 commit into from
Dec 27, 2024
Merged

feat(imagebind): add imagebind #21958

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ cloud-platform/aws/aws-secrets-manager/secrets/*-credentials.json
cloud-security/hm-prowler/output
computational-fluid-dynamics/openfoam/simulations/*/0.*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh
computer-vision/hm-imagebind/.checkpoints
computer-vision/hm-imagebind/data
data-analytics/hm-geopandas/data
data-ingestion/fluent-bit/*/data
data-ingestion/vector/*/data
Expand Down
5 changes: 4 additions & 1 deletion .github/workflows/.static-type-check.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ jobs:
uv run poe static-type-check-python --package=cloud-computing.hm-ray.applications.process-flight-data
uv run poe static-type-check-python --package=cloud-platform.aws.amazon-sagemaker.pytorch-mnist
uv run poe static-type-check-python --package=cloud-platform.aws.aws-parallelcluster.pcluster
uv run poe static-type-check-python --package=computer-vision.hm-imagebind
uv run poe static-type-check-python --package=computer-vision.hm-open3d
uv run poe static-type-check-python --package=computer-vision.hm-pyvista.mount-saint-helens
uv run poe static-type-check-python --package=data-analytics.hm-cudf
Expand All @@ -140,10 +141,12 @@ jobs:
uv run poe static-type-check-python --package=data-storage.delta-lake.write-to-delta-lake
uv run poe static-type-check-python --package=data-storage.hm-duckdb.query-duckdb
uv run poe static-type-check-python --package=data-storage.hm-duckdb.query-protobuf
uv run poe static-type-check-python --package=data-storage.hm-lancedb
uv run poe static-type-check-python --package=data-storage.hm-protobuf
uv run poe static-type-check-python --package=data-visualization.grafana.hm-dashboard
uv run poe static-type-check-python --package=data-visualization.iads.iads-data-manager.iads-config-reader
uv run poe static-type-check-python --package=data-visualization.iads.iads-data-manager.iads-data-reader
uv run poe static-type-check-python --package=data-visualization.iads.iads-data-manager.iads-data-reader
uv run poe static-type-check-python --package=embedded.decode-can-data
uv run poe static-type-check-python --package=embedded.format-can-data
uv run poe static-type-check-python --package=embedded.hm-serial
Expand All @@ -154,9 +157,9 @@ jobs:
uv run poe static-type-check-python --package=hm-xxhash
uv run poe static-type-check-python --package=machine-learning.convolutional-neural-network
uv run poe static-type-check-python --package=machine-learning.dali
uv run poe static-type-check-python --package=machine-learning.hm-docling
uv run poe static-type-check-python --package=machine-learning.feature-store
uv run poe static-type-check-python --package=machine-learning.graph-neural-network
uv run poe static-type-check-python --package=machine-learning.hm-docling
uv run poe static-type-check-python --package=machine-learning.hm-gradio.applications.classify-image
uv run poe static-type-check-python --package=machine-learning.hm-kubeflow.pipelines.calculate
uv run poe static-type-check-python --package=machine-learning.hm-kubeflow.pipelines.classify-mnist
Expand Down
37 changes: 37 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ jobs:
hm-gradio-classify-image: ${{ steps.filter.outputs.hm-gradio-classify-image }}
hm-streamlit-live-line-chart: ${{ steps.filter.outputs.hm-streamlit-live-line-chart }}
hm-streamlit-map: ${{ steps.filter.outputs.hm-streamlit-map }}
hm-imagebind: ${{ steps.filter.outputs.hm-imagebind }}
hm-jax: ${{ steps.filter.outputs.hm-jax }}
hm-kubeflow-calculate: ${{ steps.filter.outputs.hm-kubeflow-calculate }}
hm-kubeflow-classify-mnist: ${{ steps.filter.outputs.hm-kubeflow-classify-mnist }}
Expand Down Expand Up @@ -259,6 +260,9 @@ jobs:
hm-gradio-classify-image:
- '.github/workflows/test.yml'
- 'machine-learning/hm-gradio/applications/classify-image/**'
hm-imagebind:
- '.github/workflows/test.yml'
- 'computer-vision/hm-imagebind/**'
hm-jax:
- '.github/workflows/test.yml'
- 'high-performance-computing/hm-jax/**'
Expand Down Expand Up @@ -2333,6 +2337,39 @@ jobs:
with:
directory: hardware-in-the-loop/national-instruments/veristand/hm-veristand

imagebind-test:
name: ImageBind | Test
needs: detect-changes
if: ${{ needs.detect-changes.outputs.hm-imagebind == 'true' }}
runs-on: ubuntu-24.04
environment: test
timeout-minutes: 10
steps:
- name: Checkout
uses: actions/checkout@v4.2.2
- name: Install uv
uses: astral-sh/setup-uv@v5.1.0
with:
version: 0.5.11
enable-cache: true
cache-dependency-glob: computer-vision/hm-imagebind/uv.lock
- name: Set up Python
uses: actions/setup-python@v5.3.0
with:
python-version-file: computer-vision/hm-imagebind/pyproject.toml
- name: Install dependencies
working-directory: computer-vision/hm-imagebind
run: |
uv sync --dev
- name: Test
working-directory: computer-vision/hm-imagebind
run: |
uv run poe test-coverage
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5.1.2
with:
directory: computer-vision/hm-imagebind

open3d-test:
name: Open3D | Test
needs: detect-changes
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ cloud-platform/aws/aws-secrets-manager/secrets/*-credentials.json
cloud-security/hm-prowler/output
computational-fluid-dynamics/openfoam/simulations/*/0.*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh
computer-vision/hm-imagebind/.checkpoints
computer-vision/hm-imagebind/data
data-analytics/hm-geopandas/data
data-ingestion/fluent-bit/*/data
data-ingestion/vector/*/data
Expand Down
2 changes: 2 additions & 0 deletions .markdownlint-cli2.jsonc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@
"cloud-security/hm-prowler/output",
"computational-fluid-dynamics/openfoam/simulations/*/0.*",
"computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh",
"computer-vision/hm-imagebind/.checkpoints",
"computer-vision/hm-imagebind/data",
"cloud-security/hm-prowler/output",
"data-analytics/hm-geopandas/data",
"data-ingestion/fluent-bit/*/data",
Expand Down
3 changes: 3 additions & 0 deletions .mergify.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,9 @@ pull_request_rules:
- or:
- check-success=National Instruments (tdms) | Test
- check-skipped=National Instruments (tdms) | Test
- or:
- check-success=ImageBind | Test
- check-skipped=ImageBind | Test
- or:
- check-success=Open3D | Test
- check-skipped=Open3D | Test
Expand Down
2 changes: 2 additions & 0 deletions .prettierignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ cloud-platform/aws/aws-secrets-manager/secrets/*-credentials.json
cloud-security/hm-prowler/output
computational-fluid-dynamics/openfoam/simulations/*/0.*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh
computer-vision/hm-imagebind/.checkpoints
computer-vision/hm-imagebind/data
data-analytics/hm-geopandas/data
data-ingestion/fluent-bit/*/data
data-ingestion/vector/*/data
Expand Down
2 changes: 2 additions & 0 deletions .rubocop.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ AllCops:
- 'cloud-security/hm-prowler/output/**/*'
- 'computational-fluid-dynamics/openfoam/simulations/*/0.*/**/*'
- 'computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh/**/*'
- 'computer-vision/hm-imagebind/.checkpoints/**/*'
- 'computer-vision/hm-imagebind/data/**/*'
- 'data-analytics/hm-geopandas/data/**/*'
- 'data-ingestion/fluent-bit/*/data/**/*'
- 'data-ingestion/vector/*/data/**/*'
Expand Down
2 changes: 2 additions & 0 deletions .ruff.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,8 @@ exclude = [
"cloud-security/hm-prowler/output",
"computational-fluid-dynamics/openfoam/simulations/*/0.*",
"computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh",
"computer-vision/hm-imagebind/.checkpoints",
"computer-vision/hm-imagebind/data",
"data-analytics/hm-geopandas/data",
"data-ingestion/fluent-bit/*/data",
"data-ingestion/vector/*/data",
Expand Down
2 changes: 2 additions & 0 deletions .solhintignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ cloud-platform/aws/aws-secrets-manager/secrets/*-credentials.json
cloud-security/hm-prowler/output
computational-fluid-dynamics/openfoam/simulations/*/0.*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh
computer-vision/hm-imagebind/.checkpoints
computer-vision/hm-imagebind/data
data-analytics/hm-geopandas/data
data-ingestion/fluent-bit/*/data
data-ingestion/vector/*/data
Expand Down
2 changes: 2 additions & 0 deletions .sqlfluffignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ cloud-platform/aws/aws-secrets-manager/secrets/*-credentials.json
cloud-security/hm-prowler/output
computational-fluid-dynamics/openfoam/simulations/*/0.*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh
computer-vision/hm-imagebind/.checkpoints
computer-vision/hm-imagebind/data
data-analytics/hm-geopandas/data
data-ingestion/fluent-bit/*/data
data-ingestion/vector/*/data
Expand Down
2 changes: 2 additions & 0 deletions .stylelintignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ cloud-platform/aws/aws-secrets-manager/secrets/*-credentials.json
cloud-security/hm-prowler/output
computational-fluid-dynamics/openfoam/simulations/*/0.*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh
computer-vision/hm-imagebind/.checkpoints
computer-vision/hm-imagebind/data
data-analytics/hm-geopandas/data
data-ingestion/fluent-bit/*/data
data-ingestion/vector/*/data
Expand Down
2 changes: 2 additions & 0 deletions .textlintignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ cloud-platform/aws/aws-secrets-manager/secrets/*-credentials.json
cloud-security/hm-prowler/output/**/*
computational-fluid-dynamics/openfoam/simulations/*/0.*/**/*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh/**/*
computer-vision/hm-imagebind/.checkpoints/**/*
computer-vision/hm-imagebind/data/**/*
data-analytics/hm-geopandas/data/**/*
data-ingestion/fluent-bit/*/data/**/*
data-ingestion/vector/*/data/**/*
Expand Down
2 changes: 2 additions & 0 deletions .yamllint.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ ignore: |
cloud-security/hm-prowler/output
computational-fluid-dynamics/openfoam/simulations/*/0.*
computational-fluid-dynamics/openfoam/simulations/*/constant/polyMesh
computer-vision/hm-imagebind/.checkpoints
computer-vision/hm-imagebind/data
data-analytics/hm-geopandas/data
data-ingestion/fluent-bit/*/data
data-ingestion/vector/*/data
Expand Down
4 changes: 3 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,7 @@ static-type-check-python:
uv run poe static-type-check-python --package=cloud-computing.hm-ray.applications.process-flight-data
uv run poe static-type-check-python --package=cloud-platform.aws.amazon-sagemaker.pytorch-mnist
uv run poe static-type-check-python --package=cloud-platform.aws.aws-parallelcluster.pcluster
uv run poe static-type-check-python --package=computer-vision.hm-imagebind
uv run poe static-type-check-python --package=computer-vision.hm-open3d
uv run poe static-type-check-python --package=computer-vision.hm-pyvista.mount-saint-helens
uv run poe static-type-check-python --package=data-analytics.hm-cudf
Expand All @@ -357,6 +358,7 @@ static-type-check-python:
uv run poe static-type-check-python --package=data-storage.delta-lake.write-to-delta-lake
uv run poe static-type-check-python --package=data-storage.hm-duckdb.query-duckdb
uv run poe static-type-check-python --package=data-storage.hm-duckdb.query-protobuf
uv run poe static-type-check-python --package=data-storage.hm-lancedb
uv run poe static-type-check-python --package=data-storage.hm-protobuf
uv run poe static-type-check-python --package=data-visualization.grafana.hm-dashboard
uv run poe static-type-check-python --package=data-visualization.iads.iads-data-manager.iads-config-reader
Expand All @@ -372,9 +374,9 @@ static-type-check-python:
uv run poe static-type-check-python --package=hm-xxhash
uv run poe static-type-check-python --package=machine-learning.convolutional-neural-network
uv run poe static-type-check-python --package=machine-learning.dali
uv run poe static-type-check-python --package=machine-learning.hm-docling
uv run poe static-type-check-python --package=machine-learning.feature-store
uv run poe static-type-check-python --package=machine-learning.graph-neural-network
uv run poe static-type-check-python --package=machine-learning.hm-docling
uv run poe static-type-check-python --package=machine-learning.hm-gradio.applications.classify-image
uv run poe static-type-check-python --package=machine-learning.hm-kubeflow.pipelines.calculate
uv run poe static-type-check-python --package=machine-learning.hm-kubeflow.pipelines.classify-mnist
Expand Down
1 change: 1 addition & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,7 @@ The diagram illustrates the repository's architecture, which is considered overl

## Computer Vision

- **ImageBind** - Multimodal embedding model for images, video, audio, text, depth, thermal, and inertial measurement units (IMUs)
- **OpenCV** - Computer vision library
- **supervision** - Computer vision library
- **Ultralytics YOLOv8** - Object detection model
Expand Down
13 changes: 13 additions & 0 deletions computer-vision/hm-imagebind/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
uv-install-python::
uv python install
uv-update-lock-file:
uv lock
uv-install-dependencies:
uv sync --dev

uv-run-dev:
uv run poe dev
uv-run-test:
uv run poe test
uv-run-test-coverage:
uv run poe test-coverage
Empty file added computer-vision/hm-imagebind/data/.gitkeep
View file
Open in desktop
Empty file.
31 changes: 31 additions & 0 deletions computer-vision/hm-imagebind/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
[project]
name = "hm-imagebind"
version = "1.0.0"
requires-python = "~=3.10.0"
dependencies = [
"httpx==0.28.1",
"gradio==5.9.1",
"imagebind",
"mayavi",
"lancedb==0.17.0",
"pandas==2.2.1",
]

[dependency-groups]
dev = [
"poethepoet==0.31.1",
"pytest==8.3.4",
"pytest-cov==6.0.0",
]

[tool.uv]
package = false

[tool.uv.sources]
imagebind = { git = "https://github.com/raghavdixit99/ImageBind.git" }
mayavi = { git = "https://github.com/enthought/mayavi.git" }

[tool.poe.tasks]
dev = "python src/main.py"
test = "pytest --verbose --verbose"
test-coverage = "pytest --cov=. --cov-report=xml"
3 changes: 3 additions & 0 deletions computer-vision/hm-imagebind/src/dummy_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
class TestDummy:
def test_dummy(self):
assert 1 + 1 == 2
Loading
Loading