Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Making Naoki's suggested changes and fixing configs #16

Merged
merged 10 commits into from
Oct 16, 2023
Merged

Making Naoki's suggested changes and fixing configs #16

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
159c5a8
Fixes for things that go wrong when running. Caption that doesn't get…
kashton-bdai Oct 16, 2023
2bd198d
Merge branch 'main' into ka/fixes
kashton-bdai Oct 16, 2023
b4cc660
Remove None check following Naoki's feedback (tested)
kashton-bdai Oct 16, 2023
44df2f0
Merge branch 'ka/fixes' of https://github.com/bdaiinstitute/vlfm into…
kashton-bdai Oct 16, 2023
10ec95f
Update grounding_dino.py
naokiyokoyama Oct 16, 2023
8b2cb20
Merge branch 'ka/fixes' of https://github.com/bdaiinstitute/vlfm into…
kashton-bdai Oct 16, 2023
5619066
Move copyright so yaml files work
kashton-bdai Oct 16, 2023
9696750
Merge branch 'main' into ka/fixes2
kashton-bdai Oct 16, 2023
31cb4a9
Fix black pre-commit issue as Naoki suggested
kashton-bdai Oct 16, 2023
920d388
merge
kashton-bdai Oct 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion config/experiments/reality.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# @package _global_

# Copyright (c) 2023 Boston Dynamics AI Institute LLC. All rights reserved.

# @package _global_
defaults:
- /policy: zsos_config_base
- _self_
Expand Down
4 changes: 2 additions & 2 deletions config/experiments/ver_pointnav.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Copyright (c) 2023 Boston Dynamics AI Institute LLC. All rights reserved.

# @package _global_

# Copyright (c) 2023 Boston Dynamics AI Institute LLC. All rights reserved.

defaults:
- /tasks: pointnav_depth_hm3d
- /habitat_baselines: habitat_baselines_rl_config_base
Expand Down
4 changes: 2 additions & 2 deletions config/experiments/vlfm_objectnav_hm3d.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Copyright (c) 2023 Boston Dynamics AI Institute LLC. All rights reserved.

# @package _global_

# Copyright (c) 2023 Boston Dynamics AI Institute LLC. All rights reserved.

defaults:
- /habitat_baselines: habitat_baselines_rl_config_base
- /benchmark/nav/objectnav: objectnav_hm3d
Expand Down
3 changes: 2 additions & 1 deletion vlfm/policy/base_objectnav_policy.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,7 +350,8 @@ def _update_object_map(

# If we are using vqa, then use the BLIP2 model to visually confirm whether
# the contours are actually correct.
if (self._use_vqa is not None) and self._use_vqa:

if self._use_vqa:
contours, _ = cv2.findContours(
object_mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE
)
Expand Down
5 changes: 3 additions & 2 deletions vlfm/vlm/grounding_dino.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ def __init__(
self.box_threshold = box_threshold
self.text_threshold = text_threshold

def predict(self, image: np.ndarray, caption: str = "") -> ObjectDetections:
def predict(self, image: np.ndarray, caption: Optional[str] = None) -> ObjectDetections:

"""
This function makes predictions on an input image tensor or numpy array using a
pretrained model.
Expand All @@ -58,7 +59,7 @@ def predict(self, image: np.ndarray, caption: str = "") -> ObjectDetections:
image_transformed = F.normalize(
image_tensor, mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]
)
if caption == "":
if caption is None:
caption_to_use = self.caption
else:
caption_to_use = caption
Expand Down