better detect dropdown menu

[LawyZheng]

🚀	This description was created by Ellipsis for commit f5e7658

Summary:

Improved dropdown menu detection and interaction in Skyvern by adding relevance checks, enhancing error handling, and updating JavaScript utilities for better element identification.

Key points:

• Added relevant field in JSON response in skyvern/forge/prompts/skyvern/custom-select.j2.
• Removed element_trees parameter from handle_input_text_action and handle_select_option_action in skyvern/webeye/actions/handler.py.
• Enhanced error handling and logging in handle_input_text_action and handle_select_option_action.
• Updated select_from_dropdown to include should_relevant parameter for relevance checking.
• Improved dropdown menu detection logic in locate_dropdown_menu by checking for ul or listbox roles.
• Modified buildElementObject in skyvern/webeye/scraper/domUtils.js to include purgeable attribute.
• Adjusted buildElementTree to support full tree building with purgeable elements.
• Updated json_to_html in skyvern/webeye/scraper/scraper.py to handle purgeable elements.
• Added get_element_visible method in skyvern/webeye/utils/page.py for visibility checks.
• Added max_steps parameter to generate_curl_params and create_task in streamlit_app/visualizer/api.py.

---

Generated with ❤️ by ellipsis.dev

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to f5e7658 in 1 minute and 7 seconds

More details

• Looked at 415 lines of code in 7 files
• Skipped 0 files when reviewing.
• Skipped posting 6 drafted comments based on config settings.

1. streamlit_app/visualizer/api.py:19

• Draft comment:
  The max_steps parameter is added to allow overriding the maximum steps in the request header. Ensure this is intentional and documented.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The PR introduces a new parameter max_steps to the generate_curl_params and create_task methods. This parameter is used to set a header x-max-steps-override if provided. This change is consistent across the methods where these functions are used.

2. skyvern/webeye/actions/handler.py:1255

• Draft comment:
  The element_trees parameter is removed as it was unused. Ensure this parameter is not needed elsewhere.
• Reason this comment was not posted:
  Confidence changes required: 20%
  The element_trees parameter is removed from the select_from_dropdown and locate_dropdown_menu functions. This parameter was not used in the function body, so its removal is appropriate.

3. skyvern/webeye/actions/handler.py:1437

• Draft comment:
  The element_trees parameter is removed as it was unused. Ensure this parameter is not needed elsewhere.
• Reason this comment was not posted:
  Confidence changes required: 20%
  The element_trees parameter is removed from the locate_dropdown_menu function. This parameter was not used in the function body, so its removal is appropriate.

4. skyvern/webeye/scraper/domUtils.js:891

• Draft comment:
  The purgeable attribute is added to indicate elements used only for tree relationships. Ensure this is documented and used correctly.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The purgeable attribute is added to the element object to indicate if an element is only used for building the tree relationship. This is a new feature that should be documented.

5. skyvern/webeye/scraper/scraper.py:112

• Draft comment:
  The purgeable attribute is checked to determine if the element should be converted to HTML. Ensure this is documented and used correctly.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The purgeable attribute is checked in the json_to_html function to determine if the element should be converted to HTML. This is a new feature that should be documented.

6. skyvern/webeye/scraper/scraper.py:341

• Draft comment:
  The get_incremental_element_tree function no longer returns element_trees. Verify this change doesn't affect other parts of the code.
• Reason this comment was not posted:
  Confidence changes required: 30%
  The get_incremental_element_tree function no longer returns element_trees. This change should be verified to ensure it doesn't affect other parts of the code.

Workflow ID: wflow_K0JCHHzKs69ve0wu

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.