Requirement ideas towards a major refactoring of Grasplan

[oscar-lima]

Description

Proposed changes and improvements for the next iteration of the Grasplan framework:

MTC

• Use MTC (MoveIt Task Constructor) as manipulation backend vs old deprecated pick n place MoveIt capability
• Use code inspiration from roscat moveit_task_constructor_demo panda_config.yaml

SOTA

• Incorporate ideas from the GraspIt! simulator:
  • https://github.com/graspit-simulator
  • e.g., automated grasp generation
• Incorporate ideas from MoveIt Grasps:
  • e.g., add grasp min/max depth (see image on moveit grasps website)
  • e.g., filtering grasps by color:
    RED - grasp filtered by ik
    PINK - grasp filtered by collision
    MAGENTA - grasp filtered by cutting plane
    YELLOW - grasp filtered by orientation
    BLUE - pregrasp filtered by ik
    CYAN - pregrasp filtered by collision
    GREEN - valid
  • https://moveit.picknik.ai/main/doc/examples/moveit_grasps/moveit_grasps_tutorial.html
  • grasp waypoint naming: retreat grasp, lifted grasp, Object Pose, EE Pose
• Add previously unseen object grasping

Software development

• Keep the usage of precommit
• Decouple code from ROS (ROS-agnostic core) + make ROS1 and ROS2 wrappers
• Single core file managing common functionalities (pose selection, planning scene, etc.) instead of multiple wrappers
• Provide services for adding objects to planning scene
• Allow for libraries to be imported outside of Grasplan
• Have a central node, e.g., like move_group in MoveIt, perhaps called "grasp_group"
  • and give, e.g.: pose selector access, planning scene access
• Support better finger trajectory control at the object
• Consider creating a dedicated Grasplan GitHub organization
• automatic elimination of unfeasible grasps (e.g. those under the table)
  • MoveIt might already do this...
• Explore LLM integration in a separate package
• Add more test cases
• Develop a GUI setup assistant

Visualization

• Implement visual grasp filtering based on color codes from MoveIt Grasps

Test with multiple robots

• Integrate at least 2 robots to show that the Software is flexible and robot agnostic

Desirable major feature upgrades

• Support grasp priority metrics (stability, reachability)
  • e.g. if obj is requested to be placed very far away it should not even try
  • stability: rank higher grasps that are more stable
  • Integrate reachability checks for requested target poses
  • score higher grasps which have higher probability of success based on reachability analysis
• Include MoveIt constraints, approach directions, and grasp semantics (top, side)
  • e.g., make eef point always up
• Improve insert pipeline using force feedback (if available)
• Add support for suction grippers
• Include semantic reasoning to reduce manual collision ignore parameters
  • reduce parameterization to ignore collisions with certain objects without explicit mention
    (e.g., picture a multimeter inside a box, now you want to pick the box, multimeter needs to be ignored from ps)
  • reduce parameterization on having to specify a surface where objects are lying on

Ambitious features that would probably not implement

• Grasp dynamics