bind method of ChatVertexai is not implemented for ToolExecutor(tools)

[jorge2najo]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

class RecordDog(BaseModel):
    """Record some identifying information about a dog."""

    name: str = Field(..., description="The dog's name")
    color: str = Field(..., description="The dog's color")
    fav_food: Optional[str] = Field(None, description="The dog's favorite food")

class RecordDogTool(BaseTool):
    name = "record_dog_tool"
    description = "useful for when you need to record information about a dog"
    args_schema: Type[BaseModel] = RecordDog
    return_direct: bool = False
    
    def _run(
        self, 
        name: str = "Harry", 
        color: str = "brown", 
        fav_food: Optional[str] = "chicken"
    ) -> str:
    
        """Use the tool.""" 

        return f"Recorded {name} who is {color} and loves {fav_food}."


class RecordPerson(BaseModel):
    """Record some identifying information about a person."""

    name: str = Field(..., description="The person's name")
    age: int = Field(..., description="The person's age")
    fav_food: Optional[str] = Field(None, description="The person's favorite food")


class RecordPersonTool(BaseTool):
    name = "record_person_tool"
    description = "useful for when you need to record information about a person"
    args_schema: Type[BaseModel] = RecordPerson
    return_direct: bool = False
    
    def _run(
        self, 
        name: str = "Pedro", 
        age: int = 30, 
        fav_food: Optional[str] = "Pizza"
    ) -> str:
    
        """Use the tool.""" 

        return f"Recorded {name} who is {age} years old and loves {fav_food}."

# Creation of the list of tools
tools = [RecordDogTool(), RecordPersonTool()]
functions = [convert_to_openai_function(t) for t in tools]
tool_executor = ToolExecutor(tools)

# # For ChatOpenAI
# llm = ChatOpenAI(model="gpt-4-turbo-preview", temperature=0)
# llm_with_tools = llm.bind_functions(functions)

# # For ChatAnthropicTools
# llm = ChatAnthropicTools(model_name="claude-3-opus-20240229", temperature=0)
# llm_with_tools = llm.bind_tools(tools=tools)

# For ChatVertexAI
llm = ChatVertexAI(model_name="gemini-pro", temperature=0)
llm_with_tools = llm.bind(functions)

result = llm_with_tools.invoke("Record a dog named Fido who is black and loves bones.")
print(result)

Description

Issue Description:

I'm attempting to extract the function_name and associated arguments using a specific structure that integrates seamlessly with ChatOpenAI or ChatAnthropicTools. This process facilitates the utilization of the ToolExecutor module from langgraph.prebuilt.tool_executor to execute the extracted function and arguments.

However, when trying to apply the same methodology with ChatVertexAI, I encounter difficulties. Could you confirm if this functionality is currently supported or implemented for ChatVertexAI? If so, guidance on how to properly achieve this would be greatly appreciated.

Expected output

Output ChatOpenAI
'''
content='' additional_kwargs={'function_call': {'arguments': '{"name":"Fido","color":"black","fav_food":"bones"}', 'name': 'record_dog_tool'}} response_metadata={'finish_reason': 'function_call', 'logprobs': None}
'''

Output ChatAnthropicTools
'''
content='' additional_kwargs={'tool_calls': [{'function': {'name': 'record_dog_tool', 'arguments': '{"name": "Fido", "color": "black", "fav_food": "bones"}'}, 'type': 'function'}]}

'''

For ChatVertexAI should be something similar like this:

content= additional_kwargs={name: 'record_dog_tool', 'arguments':{'name': 'Fido', 'color': 'black', 'fav_food': 'bones', 'type': function'}}

System Info

System Information

OS: Linux
OS Version: #110~20.04.1-Ubuntu SMP Tue Feb 13 14:25:03 UTC 2024
Python Version: 3.11.8 (main, Feb 25 2024, 16:41:26) [GCC 9.4.0]

Package Information

langchain_core: 0.1.32
langchain: 0.1.12
langchain_community: 0.0.28
langsmith: 0.1.23
langchain_anthropic: 0.1.4
langchain_cli: 0.0.21
langchain_experimental: 0.0.51
langchain_google_genai: 0.0.9
langchain_google_vertexai: 0.1.1
langchain_groq: 0.0.1
langchain_mistralai: 0.0.5
langchain_openai: 0.0.8
langchain_text_splitters: 0.0.1
langchainhub: 0.1.14
langgraph: 0.0.24
langserve: 0.0.41