Support for Gemini tool calling (plus general fixes and updates)

[marklysze]

Why are these changes needed?

The Gemini class doesn't currently support tool calling.

This PR adds tool calling.

Additional updates:

• Removal of role alternation, this no longer appears necessary
• Image input corrected for non-Vertex AI
• Costs updated

NOTE: This hasn't been tested with Vertex AI authentication, if someone can test it that would be good. Please try the client test files in my repo.

No documentation added, yet. TODO

Related issue number

Noted in Discord issue

Checks

• I've included any doc changes needed for https://autogen-ai.github.io/autogen/. See https://autogen-ai.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 0% with 206 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
autogen/oai/gemini.py	0.00%	206 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Flag	Coverage Δ
unittests	31.22% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
setup.py	0.00% <ø> (ø)
autogen/oai/gemini.py	0.00% <0.00%> (ø)

[marklysze]

Hey @yiranwu0, would you be able to assist in identifying why the ContribGraphRagTests test is failing?

[yiranwu0]

I am not familiar with this PR, mayber @randombet can help?

[marklysze]

I am not familiar with this PR, mayber @randombet can help?

I've found out that for graphrag_sdk they've changed some terminology since moving to V2, Schema is now called Ontology. I've updated the test.

[sonichi]

There's one small issue in the workflow. Otherwise lgtm.