feat(models): Add OCI Generative AI provider for Google Gemini on OCI#5285
Open
fede-kamel wants to merge 2 commits intogoogle:mainfrom
Open
feat(models): Add OCI Generative AI provider for Google Gemini on OCI#5285fede-kamel wants to merge 2 commits intogoogle:mainfrom
fede-kamel wants to merge 2 commits intogoogle:mainfrom
Conversation
de72195 to
3d70b80
Compare
Adds first-class support for Google Gemini models hosted on Oracle Cloud Infrastructure (OCI) Generative AI service — a native Google × OCI model partnership that makes Gemini available directly through OCI's inference endpoints. Key design points: - Subclasses BaseLlm following the anthropic_llm.py pattern - Uses the OCI Python SDK directly (no LangChain dependency) - Optional dependency: pip install google-adk[oci] - Supports API_KEY, INSTANCE_PRINCIPAL, and RESOURCE_PRINCIPAL auth - Both non-streaming (_call_oci) and streaming (_call_oci_stream) paths share setup code via _build_chat_details(); streaming collects OCI's OpenAI-compatible SSE events in a thread pool (asyncio.to_thread) and yields partial then final LlmResponse - Registers google.gemini-* (and other OCI-hosted) model patterns in LLMRegistry via optional try/except in models/__init__.py - 37 unit tests (fully mocked, no OCI account needed) - 10 integration tests (skipped when OCI_COMPARTMENT_ID is unset) Supported models: google.gemini-*, google.gemma-*, meta.llama-*, mistralai.*, xai.grok-*, nvidia.*
3d70b80 to
8f40e5c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5069
What this PR does
Adds
OCIGenAILlm— a first-class ADK model provider for Google Gemini models hosted on Oracle Cloud Infrastructure (OCI) Generative AI. Gemini 2.5 Flash, Gemini 2.5 Pro, and Gemini 2.5 Flash Lite are available as first-party models through OCI's inference endpoints — this is a native Google × OCI model partnership, not a third-party wrapper.Why this belongs in
adk-python, not a community package1. This is a model provider primitive, not a tool or connector
The community repo and integrations catalog host tools and connectors — BigQuery, Chroma, MongoDB, ElevenLabs, Slack, etc. All 60+ entries in that catalog are tool integrations. There is zero precedent for a model provider (
BaseLlmsubclass) living in the community repo — the community repo itself only containsmemory/andsessions/modules, no model providers.Every existing model provider in ADK —
Gemini,Gemma,Claude,LiteLlm,ApigeeLlm— lives insrc/google/adk/models/in this repo. This PR follows that exact pattern.2. Model providers cannot be external without degrading the SDK
LLMRegistry,BaseLlm, and the model dispatch layer are core SDK primitives. A model provider sitting in a standalone package creates:BaseLlm,LlmRequest,LlmResponse, orLLMRegistrycan silently break an external provider with no CI coveragepip install google-adk[oci](an optional extra, like[extensions]for Claude) is the established pattern; requiring a separate package for one cloud provider while others are built-in is an inconsistency that confuses users3. OCI is an official Google model distribution channel
OCI Generative AI hosts Google's own Gemini models through a direct partnership. This is the same class of integration as Vertex AI or Apigee AI Gateway — a Google-authorized inference endpoint, not a community hobby project. AWS Bedrock and Azure are absent from the SDK because no one has contributed them yet, not because multi-cloud model providers belong in a tools catalog.
4. The implementation follows established precedent exactly
AnthropicLlm(Claude)OCIGenAILlm(this PR)BaseLlmLLMRegistrytry/exceptin__init__.pypip install google-adk[extensions]pip install google-adk[oci]anthropicpackageocipackageDesign
BaseLlm, registered inLLMRegistryforgoogle.gemini-*and other OCI model patternspip install google-adk[oci](safe to import withoutociinstalled)_call_oci()runs the synchronous OCI SDK call inasyncio.to_thread_call_oci_stream()collects OCI's OpenAI-compatible SSE events in a thread, then yieldspartial=Truechunks followed by apartial=Falsefinal response with aggregated content and usage metadataAPI_KEY(default),INSTANCE_PRINCIPAL,RESOURCE_PRINCIPALFiles changed
src/google/adk/models/oci_genai_llm.pyOCIGenAILlmimplementationsrc/google/adk/models/__init__.pyLLMRegistrypyproject.toml[oci]optional dependency extratests/unittests/models/test_oci_genai_llm.pytests/integration/models/test_oci_genai_llm.pyOCI_COMPARTMENT_ID)Test results (rebased on current
main)tests/unittests/models/suitegoogle.gemini-2.5-flash, us-chicago-1): text generation, streaming, tool calls, system instructions, multi-turn, concurrent callsimport google.adksucceeds withoutociinstalled (optional dependency guard)Request to maintainers
Issue #5069 is open, labeled
models+needs review, and a maintainer (@sanketpatil06) confirmed it is "under review by our team." I am asking that this PR be reviewed on its technical merits and that the routing question (core vs. community) be evaluated against the evidence above — specifically the zero precedent for external model providers, the SDK degradation risks, and the partnership-level nature of OCI Gemini hosting.Happy to address any code review feedback directly.