Skip to content

opentelemetry-instrumentation-vertexai: support StreamGenerateContent #3297

Open
@codefromthecrypt

Description

@codefromthecrypt

What problem do you want to solve?

Right now, if you run use langchain with VertexAI in a chatbot, the initialization will appear like this:

    return ChatVertexAI(
        model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
    )

When a generation occurs, if you have bootstrapped your deps, you'll see a normal platform span like this:

google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent

You won't yet see a genai span as while non-streaming has, streaming hasn't yet been implemented.
https://github.com/open-telemetry/opentelemetry-python-contrib/blob/opentelemetry-instrumentation-vertexai%3D%3D2.0b0/instrumentation-genai/opentelemetry-instrumentation-vertexai/src/opentelemetry/instrumentation/vertexai/__init__.py#L81-L82

Describe the solution you'd like

I'd like the next release of opentelemetry-instrumentation-vertexai to include google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent

Describe alternatives you've considered

Currently, we use langtrace as the data is most similar to the semantic conventions.

    from langtrace_python_sdk.instrumentation import VertexAIInstrumentation

    VertexAIInstrumentation().instrument()
    return ChatVertexAI(
        model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
    )

Additional Context

cc @aabmass and FYI this is the specific code I would like to remove https://github.com/elastic/elasticsearch-labs/blob/main/example-apps/chatbot-rag-app/api/llm_integrations.py#L23-L26

Would you like to implement a fix?

None

Metadata

Metadata

Assignees

Type

No type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions