Description
What problem do you want to solve?
Right now, if you run use langchain with VertexAI in a chatbot, the initialization will appear like this:
return ChatVertexAI(
model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
)
When a generation occurs, if you have bootstrapped your deps, you'll see a normal platform span like this:
google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent
You won't yet see a genai span as while non-streaming has, streaming hasn't yet been implemented.
https://github.com/open-telemetry/opentelemetry-python-contrib/blob/opentelemetry-instrumentation-vertexai%3D%3D2.0b0/instrumentation-genai/opentelemetry-instrumentation-vertexai/src/opentelemetry/instrumentation/vertexai/__init__.py#L81-L82
Describe the solution you'd like
I'd like the next release of opentelemetry-instrumentation-vertexai to include google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent
Describe alternatives you've considered
Currently, we use langtrace as the data is most similar to the semantic conventions.
from langtrace_python_sdk.instrumentation import VertexAIInstrumentation
VertexAIInstrumentation().instrument()
return ChatVertexAI(
model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
)
Additional Context
cc @aabmass and FYI this is the specific code I would like to remove https://github.com/elastic/elasticsearch-labs/blob/main/example-apps/chatbot-rag-app/api/llm_integrations.py#L23-L26
Would you like to implement a fix?
None
Metadata
Metadata
Assignees
Labels
Type
Projects
Status