Basic Setup within Datrabricks using azure-openai

[GrahamJWills]

I am trying to run LangTest within Databricks. I have an Azure OpenAI deployment I've used for a number of other purposes so I'm confident it's set up correctly. But when I run a simple Harness, it fails to invoke the langchain at the rpediction step with this error:

  ERROR:langtest:[E089] An error occurred during prediction: Completions.create() got an unexpected keyword argument 'deployment_name'

Here is my setup within my Databricks notebook:

  API_KEY = [REDACTED]
  VERSION =  "2024-05-13"
  ENDPOINT = "https://[REDACTED].openai.azure.com/"
  DEPLOYMENT = 'gpt-4'
  
  os.environ['OPENAI_API_KEY'] = API_KEY
  os.environ['OPENAI_API_VERSION'] = VERSION
  os.environ['AZURE_OPENAI_ENDPOINT'] = ENDPOINT
  os.environ['DEPLOYMENT_NAME'] = DEPLOYMENT
  
  openai.api_type = "azure"
  openai.api_base = ENDPOINT
  openai.api_endpoint = ENDPOINT
  openai.api_key = API_KEY
  openai.api_version = VERSION
  openai.deployment_name = DEPLOYMENT

and here is the code which throws an error in the run call:

  # Create a Harness object
  h = Harness(task="question-answering",
              model={"model": "gpt-4o", 'hub':'azure-openai', 'max_tokens':4096},
              data={"data_source" :"./test.csv"}
  )
  h.generate().run().report()

It can be quite tricky to ensure that all the various versions of the packages are aligned correctly, especially as Databricks provides some defaults which often need overriding, so I suspect that might be the issue. But since LangTest doesn't explicitly define which versions of the packages it needs, I'm finding it hard to get going. Any help would be strongly appreciated.

[chakravarthik27]

Hi @GrahamJWills ,

I found the bug and will fix it as soon as possible next release.

[chakravarthik27]

This issue has been fixed in LangTest 2.5.0. Please upgrade to the latest version and let us know if you encounter any further issues. 🚀

Quick Guide for Databricks Integration:

Key Features:

• Support for Spark DataFrames and Delta Live Tables for seamless integration and efficient data processing.
• Enhanced testing and evaluation capabilities for robust workflows.

Example Workflow:

1. Load Data into Spark DataFrame:

   df = spark.read.json("<FILE_PATH>")
   df.printSchema()

2. Test Configuration:

   from langtest.types import HarnessConfig
   
   prompt_template = "Your task is to choose the correct answer to the question: {question}\nOptions: {options}\nAnswer: "
   test_config = {
       "evaluation": {"metric": "llm_eval", "model": "gpt-4o", "hub": "openai"},
       "tests": {"robustness": {"add_typo": {"min_pass_rate": 0.8}}},
   }

3. Model and Dataset Config:

   input_data = {"data_source": df, "source": "spark", "spark_session": spark}
   model_config = {"model": {"endpoint": "databricks-meta-llama-3-1-70b-instruct"}, "hub": "databricks", "type": "chat"}

4. Run Harness:

   from langtest import Harness
   
   harness = Harness(task="question-answering", model=model_config, data=input_data, config=test_config)
   harness.generate().run().report()

Let us know if this helps! 😊

---