A RAG system that uses vector search to show users similar engineering issues that have occurred in the past and uses these as context to suggest how to solve a new issue.
A streamlit UI is used to interact with the search engine.
This was part of the submission for the 2024 Databricks Generative AI World Cup, which resulted in our team winning the 3rd place in the EMEA region.
The team consisted of Gabriele Calvo, Lewis Harvey, Abbie Rhodes and Yousif Kilano
Make sure you have a profile setup for the .databrickscfg file in your user home directory:
[dev-eng-rec-helper]
host = https://#####.cloud.databricks.com
auth_type = databricks-cli
serverless_compute_id = auto
to run the all linting and testing for all subprojects use: uvx invoke cleanup
To deploy the processing jobs in the dev environment use:
uvx invoke deploy-processingFor the data processing, we are using asset bundles. To trigger the run of processing jobs in the dev environment use:
databricks bundle run -t dev summarization
databricks bundle run -t dev vector-index-creationFor the web app we are using databricks app and pyinvoke:
uvx invoke deploy-webapp