WIP: Add get metadata

[ahuang11]

Just an initial set up to explore how get_metadata could be implemented and used in Lumen AI.

Right now, it's only used in TableLookup, but I think it can be used more widely, like in ChatAgent / SQLAgent, or maybe just Planner?

I imagine it can be used similarly like DocumentLookup

I was using this for testing since these were the only tables that were not sensitive and had descriptions...

from lumen.sources.snowflake import SnowflakeSource

source = SnowflakeSource(
    account=account
    authenticator="externalbrowser",
    database="SNOWFLAKE_SAMPLE_DATA",
    user=user
)

tables_to_get = {
    "APPLICABLE_ROLES": "APPLICABLE_ROLES",
    "COLUMNS": "COLUMNS",
    "DATABASES": "DATABASES"
}

all_metadata = source.get_metadata(tables_to_get)

all_metadata

[codecov]

Codecov Report

Attention: Patch coverage is 5.79710% with 65 lines in your changes missing coverage. Please review.

Project coverage is 57.28%. Comparing base (e352271) to head (b9d2f20).

Files with missing lines	Patch %	Lines
lumen/ai/tools.py	8.88%	41 Missing ⚠️
lumen/sources/snowflake.py	0.00%	24 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1074      +/-   ##
==========================================
- Coverage   57.51%   57.28%   -0.24%     
==========================================
  Files         109      109              
  Lines       14291    14357      +66     
==========================================
+ Hits         8220     8224       +4     
- Misses       6071     6133      +62     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.