update dagster asset check to fail"better"

[cnolanminich]

This PR updates the asset check on the RAW_DATA.users asset in a few ways

• compares the types to a set (previously it would almost always fail because lists are unordered)
• forces it to fail by omitting one of the observed companies in the expected companies, and by adding an expected company that is not in observed
• this one I want feedback on -- I added a markdown data element (see the image below) after following along with some of the cool metadata output types @slopp wrote up in this GitHub Discussion. I wanted to show the "in expected not observed" and "in observed not expected" in some sort of way to show off how powerful the metadata exposures can be, but the data frame --> markdown I did here leaves a lot to be desired.

Will pull this out of draft mode once we get some consensus on the metadata output for this

[github-actions]

Your pull request is automatically being deployed to Dagster Cloud.

Location	Status	Link	Updated
demo_assets		View in Cloud	Mar 11, 2024 at 06:22 PM (UTC)
snowflake_insights		View in Cloud	Mar 11, 2024 at 06:22 PM (UTC)
basics		View in Cloud	Mar 11, 2024 at 06:22 PM (UTC)
data-eng-pipeline		View in Cloud	Mar 11, 2024 at 06:22 PM (UTC)
batch_enrichment		View in Cloud	Mar 11, 2024 at 06:22 PM (UTC)

[slopp]

Yea I like the idea of using markdown output metadata in an asset check, but I think this particular data frame is fairly confusing because what you are presenting isn't really a dataframe, the rows aren't meaningful across columns.

I think of this data check more as looking for bad company data, not necessarily checking that every expected company was observed. So maybe we could simplify it as:

MetadataValue.md(f"Observed the following unexpected companies: {list(unique_companies - expected_companies)}") 

[cnolanminich]

Oh yeah I agree! I'll push that change and then this one will be in good shape

[cnolanminich]

Fixed!

[slopp]

LGTM, one comment on removing what is now dead code