This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.

Still relevant. @edgarrmondragon this might be something we want to have available as an option in Arch so having a way to control it externally would be nice eventually.

Still relevant. @edgarrmondragon this might be something we want to have available as an option in Arch so having a way to control it externally would be nice eventually.

Makes sense. Should be pretty quick to implement too fwiw.

Related:

• refactor: Use a single source of truth for built-in capabilities #2505

Would love to be able to modify my schemas outside of meltano and just have meltano "bootstrap" schemas, and then adjust accordingly as I manage things on my end.

Load it all in as text initially for example, but if I come and change it to int later, just roll with it and follow the new rule(s).

Would love to be able to modify my schemas outside of meltano and just have meltano "bootstrap" schemas, and then adjust accordingly as I manage things on my end.

Load it all in as text initially for example, but if I come and change it to int later, just roll with it and follow the new rule(s).

Thanks for your input @atrauzzi! That makes me think we would need 3 schema update modes:

• adapt: the incoming type takes priority and the existing column type is updated
• preserve (your suggestion): the existing type takes priority and the incoming type is ignored
• fail: if there's a mismatch between the incoming and existing types, we fail

Wdyt @pnadolny13?

Yeah that sounds good. The overall goal in my scenarios are to not have to configure my schema explicitly in meltano. Instead, I want to delegate that entirely to meltano. That way if my source data schema changes, I'm not having to make config changes.

I might rename your two options to be a bit more explicit, as the effect can be ambiguous depending on perspective:

• sync-target-schema
• prefer-target-schema

Also, two friendly reminders:

• With prefer-target-schema, I would still rely on meltano to create the newly discovered columns.
• I might make columns nullable when I'm manually adjusting types. Might be nice if meltano could support that, versus empty strings