dbt rpc "_and_wait" solids should be composite solids

[dwallace0723]

There are a number of dbt rpc "_and_wait" solids that tightly couple the triggering of a dbt operation and the subsequent polling of the operation into a single solid.

Most of the time, this is the desired functionality (i.e. I want to trigger a dbt operation to happen and then wait for it to finish). However, there are instances where the RPC server will return what could be considered "transient" error codes that result in RetryRequested events (see here) during solid execution. If these RetryRequested events fire during the polling process, the current implementation will result in the entire dbt operation being executed again.

In practice, this is probably not what we want to do. Instead, we should only retry a specific portion of the solid based on when the error was received. This could be achieved by transforming the _and_wait solids into composite solids that consist of an "operation" solid (run, test, etc.) and polling solid. If a RetryRequested event was triggered during the polling portion, we would only attempt to retry the polling solid instead of retrying the operation solid as well.

[catherinewu]

Apologies for the delay in responding. This is a great idea, thank you for sharing it here. Tracking as a GH issue so we can prioritize it: #3629