Add support for monitors in DD metricprovider

[indeed-kelvin]

Summary

Enhance the Datadog metric provider to support using the state of a Datadog monitor as the source for analysis results, in addition to the current support for Datadog queries. If this enhancement request is accepted, we are willing to contribute to its implementation to make it happen.

Use Cases

This feature would be particularly useful in scenarios where the desired metric for analysis is already encapsulated in a Datadog monitor. Using the state of a Datadog monitor provides several advantages:

1. Reusability: Teams can leverage existing Datadog monitors without the need to rewrite or duplicate query logic specifically for Argo Rollouts.
2. Consistency: Ensures alignment between monitoring alerts in Datadog and analysis conditions in Argo Rollouts by referencing the same source of truth.
3. Ease of Use: Simplifies configuration by abstracting complex query logic into monitors, which are often easier to manage and version-control.
4. Flexibility: Allows teams to incorporate complex monitor states (e.g., composite monitors) into rollout analysis without additional setup.

Example use case: A team has a composite Datadog monitor that assesses service health across multiple dimensions, such as latency, error rates, and throughput. Instead of recreating these dimensions in Argo Rollouts, they could directly use the monitor's state to determine the rollout's health.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.

[zachaller]

#3892