Improve monitoring strategy based on extension of /healthcheck endpoint

[jimmie]

💡 Description

The routine exercising of the /healthcheck endpoint by ECS (once deployed) provides an excellent opportunity to vastly improve automated monitoring capabilities via alerting. Consider the following workflow:

1. ECS submits a request to the /healthcheck endpoint
   a. this request specifies thresholds (e.g. % of available JVM memory, % of free space on any Opensearch shard)
2. As part of fulfilling this request, the service:
   a. Sends informative, well-defined messages to stdout
   b. These messages are captured in the service log file, captured in Cloudwatch
   c. Messages indicate any errors, such as violation of any of the thresholds specified in 1a or inability to access Opensearch, etc.
3. Alerts are placed on the Cloudwatch logs that detect any of the error message patterns produced in CICD did not publish the jar on artifactory #2
4. Alert messages are sent to operations to be addressed or at least made aware of situations needing attention (e.g. increase disk space)