Provide timestamp per metric not per metrics set

[Dunedan]

Currently powertools doesn't track when a metric got added and uses the time of flushing as timestamp in the EMF metadata. This causes inaccurate time information when the function annotated with log_metrics runs longer than the minimal resolution for metrics in CloudWatch Metrics.

Here is sample code to reproduce this problem:

#!/usr/bin/env python3

import json
import time

from aws_lambda_powertools import Metrics
from aws_lambda_powertools.metrics import MetricUnit

metrics = Metrics(namespace="Test", service="Test")

@metrics.log_metrics
def handler(event, context):
    metrics.add_metric(name="TestMetric1", unit=MetricUnit.Count, value=1)
    time.sleep(65)
    metrics.add_metric(name="TestMetric2", unit=MetricUnit.Count, value=1)

handler(None, None)

This code produces the following output:

{
    "_aws": {
        "Timestamp": 1600326158339,
        "CloudWatchMetrics": [
            {
                "Namespace": "Test",
                "Dimensions": [
                    [
                        "service"
                    ]
                ],
                "Metrics": [
                    {
                        "Name": "TestMetric1",
                        "Unit": "Count"
                    },
                    {
                        "Name": "TestMetric2",
                        "Unit": "Count"
                    }
                ]
            }
        ]
    },
    "service": "Test",
    "TestMetric1": 1.0,
    "TestMetric2": 1.0
}

Instead of a single log record with a single timestamp, I'd expect to get two distinct log records with timestamps 65 seconds apart.

I also already reported the same problem for aws-embedded-metrics-python: awslabs/aws-embedded-metrics-python#53

[heitorlessa]

Hey Daniel, Thanks for reaching out and raising this with the EMF team - I’ve added a comment there now. I agree timestamp should be more accurate to when metric was added, however I disagree with logging multiple objects due to the cost customers will pay for that on Log ingestion and storage — Extending EMF to support that, along with things like Statistics would be best long term

…

On Thu, 17 Sep 2020 at 09:04, Daniel Roschka ***@***.***> wrote: Currently powertools doesn't track when a metric got added and uses the time of flushing as timestamp in the EMF metadata. This causes inaccurate time information when the function annotated with log_metrics runs longer than the minimal resolution for metrics in CloudWatch Metrics. Here is sample code to reproduce this problem: #!/usr/bin/env python3 import json import time from aws_lambda_powertools import Metrics from aws_lambda_powertools.metrics import MetricUnit metrics = Metrics(namespace="Test", service="Test") @metrics.log_metrics def handler(event, context): metrics.add_metric(name="TestMetric1", unit=MetricUnit.Count, value=1) time.sleep(65) metrics.add_metric(name="TestMetric2", unit=MetricUnit.Count, value=1) handler(None, None) This code produces the following output: { "_aws": { "Timestamp": 1600326158339, "CloudWatchMetrics": [ { "Namespace": "Test", "Dimensions": [ [ "service" ] ], "Metrics": [ { "Name": "TestMetric1", "Unit": "Count" }, { "Name": "TestMetric2", "Unit": "Count" } ] } ] }, "service": "Test", "TestMetric1": 1.0, "TestMetric2": 1.0 } Instead of a single log record with a single timestamp, I'd expect to get two distinct log records with timestamps 65 seconds apart. I also already reported the same problem for aws-embedded-metrics-python: awslabs/aws-embedded-metrics-python#53 <awslabs/aws-embedded-metrics-python#53> — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#166>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBFWQSWJF6IISMI2NELSGGYHDANCNFSM4RP63T6Q> .

[Dunedan]

The records I expected to be written were based on the current EMF specification. I wasn't aware that changes to the EMF specification are a possibility as well and I agree, extending the specification might be a better solution. 👍

[nikevp]

I'm running into this issue as well. For the majority of my team's metrics, we can live with grouping under one timestamp. However, I would like to break out one metric that helps us monitor TPS of a 3rd party api. If I wanted one metric (units=MetricUnit.CountPerSecond) to have its own timestamp would the solution be to manually flush metrics?

[nikevp]

Going with PutMetricData instead of EMF for this use case.

[heitorlessa]

Closing - last update is that CloudWatch EMF didn't prioritize this feature (if you're reading this later, please work with your AWS Account representatives to ask for a +1)