broken header streamer after v3.48.0

[jiaquan1]

The ansi-timestamps experiment is enabled in v3.48.0 by default. The header line begins with _bk;t=1698038886128~~~ , the header streamer for header regex https://github.com/buildkite/agent/blob/24c562342c8dd3944b16bc6b87b501e304490e67/agent/header_times_streamer.go#L167C25-L167C74 broken, no header streamer anymore. The regex needs to be updated to headerRE = regexp.MustCompile(^(_bk;t=\d+(?:---|~~~|+++)\s(.+))$) to fix the header streamer.

[moskyb]

hi there @jiaquan1! the old behaviour can be retained by specifying --no-ansi-timestamps in your agent configuration - this will remove the ansi-timestamps and go back to untimestamped output, which should match the header time regex.

ANSI timestamped output is sent without header times by design; the backend had to do a fair bit of work to untangle them and pull time between headers out for displaying the little histogram on the righthand side of the job logs:

the ansi timestamps feature attaches timestamps to lines individually, so it removes the need for the agent or the backend to do this extra work.

can i ask what your specific need for header times is, and how you were accessing them?

[jiaquan1]

We would like to have the timestamp and header times at the same time. In our log system, we get header streamer and the length of header time set in one job log, then our system will have to identify the same number of headers from this log. We use this log span tool to track how much time in each step (between each header). Now the length of header time set from our buildkite client API is 0, then our system won't work either.
Is there anyway to have ansi-timestamp steamer, similar as the header streamer in the past? We need to count how many headers in one job log, then identify the same number of the headers.

[moskyb]

hey, thanks for updating us - this really helps us out.

i think ideally, we'd like to avoid header-timestamping ANSI-timestamped output, as it requires our backend to do a bunch of work on job logs that we'd ideally like to avoid, but i can see how this would break your use case.

i presume that at some point in your log span tool, you're inspecting the logs themselves to match them up to the header times - would it be possible to alter that logic to look for headers in the log, and use the ANSI timestamp as the "header time" in this case?