There's also the added question of how batch should handle records where a single unsplittable record exceeds send_batch_max_size_bytes. This isnt an issue in the current item count based implementation, but may become an issue if/when the code is fixed to count bytes instead of items.

Counting the bytes is very exporter protocol specific unfortunately and there is no easy way for us to do in a way that will work for all protocols, also even for OTLP is pretty expensive to calculate size, so it will be good if this will be an additional component that splits by size instead of the core batch processor.

Hey Bogdan, thanks for your response! If counting size isn't desired in this batch processor, I think the config variable names should still be updated to something like send_batch_min_item_count and send_batch_max_item_count to avoid confusion, as size with a default value of 8192 for one of the configs reads and looks like it's referring to bytes/memory rather than item count, and the tests also count bytes for size rather than items.

I totally understand that counting bytes is pretty exporter specific and can be expensive to do so adding it to this module may not be desired. I think there's a use case for a core processor that does batch by size though, as many of the core exporters (otlp, otlphttp, jaeger, etc) may send data to an endpoint behind a reverse proxy like nginx or aws api gateway that limits payload sizes. It wouldnt be perfect for sure, and if the limit on the gateway is 5MB then people may have to set their batch size limit to 4.5MB, but I think a central size-based batcher is still helpful/necessary given how many exporters are likely to have some sort of payload size limitation in place, and it would be easier to build a central batch size processor than add size/splitting logic to all of the relevant exporters.

If I'm not mistaken the Payload size for API GW is 10MB and for Lambda its 6MB

Can a byte size limiting option be implemented by counting the bytes of the in memory representation of the records? Obviously being a generic component its not possible for the batch processor to know the final serialized size of an unknown downstream exporter, but each exporter likely has a semi predictable translation ratio between the size of the in memory representation and the serialized representation. E.g. maybe on average, the OTLP representation is 1.2x the size of the in memory representation.

This would give a better, although still imperfect, way of limiting the size of the exported payload.

I think being able to batch by in-memory bytes, although it wouldn't be accurate without knowing the exporter representation, would give a more accurate way of defining batch limits than item count when the receiver expects payloads to be of a given size.

Systems that behave better with a more constant payload size would also benefit from this as well, regardless of payload limits, as currently the distribution of payload sizes can vary tremendously depending on the nature of the spans within them. Sending small (in byte size) batches can affect performance.

I'm considering 3 approaches for this task:

• Implementing a function similar to (ms Metrics) DataPointCount() but to calculate the byte count. This method requires each metric type to incorporate a Size() function, which would determine the in-memory byte size of the metric. Think of it as using unsafe.Sizeof() for each field.

• Adding first 5 datapoints (this is just a number, in reality let's say x datapoints) using the add() function, then using the sizer to determine the byte size per added datapoints. let's say we have a limit of 10000 bytes, first 5 points size is 200 bytes, then per datapoints is 40 bytes. Subsequently, we employ a binary check, with the next checkpoint triggered when the number of datapoints reaches 0.5 times the size limit divided by the bytes per datapoint. In my example, the next checkpoint would be at 10000*0.5/40 = 125 datapoints. We progressively update the checkpoint to 75%, 87.5, 94% ( here the bytes per datapoint is updated with more datapoints every time in order to get better precision), and so forth, until passing 90% or 95%, in the end, we consider it is good enough to batch and send.

• Simplify option 2, Just do the estimation at 50% checkpoint above and define total datapoints allowed. take the min(send_batch_max_item_count, the total datapoints allowed we got)

The second idea comes from the assumption that the cost of calling bm.sizer.MetricsSize(req) to get request size is the biggest concern in this thread, and we would like to minimise this but during the same time still have a good enough solution.

Do we actually have a preference on the 2 approaches?