Audit for use of size_hint coming from untrusted data

[dtolnay]

@nox pointed out in IRC that Bincode sequence and map deserializers provide a size_hint that comes straight from the input, and untrusted Bincode data could put a fake large number there. If a Deserialize type takes that hint and uses it for Vec::with_capacity, it could be bad.

Let's look through existing formats to see how prevalent this is and try to come up with guidelines for avoiding this problem.

[sfackler]

See rust-lang-deprecated/rustc-serialize#150 for some similar discussion.

[oli-obk]

we can never protect against oom. If enough data comes from a stream, we will end up allocating too much memory at some point.

[oli-obk]

do bincode and similar check whether the number of hinted elements is also the number of deserialized elements? Otherwise another attack is to always suggest 4096 elements (or whatever the limit is), and only deserialize one element. Since the allocations are never shrunk, we waste a lot of memory.

[nox]

Bincode can only succeed if 4096 elements were deserialised in that case.

[dtolnay]

Let's continue to track this under #850.