LLaMa 3.2 vision - Clarification on User Input Masking During Training

[ronyadgar]

I'm noticing that during training, user input always appears to be masked. For context, you can refer to this section of the source code: https://github.com/pytorch/torchtune/blob/main/torchtune/data/_messages.py#L931C8-L931C53.

Could someone explain the reasoning behind this design? Specifically, I'm curious why the system doesn't support training on unmasked user input (where user input here refers strictly to text and not vision data). Any insights or further information on this design choice would be greatly appreciated.