How do you do LoRA on torchtune with only lora applied to MLP?

[Sinestro38]

For llama-3.1-8b LORA, I want to do the equivalent of:

model:
  _component_: torchtune.models.llama3.lora_llama3_8b
  apply_lora_to_mlp: true
  apply_lora_to_output: false
  lora_alpha: 64
  lora_attn_modules: []    # <-- the issue
  lora_dropout: 0.05
  lora_rank: 32

but I get an error like

ValueError: Must pass one or more of typing.Literal['q_proj', 'k_proj', 'v_proj', 'output_proj'] as lora_modules

Why is this? On HF PEFT, I'm able to do LoRA on just the down_proj and up_proj layers, leaving the attn modules untouched.

[Sinestro38]

@pbontrager perhaps you're aware? Thanks!

[pbontrager]

It's just the result of this check we include here in the model definition. I don't see any reason it has to be there. If you put up a PR removing it, and tag me I'll approve it.

[Sinestro38]

Thanks I will!

[Sinestro38]

Done!

[Sinestro38]

Hi @pbontrager , just following up on this, the PR is up and ready to go :)

[Sinestro38]

Okay this has been merged following #2875