Bump peft from 0.14.0 to 0.15.2

[dependabot]

Bumps peft from 0.14.0 to 0.15.2.

Release notes

Sourced from peft's releases.

v0.15.2

This patch fixes a bug that resulted in prompt learning methods like P-tuning not to work (#2477).

v0.15.1

This patch includes a fix for #2450. In this bug modules_to_save was not handled correctly when used in conjunction with DeepSpeed ZeRO stage 3 which resulted in those modules being placeholder values in the saved checkpoints.

Full Changelog: huggingface/peft@v0.15.0...v0.15.1

v0.15.0

Highlights

New Methods

CorDA: Context-Oriented Decomposition Adaptation

@​iboing and @​5eqn contributed CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuning . This task-driven initialization method has two modes, knowledge-preservation and instruction-preservation, both using external data to select ranks intelligently. The former can be used to select those ranks that correspond to weights not affiliated with knowledge from, say, a QA dataset. The latter can be used to select those ranks that correspond most to the task at hand (e.g., a classification task). (#2231)

Trainable Tokens: Selective token update

The new Trainable Tokens tuner allows for selective training of tokens without re-training the full embedding matrix, e.g. when adding support for reasoning / thinking tokens. This is a lot more memory efficient and the saved checkpoint is much smaller. It can be used standalone or in conjunction with LoRA adapters by passing trainable_token_indices to LoraConfig. (#2376)

Enhancements

LoRA now supports targeting multihead attention modules (but for now only those with _qkv_same_embed_dim=True). These modules were tricky as they may expose linear submodules but won't use their forward methods, therefore needing explicit support. (#1324)

Hotswapping now allows different alpha scalings and ranks without recompilation of the model when the model is prepared using a call to prepare_model_for_compiled_hotswap() before compiling the model. (#2177)

GPTQModel support was added in #2247 as a replacement for AutoGPTQ which is not maintained anymore.

Changes

• It's now possible to use all-linear as target_modules for custom (non-transformers) models (#2267). With this change comes a bugfix where it was possible that non-linear layers were selected when they shared the same name with a linear layer (e.g., bar.foo and baz.foo).
• The internal tuner API was refactored to make method registration easier. With this change the number of changes to numerous files is reduced to a single register_peft_method() call. (#2282)
• PEFT_TYPE_TO_MODEL_MAPPING is now deprecated and should not be relied upon. Use PEFT_TYPE_TO_TUNER_MAPPING instead. (#2282)
• Mixed adapter batches can now be used in conjunction with beam search. (#2287)
• It was possible that modules_to_save keys wrongly matched parts of the state dict if the key was a substring of another key (e.g., classifier and classifier2). (#2334)
• Auto-casting of the input dtype to the LoRA adapter dtype can now be disabled via disable_input_dtype_casting=True. (#2353)
• The config parameters rank_pattern and alpha_pattern used by many adapters now supports matching full paths as well by specifying the pattern with a caret in front, for example: ^foo to target model.foo but not model.bar.foo. (#2419)
• AutoPeftModels do not reduce the embedding size anymore if the tokenizer size differs from the embedding size. Only if there are more tokens in the tokenizer than in the embedding matrix, the matrix will be resized. This is to prevent resizing of embedding matrices in models that have 'spare' tokens built-in. (#2427)

What's Changed

• FIX: Ensure Device Compatibility for BOFT Forward/Merging by @​d-kleine in huggingface/peft#2242
• MNT: Bump version to 0.14.1.dev0 by @​BenjaminBossan in huggingface/peft#2263
• ENH: fix library interface by @​bluenote10 in huggingface/peft#2265
• FIX: Add warning for adapter_name conflict with tuner by @​pzdkn in huggingface/peft#2254
• ENH: FIX: Allow "all-linear" to target custom models by @​BenjaminBossan in huggingface/peft#2267
• MNT: apply sorting of exported symbols in __all__ by @​bluenote10 in huggingface/peft#2280
• MNT: apply sorting of imports by @​bluenote10 in huggingface/peft#2279
• FIX: Adoption prompt: New way to obtain position embeddings by @​BenjaminBossan in huggingface/peft#2276
• FIX: Int8 check for torchao v0.7.0 by @​BenjaminBossan in huggingface/peft#2284

... (truncated)

Commits

• 3c7b6e7 Breakout fix of #2481 for #2477, patch release prep (#2497)
• c42eb22 Prepare 0.15.1 patch (#2459)
• b34d8a2 Release 0.15.0 (#2435)
• 48e0c5d Fix #2422: Modules to save with multiple adapters (#2430)
• b2b34fd FIX Minimal target module optimization bug w/ IA³ (#2432)
• 7320bb9 FIX AutoPeftModels never reduce embedding size (#2427)
• 2f063e6 ENH: Extend the regex for rank/alpha pattern (#2419)
• 37266c1 FIX Revert optimization for LoRA scaling == 1 (#2416)
• 8edaae9 TST Add missing .eval() calls to inference tests (#2408)
• e1c7e8c FIX Reset the FP32 matmul precision in tests (#2411)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)