Fixed the model conversion bug caused by minicpm's GQA structure。After testing minicpm's GQA, the converted model generates all <h>. This is because the number of k and v matrices of Gqa should be the same as kv_head, not the same as head/kv_head.

[LDLINGLINGLING]

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

[ngxson]

Does this only affect minicpm 2, or both version 1 and 2?

[LDLINGLINGLING]

only minicpm 2,Because only minicpm2 started using gqa

[ngxson]

Changing this will break minicpm 1, we should add a check somewhere to detect if the input model is minicpm 1 or 2

[LDLINGLINGLING]

Is the 1 you mentioned cpm-bee? This model has never been supported. Which model did you crash? Can I check it out?

[compilade]

Seems to be the same as #7967, therefore my comment from there applies here too.

This makes MiniCPMModel._reverse_hf_permute exactly equivalent to LlamaModel.permute. Should LlamaModel.permute (which is also a static method) be used instead in MiniCPMModel?

[ngxson]

@LDLINGLINGLING I saw that support for minicpm is added a long time ago, so I supposed it was version 1.

But to be honest, there have been quite a lot of versions of minicpm so I'm not even sure what we're talking about. Could you specify which version you've tested?

A perplexity test would be appreciated.

[LDLINGLINGLING]

Hello, I just found that the code I uploaded before is wrong. The new commit has tested all minicpm, and the results are normal. The code in the original llama.cpp cannot effectively convert the minicpm of the 1b parameter. , it is worth noting that in your figure, the model with the letter v is multi-modal, which is not supported by llama.cpp.