Search results

ggml-quants : weighted rounding algorithms with cumulative search

generation qualityQuality of model outputQuality of model output

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Less than 4 bitsEfforts related to viable quantized models using <4 bitsEfforts related to viable quantized models using <4 bits

research 🔬

Review Complexity : MediumGenerally require more time to grok but manageable by beginner to medium expertise levelGenerally require more time to grok but manageable by beginner to medium expertise level

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Draft (not ready).

ggml-org/llama.cppnumber 12557

#12557 In ggml-org/llama.cpp;

· compilade opened on Mar 25, 2025

Added Arbitrary mixed quantization

Less than 4 bitsEfforts related to viable quantized models using <4 bitsEfforts related to viable quantized models using <4 bits

research 🔬

Status: Open (in progress).

ggml-org/llama.cppnumber 1834

#1834 In ggml-org/llama.cpp;

· Milkdrop opened on Jun 13, 2023

Q4_0 scale selection using RMSE

enhancementNew feature or requestNew feature or request

Less than 4 bitsEfforts related to viable quantized models using <4 bitsEfforts related to viable quantized models using <4 bits

research 🔬

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Status: Draft (not ready).

ggml-org/llama.cppnumber 835

#835 In ggml-org/llama.cpp;

· sw opened on Apr 7, 2023