Support speculative-decoding

[madroidmaq]

close #14

cURL

curl http://localhost:10240/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Unsloth-DeepSeek-R1-Distill-Qwen-32B-4bit",
+    "draft_model": "mlx-community/Qwen2.5-0.5B-Instruct-4bit",
    "messages": [
      {
        "role": "user",
        "content": "What features are typically found on smartwatches, listing the 5 most important"
      }
    ]
  }'

[madroidmaq]

Eval Result:

• Before:

prompt_tps: 71.406
generation_tps: 19.251
INFO:     127.0.0.1:62288 - "POST /v1/chat/completions HTTP/1.1" 200 OK
prompt_tps: 72.197
generation_tps: 19.100
INFO:     127.0.0.1:62444 - "POST /v1/chat/completions HTTP/1.1" 200 OK
prompt_tps: 70.344
generation_tps: 19.154
INFO:     127.0.0.1:62795 - "POST /v1/chat/completions HTTP/1.1" 200 OK

• After:

prompt_tps: 69.233
generation_tps: 18.384
INFO:     127.0.0.1:59481 - "POST /v1/chat/completions HTTP/1.1" 200 OK
prompt_tps: 67.428
generation_tps: 17.666
INFO:     127.0.0.1:59690 - "POST /v1/chat/completions HTTP/1.1" 200 OK
prompt_tps: 70.134
generation_tps: 16.917
INFO:     127.0.0.1:59909 - "POST /v1/chat/completions HTTP/1.1" 200 OK