Search results

Response parameter CompletionUsage return 0 token consumption

Status: Open.

#56 In npuichigo/openai_trtllm;

· moyerlee opened on Oct 16, 2024

Support for model in nemo format

Status: Open.

#54 In npuichigo/openai_trtllm;

· Minhhnh opened on Oct 1, 2024

openai_trtllm return 200 directly to the client when TTFT is greater than 15 seconds

Status: Open.

#53 In npuichigo/openai_trtllm;

· mynameiskeen opened on Aug 22, 2024

Support for Llama 3.1

Status: Open.

#52 In npuichigo/openai_trtllm;

· datdo-msft opened on Aug 13, 2024

all option is same as openai?

Status: Open.

#48 In npuichigo/openai_trtllm;

· dongs0104 opened on Jul 9, 2024

Feature request - Add all v1/ routes

Status: Open.

#47 In npuichigo/openai_trtllm;

· visitsb opened on Jun 17, 2024

Missing spaces

Status: Open.

#46 In npuichigo/openai_trtllm;

· Mary-Sam opened on Jun 6, 2024

llama 3 tokenizer no longer works - updated eos token

Status: Open.

#44 In npuichigo/openai_trtllm;

· avianion opened on May 13, 2024

support for llama 3

Status: Open.

#43 In npuichigo/openai_trtllm;

· avianion opened on May 12, 2024

ERROR: expected number of inputs between 1 and 3 but got 9 inputs for model

Status: Open.

#38 In npuichigo/openai_trtllm;

· samzong opened on Apr 15, 2024