html escape problem \u003c\u003e for json serialization in request gpt response results

[jokerhaos]

1. version 0.7.0+
2. path [email protected]\models_serde.go
3. request body

    "messages": [
        {
            "role": "user",
            "content": "用这几个符合“《》/<>”写一句话，10个字"
        }
    ],

4. The error occurred on line 1799, json.Marshal(objectMap), which requires html escape problem escapeHTML: true
5. At present, my fork warehouse is relatively simple, and I hope that the official can write a better performance scheme

// MarshalJSON implements the json.Marshaller interface for type ChatCompletions.
func (c ChatCompletions) MarshalJSON() ([]byte, error) {
	objectMap := make(map[string]any)
	populate(objectMap, "choices", c.Choices)
	populateTimeUnix(objectMap, "created", c.Created)
	populate(objectMap, "id", c.ID)
	populate(objectMap, "model", c.Model)
	populate(objectMap, "prompt_filter_results", c.PromptFilterResults)
	populate(objectMap, "system_fingerprint", c.SystemFingerprint)
	populate(objectMap, "usage", c.Usage)
	return JSONMarshal(objectMap)
}

func JSONMarshal(data any) ([]byte, error) {
	raw, err := json.Marshal(data)
	if err != nil {
		return nil, err
	}
	result := strings.ReplaceAll(string(raw), "\\u003c", "<")
	result = strings.ReplaceAll(result, "\\u003e", ">")
	result = strings.ReplaceAll(result, "\\u0026", "&")
	return []byte(result), nil
}

[richardpark-msft]

Hey @jokerhaos,

I'm not sure if I understand what's broken. I'm able to send the content you have above to Azure OpenAI without any issue, so it must serialize properly, and there's no requirement that this data be directly usable as HTML.

Here's my sample program, which runs without error. Please let me know if I've misunderstood what you are trying to do:

client := newTestClient(t, azureOpenAI.ChatCompletions.Endpoint)

	resp, err := client.GetChatCompletions(context.Background(), azopenai.ChatCompletionsOptions{
		DeploymentName: &azureOpenAI.ChatCompletions.Model,
		Messages: []azopenai.ChatRequestMessageClassification{
			&azopenai.ChatRequestUserMessage{
				Content: azopenai.NewChatRequestUserMessageContent("用这几个符合“《》/<>”写一句话，10个字"),
			},
		},
	}, nil)
	require.NoError(t, err)

	t.Logf("%#v", resp)

	jsonResp, err := json.Marshal(resp)
	require.NoError(t, err)
	require.NotEmpty(t, jsonResp)

[github-actions]

Hi @jokerhaos. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[jokerhaos]

@richardpark-msft
If you look at my screenshot, the response value has html tag <>, json serial number will be escaped

[richardpark-msft]

Ah, okay, I understand now. We don't have anything in the SDK that's attempting to convert these characters, so this appears to be more an issue in what the model is returning. That is out of the purview of the SDK, and will be something you'll need to address in your prompt, and may even change depending on the model you use.

There are features in OpenAI that can help with providing outputs that are more strictly specified, like Structured Outputs, but we don't provide anything like that directly in the SDK.