Help passing uploaded image (blob) into server action

[nikitavoloboev]

I have this code:

									onChange={async (e: React.ChangeEvent<HTMLInputElement>) => {
										e.stopPropagation()
										if (e.target.files && e.target.files[0]) {
											const uploadedFile = e.target.files[0]
											local.uploadedImage.set(uploadedFile)
											await describeImageAction({
												imageBlob: e.target.files[0],
												huggingFaceToken:
													"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",
											})
										}
									}}
									```
									
									
Code for it here: https://github.com/kuskusapp/kuskus/blob/main/components/AddPostModal.tsx#L125

Action here: 

```js
const describeImageSchema = z.object({
	imageBlob: z.any(),
	huggingFaceToken: z.string(),
})
export const describeImageAction = actionClient
	.schema(describeImageSchema)
	.action(async ({ parsedInput: { imageBlob, huggingFaceToken } }) => {
		console.log("wtf try run it")
		const session = auth.getSession()
		console.log(session, "sesh")
		if (session) {
			try {
				let imageDescription = await describeImage(imageBlob, huggingFaceToken)
				console.log(imageDescription)
				return imageDescription
			} catch (err) {
				return { failure: "Error describing image:", errorDetails: err.message }
			}
		}
	})
	```
	
	Code: https://github.com/kuskusapp/kuskus/blob/main/app/actions.ts#L52
	
	Currently it just silently breaks. No logs.

[nikitavoloboev]

getting this now

trying to do:

[nikitavoloboev]

ok now getting an issue

my input:

calls this action:

it fails with bad request

for some reason, i pass base64 string to it

then try pass it to endpoint (https://huggingface.co/nlpconnect/vit-gpt2-image-captioning?inference_api=true)

which accepts a Buffer

but for some reason const buffer = Buffer.from(imageAsBase64, "base64") returns a wrong buffer or something and I get 400 request

[nikitavoloboev]

https://next-safe-action.dev/docs/usage-with-forms trying to make it work with forms, maybe that works

[nikitavoloboev]

my latest code looks like this, trying to make it work

pushed it to https://github.com/kuskusapp/kuskus

[nikitavoloboev]

everyone is confused too

export const describeImageAction = actionClient
	.schema(describeImageSchema)
	.action(async ({ parsedInput: { imageAsBase64 } }) => {
		try {
			// assumes image is a JPEG
			// TODO: support more types by: adjust the MIME type if it's different
			const dataUrl = `data:image/jpeg;base64,${imageAsBase64}`
			console.log(dataUrl, "data url")
			const response = await openai.chat.completions.create({
				model: "gpt-4o",
				messages: [
					{
						role: "user",
						content: [
							{ type: "text", text: "What’s in this image?" },
							{
								type: "image_url",
								image_url: {
									url: dataUrl,
								},
							},
						],
					},
				],
			})
			console.log(response, "response")
			console.log(response.choices[0], "ai description of image")
			console.log("finished")
			```
			
			in above code, those 3 last logs, never get run
			
			
![Uploading image.png…]()

just see this

[nikitavoloboev]

https://next.next-safe-action.dev/docs/safe-action-client/custom-server-error-handling tried finding docs on it in v7, found 404

[nikitavoloboev]

												const resp = await describeImageAction({
													imageAsBase64: base64Image,
												})
											 console.log(resp, "response from action")
											 console.log(resp.serverError, "server error")
											 ```

[nikitavoloboev]

oh ok

[TheEdoRan]

This is the right URL: https://next.next-safe-action.dev/docs/safe-action-client/initialization-options#handlereturnedservererror

next-safe-action catches errors that occur during execution, on the server. If that happens, the serverError property of the action result object gets set.

If you want to rethrow all server errors, you can easily achieve that behavior by doing that inside handleReturnedServerError function.

[nikitavoloboev]

from https://discord.com/channels/974519864045756446/1037561178286739466/1242099532523634770

[nikitavoloboev]

pushed code in: https://github.com/kuskusapp/kuskus/blob/main/app/actions.ts

super confused, open to call to resolve this if possible 🙏

[nikitavoloboev]

ok thank you @TheEdoRan

const resp = await describeImageAction({
imageAsBase64: base64Image,
})
console.log(resp, "response from action")
console.log(resp.serverError, "server error")

this helped me get the error

so happy