Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(scripting/mono-rt2): Fix UTF8 encoding issues for surrogate pairs and Korean characters #3267

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

poco8537
Copy link
Contributor

Goal of this PR

Ensure that UTF8 encoding in the Mono v2 scripting environment correctly handles surrogate pairs and Korean characters, preventing string corruption during native interop.

How is this PR achieving the goal

This PR updates the UTF8 encoding functions to correctly handle surrogate pairs by encoding valid pairs as 4-byte sequences and replacing invalid or missing surrogates with the replacement character (U+FFFD), ensuring that Korean and other Unicode characters are encoded properly.

This PR applies to the following area(s)

ScRT: C#

Successfully tested on

Game builds: ..

Platforms: Windows, Linux

Checklist

  • Code compiles and has been tested successfully.
  • Code explains itself well and/or is documented.
  • My commit message explains what the changes do and what they are for.
  • No extra compilation warnings are added by these changes.

Fixes issues

fixs #3263

Sorry, something went wrong.

@github-actions github-actions bot added ScRT: C# Issues/PRs related to either C# scripting runtimes triage Needs a preliminary assessment to determine the urgency and required action labels Mar 20, 2025
@prikolium-cfx prikolium-cfx added the status:needs planning Hard issues that take time as there's no 'one' obvious way. label Apr 4, 2025
@prikolium-cfx
Copy link
Collaborator

Thank you for the contribution. We need some time to verify that this change doesn't break anything else.

@prikolium-cfx prikolium-cfx self-requested a review April 4, 2025 12:20
@DaniGP17
Copy link
Contributor

DaniGP17 commented Apr 4, 2025

I said in the original issue(#3263) that maybe the cause was CString but looking into it in depth I think this won't fix it

@poco8537
Copy link
Contributor Author

poco8537 commented Apr 4, 2025

I said in the original issue(#3263) that maybe the cause was CString but looking into it in depth I think this won't fix it

When I tested it, it worked fine.
image

@DaniGP17
Copy link
Contributor

DaniGP17 commented Apr 4, 2025

When I tested it, it worked fine.

Oh, sorry, when I checked it I thought it was unrelated. It seems I didn't investigate well enough, and when I saw that the "Code compiles and has been tested successfully." check was unchecked, I thought you hadn't tested it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ScRT: C# Issues/PRs related to either C# scripting runtimes status:needs planning Hard issues that take time as there's no 'one' obvious way. triage Needs a preliminary assessment to determine the urgency and required action
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants