PPU LLVM: Precompile encrypted modules

[cipherxof]

Addresses: #12320

Also fixes an issue where the progress bar was not updating correctly when a self failed to decrypt.

Possible solutions to precompile encrypted modules:

1. Get the key at runtime df0d4a4
2. Use pattern matching 77812a7
3. module hash / key or elf address lookup (this is probably the best solution)

[elad335]

What does it achieve though? you are still compiling it ingame.

[cipherxof]

What does it achieve though? you are still compiling it ingame.

PPU modules will compile every time you enter a new area in MGS4, because each stage is it's own executable. This is quite annoying, especially in certain acts where you transition between areas quickly. While this PR technically does precompile while you are in-game, it only happens a single time and it's before you even see the first screen.

[elad335]

The user does not know it, it will create confusion possibly with other titles in which it happens way further in execution.

[cipherxof]

The user does not know it, it will create confusion possibly with other titles in which it happens way further in execution.

Understandable.

Maybe we can check the amount of time that has passed since initial EBOOT.bin has executed? At least with MGS4, it happens nearly instantly and this way it won't interfere with other games. It does feel a little hacky, although I'm unsure of any other way to dynamically retrieve the encryption key. Not being able to precompile PPU modules for MGS4 is a pretty bad experience.

[woj1993]

Maybe add it as an option "Precompile dynamically generated modules while game is running" with description "Some games generate/decrypt elf files that contains executable code. Normally we would compile it when given code needs to be run but some games generate gigantic amount of code that needs to be compiled at the same time. This switch allows us to precompile these modules before they are needed possibly elimination shutter/pauses while in game"

Edit: And then set it to disabled by default with ability to turn it on. When it is turned on just precompile every child ELF process that is spawned.

[cipherxof]

Maybe add it as an option "Precompile dynamically generated modules while game is running" with description "Some games generate/decrypt elf files that contains executable code. Normally we would compile it when given code needs to be run but some games generate gigantic amount of code that needs to be compiled at the same time. This switch allows us to precompile these modules before they are needed possibly elimination shutter/pauses while in game"

Edit: And then set it to disabled by default with ability to turn it on. When it is turned on just precompile every child ELF process that is spawned.

I would be interested to know how many other games would even benefit from this, or if it's just MGS4. I believe some games dynamically execute SPRXs at runtime? I do understand how runtime precompilation could cause confusion, and it might not even be worth it if no other game actually benefits from it.

However, the distinguishing factor for MGS4 is the EBOOT.bin is essentially a bootstrapper for MGS4.self. The runtime is less than half a second on my machine before precompilation would kick in. We could conceivably limit the "runtime" precompilation by checking some things before compiling modules, such as time between initial eboot.bin->current exitspawn, file size of eboot.bin, or checking import table for cellGcm calls.

I don't know if there is a perfect solution. I don't think it's really feasible to statically analyze the EBOOT.bin or hardcode the klics and finding the klic at runtime brings it's own oddities. Personally, I think it's worth the tradeoff but I can understand if RPCS3 doesn't want to add this feature natively.

[cipherxof]

#16748

Pushed a new commit with a proof of concept pattern match for klic (this can probably be marked as a draft, at least until elad gives more feedback or creates his own PR).

There are likely some issues with this PR in it's current state. For example, the hardcoded behavior when looking up npDrmProcessExitSpawn2. Ideally the pattern matcher would be able to dynamically match against branches to imported functions by fnid or name.

Creating the LLVM cache via the context menu does not work at the moment either. You must actually boot the game.

EDIT: I do sort of feel like using pattern matching for this is a bit of a bad solution. For example, lets take a look at at castlevania (another game that fails to precompile some encrypted modules).

  puVar7 = *(undefined4 **)(&stack0x00000000 + iVar5);
  if (puVar7 == DAT_00cbba5c) {
    iVar5 = FUN_007a3d70();
    if ((iVar5 != 0) && (uVar10 = 0, DAT_00b78220 != 0)) {
      iVar5 = 0;
      do {
        iVar6 = sceNpDrmIsAvailable(DAT_00b7821c + iVar5,param_2);
        if (iVar6 == 0) break;
        uVar10 = uVar10 + 1;
        iVar5 = iVar5 + 0x10;
      } while (uVar10 < DAT_00b78220);
    }
    uVar1 = sys_prx_load_module(param_2,0,0);
    iVar5 = sys_prx_start_module(uVar1,4,local_98,auStack_b0,0,0);

Pattern matching would be a nightmare here, and the value that it's pointing to is actually zeroed out and gets written to during runtime. This is the code that writes the klic:

  iVar5 = FUN_0077ac54();
  uVar7 = ZEXT48(PTR_DAT_00b75f58);
  *(undefined4 *)(iVar5 + 0x4094) = 0x3f800000;
  FUN_007bf9d8(uVar7 + 4,0x17);

Again, a nightmare. There's no imported calls to match against, and it's using arithmetic like the previous section. I suspect other games will end up being just as convoluted.

Is it not feasible to just have an address lookup based on the module hash that we know we can read the klic from? This would work for more games and would be less complex.

const std::map<std::string, u32> module_klic_offsets
{
  {"dc111ca4441b553ab5214de56c26e5365afd6498", 0x00b77f04},
  // etc...
};

[elad335]

See #16764

What I believe should be done in this pull request, is to use Emu.klic decryption as a debug method.
So if an executable is able to be decrypted only by Emu.klic, you should log an error saying "Possible missed KLIC for precompilation for file '%s', please report to developers."