A word on LINUX Hacking of the cheap variety: Running Carrion on cheap devices and beyond.

01-Word-On-Cheapo-Hacking.md

A word on LINUX Hacking of the cheap variety: Running Carrion on cheap devices and beyond.

Say that you have an ARM Linux Device (or many) that you got on the cheap. One of the things that you'll quickly realize is how little support there is for devices that are outside of the "Raspberry Pi" ecosystem, specially those without display servers and/or running limited graphical stacks, such as Mali proprietary drivers.

This is both a blessing and a curse:

• A curse because your favorite titles are not immediately available (and maybe never will be).
• A blessing because this presents us a very ludic learning opportunity poking & hacking video games and their technology stacks.

Having a solid understanding of the Software Stack involved is a key part of getting software to run on these devices. You are, however, not required to have it at first. Matter of fact, this is a great way to learn many different languages, engines, frameworks and developing the analytic thinking skills necessary for the hacking hobby/trade we all love.

Having fun, perseverance and realistic expectations are your only pre-requirements for success.

Carrion:

Carrion, developed by Phobia Game Studio and published by Devolver Digital is a horror title where the roles are reversed, it's a fun game where you're tasked with terrorizing a facility in which you were trapped into.

This game is, unfortunately, not available in our platform of choice, can it be made to run on these anyways? We'll tackle this game as a case study in order to present a novel case study for which there isn't a whole lot of documentation for.

Understanding the problem.

A game is software which iteratively simulates a set of rules, and then presents these updates to the players and/or spectators. Your average video game software is, in simple terms, built out of a "game loop":

• Update game
  • Poll for inputs
  • Simulate the next state
    • Physics
    • Game Rules
    • <..>
  • Present the next state
    • Play audio and music
    • Render graphics

There are multiple ways developers approach this problem, such as engines (e.g. gamemaker, unity, etc, where this is heavily abstracted from you), frameworks (e.g. XNA/FNA, PyGame, LOVE2D, where this is reasonably abstracted from you) and Hardware Abstraction Layers (such as GLFW, LWJGL, SDL2, where this is lightly abstracted from you).

Understanding how the puzzle pieces go together is important, in order to gauge hardware/software requirements and the work necessary to fill in the missing puzzle pieces. Let's open the game's folder and just get a feel for it.

From previous experience, this folder structure stands out to me for a few reasons:

• Presence of a folder name "Content/" (Common for games based on XNA, FNA and MonoGame)
• FMOD Libraries are used for audio (usually an interesting pain point...)
• A Carrion.dll.config file with no Carrion.dll¹ anywhere (probably not a CIL/.NET Assembly, this is bad.)
• A Carrion.runtimeconfig.json file, containing the line "tfm": "netcoreapp3.1", (it's a .NET Core 3.1 application.
• A worrying lack of the many .NET .dll files that usually accompany games compiled to CIL instead of Native Code.

If you search for modding resources for C# Mono/.NET games, you'll find that tools such as Harmony, ILSpy, DNSpy and MonoMod are incredibly useful tools for understand and modifying the behavior of said software.

We're going to analyze the files we have available using ILSpy:

This won't do... Maybe this is some kind of self-contained deployment?

Nope. Our only available file is a precompiled CoreRT binary, this will only run on it's native platform or via emulation. This won't do...

Let's go for the next approach: SteamDB, an interesting website with a lot of information on games available on Steam such as lists of games per technology stack, and most importantly, it has plenty of information on the game and the available Depots on Steam.

SteamDB seems to agree with our guesswork - Carrion is likely to be a MonoGame title, but having to run this through a layer such as Box86 has a significant impact on lower end platforms.

Let's continue looking, maybe there are interesting files on other published depots?

If you look around the depots, you'll quickly notice most of them are published on the same manner... However there is one platform that stands out, Windows 32bit.

The file Carrion.dll and multitude of .NET Assembly DLLs on that folder suggests that there might be a .NET Assembly for this game available on that specific depot. We're going to open the Steam Console and download it.

Let's test our hunch by opening the Carrion.dll file on ILSpy:

Great! Our game is available in a moddable and hackable form. Let's check what's the software stack for Carrion:

Notable information you can see from this listing:

• Our runtime is .NETCoreApp 3.1 and this runtime is available for Linux ARM64.
• FMODWrapper is some kind of library to interact with FMOD from .NET. (We will get there in time, this will be a subquest of it's own.)
• MonoGame.Framework, as we've previously stablished.
• There are no apparent Steam API calls (probably no DRM, nice!²)

This is an interesting candidate. Let's download the runtime, copy the files and see if it runs out of the box. I'll be doing the initial reconnaissance with a Radxa Rock 5B board running Armbian.

First we're going to copy over the game data somewhere and also extract the .NETCore3.1 runtime somewhere so we can start seeing if we're missing anything.

Interesting error. Let's look for the runtimeconfig.json documentation...

The difference here is incredibly peculiar, maybe if we edit this file...?

Note the DISPLAY=:0 variable here, it's there for a reason

Bingo! We're not there, but this is solid progress. For most projects this is where we start the fun part of poking the game with ILSpy, dnSpy, Harmony and MonoMod, but... the error message suggests that this "An attempt was made to load a program with an incorrect format", that's worrying, wasn't CIL cross-platform and portable?

Kinda.

Open Heart Surgery.

Our plans were foiled, there are project settings that causes the .NET Assemblies to be platform dependent, despite the CIL being largely platform independent. What the hell?

I could not find anything in regards to what this actually means on a conceptual level. It's reasonable that a library wrapper that interfaces with native code (such as a FMOD Wraper...) has a specified TargetPlatform that severely narrows down the acceptable environment it wants to execute in.

But conceptually, it's still just CIL code right? Maybe we can just work around the incompatibilities as we go? I'm sure this limitation is reflected somewhere because the runtime (and ILSpy) has to actually query this information somehow. Let's open our patient's chest.

The document above gives us a good overview of the metadata contained on a .NET Assembly. One such interesting structure is the The .NET Directory which describes the metadata that is specific to CLI that is relevant to our problem, such as the CLI Header³, is there anything interesting in those members...?

Does this MEAN what I think it does? What are the odds unsetting this specific bitflag just... allows it to continue? Let's find where the scalpel goes. Can we locate this CLI Header on the file? A few searches away and you can find this interesting article on the PE Format.

4.3. Converting from “Raw Address” to “Virtual Address” and back

Tasks that will frequently appear are conversions from “Raw Address” (how bytes are aligned in the file) to “Virtual Address” (how bytes are aligned in the memory) and back, using Relative Virtual Address (RVA).

That's... a lot. To simplify this:

• RVA is where something should go in memory.
• Raw Address is where something is in the PE file.

So, to find where CLI Header is in the file:

• CLI Header Raw Address = Raw Data + (CLI Header RVA − Section RVA)

There are many tools that can analyze PE Files (including ILSpy itself). I'm going to use a different one (Python's pefile) here just for illustration.

~/Carrion$ python3 -m pefile gamedata/FMODWrapper.dll
----------DOS_HEADER----------
<...>
----------PE Sections----------

[IMAGE_SECTION_HEADER]
0x178      0x0   Name:                          .text
0x180      0x8   Misc:                          0x47AE8
0x180      0x8   Misc_PhysicalAddress:          0x47AE8
0x180      0x8   Misc_VirtualSize:              0x47AE8
0x184      0xC   VirtualAddress:                0x2000        <-------------- Section RVA
0x188      0x10  SizeOfRawData:                 0x47C00
0x18C      0x14  PointerToRawData:              0x200         <-------------- Raw Data
0x190      0x18  PointerToRelocations:          0x0
0x194      0x1C  PointerToLinenumbers:          0x0
0x198      0x20  NumberOfRelocations:           0x0
0x19A      0x22  NumberOfLinenumbers:           0x0
0x19C      0x24  Characteristics:               0x60000020
<...>

----------Directories----------
<...>
[IMAGE_DIRECTORY_ENTRY_COM_DESCRIPTOR]
0x168      0x0   VirtualAddress:                0x2008        <-------------- CLI Header RVA
0x16C      0x4   Size:                          0x48

So, our address on the file should be something like:

• 0x200 + (0x2008 − 0x2000) => 0x208

Now, to find the flags, we do some back-of-the-napkin math. For this refer to the documentation on what are the Data Types used by the Win32 API, and also for other fields which we don't know of.

    DWORD                   cb;                  // +4 bytes
    WORD                    MajorRuntimeVersion; // +2 bytes
    WORD                    MinorRuntimeVersion; // +2 bytes
   
    // Symbol table and startup information
    IMAGE_DATA_DIRECTORY    MetaData;            // +8 bytes
    DWORD                   Flags;               // Flags Relative Offset = 4+2+2+8 = 16 bytes (0x10)
                                                 // Final Offset = 0x208 + 0x10 = 0x218

So, 0x208 + 0x10 = 0x218. Let's make a backup of the file and open it in a Hex Editor of our choice, and then navigate to the offset we calculated above.

Notice the flag is currently set to 3, in binary:

00000000.00000000.00000000.00000011
                                 ||
                                 |-- COMIMAGE_FLAGS_ILONLY
                                 --- COMIMAGE_FLAGS_32BITREQUIRED

If you unset the COMIMAGE_FLAGS_32BITREQUIRED flag (There are programming calculators that lets you do this visually), our new value will be 1. Change that in the file and save it.

Now let's run the application again.

We got a new error, System.DllNotFoundException: Unable to load shared library 'libfmod.so.10.9' - which means FMODWrapper.dll was loaded successfully and tried finding FMOD, which promptly fails. The next steps are providing FMOD and other random dependencies that the game might require, and maybe resolve incompatibilities that might arise.

Footnotes

1. .dll/.exe file extensions are PE files. Those can contain Native code and/or .NET Assembly/CIL. We want a PE File with CIL available, because unlike native code those are actually cross-platform. ↩

2. DRM-free gives you a lot of freedom in how you choose to consume media you paid for. Acknowledge and respect this decision by supporting the developers. Don't ever take this for granted. ↩

3. Annoyingly, this structure has many aliases, such as COR20 Header, CLR Header or COM Descriptor. ↩

02-Porting-FMOD-Studio1x.md

Ooops, no FMOD Studio!

In a previous episode, we've managed to boot Carrion mid-way through and removed some architecture-dependent flags from certain Assemblies. This isn't enough to get a fully working game.

FMOD is a widely used, proprietary sound system created by Firelight Technologies Pty Ltd. Thankfully the runtime is available for download on their website, problem is, the specific version we need isn't available for our platform.

Can we use a newer version, one that is available for Linux ARM64, with older sound banks?

Skin Grafts and You.

There are ample methods which are available for you to mod .NET games, for this project we're going with the ever so popular MonoMod. We're using it to hook, instrument and compose functionality that is either missing, changed or requires some level of tweaking.

First, let's open the game on ILSpy and search for the affected areas of code.

If you read through, you'll notice that this is just a function that loads the FMOD dynamic libraries for you. If we're going to modify the code that loads this library, this is as good a hooking spot as any. Refer to the MonoMod documentation if you need to get a feel for how this works.

Now that we have an idea of what our approach is, let's create a .NET Library project named "FMODWrapper.mm" targetting netcoreapp3.1 for AnyCPU, and we're going to add dependencies on FMODWrapper and MonoMod, let's see if we can get the program to print Hello World before ForceLoadLibs runs by instrumenting it:

using System;
using MonoMod;

[MonoModPatch("FMOD.FMODInit")]
public static class FMODInitMod
{
    // We need an early enough point for us to be able to change how the loading works
    public extern static void orig_ForceLoadLibs();
    public static void ForceLoadLibs()
    {
        Console.WriteLine("Hello World!");
        orig_ForceLoadLibs();
    }
}

We're going to take the resulting DLL and use it to patch our FMODWrapper.dll:

~/Carrion$ dotnet/dotnet monomod/MonoMod.dll gamedata/FMODWrapper.dll
MonoMod 22.7.31.1
[MonoMod] Reading input file into module.
[MonoMod] [Main] Scanning for mods in directory.
[MonoMod] [ReadMod] Loading mod dir: /home/johnny/Carrion/gamedata
[MonoMod] [ReadMod] Loading mod: /home/johnny/Carrion/gamedata/FMODWrapper.mm.dll
No usable version of libssl was found
Aborted

The hell does that mean? If you search google you'll find that older versions of dotnet wants to load older versions of libssl... The suggestion to downgrade is dangerous so we're not going down that road, instead, we're going to just take those files from an older debian package without installing it.

We'll extract those files (you can use 7zip or similar for this) and point dotnet to them, in the Shared Library HOWTO there's an old trick for this called LD_LIBRARY_PATH.

~/Carrion$ LD_LIBRARY_PATH=libs.arm64/ dotnet/dotnet monomod/MonoMod.dll gamedata/FMODWrapper.dll
MonoMod 22.7.31.1
[MonoMod] Reading input file into module.
[MonoMod] [Main] Scanning for mods in directory.
[MonoMod] [ReadMod] Loading mod dir: /home/johnny/Carrion/gamedata
[MonoMod] [ReadMod] Loading mod: /home/johnny/Carrion/gamedata/FMODWrapper.mm.dll
[MonoMod] [Main] mm.AutoPatch();
[MonoMod] [AutoPatch] Parsing rules in loaded mods
[MonoMod] [AutoPatch] PrePatch pass
[MonoMod] [AutoPatch] Patch pass
[MonoMod] [AutoPatch] PatchRefs pass
[MonoMod] [PostProcessor] PostProcessor pass #1
[MonoMod] [Write] Writing modded module into output file.
[MonoMod] [Main] Done.

We can copy the MONOMODDED_FMODWrapper.dll file generated and overwrite our FMODWrapper.dll file for testing, we'll make a copy of the original file first, and use this to automate the process with this little bash script:

# Set DISPLAY and LD_LIBRARY_PATH for all processes ran by this script
export DISPLAY=:0
export LD_LIBRARY_PATH=libs.arm64/

# Restore, patch and overwrite
cp gamedata/og_FMODWrapper.dll gamedata/FMODW rapper.dll
dotnet/dotnet monomod/MonoMod.dll gamedata/FMODWrapper.dll
cp gamedata/MONOMODDED_FMODWrapper.dll gamedata/FMODWrapper.dll

# Run the game
dotnet/dotnet gamedata/Carrion.dll

Let's save it as patcher.sh and run it:

Got 'em! We're also copying the FMOD and FMOD Studio libraries into our libraries folder. Is there any way for .NET to look for alternative paths for these libraries? If you search for this, you'll run into the Native library loading documentation, which starts with an interesting tidbit:

This article explains which paths the runtime searches when loading native libraries via P/Invoke. It also shows how to use SetDllImportResolver.

What is this SetDllImportResolver? Let's read the documentation.

Sets a callback for resolving native library imports from an assembly.

Interesting, let's use this in our mod.

    // We want to be able to control the version of FMOD we're resolving
    static IntPtr DllImportResolver(string libraryName, Assembly assembly, DllImportSearchPath? searchPath)
    {
        string newlibraryName = libraryName.Replace(".so.10.9", ".so.13.6");
        return NativeLibrary.Load(newlibraryName);
    }

    // We need an early enough point for us to be able to change how the loading works
    public extern static void orig_ForceLoadLibs();
    public static void ForceLoadLibs()
    {
        NativeLibrary.SetDllImportResolver(Assembly.GetExecutingAssembly(), DllImportResolver);
        orig_ForceLoadLibs();
    }

Run this and... Failure?

Unhandled exception. System.DllNotFoundException: Unable to load shared library 'libfmod.so.13.6' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibfmod.so.13.6: cannot open shared object file: No such file or directory

Follow their advice, set LD_DEBUG=libs and read the logs, notice:

   1400162:       trying file=libs.arm64/libfmod.so.13.6.so

Fun, it's looking for a file on the current directory and it's not finding it... What if we use absolute paths for LD_LIBRARY_PATH? Let's change it to: export LD_LIBRARY_PATH=$(pwd)/libs.arm64/ and run the script again:

Unhandled exception. Monster.Sounds.SoundException: FMOD error: ERR_HEADER_MISMATCH
   at Monster.Sounds.SoundSystem.AssertThrow(RESULT result)
   at Monster.Sounds.SoundSystem..ctor(EngineSystem engine)
   at Monster.EngineSystem.LoadingProcess()+MoveNext()
   at Monster.LoadingScreen.Update(GameTime time)
   at Monster.EngineSystem.Update(GameTime gameTime)
   at Microsoft.Xna.Framework.Game.DoUpdate(GameTime gameTime)
   at Microsoft.Xna.Framework.Game.Tick()
   at Microsoft.Xna.Framework.SdlGamePlatform.RunLoop()
   at Microsoft.Xna.Framework.Game.Run(GameRunBehavior runBehavior)
   at Monster.Program.Main(String[] args)

More challenges. Let's look for Monster.Sounds.SoundSystem..ctor on ILSpy, this should be the constructor for SoundSystem.

Let's rewrite that function so that we change the Header reported to FMOD by FMODWrapper, we can achieve this by modding StudioSystem.create as follows:

[MonoModPatch("FMOD.Studio.StudioSystem")]
public class StudioSystemMod
{
    [DllImport("libfmodstudio.so.13.6")]
    extern static RESULT FMOD_Studio_System_Create(out IntPtr studiosystem, uint headerversion);

    new public static RESULT create(out StudioSystem studiosystem)
    {
        RESULT rESULT = RESULT.OK;
        studiosystem = null;
        //rESULT = FMOD_Studio_System_Create(out var studiosystem2, 69632u);
        rESULT = FMOD_Studio_System_Create(out var studiosystem2, 0x00020206);

        if (rESULT != 0)
        {
            return rESULT;
        }
        studiosystem = new StudioSystem(studiosystem2);
        return rESULT;
    }
}

Now we get:

Unhandled exception. System.EntryPointNotFoundException: Unable to find an entry point named 'FMOD_Studio_System_GetLowLevelSystem' in shared library 'libfmodstudio.so.10.9'.
   at FMOD.FMOD_API_LINUX.FMOD_Studio_System_GetLowLevelSystem(IntPtr studiosystem, IntPtr& system)
   at FMOD.Studio.StudioSystem.getLowLevelSystem(FMODSystem& system)

The Changelog in FMOD's documentation tells us FMOD_Studio_System_GetLowLevelSystem was renamed to FMOD_Studio_System_GetCoreSystem, we add another patch to StudioSystemMod:

    new public RESULT getLowLevelSystem(out FMODSystem system)
    {
        system = null;

        IntPtr systemPtr = default(IntPtr);
        RESULT result = FMOD_Studio_System_GetCoreSystem(rawPtr, out systemPtr);

        if (result != 0)
        {
            return result;
        }

        system = new FMODSystem(systemPtr);
        FMODPatch.system = system;
        return result;
    }

It won't build. We need to find a way to access rawPtr, the MonoMod documents tells us to extend the class we want to change, so let's do that:

[MonoModPatch("FMOD.Studio.StudioSystem")]
public class StudioSystemMod: FMOD.Studio.StudioSystem
{
    public StudioSystemMod(IntPtr raw) : base(raw) {  }

    <...>

We go a bit further now, the game successfully boots into the content selection screen:

Getting the game to run properly means doing this over and over again, until no more inconsistencies exist. You can take the FMOD Headers for 1.x and 2.x and compare them, all of the changes are likely to cause problems. As an example, there's a new argument added to FMOD_Studio_System_SetListenerAttributes on FMOD 2.x, and we need to take that into account:

    // Missing `attenuationposition` causes missing sound effects...
    new public RESULT setListenerAttributes(int listener, FMOD.Studio._3D_ATTRIBUTES attributes)
    {
        return FMOD_Studio_System_SetListenerAttributes(rawPtr, listener, ref attributes, IntPtr.Zero);
    }

Figuring this out took hours of clicking around ILSpy trying to understand how positional sound and occlusion is implemented on Carrion, finding setListenerAttributes and noticing that the implementation details for them differ on both version:

- FMOD_RESULT F_API FMOD_Studio_System_SetListenerAttributes(FMOD_STUDIO_SYSTEM *system, int index, FMOD_3D_ATTRIBUTES *attributes);
+ FMOD_RESULT F_API FMOD_Studio_System_SetListenerAttributes(FMOD_STUDIO_SYSTEM *system, int index, const FMOD_3D_ATTRIBUTES *attributes, const FMOD_VECTOR *attenuationposition);

Once you're done fixing the differences and the game works, you might want to look for means to redistribute your mod. There are examples of tooling that achieves something to this sort. It's also important to know if you can actually redistribute those libraries at all.

03-Thank-yous.md

Finishing words

That was a lot, we've seen a wide range of topics such as CIL Internals, MonoMod patching, etc. All of those tools are incredibly useful in many other contexts. Those skills apply to legacy software maintenance (we're kinda doing this, no?), software testing (test injection via instrumentation) and others.

I hope you enjoyed this read as much as I'm enjoying this project. Remember to support the developers. A thank you to the many folks responsible for what you've seen in this article:

• Phobia Game Studio: For developing and releasing this great game as a DRM Free product.
• Firelight Technologies: For FMOD's ample documentation, samples and great support.
• MonoGame Foundation: For developing and maintaining the cross-platform MonoGame project.
• Many Articles & Documentation Authors: For freely sharing with the necessary knowledge to succeed.
• ILSpy: For developing tooling that allows us to poke at the internal of .NET applications.
• Portmaster Community: For all the banter and sharing the love for homebrew and indie on cheap devices.
• You: For sitting through this article to it's end, kudos.