Reverse Engineering RG35XX Stock Firmware

A friend gave me an Anbernic RG35XX to hack. This is a retro gaming device (it just means that it is designed to emulate old consoles). It comes with two OS: Stock OS and Garlic OS. Stock OS is closed source, and Garlic OS is open source, except for the kernel part (all are closed source). You can switch from one OS to another via a menu.

Stock OS starting my custom binary.

In my opinion, the stock OS is fast and quite user-friendly but is not customizable, although many people like Garlic OS more because it can emulate more systems.

Kernel part

Anbernic won’t release the source for ATM7039S, and no datasheet is found for this SOC. The stock RG35XX OS uses a slightly different kernel compared to the Garlic OS.

Someone was able to compile the kernel from an old S500 device and have the GPU to work. Koriki for RG35XX was based on this kernel, but from the information in Discord, the latest Koriki release uses a stock kernel.

There is no serial port accessible and no debug interface available, so trying to hack the kernel will be a painful experience.

Stock RG35XX boot sequence

The kernel is stored as a uImage file on the first partition (FAT32). The built-in bootloader (u-boot) will boot load this file, and it will mount ramdisk.img. Inside ramdisk.img , we can find: /init, /init.rc, The kernel will start /init, which is based on Android init (it uses bionic libc). /init will load /init.rc, and on the last lines, it contains instructions to start

service loadapp /system/bin/logwrapper /
    class core will load /system/dmenu/dmenu_ln. The dmenu_ln can be found on the second partition (ext4), and this is just a shell script that will start /mnt/vendor/bin/dmenu.bin that can also be found on the second partition.

dmenu.bin is the main shell for the OS. This is written in C using SDL1.2, but it uses custom input handling instead of using SDL_WaitEvent.

Custom Input Handling

Some people swear that the input handling in the Stock RG35XX OS is faster than the other alternative OS. I can’t feel it, but the Stock OS does process input events manually.

To reverse engineer how it works, I use Ghidra. Since this is not security-related software, there is no protection or obfuscation so the code can be decompiled quite cleanly.

Reverse engineering

It starts by opening /dev/input/ to find a device that has a name: gpio-keys-polled (this name is obtained using ioctl call with request EVIOCGNAME). Then, it will start a thread (using pthread) to poll this device. The power button is a separate device from all other buttons, and the reset button (under the power button) is hardwired to reset the console.

Emulator modification

Inside appres/bin/game on the second partition, we can see several binaries for each emulator. All of them have been modified by Anbernic:

  • They use custom error handling
  • The menu button is set to display the menu (so all emulators have the same interface)
  • Added Video filter effect (such as dot-matrix) implemented in C (not using GPU)

Compiling for RG35XX stock OS

Usually, we will need an SDK to compile an app, but since we know the target architecture, calling convention, and the libraries used, we can work around this problem. To compile a simple SDL app that will run on the Stock OS, we will need a compiler, header files, and some libraries.

For the compiler, download Linaro toolchain 4.7 (closest to existing binaries on the system) from here (choose gnueabihf):

For the headers, download the latest SDL1.2 and use the default SDL config. And for the libraries, we can use files from /lib on the second partition. Remove and, these two are bionic files and will cause errors. Then, add files from usr/local/lib/arm-linux-gnueabihf (also from the second partition).

Then, you should be able just to compile everything manually.

Outputs to stdout/stderr will not be visible, so use dup2 to redirect these to files.

Small Demo App

In this repository, you can see my small demo app. I included all the libraries to make it easy for anyone to start (please change CC path in Makefile to your installation directory).

This is a very simple app to replace dmenu.bin (please rename the original dmenu.bin to orig.bin), it only provides three functions:

  • Testing key events
  • Starting ADB (useful for transferring files and debugging), I Included my own which needs to be copied to the same location as dmenu.bin
  • Starting the original launcher (now named orig.bin)

I am not planning to develop this. Maybe someone can make a better launcher based on this.