Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Outdated / misleading information in N64 section of the site #111

Open
ethteck opened this issue Aug 19, 2023 · 2 comments
Open

Outdated / misleading information in N64 section of the site #111

ethteck opened this issue Aug 19, 2023 · 2 comments

Comments

@ethteck
Copy link

ethteck commented Aug 19, 2023

Hi! I'm the creator of splat, a tool that's commonly used nowadays to handle rom splitting and linker script generation for N64 decomp projects. I made it a few years ago after learning how n64split works and realizing that what I wanted in a tool kind of went a bit beyond what n64split is capable of. I've spoken to queueRAM, creator of n64split, about this, and he was supportive of my aspirations. The community tries to steer people away from using n64split at this point, because it's not really suitable for arbitrary projects and was designed with a specific game (sm64) in mind.

However, your site has a guide which pushes people to try creating new projects with n64split. This has sent numerous people to n64 decomp communities on Discord, asking for help with what is essentially a deprecated piece of software. We have to continuously tell them that your website is outdated and to use splat instead. I realize that it's work to keep this kind of thing up-to-date, so if you're not interested in maintaining the information here, I think it would be best if the outdated info was marked as such or removed.

Similarly, there's some info on the site about a mupen RE version, but I've tried and failed to build it, and I've seen many others try and fail to build it as well. There's also no guide on how to use it or indication that any of these RE features actually exist. I've had to explain to several people that they won't be able to use this thing on your site. I think mentions of this should be removed, as it seems unusable and is causing many people to waste their time.

I don't mean to sound unappreciative of the service you provide, but the outdated info here has been more of a hindrance than a help to both the people looking to start new projects and those of us trying to help them. If we were to make a PR replacing n64split references with a pointer to the splat repo and removing the mupen RE stuff, would that be accepted?

Also, if you'd like to discuss this in a more casual setting, feel free to reach out to me on discord. I'm ethteck there.

@RetroGameDeveloper
Copy link
Contributor

Hey ethteck! Thanks for reaching out, I was unaware of your new splat splitter which looks very useful. You raise some very good points about n64split so I have:

  • deleted the page CleanUpN64Split.md

No the N64 Reversing introduction page we have the following content:

# Part 2 - Splitting up the ROM

## Introduction to N64Split
N64 Split is a tool initially developed for Super Mario 64 for extracting files out of the n64 rom but it is now general enough to be used for any N64 game.

This is the tool we will be using to start reversing the game of your choice.

You can download N64 Split here: [queueRAM/sm64tools: Collection of tools for Super Mario 64 ROM hacking](https://github.com/queueRAM/sm64tools)

## Find existing N64Split configs
If someone has already started reversing a game you may be lucky and they could have generated a config file that we can use in N64Split:
https://github.com/queueRAM/sm64tools/tree/master/configs

## Creating a new N64Split config
If the game you want to reverse engineer does not yet have a split file config then we can create one using the following template:

# ROM splitter configuration file
name: "Your Game Name (Region)"

# checksums from ROM header offsets 0x10 and 0x14
# used for auto configuration detection
checksum1: 0xA03CF036
checksum2: 0xBCC1C5D2

# base filename used for outputs - [please, no spaces)
basename: "yourGameName.region"

# ranges to split the ROM into
# types:
#   asm      - MIPS assembly block.  Symbol names are in 'labels' list below
#   behavior - behavior script
#   bin      - raw binary, usually data
#   header   - ROM header block
#   instrset - instrument set
#   level    - level commands
#   m64      - M64 music sequence bank
#   mio0     - MIO0 compressed data block.  may have texture breakdown
#   ptr      - RAM address or ROM offset pointer
#
#   textures types:
#      rgba   - 16-bit RGBA - [5-5-5-1)
#      ia     - 16/8/4/1-bit greyscale
#      skybox - grid of 32x32 16-bit RGBA
ranges:
   # start,  end,      type,     label
   - [0x000000, 0x000040, "header", "header"]
   - [0x000040, 0x001000, "bin",    "boot"]
   - [0x001000, 0x0B6A40, "asm", "main", 0x80241800]
# Labels for functions or data memory addresses
# All label addresses are RAM addresses
# Order does not matter
labels:
  - [0x80241800, "EntryPoint"]


Save this with a filename in the form: `gameName.region.yaml` in the config folder next to n64split executable.

### Set checksum values
You will notice in the above template that there are 2 checksum addresses:

checksum1: 0xA03CF036
checksum2: 0xBCC1C5D2

These are currently set to the checksum for Mario64 which will be no good for any other game, so lets get some proper values for this!

You will need to change the checksum1 and checksum2 values to the values of your specific rom. 

You can find the values by opening the Z64 ROM in a hex editor and going to address 0x10 and copying the hex value at that location. The same for checksum2 at 0x14 in the ROM.

### Set Entry point values
In two sections of the above template we have reference to the hex address `0x80241800`.
One is in the labels section:

labels:
  - [0x80241800, "EntryPoint"]


The other is in the main range section:

   - [0x001000, 0x0B6A40, "asm", "main", 0x80241800]


 This is called the Entry Point of a game and it is start point for the game code in RAM, the value above is for the Super Mario ROM.

So we need to replace this value with the start address of the game you are wanting to reverse.

Again open your ROM in a hex editor and go to address: 0x08 and use the 4 bytes at that address to set the entry point for you rom.

### Set end of assembly
This is much trickier, so while we know that the assembly code starts at 0x001000 we don’t know where it finishes. This will require some disassembling and detective work which is what we will do in the next section so for now keep it at `0x0B6A40` which is where the initial Mario code ends. We will find the proper value for this later on.

### Run N64Split
If you have put your new config file inside the config folder beside the n64split executable then you should be able to run this:

./n64split GameName.z64


It detects the correct config file using the checksum settings we defined earlier.

If however you would rather keep the config file in a different directory you can directly specify it like so:

./n64split -c gameName.region.yaml  GameName.z64


If all goes well the output should be similar to:

root@69b99468cc18:/sm64tools# ./n64split GameName.z64
Using config file: configs/gameName.region.yaml
Splitting into "gameName.region.split" directory

ROM split statistics:
Total decoded section size:  B5A80/800000 (8.87%)


This will create a `gameName.region.split` directory, where gameName and region are replaced with your specific rom settings. Don’t worry about that low percentage of decoded section we still have a few more ticks which can help increase the percentage decoded quickly.


### N64Split Output

Lets briefly look at the output of our initial run of n64split, if you open the directory it created it contains a number of different files listed below:

Name | Description
--- | ---
Makefile | 
Makefile.split |
bin | This is where the binary blobs get extracted to, initially you will have `yourGameName.region.000040.boot.bin` which contains the boot code common in all roms
geo | geometry will be extracted here (currently none)
geo_commands.inc |
globals.inc |  This is where all the labels you defined will be placed allowing compilers/assemblers to refer to them
levels  | Game Levels will be placed here when we tell n64split how to decode the levels
macros.inc  |
models  | 3D Model data will be placed in this folder
textures  | Game textures will be placed here when we tell n64split where in the rom the textures are
yourGameName.region.ld  |
yourGameName.region.s |

### The MakeFile
The MakeFile generated by n64 split is a great start but you will notice running `make` doesn’t actually manage to rebuild the rom as certain settings will need to be changed first.

While changing these there is also a number of other modifications to do to the makeFile so I tend to just replace the entire makefile with one based on the Sm64 project. This allows us to set it up to compile with the IDO compiler initially and we can easily change to a n64 version of gcc when we find out which compiler is used for this ROM.

Here is the new MakeFile:

Notice you can now run `make` and it will try to create the ROM from the split sections, cool! 

Note that currently it will fail with lots of errors when you run `make` but this is actually good as we will cover this in the next section as it is giving us hints as to where the assembly code stops and the data section begins!

This will form the basis of your reversing project, soon it will be filled with .c files, 3d models and other data and will all be able to be rebuilt back into the original rom using `make`!

However the current content of the folder is not very useful at the moment as we haven’t actually split the rom into any proper areas yet but that is what we will do in the Next section.

So now at the end of this section you will have a folder with files that will eventually become the reversed source code of the game, we just need to tell n64split where the code, models, levels and other assets are in the game. To do that we will need to do some digging to understand the internals of your chosen game rom.

Would you be able to write the equivalent for splat and we can get rid of this n64split section?
Note that although outdated this section does give useful information that does not just apply to sm64, e.g N64SplitConfigs

If you don't have time to do that then that is understandable I will try to get some time in the near future to learn about slat and how to move n64split configs to that tool instead.

The mupen64Re version only builds on MacOS so far but I will upload a compiled binary as I have had to send it to a few people recently who don't want to build it themselves. Windows/Linux builds would be a nice thing to have but I just have trouble finding the time at the moment (Pull Requests are welcome).

Also this site is wiki-like, if you find any information you would like to change/add feel free to submit a Pull Request, we promise to be friendly :)

@ethteck
Copy link
Author

ethteck commented Sep 24, 2023

@RetroGameDeveloper thank you for the thoughtful and kind reply, and I'm so sorry it took me this long to get back to you. I think it might make more sense to link to the splat quickstart guide and docs, but if you prefer we have a mini guide on your site, I understand. I can't say writing said guide would be at the top of my to-do list, since I am always balancing a ton of things at once 😅 but hopefully someone could get to it at some point.

Thanks for addressing the mupen64Re thing as well! I'm happy to hear this tool in fact exists and does build at least somewhere. It sounds really cool. Hopefully we can get it building on other platforms at some point or upstream your changes so they're easier to work with in the main project.

Thanks again :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants