Updated guide on how to patch the game using the GL6 repository files Part 2 (Dumping the ISO)

I actually took a look at the Growlanser 1 PS1 and PSP game contents, to see what kind of issue other people might have had to translate the game and i came across the problem of file compression. As far as i can tell, both the PS1 and PSP version use some kind of compression algorithm to reduce the filesize of the games ISO.
Because the games (Growlanser 5 and 6) .DAT files do not use any kind of compression we were able to dump the scripts of the game rather easily.

Heres a quick overview on how to compile all of the repository scripts into the game files.

1. Dump the ISO using "Xpert2"
2. extract the contents of the .DAT files with "quickBMS" using the Growlanser script
3. use "abcde" to re-insert the modified script files into the games .SCEN/.SCEC/.STXT files
4. use "quickBMS" to reinsert the modified files into the games .DAT files
5. use "Xpert2" to create a new ISO with the modified files

After dumping the ISO using Xpert2 or some other program you will have the following (on both Growlanser 5 and 6):

SLPM_667.16 is the ELF of the game
SYSTEM.CNF some game config file
DUMMY.DAT dummy file containing dummy data
GL6_BTL.DAT no clue about that one
GL6_CHAR.DAT seems to store the character model files
GL6_FACE.DAT probably contains all of the games face image files
GL6_FILE.DAT contains all of the image files (for example, the games character menu) and some other stuff
GL6_MAP.DAT Seems to contain the map data, also has a file that has all of the different map information stored in it
GL6_MOV.DAT contains alls of the movies without sound
GL6_SCEN.DAT contains all of the games script files
GL6_SND.DAT contains the sound files
GL6_VOI.DAT contains all of the voice data


1. First you have to download Xpert2 and use it to dump the ISO file (a little bit out of date guide but its still pretty much the same):

https://growlanser6english.blogspot.com/2019/03/growlanser-6-unpack-iso.html


2 Download quickBMS and extract the it in a folder, put the Growlanser quickBMS script into the same folder.
The quickBMS Growlanser 5/6 script can be found here:

https://bitbucket.org/Risae/growlanser-6-english-translation/src/master/GL%205%20and%206%20quickBMS%20scripts/


3. Use quickBMS to dump the contents of "GL6_SCEN.DAT":

https://growlanser6english.blogspot.com/2019/03/growlanser-6-unpack-iso.html

All of the contents of SCEN.DAT will be extracted from the .DAT file.
Now you can pretty much do the following

- We have the repository folder that contains the abcde script files (/GL6_SCEN.DAT/)
- We have a folder that contain the extracted GL6_SCEN.DAT (game) files
- We have the original GL6_SCEN.DAT file

We use abcde to import the repository script files into the original .SCEN files.
And after that we use quickbms to import the modified .SCEN files to the original .DAT files.
(And after that you just have to use armips to import the variable width code into the ELF file)

Repository script (abcde) > original .SCEN/SCEC/STXT (quickBMS) > original GL6_SCEN.DAT (Xpert 2) > dump the iso and switch out the files, rebuild the ISO

Updated guide on how to patch the game using the GL6 repository files Part 1 (Overview)

Over the last year the way we compile our repository scripts changed a little bit, so heres an updated How-To guide on how to compile the repository scripts into the games .DAT files.
But first, a quick overview on the different tools we use for our project:


- PCSX2
https://pcsx2.net/
I personally use the PS2 emulator to test the game


- PCSX2 debugger
a useful debugger for checking the MIPS code that is being processed in the background while the game is running.


- PCSX2dis (ps2dis)
https://forums.pcsx2.net/Thread-PCSX2dis-v1-1-A-ps2dis-inspired-PCSX2-enabled-Game-Hacking-Tool-W-I-P-13-02-2015
Another useful tool:
"PCSX2dis is an ambitious new hacking tool based on old hackers' favourite ps2dis. Created for advanced hackers and beginner/intermediate hackers who wish to learn ASM hacking, this is an all-purpose tool fixed to a modded version of PCSX2, designed to make PS2 game hacking easier than ever, all in realtime!"
Afaik, only works with PCSX2 1.2.1.


- bitbucket
https://bitbucket.org/Risae/growlanser-6-english-translation/src/master/
We are using the repository tool "Bitbucket" to share and store our project.
It contains all the games script files and also files for various software that we are using.
We are using GitHub Desktop to sync our repository files to our local hard drive:

https://medium.com/@rvrajavijay/how-to-use-bitbucket-repository-in-the-github-desktop-fae2ef61135d

With this you can download the repository to your local computer and work on the files in there.
Any changes you make you can "commit" to the repository for all of the team to access.
The changes will be documented and everybody can see what changed.
A lot easier than working on the scripts files inside the bitbucket web view.


- abcde (Cartographer and Atlas)
https://www.romhacking.net/utilities/1392/
abcde is a tool to dump (Cartographer) and re-import (Atlas) the Growlanser 6 script file text.
I already dumped all of the script files that exist in Growlanser 6, so currently we only use its Atlas function to reinsert it into the games files.
The control codes for abcde are stored inside "/GL 5 and 6 abcde scripts/abcde.tbl" and so far i think i have all of the ones that are used inside the table file.


- quickBMS
https://aluigi.altervista.org/quickbms.htm
quickBMS is a tool to dump and reinsert game specific files.
For example, Growlanser 6 uses ".DAT" archives that stores the game files.
We are using quickBMS to dump all of the files that are inside the .DATs using a custom script which is saved in "/GL 5 and 6 quickBMS scripts/growlanser.bms"


- Xpert 2
http://xpert2.blogspot.com/
Xpert 2 is a tool to dump the contents of the games ISO and rebuild it.
Since the filesizes of the modified .DAT files are going to be bigger than the original, Xpert lets us create a new ISO with the bigger files.


- Apache2.0
https://www.psx-place.com/resources/apache.697/
Apache was another ISO tool to dump and hotswap files inside the games ISO.
Because of the fact that most of the game files are going to be bigger in size we sadly can't use it anymore.


- deltapatcher
Deltapatcher is a useful tools to create patched for the ISO, but this only works if the modified ISO is not bigger in size than the original.
Because of the new quickBMS script we don't have any script size limitation anymore, but this also means that we can't use deltapatcher because the filesize of the original ISO will increase.


- rainbow
https://github.com/marco-calautti/Rainbow
Rainbow is a tool to view the TIM2 (.tm2) image files


- Ghidra with the Emotion Engine processor
https://github.com/NationalSecurityAgency/ghidra
https://github.com/beardypig/ghidra-emotionengine
Ghidra lets us decompile the "ELF" file (SLPM_667.16).
The ELF file contains various text that only can be translated in there, but its almost all been translated.


- armips
https://github.com/Kingcom/armips
We are using armips to quickly modify the ELF file, mainly for the Variable Width Font code.


- TileShop
https://www.romhacking.net/forum/index.php?topic=30404.0
A tool to view the Growlanser 5/6 font files.
Uses XML files to tell the program how the font file is build up.


- wxMEdit
https://wxmedit.github.io/
A bit outdated Hex editor which supports Shift-JIS.

No more size limits with our script files!

The last few weeks i have been working a lot on inserting the Growlanser 6 Realm script into the games script files.
Sadly the script only has the english translation without the japanese original, so i have been playing through the game side by side translating the strings.
It's a bit tedious, but i have been making a lot of progress and i am about half-way through the script.
But, after working on a lot of different script files i always came across the issue that our script is a lot bigger than the original.

The translated script gets put back into the games .FLK file using the tool called "abcde".
Whenever the filesize, because of the re-import, got bigger than the original, i wasn't able to use quickBMS to re-insert the .FLK file into the games GL6_SCEN.DAT file.
I had to manually "trim" the script files and delete various things, for example the debug text or squish some dublicate text into one text pointer.
Doing that takes a lot of time, so i have been looking for ways to be able to expand the script files without having any space issues.
And lo and behold, quickBMS actually implemented such a feature last year!:

From version 0.10.0 QuickBMS has an additional mode called REIMPORT3,
it's identical to REIMPORT2 with the only difference that the archive
is shrinked or enlarged if there is no offset field used in the archive
and the size of the input file differs than the original.
This method "may" be useful with some language files and some archives
with sequential data.

I immediatly gave it a try and put all of the original GL6_SCEN.DAT files into one folder, inserted all of the scripts into the original .FLK files using abcde and reimported it with quickbms reimport3.
All of the files are properly imported and are in the right place, just the GL6_SCEN.DAT file is a little bit bigger now!
Doing a little test ingame also didn't show any problems with the translated text. Everything seems to be in the right place.

With that, we now can:

- Download the repository files

- insert the script files into the .FLK files using abcde bulk import

- reimport all the modified + unmodified GL6_SCEN.DAT files into the .DAT

- Dump the ISO with Xpert and switch out the files, rebuild it with Xpert

In a matter of minutes! No more tedious trimming of the script files because of size issues!

Sadly, we are not able to use the cool tool called "Apache2" anymore, since the original size of  GL6_SCEN.DAT increased as a result. Which also means right now we cannot create an xDelta patch.

But not having a size limit in the script files is a really big help in being able to create a proper translation.

Update:
Meh, all the files that are not increasing in size work without a problem, but the ones that do pretty much break the game:

Update 2:

So, after consulting with the super hacker Ethanol Pwned he found out that

I figured it out! After blankly staring at hex for too long I figured this thing out.

Remember all those silly spaces at the end of every file? They are CRITICAL, all of them pad out the files for them to snuggly fit inside an LBA (0x800 chunks that divide the game disc logically) the game will simply refuse to read in-between LBA's for speed (and because they have ~4GB to work with anyways :P) as such, if the file doesn't start at a disc location where 0xLocation % 0x800 == 0 the game will just discard the part that isn't and will do direct copy starting from a location that is multiple of 0x800.

THAT'S why it breaks when the changes are big enough, but leaves a little wiggle room where if small enough it can work by pure luck, and that's also why the defined .DAT size isn't really honoured.

Every .DAT file is like this and that explains why the .flk's inside were multiples of 2, now I haven't tested this yet, but a comparison of two instances of the game running confirms this suspicion (and also confirms that the game can handle relocating bigger files, assuming enough memory is available) so we only need to automate this.

And lastly some funny observation, I actually pointed at something like this in a previous email, past me basically had it resolved xd

And it seems like another scripting God on the quickBMS forums was able to create a quickBMS script that does exacly that and the game so far runs perfectly:

https://zenhax.com/viewtopic.php?f=13&t=13696

So, heres hoping i'm not coming across any more crashes, and if that is the case we finally have a way to increase the script size without a worry in the world!

This is what happens when the game doesn't like you.

It trolls you into writing and deleting random letters...

Equipment Window now properly translated and GL5 VWF table found!

Dear readers,

i already made a post in the Romhacking thread (https://www.romhacking.net/forum/index.php?topic=30284.0), but i also want to make a new post here since its been  some time.

Thanks to the big help of "Ethanol Pwned" we are now able to modify the Y position of various text windows!
He made a really cool tutorial on how the text window is build up:

https://gist.github.com/Mc-muffin/5d121fa01d3be9199e6f66f853b6ff4b

Thanks to that, i was able to make sense of the Assembly code and change the values for those windows:

Before and after:

There are a few more windows that need to be worked on, for example the selling window:

and the spell casting message:

Those seem to have slight variantions in regards to their Y position code, but i believe i can figure out how the code is build up and change those too.

And, the even bigger news is that Ethanol Pwned took a closer look at the Text Routine of Growlanser 5 and managed to find the VWF table!
He too believes that the english publishers made a VWF for the game and because both Growlanser 5 and 6 are 90% identical in terms of game code and its engine, it should be possible to port it over to Growlanser 6.

The location of the VWF Table is 0x314C68 inside the GL5s ELF:

I stumbled upon this spot in the past already, but i didn't believe that it would have anything to do with the text.
The width values are 1 byte each, and to get to the letter that you want to modify you just have to add the hex value of the letter to 0x314C68.
As a little test i ported the GL6 font back into GL5 and changed all the values to see what happens:

Porting the GL5 font over to GL6 is now one step closer.
Big, big thanks to Ethanol Pwned for taking the time to help with this and being able to figure out so much in such a short time!

We are still looking for Japanese > English translators. If you would like to help us translate Growlanser 6 to english, please let us know!
You can check out our current progress on our repository:

https://bitbucket.org/Risae/growlanser-6-english-translation/src/master/