Welcome to the Shining Force Central Forums!
SFC Forums Index Shining Forums Shining Force II
Register for your free forum account now or Login to remove this advert.

SF2 Disassembly Tool Suite

General thread for quick links, status reports and support

Discussion about this classic Genesis/Mega Drive game.

SF2 Disassembly Tool Suite

Postby Wiz » Mon Feb 13 2017 1:53pm

Here is a thread to share current releases, work in progress and the roadmap of the tools I'm slowly (but surely ! ;)) building with only one distant goal in mind : to play whole new Shining Force adventures !

A few months ago I wanted to explain the whole creation process I have in mind with tools deduced from the game's data structures, and then I got lost into real life again, so this time I'll just make a quick list of existing tools and enrich the original post later.


Just one quick detail to make things clear :
These tools are intended to be used on the game's disassembly, not on the ROM directly.
Current and future "low-level" tools may allow to work on ROMs when possible, because handling basic assets is still feasible quite easily, as long as original data length is not exceeded.
However, direct ROM hacking is not flexible enough to handle "high-level" data structure editing.
This is why most future and more complex tools will work exclusively with a disassembly.
Some people may find it harder to use than just hacking a ROM file, but this is well worth the effort.
The sonic hacking community is the best example there is to show how powerful this approach can be.
With the disassembly, anything will be possible. We're the developers now ! :)


Disassembly quick start guide :
WARNING: SPOILER!


Tools :
Please note that all of these tools also work as Java APIs for anybody to include them in their own tools.

SF2TextManager : https://github.com/ShiningForceCentral/SF2TextManager/releases/latest
WARNING: SPOILER!

SF2VWFontManager : https://github.com/ShiningForceCentral/SF2VWFontManager/releases/latest
WARNING: SPOILER!

SF2PaletteManager : https://github.com/ShiningForceCentral/SF2PaletteManager/releases/latest
WARNING: SPOILER!

SF2GraphicsManager : https://github.com/ShiningForceCentral/SF2GraphicsManager/releases/latest
WARNING: SPOILER!

SF2IconManager : https://github.com/ShiningForceCentral/SF2IconManager/releases/latest
WARNING: SPOILER!

SF2MapSpriteManager : https://github.com/ShiningForceCentral/SF2MapSpriteManager/releases/latest
WARNING: SPOILER!

SF2BackgroundManager : https://github.com/ShiningForceCentral/SF2BackgroundManager/releases/latest
WARNING: SPOILER!



TODO roadmap with future tools to give a global view of the tool suite architecture.
Last edited by Wiz on Mon Jul 17 2017 1:38am, edited 11 times in total.
Wiz

User avatar
Member
Member
 
Posts: 83
Joined: Sun Mar 18 2007 2:43pm

Re: SF2 Disassembly Tool Suite

Postby SirHedge » Mon Feb 13 2017 7:13pm

Is there a good online tutorial on how to read and reassemble disassembly?
SirHedge

User avatar
Shining Hero
Shining Hero
 
Posts: 825
Joined: Sat Mar 15 2008 4:10am

Re: SF2 Disassembly Tool Suite

Postby Wiz » Mon Feb 13 2017 7:57pm

Good question, SirHedge !
It's not so intuitive after all :) ... and I must admit I didn't take the time to write it down in details.
Currently the only details I remember are in SF2DISASM's main README, here :
https://github.com/ShiningForceCentral/SF2DISASM/blob/master/README.md
Re-assembly : By providing the original US rom file of the game, one will be able to use the provided tools in order to :

with split/split.bat, split the original rom file into a lot of small binary data files, to extract data chunks which are not included in the disassembly
with build/build.bat, assemble the game from its disassembly and the extracted data chunks The assembled game will be perfectly identical to the original one.


Let's see if these more detailed instructions are good enough :

1. Get SF2DISASM :
- Get the latest revision with the repository's download link :
https://github.com/ShiningForceCentral/SF2DISASM/archive/master.zip
(In a near future there may be proper releases here : https://github.com/ShiningForceCentral/SF2DISASM/releases/latest)
- Unzip it anywhere you want
(not sure if space characters in path will work though)

2. Split :
- Put the original US ROM in BIN format (not SMD) in the split/ directory, with the following filename : sf2.bin
- Then run split/split.bat. This may take from a few seconds to around a minute.
This will split the ROM's original data in thousands of tiny binary files in the disasm/ directory.

3. Build :
- Also put sf2.bin into the build/ directory, as a comparison reference for new builds.
- Run build/build.bat. This may take from a few seconds to a few minutes.
This will assemble the game from the content of the disasm/ directory only.
With the original content of disasm/, the resulting ROM file is supposed to be a bit-perfect replica of sf2.bin, hence the comparison at the end of the build to make sure of SF2DISASM's integrity.
When disasm/'s content has been modified, the build will naturally notice differences compared to sf2.bin and you can ignore it.

4. Play with your disassembly !
According to your understanding of how ASM 68K works and how data is organized, you can modify the game's content step by step and see if/how it builds.
Just be aware that extensive modifications (with more content than original content) imply to extend the ROM's layout from a 2MB ROM to a 4MB one, and this involves a bit of ASM hacking which should be covered in a future tutorial, I guess. :)



Please guys tell me if that's enough instructions here, and then I'll put it in the original post.
Wiz

User avatar
Member
Member
 
Posts: 83
Joined: Sun Mar 18 2007 2:43pm

Re: SF2 Disassembly Tool Suite

Postby greyskies » Tue Feb 14 2017 9:41am

Any hints what the "Graphics Manager" is going to be? :D
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
greyskies

User avatar
Shining Member
Shining Member
 
Posts: 355
Joined: Wed Feb 02 2011 11:55am
Location: Seebenstein, Austria

Re: SF2 Disassembly Tool Suite

Postby Wiz » Tue Feb 14 2017 10:29am

:shifty: This is the generic low-level layer for read-decode/encode-write functions, for any type of graphical data, compressed or not.
This layer manipulates graphics data as an array of 8x8 tiles associated to a palette loaded with SF2PaletteManager, which is included as an API.
This array will then be transmitted to future higher-level tools which will arrange/display/export/edit graphics in a more specific context, with the corresponding tile layout :
SF2IconManager, SF2TilesetManager, SF2SpriteManager, SF2BattleSpriteManager, SF2PortraitManager, SF2BackgroundManager, SF2SpecialScreenManager ...
... which then could be used by highest-level tools to display a whole map, cutscene, battle with SF2MapEditor, SF2CutsceneEditor, SF2BattleEditor, SF2BattleSpriteAnimationEditor, SF2SpellAnimationEditor, SF2ItemEditor, SF2CharacterEditor ...
But we're still very far from that, and I'm really slow compared to rubixcuber or BNC, so we're talking about years of good use of my precious free time here.
Don't expect anything near the Caravan's possibilities anytime soon ! ;)

Currently SF2GraphicsManager just reads uncompressed data, which was quite easy since it's just 4BPP with nothing special. I might put it in good use in an SF2IconManager as a proof-of-concept first, and then get into compressed graphics, which I expect to be quite hard to figure out and implement :damnit: ...
Wiz

User avatar
Member
Member
 
Posts: 83
Joined: Sun Mar 18 2007 2:43pm

Re: SF2 Disassembly Tool Suite

Postby greyskies » Tue Feb 14 2017 3:27pm

I see! Yeah, I know what you mean, it took me years just to get a hack finished. Just know that your effort is being appreciated ;)
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
greyskies

User avatar
Shining Member
Shining Member
 
Posts: 355
Joined: Wed Feb 02 2011 11:55am
Location: Seebenstein, Austria

Re: SF2 Disassembly Tool Suite

Postby SirHedge » Mon Feb 20 2017 2:14am

Thanks for the further information Wiz. I was actually talking about understanding what each chunk actually means, though. For example, is there something I can read that would explain what each of these lines mean?

Code: Select all
sub_234C8:
      
      movem.l d0-d2/a0-a1,-(sp)
      move.b  ENTITYDEF_OFFSET_FACING(a1),d6
      ext.w   d6
      move.b  byte_2353E(pc,d6.w),d6
      bne.s   loc_234DA
      addq.w  #2,d6
SirHedge

User avatar
Shining Hero
Shining Hero
 
Posts: 825
Joined: Sat Mar 15 2008 4:10am

Re: SF2 Disassembly Tool Suite

Postby Wiz » Tue Feb 21 2017 12:42am

Oh I see, you mean how to read assembly language, ASM 68K here.

I think I'll do some research to get the best introduction resources I can find about :
- A general overview of assembly language : the CPU's basic instructions to manipulate registers and memory locations.
- A specific overview of the Megadrive's hardware architecture around it's CPU : Motorola 68000.
- And then a reference to ASM 68K's language in the Megadrive's context : specific registers and memory addresses.
- ... And then, I should do a specific introduction to SFII's ASM, with a global view of the game's execution flow (main flow and technical interrupts after every frame), and detailed examples.

... actually, this deserves its own thread ! Not sure about when I'll have the time to do that, but I'll definitely do it if it can help people not to get too intimidated by the technical aspects of the disassembly !
Wiz

User avatar
Member
Member
 
Posts: 83
Joined: Sun Mar 18 2007 2:43pm

Re: SF2 Disassembly Tool Suite

Postby SirHedge » Tue Feb 21 2017 5:03am

I probably don't have time right now to understand a full rundown, unless it is as complex as I expect it to be (though I won't say no to it being available -- I just don't want to feel guilty that I'm not taking advantage of it immediately). What I'm looking for now is more just enough to make basic sense of what I'm looking at and the ability to identify simple changes. For example, I would like to make it so that items always cracked on first use, but auto-repaired after battle. The first would be reasonably simple to do, I would think, and for all I know the second may not be as difficult as imagined. But getting some frame of reference for things like that would be helpful. Hopefully that makes sense.
SirHedge

User avatar
Shining Hero
Shining Hero
 
Posts: 825
Joined: Sat Mar 15 2008 4:10am

Re: SF2 Disassembly Tool Suite

Postby Wiz » Sat Jul 15 2017 3:51pm

Quick bump to let you guys know that SF2GraphicsManager has been released with version 1.0.0 :
https://github.com/ShiningForceCentral/SF2GraphicsManager/releases/tag/1.0.0

Image

Original post updated.

Most graphics can now be edited with the proper use of this editor, which can be more or less tedious depending on what you want to edit and how.
This is why I'm starting to develop more specific tools for specific graphic objetcs : SF2MapSpriteManager and SF2IconManager already exist, and others will follow :
- SF2PortraitManager
- SF2BattleSpriteManager
- SF2WeaponSpriteManager
- SF2BackgroundManager
- SF2GroundManager
- SF2SpecialScreenManager (title, witch, etc.)
Then I'll think about map tilesets and map content more globally, when the time comes.

For curious people, here are the graphics compression details :
- Stack compression : https://github.com/ShiningForceCentral/SF2DISASM/blob/master/disasm/data/graphics/compression.txt
- Basic compression : https://github.com/ShiningForceCentral/SF2DISASM/blob/master/disasm/data/graphics/mapsprites/mapsprites.txt
Wiz

User avatar
Member
Member
 
Posts: 83
Joined: Sun Mar 18 2007 2:43pm

Re: SF2 Disassembly Tool Suite

Postby greyskies » Sun Jul 16 2017 10:00am

Could you give a more detailed (read: more noob-friendly) description what this does and how to work with it? :)

For example, if I wanted to pull out the endingkisstiles-"picture" of the ROM as displayed in the image above, what would I have to do? I am assuming "entering the correct palette/graphics offset data" before importing, but if so, how do I know these?
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
greyskies

User avatar
Shining Member
Shining Member
 
Posts: 355
Joined: Wed Feb 02 2011 11:55am
Location: Seebenstein, Austria

Re: SF2 Disassembly Tool Suite

Postby Wiz » Mon Jul 17 2017 2:14am

greyskies,

With this tool there are two approaches :
- Either keep the classic ROM approach and apply specific offsets and lengths, which you have to know indeed.
You can find this kind of information from this ROM map, for example :
https://github.com/ShiningForceCentral/SF2DISASM/wiki/ROM-Map
But this ROM Map is not exhaustive enough to give you the offset and length of every piece of compressed tiles individually, so this method may be tedious ...
- Or use a disassembly as explained in my first post, and open binary files containing graphics data.
Once again, you have to know which files contain graphics, and if they are compressed or not, so this still implies some technical knowledge, but well, let's summarize the main graphic assets :
Icons are not compressed, map sprites use basic compression, and almost everything else use stack compression : map tilesets, title screen, witch screen, battle sprites, battle scene backgrounds, grounds, weapons, etc.

But even when you know this, another aspect comes into play : tile layout.
Depending on graphic assets, the tiles will not be arranged in the same layout.
This is something that SF2GraphicsManager is not managing, as it only loads graphics as an array of tiles.
You can input an image witdh (in number of tiles), but in most cases this will not be enough, as tile layouts can be quite complex.
Battle scene background layout for example, arranges its 384 tiles in 4*4 blocks and then arranges these blocks in 4*3 screen halves :
WARNING: SPOILER!

... hence what you get when loading a background from SF2GraphicsManager :
(tricky use of the ROM import for a disassembly binary file, as background files actually contain more than a compressed tileset : 3 pointers, one palette, 2 tilesets)
Image
You can always export those tiles in PNG files if you're able to work on them like that, but that's not really practical. :D

Actually, the ending kiss graphics are just a lucky find : they are stored in a very simple way, which explains why they appear properly arranged here.

... so yeah, unless you know what you're doing, you may prefer to wait for more specific tools ! :)
Just be aware that these specific tools will be disassembly-oriented only.

And talking about specific tools ... I just uploaded SF2BackgroundManager. OP updated ! :)
Image
Last edited by Wiz on Mon Jul 17 2017 1:10pm, edited 1 time in total.
Wiz

User avatar
Member
Member
 
Posts: 83
Joined: Sun Mar 18 2007 2:43pm

Re: SF2 Disassembly Tool Suite

Postby greyskies » Mon Jul 17 2017 9:56am

welp, then I guess I will have to learn how this disassembly thing works sometime. Thanks for your explanation!

Also, it is great to see that you're on a SF2-tool-creation roll! :)
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
.
Check out Return to Grans - A Shining Force II hack which features a completely
reworked story, numerous visual improvements, and tons of balance changes.

Also available as storymode hack only.
greyskies

User avatar
Shining Member
Shining Member
 
Posts: 355
Joined: Wed Feb 02 2011 11:55am
Location: Seebenstein, Austria


Return to Shining Force II

Who is online

Users viewing this topic: No registered users and 0 guests

cron