MAMORY

Rom Management Library

Pierre Mazière


Table of Contents
Introduction
I. The Mamory CLI : mamory
1. Source of roms information
2. Parsing GamesSrc
3. Getting options list
4. Dumping GamesSrc to file
5. Selecting storage mode
6. Ignoring nodump roms
7. Adding sources to GamesSrc
8. Comparing different GamesSrcs
9. Dumping standard output in a file
10. Synchronizing a roms set with GamesSrc2
11. Deleting incomplete games
12. Backing up the modified roms
13. Testing synchronization
14. Exporting a roms diff set
II. The Mamory library: libmamory

Introduction

Mamory is a rom management library. It allows developpers of emulation related projects to access useful functions in order to maintain roms sets. It's based on the comparison on CRC32, size and name of the roms. Therefore, it needs a source of informations that describes the roms set to maintain.

The library is called "libmamory". To demonstrate the functionalities offered by libmamory, Mamory is distributed with a command line interface (CLI) called "mamory".


The Mamory CLI : mamory

This should be the part that most people will be looking for, at least until libmamory is used through the graphical user interface of a generic roms management application. I will try to explain how to use the Mamory CLI in order to manage a set of roms corresponding to any kind of emulators.


1. Source of roms information

To be able to determine if your files are correct roms, mamory need to have one or several sources of informations. More precisely, it needs at least the name of games and the name, the CRC and the size of the roms composing these games. the CLI can also use informations about dependencies of a given game set for another one. In the rest of this document these sources of information will be named "GamesSrc"

These sources can be of 3 kinds:

  • executable. some emulators have option that prints useful information about the roms they need to do their job; for example, MAME (Multi Arcade Machine Emulator) prints games name, roms name, roms CRC, roms size and many more information through the option "-listinfo" and "-listxml". for the moment, mamory only support XMAME executables.

  • file. certain web sites provide files that give the required roms informations; these files generally use one of the following format: ClrMamePro, RomCenter . For the moment, mamory can only understand the ClrMamePro format.

  • directory content. when you give a directory as GamesSrc, mamory will look for files and treat them as potential roms: it stores names, CRCs and sizes. for the moment, mamory can only handle zipped files, but in the future, it will be able to look for raw files and for sudirectories. the name of a .zip file is considered as the game name, and the files zipped into as the roms.


2. Parsing GamesSrc

the simplest mamory usage

When GameSrc is given as single argument to mamory, this source is parsed. If the source is valid, mamory will output the number of RomContents on the standard error stream.

# mamory /usr/local/bin/xmame.x11-0.60.1
Mamory v0.1.3 (c) 2002 Pierre Mazière

Extract Games Infos from xmame (X11) version 0.60 (Oct 24 2002) ... done
	Nbr of RomContents: 32231
# 

The number of RomContents is the number of roms that have a unique non null CRC. When GamesSrc is a ClrMamePro or RomCenter format file, this number excludes roms that need to be redumped (stored with a null CRC in the file). It gives you an indication on the size of your set. As several games can use the same rom under different names, it's almost certain that your actual roms set contain more files than RomContents.

This argument will always be considered as your main source of roms information. For the moment, mamory can only handle one main source of roms. In the common use of mamory, this main source is generally the directory where you store your roms. If you store your roms in several directories(one for NeoGeo, one for CPS2 and one for the rest), you will have to maintain these directories individually until mamory supports multi directory roms sets.


3. Getting options list

option "--help" ( "-h" )

To have the list of the existing mamory options, you have to use the classical "--help" option (short option "-h").


4. Dumping GamesSrc to file

option "--list-games file" ( "-l file" )

If you want to get the informations of any GamesSrc, you can use the option "--list-games" (short option "-l"), followed by the name of the file where you want to dump the informations. This file will be in what I call the "Advanced Mame -li" format, which is compatible with MAME -li option output and ClrMamePro with additional data usefull to mamory.

Examples :

  • mamory -l ./ExecGamesSrc /usr/local/bin/xmame.x11

    will dump the xmame games informations into the file "ExecGamesSrc"

  • mamory -l ./FileGamesSrc "./MAME v0.62(cm).dat"

    will dump the games informations of the ClrMamePro format file "MAME v0.62(cm).dat" into the file "FileGamesSrc"

  • mamory -l ./DirGamesSrc /usr/local/games/MAME/

    will dump the games informations of the files contained in the /usr/local/games/MAME directory into the file "DirGamesSrc"


5. Selecting storage mode

option "--storage-mode" ( "-m" )

With version 0.2.0, Mamory allows to choose how roms sets are stored:

  • SPLIT storage mode: original games and their clones are stored in their respective slots (zip or directory) with their exculsive roms.

  • MERGE storage mode: original games and their clones are stored in the same slot (zip or directory).

  • FULL storage mode: original games and their clones are stored in their respective slots (zip or directory) with all the necessary roms.

In order to select one of these modes, you have to use the "--storage-mode" option (short option "-m") followed by one of these three explicit keywords: split, merge or full. All GamesSrcs will be converted to the corresponding mode before any other actions. Therefore, if you use the following command:

mamory -m merge -l GamesSrc_merge GamesSrc

where GamesSrc is described as a SPLIT or FULL storage mode, GamesSrc_merge will describe the same roms set in the MERGE storage mode

WARNING: The default value for this option is SPLIT


6. Ignoring nodump roms

option "--ignore-nodump" ( "-i" )

No dump roms are data that have not been dumped. They usually are associated with Null CRC value, and if they exist in a set, are filled with junk data. With this option, you choose to ignore the lack of these files: they won't appear as missing.


7. Adding sources to GamesSrc

option "--additional-sources GamesSrcA:GamesSrcB:..." ( "-a GamesSrcA:GamesSrcB:..." )

If you want to virtually add roms informations that comes from other source to the main GamesSrc, you can use the option "--additional-sources" (short option "-a"), followed by the sources you need to add, separated by a colon ':'.

Example :

if you have your MAME roms set in several directories, such as ./main, ./NeoGeo and ./CPS2, and you want to dump the roms informations into the file "TotalSet", you'll have to type the following:

mamory -l ./TotalSet -a ./NeoGeo:./CPS2 ./main

These additional sources will be considered only as source of information: it will never be modified in any way by the synchronization feature of mamory (cf. I-8). Only the GamesSrc given as main source of information can be modified, in the case of a directory.


8. Comparing different GamesSrcs

option "--diff-with GamesSrc2" ( "-d GamesSrc2" )

In order to know if your roms set is conform to a theoretical roms set, you can use the option "--diff-with" (short option "-d" ), followed by the theoretical roms set description named hereafter GamesSrc2. This will print, on the standard output, the modifications you have to bring to your roms set in order to synchronize it with the theoretical roms set. More generally, this option is used to compare two roms sets.

Example :

if you want to compare the roms set located in /usr/local/games/MAME with the last version of xmame, you'll have to type the following:

mamory -d /usr/local/bin/xmame.x11 /usr/local/games/MAME/

The output will look like the following:

game1_name:
        rom1_name (CRC: XXXXXXXX) -> CHANGED NAME FROM rom1_oldname
        rom2_name (CRC: XXXXXXXX) -> TRANSFERED IN
        rom3_name (CRC: XXXXXXXX) -> TRANSFERED IN AND CHANGED NAME FROM rom3_oldname
        rom4_name (CRC: XXXXXXXX) -> TRANSFERED OUT
        rom5_name (CRC: XXXXXXXX) -> UNUSED
        rom6_name (CRC: XXXXXXXX) -> MISSING

game2_name:
        ...

which means:

CHANGED NAME FROM :

the rom exists in the existing game1_name.zip file, but needs to change its name to rom1_name

TRANSFERED IN :

the rom exists in another zip file, and need to be imported in game1_name.zip file

TRANSFERED IN AND CHANGED NAME FROM :

the rom exists in another zip file but under another name, and need to be imported in game1_name.zip file under the name rom3_name

TRANSFERED OUT :

the rom is not used in this game, but exists in others and can be removed from this one

UNUSED :

the rom is not used in any games of this set and can be deleted from the set

MISSING :

the rom is needed in this game, but cannot be retrieved in any other zip file


9. Dumping standard output in a file

option "--output file" ( "-o file" )

To dump the standard output into a file, you can use the option "--output" (short option "-o"), followed by the name of the file to create. This can be used to store the result of the "--diff-with" option.

Example :

mamory -o output_file -d /usr/local/bin/xmame.x11 /usr/local/games/MAME

will store the result of the comparison between the roms set located in the directory /usr/local/games/MAME and the roms set provided by /usr/local/bin/xmame.x11, into the file "./output_file"


10. Synchronizing a roms set with GamesSrc2

option "--synchronize" ( "-s" )

To maintain your roms set uptodate with a theoretical GamesSrc, you can use the option "--synchronize" (short option "-s") concurently with the option "--diff-with". It will perform the modifications indicated by the "--diff-with" option into the directory provided as main GameSrc. For the moment, the synchronization can only be performed on a single directory. I hope to change this, and support multidirectories synchronization.

After the comparison, mamory will evaluate the required disk space and ask the user if he decides to perform it or to abort. If the user choose to continue, the zip files will be modified in order to reflect the modifications indicated by the "--diff-with" option.

At the end of the synchronization process, the modifications that are still needed (in fact missing roms) are printed.

During the synchronization, the roms that are removed from filename.zip file are saved in a zip file called ".mamory_filename.zip" in the same directory as the main GamesSrc. If mamory segfaults for any reason, the currently modified game will still be in the /tmp directory, and the ".mamory_filename.zip" will have to be removed.

Examples:

  • mamory -s -d /usr/local/bin/xmame.x11 /usr/local/games/MAME/

    will synchronize the /usr/local/games/MAME roms set with the informations provided by /usr/local/bin/xmame.x11 using the SPLIT storage mode

  • mamory -m merge -s -d /usr/local/bin/xmame.x11 /usr/local/games/MAME/

    will synchronize the /usr/local/games/MAME roms set with the informations provided by /usr/local/bin/xmame.x11 using the MERGE storage mode

  • mamory -s -d /usr/local/bin/xmame.x11 -a ./new-roms /usr/local/games/MAME/

    will synchronize the /usr/local/games/MAME roms set with the informations provided by /usr/local/bin/xmame.x11 using the SPLIT storage mode. The ./new-roms and /usr/local/games/MAME/ informations are joined in a virtual GamesSrc. This virtual GamesSrc is compared to the /usr/local/bin/xmame.x11 informations, and the roms missing in /usr local/games/MAME are imported from the ./new-roms directory.


11. Deleting incomplete games

option "--remove-incomplete-games" ( "-r" )

Some roms may be shared by several games, including games that are not part of your own collection. In that case, the zip files corresponding to these games will be created and will contain the shared roms. Using the option "--remove-incomplete-games" (short option "-r") with the "-s" option, those games and all games whose rom set is not complete according to the GameSrc2 given by option "-d" will be deleted. As a side effect, it means that when you are updating your rom collection using the merge storage mode, if roms of the original and the clones are not all available, then the whole set will be deleted.


12. Backing up the modified roms

option "--backup-path BackUpPath" ( "-b BackUpPath" )

If you want to backup the zip files that are modified during the synchronization, you can use the option "--backup-path" (short option "-b"), followed by the path where the backup directory have to be created. The directory "mamory_roms_backup-YYYYMMDDhhmmss" will be created in the directory "BackUpPath". The modified zip files will be copied in this directory before to be modified.


13. Testing synchronization

option "--test" ( "-t" )

If you're not confident in mamory, you can test the synchronization before performing it. You only need to use the option "--test" (short option "-t"). It will not modified any files of the main GamesSrc. Because of the way mamory perform the comparison, it needs some disk space. Therefore, the required disk space evaluation will not give you a null value.

Example :

mamory -t -s -d /usr/local/bin/xmame.x11 /usr/local/games/MAME/

will test the synchronization of the /usr/local/games/MAME roms set with the informations provided by /usr/local/bin/xmame.x11 using the SPLIT storage mode.


14. Exporting a roms diff set

option "--export-diffset-using RomsDir1:RomsDir2:..." ( "-e RomsDir1:RomsDir2:..." )

To generate a roms set which contains the missing roms between two theoretical GamesSrc, you can use the "--export-diffset-using" option (short option "-e"), followed by the directories containing the roms, concurently with the "--diff-with" option. It will determine the roms missing between the two theoretical roms set and uses the Roms1Dir:Roms2Dir:... as sources for these roms. The resulting set will be placed in "./mamory-diffset"

Example :

mamory -e ./MAME-games-0.61 -d /usr/local/bin/xmame.x11-0.61 ./xmame.x11-0.60

will store the roms that miss between /usr/local/bin/xmame.x11-0.61 and ./xmame.x11-0.60 into the directory "./mamory-diffset", using "./MAME-games-0.61" as roms sources, in the SPLIT storage mode.


The Mamory library: libmamory

(have to be written)