Orders from the Command Bunker #3 : Emulation Station Metadata
Do you want your games to display interesting content within the Emulation Station user interface (UI) vs. a simple list of titles?
If something like this interests you, here's some helpful guidance to get you moving in the right direction.
1) Download one or more themes you find appealing. Via the Emulation Station menu, here's how to download additional themes beyond the stock 'Carbon' version. There's actually quite a few choices when it comes to themes, as well as an active community constantly creating new variants. Try some out and see what best suits your aesthetic taste.
1A) Go to the Retropie Menu.
1B) Choose ES Themes.
1C) Select and install any themes you want. You can also uninstall themes from this menu as well. Once done, back out of the menu and return to Emulation Station.
2) To activate your new theme, simply do the following:
2A) Press Start (Main Menu).
2B) Select UI Settings.
2C) Select Theme Set and toggle any value you want to try out.
2D) Select Back and Restart Emulation Station.
3) Now, let's talk about metadata, or more specifically, the content you want to display along with your roms, such as release date, number of players, rating, publisher, etc. All of this content comes from an XML file located within your Pi's file system. This file is called a 'gamelist', and you will learn to hate them.
Why? Because filling out content in said gamelist is both tedious and time-consuming. Luckily, there is automation to assist with this process, but it is far from perfect. Depending on your level of OCD, even with good automation, you may still end up spending weeks to months cleaning and formatting your collection's metadata.
If you SFTP into your Pi, the gamelists (as in 1 for each core/system you emulate) are found in the following location:
HOME > PI> .EMULATIONSTATION>GAMELISTS>(Your Core/System)
So, for NES, the location would look like this:
HOME > PI> .EMULATIONSTATION>GAMELISTS>NES
Within the NES folder would be your NES' gamelist in an XML format. Though you have lots of options to open the XML file, I highly recommend you install a free 3rd party program known as 'Notepad ++'. This will allow you to more easily view the contents of this file and edit it. Plus, it's a handy program to have. You can get the program here.
Here's a sample of what an entry to the gamelist would look like. Keep in mind, each rom/game will have an entry on this list. So, if you have 1500 NES roms, you'd have 1500 log entries. If you had 10 cores/systems within your collection, all of which have 1000 roms, you can see how overwhelming this task can become.
You'll note the entry both starts off with and ends with same <game> format. This is important because it tells Emulation Station how to aggregate the content on a per rom/entry basis. Also note that each sub-field kicks off the content via a <field> and ends with </field>, where the ending has the forward slash at the start of the field name (i.e. <name>Duck Hunt (USA)</name>). This basically tells Emulation station where any given field's content starts and ends. If you violate this format, your gamelist will not display any content, so be forewarned, XML is very high maintenance.
Also, keep in mind, depending on your selected theme and desired level of bells-and-whistles, your actual gamelist may have, or be missing, several entry fields. In my example, I don't do thumbnails or video previews, though the principle is the same for any metadata content. Let's go over some the fields, and their respective quirks:
Path refers to the location of the rom. Typically, all roms are stored in your Pi's ROMS directory (unless you are using an external storage device). Simply having the format as './rom name' is fine. Just make sure your rom's file extension is correct and note the field is case-sensitive.
Name refers to the title you want the game to display as within the Emulation Station UI. You can write pretty much whatever you want here, including region or sub-title, or even translated title if the game was never released in English.
Desc or Description can be used as a text summary of your rom. You may use spaces, most punctuation (. or : or ! or ?), most symbols (like + or -) and line breaks. Just be mindful that the Pi cannot read ampersands (&). If you must use an ampersand, it must use this format (&). For example, T&E Surf Designs would actually be logged as:
<desc> T&E Surf Designs</desc>
Also, be aware that some automated scrapping services will grab non-Latin characters, such as Japanese text. I ran into a lot of display issues with this, so my recommendation is to simply delete this content from the entry.
Image refers to box art. You can use either PNG or JPG format, where the tradeoff will be image quality vs. size. It's really your call, but I ended up using PNG, despite the larger file sizes, as the crispness and clarity of the images compared to JPG were worth it in my opinion. Note that by default, the scrappers will slug in the suffix '-image' at the end of the box art entry. You don't have to keep that naming convention, but most of the automation is based on using it. Also note that the file extensions are case-sensitive, so Emulation Station won't display .PNG if the gamelist lists .png. I will prepare another, more in-depth article regarding box art, so I'll table further discussion on this topic for now. Check back for 'Orders From the Command Bunker #4' for additional guidance.
Rating refers to your personal rating of said rom. You can scrape aggregated community ratings for games if you wish, or simply rate everything yourself. Keep in mind the rating format is a 2 digit decimal (i.e. .75 would be 75% or not quite 4 out of 5 stars) within the emulation station UI.
Releasedate or Release Date is the original publication date of your game. Note the default format is YYYYMMDD. You can delete or ignore the timestamp. Even then, depending on the game, accurate day and month values may not always be possible to populate. I ended up using 0's for those entries (i.e. <releasedate>19920000</releasedate>).
Developer refers to the game's creator/designer, assuming it is known. Again, this is one of those fields where the automated scrappers tend to breakdown and come up with blank entries.
Publisher refers to the game's publisher, assuming it known. Again, this is one of those fields where the automated scrappers tend to breakdown and come up with blank entries.
Genre refers to the generic category of the game, such as racing, adventure, action, etc. Depending on the automated scrapping tool used, some designations are nonsensical or questionable. For example, one tool labeled the NES title 'Duck Hunt' as a puzzle game. Obviously, these designations are subject to interpretation. You can make up your own or use handy 'find and replace' functions within Notepad ++ to suit your specific tastes. You could even add more than 1 designation if you wanted (i.e. <genre>Action / 2D Platformer</genre>).
Players refers to the number of players the game allows. Some scrapping tools will display content as '1 to 2' or something other than a simple number. Depending on the theme you're using, it may not be possible to display any values that are greater than 2 digits, so keep that in mind. I ended up using the 'find and replace' function within Notepad ++ to simply change all values into straight numbers. Also, keep in mind the players designation is also open to interpretation somewhat, as many automated scrapping tools will count 2-player alternating games as 2 players. For instance, Super Mario Bros. 3, where Mario and Luigi take turns playing the game. While technically accurate, you may not feel that classification is representative compared to traditional, simultaneous 2-player games, such as Contra. To the system's credit, you can customize the fields and content as you deem appropriate, so keep that flexibility in mind for this field.
So, what are your options regarding populating this content into the gamelist? While you have several automated scrappers to chose from, my personal preference is an external 3rd party program known as the Universal XML Scrapper. You can get it here.
Why do I feel this is the superior option, compared to Emulation Station's built-in scrapper, or the equally popular Steven Selph's scraper? Simple - convenience and on-par, if not better accuracy. Unlike those other options, which are Pi-based, the Universal XML scrapper is PC-based, so I can do the work directly on my computer, with a mouse, via an easy-to-follow UI. To me, the ability to easily scrape, view, edit, and SFTP the content to/from the Pi via my PC was a no-brainer. I should warn you though, no scraping tool is perfect. If pressed for an estimate, I would say depending on how your rom set is named, the tool will likely only recognize about 80% of your games for automated matching. The scrapping tools do not seem to know that 'Double Dragon 2' is the same game as 'Double Dragon II', or 'The Adventures of Gilligan's Island' is the same game as 'Gilligan's Island'. They all seem to have issues scraping Japanese games as well. Even within matched games, sometimes sub-fields are simply omitted, like publisher, release date, number of players, etc. At the same time, you can run and re-run your rom set into the universal XML scrapper as many times as you want, with the system generating a log of 'failed to identify' games via a text file. In the end, you will likely end up just manually key-entering the content for these straggler games. Great sites for metadata include Moby Games and Giant Bomb, so make sure to favorite them.
Here's how you run the Universal XML Scrapper:
1) Launch either the Silent .exe or the normal Universal.exe. As the name implies, the silent version does not have the annoying startup song. I don't why the developers of this otherwise great program would think people would want to hear that crap.
2) Select Configuration > Launch Wizard.
3) Select Retropie.
4) Select your default visual choice (Simple vs. Complex, where complex is a mixture of images).
5) Select your 1st and 2nd preference for box art for each entry, assuming it can be located (Screenshot vs. Box Art vs. Title Screen vs. 3D Cover).
6) Select the location of your roms directory (Pi or PC). Note the program wants to use the core naming conventions used on the Pi's 'roms' directory, so name your folders in the same fashion - NES, SNES, Megadrive, NEOGEO, etc.
7) The system will ask you if you are registered with Screen Scraper, which is another site, and if not, would you like to register. Note, you can decline both questions and still use the program.
8 ) Confirm you want to scrape and wait for it to finish. Obviously, the more content you ask to run, the longer it will take.
9) Once completed, the system will dump the completed content, both gamelists and box art (known as downloaded images) into the following directory: C> Configs> All> Emulationstation.
I hope you found this article helpful. Collecting and editing the metadata for your rom collection is quite an undertaking. My advice is to keep focused, take regular breaks, and remember to check your XML for errors via one of the numerous free XML valuators available on the web. It took me 6 months to complete this task, using a mixture of both automation, key-entry, and editing/corrections. Was it worth the effort? I think so. It's like having an interactive video game encyclopedia on my Pi, or unearthing a time capsule straight from my childhood. If something like that sounds appealing to you, then have at it.
Emulate This is not sponsored, nor were we compensated by any of the following parties: Notepad ++, Retroarch, Emulation Station, Universal XML Scrapper, Moby Games, or Giant Bomb. We are retro gaming fans - and only wish to share our love of this hobby with others.