Prev | Project Journal Entries | Github | Next

Completing Memory Dump and Initial Trials of LLM

Today was another productive and interesting day. I can definitely say I'm learning about LLMs already. Some key insights from today:

• Impressed and almost scared by how good Github Copilot's recommendations are.
  • I hadn't used Github Copilot before since I used Augment AI and IntelliJ in my last professional role. Now I am using VSCode and CoPilot and it was incredible how accurate CoPilot's suggestions were when both creating a JSON catalog of ram addresses for the bot to use later, and when writing LUA script code.
• It will be a big challenge to train an LLM for this specific task.
  • After having an LLM ingest both a ram catalog that attempts to detail the meanings of memory addresses in the game, and a sample of RAM from the game, it is proving very difficult to get it to do even a simple task. This will be a great challenge and learning opportunity which is exactly what I was after.
• A smaller LLM running on my local machine will not perform at speeds I am used to.
  • The speed of a local LLM may end up being a limiting factor of what the FFBot can ultimately do.
• Models are measured in Billions of Parameters with notations like "1b", "3b", "8b" etc.
  • Models with more parameters are likely to be better at understanding more complex data, but will also require more resources.

A JSON Catalog of the Game Memory

My first task today was to reverse engineer some key memory locations in the game to begin a JSON catalog the LLM can use for context. I was actually amazed as I produced that JSON file how accurately Github Copilot was offering suggestions for the descriptions of the memory locations. This RAM Catalog will need to be filled out more as time goes on but I've got a good start.

Updating LUA Daemon to Dump NES Memory

I also updated the LUA Daemon to dump the NES memory to the RAMdisk. This happens twice per second, but there was no noticeable lag or strain on the game while this was happening. The daemon reads the RAM Catalog to get the full list of memory addresses we are interested in (assumption is that any/all addresses in the catalog are of interest), reads the value at each address, and prepares a JSON object whose key is the memory address and value is the value at that address. This JSON object is saved to the RAMdisk. I did encounter an issue trying to have this JSON output be sorted properly by hex key but have put that problem aside for now. As of now, the memory dump is not sorted in any meaningful way.

Ollama: Choosing an LLM

After some initial research I found the Ollama open source LLM manager and installed it. This is a really simple tool for downloading and running open source LLMs on a local machine. After learning about how models are measured in billions of parameters I am starting off with the Llama 3.2 3b model since it requires about 2GB of memory, which my 8GB laptop should be able to handle.

Initial LLM Tests Shaky

Now that I have both a RAM Catalog and a sample of game memory in JSON format, I decided to try manually feeding that into Llama 3.2 3b and have it write me a LUA script to perform what I thought was a simple task. I wanted the LLM to "increase the strength of the character 'Glyn'". The first task of the LLM will be to underdstand which character is actually Glyn, since there are 4 characters in the party. Because the game does not use ASCII data, this task involves finding the memory locations of the bytes that store each characters name, checking the RAM dump for the values at each of those locations, and then using a lookup table provided in the RAM catalog to determine which real letters the values correspond to. For example, a "G" is represented by 0x90, a lowercase "l" by 0xAF etc. So the LLM would need to understand that memory address 0x006142 corresponds to the first letter of a name, the value at that memory location is 0x90, and 0x90 means "G" by looking it up in a table. I've spent almost two hours with the LLM and still haven't been able to get it to understand how to look up the 4 letter name from memory. In fact, it still can't seem to even do simple JSON lookups or understand how my data is structured. I will likely stop here for the day and sleep on how I might rearrange the data for the LLM or give it better instructions.

Prev | Project Journal Entries | Github | Next

Demo of Today's Accomplishments