Flash: Direct Memory Access via Native Extension

A couple weeks before the Flash Gaming Summit I found an interesting blog post from Colin Northway about managing texture memory in Starling applications. In this post he describes how he forces an error in ActionScript to receive an object's memory address which he then uses as a unique id. I thought this was a pretty cool trick and was discussing it with our engine developer at the office when we suddenly had a thought: if you can get the memory address of an object in ActionScript, what's to stop you from passing that address into native code and doing whatever you want with it?

As it turns out, nothing. Fair warning: this is an advanced topic that only a select few will be interested in and even fewer will find useful. It was mainly a fun little hacking experiment with Flash.

To be perfectly honest, I don't know what sort of repercussions this may have as far as security is concerned. However, it was too exciting at the time to just ignore it. Without going into too much detail, this piece will describe the overall process and demonstrate a simple example using an array. To begin with, you need to know how to get the memory address from Flash. You can get this by performing a bad cast in AS3. I used the function from the post as a starting point and modified it slightly to give the address as a uint instead of a string. Just a quick note here: this does not work for the basic data types (int, number, boolean, string); it only works for complex objects. That said, there are a ton of objects it does work with including arrays, sprites, movieclips and even functions!

Now that you have the address you can send it down to native code. Obviously you'll need a function to accept the address and to do something with it. And because we don't know what Adobe's data types look like under the hood (yet), this will take a lot of poking around. By looking at the data in memory, and changing things, you can begin to find out more about these types. As mentioned previously, in this example we'll take a look at an array object. For now, the native function for peeking at the data is pretty simple:

FREObject accessMemory(FREContext ctx, void* funcData, uint32_t argc, FREObject argv[])
{
    uint32_t flAddress;

    FREGetObjectAsUint32(argv[0], &flAddress);

    uint8_t* ptr = (uint8_t*)flAddress;

    int x = 0;
    return argv[0];
}

In this case the function takes in just one parameter (from Flash), so argv[0] is the memory address of the object we want to look at. After placing the value into a compatible C data type, we create a byte (uint8_t) pointer to the passed in address. The next line where we set x to 0 is just there so I can add a breakpoint and examine the code after the pointer has been initialized. In order to hit the breakpoint, you'll need to know how to attach the visual studio debugger to a running process. If you don't know how to do that, you can check out my previous tutorial on debugging native extensions.

Once you've attached the debugger to the process, you need to view the memory window. You can do this in Visual Studio by going to Debug > Windows > Memory > Memory 1 as shown in the image below.

I didn't know about the memory window myself until our engine developer showed me. I think its pretty cool, and also handy, so don't be afraid to check it out. Once the memory window is open, you'll be able to see the ActionScript objects in memory. For example, when my breakpoint gets hit, I can see the memory address of my array.

As you can see in the image above, my byte pointer is pointing to the address 0x050b8281 so I can type that address into the top portion of the memory window and it will display what is in memory at that location. Click the image below to see the details.

Undoubtedly, this looks mostly like a garbled mess of numbers and occasional letters. However, by making slight changes to the data passed in and observing how the memory changes, we can begin to decipher some useful information. I'll give you a chance to see for yourself. I passed in two different arrays: the first had 3 elements, the second had 4. Below are the memory windows corresponding to each.

The highlight gives it away, but just by looking at the data you could probably have determined that the length of the array shows up multiple times. You wouldn't know this at first, but after passing in a few different arrays of varying size, you'd start to see the pattern. So with that knowledge, we can now trick Flash into believing our array is actually a different length than it really is. All we have to do is overwrite the length value in memory. To do so, we just need to modify our native function from earlier:

FREObject accessMemory(FREContext ctx, void* funcData, uint32_t argc, FREObject argv[])
{
    uint32_t flAddress;

    FREGetObjectAsUint32(argv[0], &flAddress);

    uint8_t* ptr = (uint8_t*)flAddress;

    ptr += 23;
    *ptr = 8;

    ptr += 4;
    *ptr = 8;

    ptr += 4;
    *ptr = 8;

    int x = 0;
    return argv[0];
}

What we've done above is trick our array into thinking that its length is 8. In reality, the array's length is 4, but if we run this native function and trace the array.length property in Flash, it will give us 8. Ok, so this isn't entirely useful, but it is shown as an example of accessing an ActionScript object directly in memory and modifying it. And in that regard, it is decidedly a success!

Having direct memory access to objects means you have pretty much unlimited control. There are endless possibilities with what one could do with this if one is driven enough, and talented enough, to find their way around the various ActionScript objects on the native side. As for myself, I may be neither, but it was definitely a fun little experiment and I learned a bit about memory and walking through it in the process.

Bookmark and Share

3 Responses to “Flash: Direct Memory Access via Native Extension”

  1. Fernando says:

    Great post, man! I could not help thinking about using that hack to manipulate Stage3D data. If one is able to map the memory used by a Quad, for instance, it could be possible (?) to use C to crunch the numbers and do the heavy work, avoiding some slow AS3 paths.

  2. Santanu Karar says:

    Pretty investigated stuff must say. Good writings, though I need to re-visit a multiple times I must say.

  3. Tap says:

    Pretty neat trick! I think ANE is going to be Flash’s saving grace in rapid development circles. I’m such a newbie that I do not yet know how to do any of this stuff, but I am wanting to learn, so I will fave this article for later.

Leave a Reply

Subscribe to RSS feed FGS5 Badge