Basic Overview

Self-modifying code is a technique in which the application writes or modifies portions of it's own code at run-time.

Windows 95 has higher data protection than MS-DOS. Applications are normally given access to memory for their data segments and stack, but they are not permitted to modify their own code. In order to do this we must first ask Windows 95 for permission by calling the VirtualProtect() function. When you call VirtualProtect() you pass in the address of the first byte you want to modify, the number of bytes you want to work with and a flag indicating what you want to do with the memory (ie read it, write to it, execute it etc). You also pass in the address of a variable which the function fills with the previous protect state so you can restore it when you're done.

The following is a portion of Win32 code that demonstrates self-modifying code. The assembly statement immediately after the myloop label is "mov dword ptr a,0x12345678". The preceding statements change the op code to "mov dword ptr a,0x87654321". Try placing this code inside a C/C++ function and stepping through it with a line debugger:

LPVOID address;

// Get the address of the dword we need to change

_asm mov dword ptr address,offset [myloop+3]

// Ask windows for permission to modify the code

result = VirtualProtect(address,4,PAGE_WRITECOPY,&oldprotect);

// Modify it in assembly. This is equivelelent to *(LPLONG)address = 0x87654321

_asm mov ebx,dword ptr address

_asm mov dword ptr [ebx],0x87654321

// All done

result = VirtualProtect(address,4,PAGE_EXECUTE,&oldprotect);

myloop:

_asm mov dword ptr a,0x12345678

Creating New Portions of Code

It's also possible to generate new code from scratch. This technique is particularly handy for things such as precompiled sprites and precompiled texture mapping loops. As a simple example, I'll show how to create a function that accepts two long integers and returns the sum. For convenience I'll declare a function pointer type and a variable of that type:

typedef LONG (* FunctionType)(LONG, LONG);

FunctionType ComputeSum;

ComputeSum is a variable of type FunctionType, a type that can point to functions with the following format:

LONG Function(LONG, LONG)

The first step is to determine the actual op-codes that will go inside the function. I've already done this (which I'll show next) and it turns out that we need 11 bytes of memory to store them. How you allocate the memory for your functions is up to you, I'll use the new operator:

ComputeSum = (FunctionType) new BYTE[11];

We can now fill the array with the actual op code values:

((LPBYTE)ComputeSum)[0] = 0x55; // push ebp

((LPBYTE)ComputeSum)[1] = 0x8B; // mov ebp, esp

((LPBYTE)ComputeSum)[2] = 0xEC;

((LPBYTE)ComputeSum)[3] = 0x8B; // mov eax,[bp+8]

((LPBYTE)ComputeSum)[4] = 0x45;

((LPBYTE)ComputeSum)[5] = 0x08;

((LPBYTE)ComputeSum)[6] = 0x03; // add eax,[bp+12]

((LPBYTE)ComputeSum)[7] = 0x45;

((LPBYTE)ComputeSum)[8] = 0x0C;

((LPBYTE)ComputeSum)[9] = 0x5D; // pop ebp

((LPBYTE)ComputeSum)[10] = 0xC3; // ret eax

The final step is to call VirtualProtect(). We already have the op-codes in place, so we only have to ask for access to execute the code segment. The following call will do the job in this case:

VirtualProtect(ComputeSum, 11, PAGE_EXECUTE, &oldprotect);

As before, the variable oldprotect should be of type DWORD. At this point we have a pointer to a valid function and we can call it with C code such as the following:

sum = ComputeSum(1, 2);

sum = ComputeSum(val1, val2);

The only thing that remains is for us to delete the code segment before our application terminates. This is a simple matter of restoring the access protection back to it's original value and calling the appropriate function to delete the allocated memory, eg:

VirtualProtect(ComputeSum, 11, oldprotect, &oldprotect);

delete (LPBYTE)ComputeSum;

One last point: if you want your self-modifying code to be compatible with Windows NT then you'll also need to call the FlushInstructionCache() function after modifying any code segments. This is not a requirement under Windows95 since it is a single-CPU operating system, but I strongly recommend calling it anyway to avoid compatibility problems with future Windows releases.

Copyright (c) 1997 Mark Feldman (pcgpe@oocities.com) - All Rights Reserved

This article is part of The Win95 Game Programmer's Encyclopedia

Please retain this footer if you distribute this file.

Back to the Assembly Page