So we had fun right? Those parlor tricks were neat no? Thats peas compared to the power of what you can do. For now however, we need to get back to learning, but I hope it inspired you to dig and learn on your own in addition to what I've been posting here.
Let's learn how to encode a x86 instruction shall we?
So everybody and his grandma knows that a x86 instruction has a few parts
There are a total of four prefixes, lock prefix/repeat, segment override, and 2 size override prefixes.
Your lock and repeat prefixes start at 0xF0 -> 0xF3, excluding 0xF1.
Your segment overrides:
#define PREFIX_SEGMENT_CS 0x2e
#define PREFIX_SEGMENT_SS 0x36
#define PREFIX_SEGMENT_DS 0x3e
#define PREFIX_SEGMENT_ES 0x26
#define PREFIX_SEGMENT_FS 0x64
#define PREFIX_SEGMENT_GS 0x65
Your size overrides are 0x66 and 0x67.
0x66 overrides the operand size
0x67 overrides the address size.
We could discuss the opcode/opcode2, but I feel that it's wiser to save it for another time.
Let's move onto the modrm and sib now. How do we intrepret the modrm and sib?
Well think back to the first tutorial, remember I stressed how important understanding bitmasking was? Well here is why.http://pdos.csail.mit.edu/6.828/2006/readings/i386/fig17-2.gif
Notice the register is packed into the byte using 3 bits, with the least most significant bit specifying if it is a register or a memory operation. THe most significant bit, bit 7 is the mode.
The SIB is largely self explanatory, where bits 6-7 specify the scale, 1, 2, 4, 8. The index and the base are registers usually, otherwise it could happen to be the immediate.
Seem familiar and visible from the last tutorial? I hope so.
We use the relative when doing calls/jumps. We'll cover this in the future.
The displacement can be a memory address or an offset from a memory address.
The immediate can be any value, 8 bit, 16 bit and 32 bit thereof.
The only nuance I can recall about 64 bit is that aside from the rex prefix, the only difference is that the immediates are 64 bit whereas displacements/relatives are stuck to 32 bit.
I'll edit this post to give a few nice examples of this so it's easier to comprehend.