Binary exploitation (pwn)

Pwn is gamer slang for "own", as in completely defeating something. In CTFs, it means finding a bug in a running program and using it to make the program do something its author never intended.

Reversing is about understanding a program. Pwn goes a step further and breaks it. Usually you are given a copy of the program to study, plus an address like nc pwn.challenges.example 9001 where the same program runs on a server. The flag is in a file on that server, and the only way to it is through the program.

It has a reputation as the hardest category, because you need to understand how memory works. But the core idea fits on one page, and you are about to see it.

When a program runs a function, it keeps that function's variables together in an area of memory called the stack. Picture a row of numbered boxes, with variables sitting right next to each other and nothing between them.

A buffer is a variable that reserves a fixed number of boxes, such as char name[16], which holds 16 characters. Now here is the problem: the old C function gets() reads your input into a buffer and never checks whether it fits.

So what happens if you type 20 characters into a 16-character buffer? The extra four don't disappear. They keep going, spilling into whatever is stored next door. That is a buffer overflow. If the neighbor is a variable that controls the program's behavior, then you control the program.

Below is a small vulnerable program. key sits right after name in memory. The program never lets you set key directly, but it only prints the flag if key holds one particular value. See what you can do about that.

void greet() {
    int  key = 0;
    char name[16];

    // reads input with no length limit!
    gets(name);
    printf("Hello, %s\n", name);

    if (key == 0x6b636168) {
        print_flag();
    }
}

Getting the exact value

Overwriting key is the easy part. To pass the check, it has to equal exactly 0x6b636168. That is a number written in hex, and each pair of digits is one byte: 6b 63 61 68. Look those up as characters and you get k c a h.

That looks backwards, and it is. Most computers are little-endian, meaning they store the lowest byte of a number first. So in memory, those bytes run the other way: 68 61 63 6b, which is h a c k. Whatever you type lands in memory in the order you type it.

Past the key variable sits something far more powerful: the return address, which tells a function where to jump when it finishes. Overwrite that with an address of your choosing, and you decide what the program does next. Pointing it at a function that prints the flag is a classic first exploit called ret2win.

On a real challenge, you would send the exploit from a script, because the bytes you need often can't be typed. pwntools(opens in a new tab) is the standard Python library for it:

from pwn import *

io = remote("pwn.challenges.example", 9001)
io.sendline(b"A" * 16 + b"hack")     # filler, then the value for key
print(io.recvall().decode())

As you progress, you will meet the protections that make this harder, and the techniques for getting past them: stack canaries, non-executable memory (NX), and address randomization (ASLR). Running checksec on a program tells you which are turned on. To learn it properly, pwn.college(opens in a new tab) is free and excellent.