Obfuscation Techniques

Overview

IronBrew 2 employs multiple layers of obfuscation to protect Lua code. These techniques work together to make reverse engineering extremely difficult.

Obfuscation Settings

Obfuscation is controlled via ObfuscationSettings (ObfuscationSettings.cs:3-31):

public class ObfuscationSettings
{
    public bool EncryptStrings;           // Encrypt all strings
    public bool EncryptImportantStrings;  // Encrypt specific strings
    public bool ControlFlow;              // Enable control flow obfuscation
    public bool BytecodeCompress;         // Compress bytecode
    public int DecryptTableLen;           // String decryption table size
    public bool PreserveLineInfo;         // Keep line info (debug)
    public bool Mutate;                   // Enable instruction mutation
    public bool SuperOperators;           // Enable super operators
    public int MaxMiniSuperOperators;     // Mini super operator count
    public int MaxMegaSuperOperators;     // Mega super operator count
    public int MaxMutations;              // Maximum mutations
}

Default configuration:

String encryption: disabled
Control flow: enabled
Bytecode compression: enabled
Mutations: enabled (max 200)
Super operators: enabled (120 mini, 120 mega)

Instruction Mutations

What Are Mutations?

Mutations create alternative register orderings for each instruction (Generator.cs:100-126):

public List<OpMutated> GenerateMutations(List<VOpcode> opcodes)
{
    Random r = new Random();
    List<OpMutated> mutated = new List<OpMutated>();

    foreach (VOpcode opc in opcodes)
    {
        if (opc is OpSuperOperator)
            continue;

        // Create 35-50 mutations per opcode
        for (int i = 0; i < r.Next(35, 50); i++)
        {
            int[] rand = {0, 1, 2};
            rand.Shuffle();

            OpMutated mut = new OpMutated();
            mut.Registers = rand;  // Shuffled register order
            mut.Mutated = opc;
            
            mutated.Add(mut);
        }
    }

    mutated.Shuffle();
    return mutated;
}

How Mutations Work

For a standard instruction like ADD R(A) R(B) R(C): Original:

local OP_A = Inst[2];
local OP_B = Inst[3];
local OP_C = Inst[4];
Stack[OP_A] = Stack[OP_B] + Stack[OP_C];

Mutated (registers shuffled to [1, 0, 2]):

local OP_B = Inst[2];  -- Now reads from position A
local OP_A = Inst[3];  -- Now reads from position B
local OP_C = Inst[4];  -- Still position C
Stack[OP_A] = Stack[OP_B] + Stack[OP_C];  -- Same logic

The instruction encoding is adjusted to match:

If registers are [1, 0, 2], the serializer swaps A and B positions
Each instruction instance can use a different mutation
Makes pattern recognition much harder

Mutation Folding

Mutations are applied to actual instructions (Generator.cs:128-167):

public void FoldMutations(List<OpMutated> mutations, HashSet<OpMutated> used, Chunk chunk)
{
    for (int i = 0; i < chunk.Instructions.Count; i++)
    {
        Instruction opc = chunk.Instructions[i];
        CustomInstructionData data = opc.CustomData;
        
        foreach (OpMutated mut in mutations)
            if (data.Opcode == mut.Mutated && data.WrittenOpcode == null)
            {
                if (!used.Contains(mut))
                    used.Add(mut);

                data.Opcode = mut;  // Apply mutation
                break;
            }
    }
}

Only mutations that are actually used get included in the final VM.

Super Operators

Concept

Super operators combine multiple instructions into a single virtual opcode. Instead of executing 5 separate instructions, one super operator executes all 5 at once.

Types of Super Operators

Mini Super Operators

Combine 5-10 consecutive instructions (Generator.cs:392-399):

var miniOperators = GenerateSuperOperators(_context.HeadChunk, 10)
    .OrderBy(t => r.Next())
    .Take(settings.MaxMiniSuperOperators)
    .ToList();

virtuals.AddRange(miniOperators);
FoldAdditionalSuperOperators(_context.HeadChunk, miniOperators, ref folded);

Mega Super Operators

Combine 60-80 consecutive instructions (Generator.cs:383-390):

var megaOperators = GenerateSuperOperators(_context.HeadChunk, 80, 60)
    .OrderBy(t => r.Next())
    .Take(settings.MaxMegaSuperOperators)
    .ToList();

virtuals.AddRange(megaOperators);
FoldAdditionalSuperOperators(_context.HeadChunk, megaOperators, ref folded);

Generation Process

Super operators are generated by scanning for sequences (Generator.cs:169-256):

public List<OpSuperOperator> GenerateSuperOperators(Chunk chunk, int maxSize, int minSize = 5)
{
    List<OpSuperOperator> results = new List<OpSuperOperator>();
    bool[] skip = new bool[chunk.Instructions.Count + 1];

    // Mark instructions that can't be combined
    for (int i = 0; i < chunk.Instructions.Count - 1; i++)
    {
        switch (chunk.Instructions[i].OpCode)
        {
            case Opcode.Jmp:
            case Opcode.ForLoop:
            case Opcode.ForPrep:
                skip[i + 1] = true;  // Can't combine after jumps
                break;
            
            case Opcode.Test:
            case Opcode.TestSet:
                skip[i + 1] = true;  // Can't combine after tests
                break;
        }
    }
    
    // Scan for valid sequences
    int c = 0;
    while (c < chunk.Instructions.Count)
    {
        int targetCount = maxSize;
        OpSuperOperator superOperator = new OpSuperOperator {SubOpcodes = new VOpcode[targetCount]};

        // Check if we can take targetCount instructions
        for (int j = 0; j < targetCount; j++)
        {
            if (c + j > chunk.Instructions.Count - 1 || skip[c + j])
            {
                // Reduce size if we hit a boundary
                targetCount = j;
                break;
            }
        }

        if (targetCount >= minSize)
        {
            // Create super operator from this sequence
            for (int j = 0; j < targetCount; j++)
                superOperator.SubOpcodes[j] = chunk.Instructions[c + j].CustomData.Opcode;

            results.Add(superOperator);
        }
        
        c += targetCount + 1;
    }

    return results;
}

Super Operator Code Generation

Super operators generate inline code (OpSuperOperator.cs:36-69):

public override string GetObfuscated(ObfuscationContext context)
{
    string s = "";
    List<string> locals = new List<string>();
    
    for (var index = 0; index < SubOpcodes.Length; index++)
    {
        var subOpcode = SubOpcodes[index];
        string s2 = subOpcode.GetObfuscated(context);
        
        // Extract local declarations
        Regex reg = new Regex("local(.*?)[;=]");
        foreach (Match m in reg.Matches(s2))
        {
            string loc = m.Groups[1].Value.Replace(" ", "");
            if (!locals.Contains(loc))
                locals.Add(loc);
            
            // Remove duplicate declarations
            if (!m.Value.Contains(";"))
                s2 = s2.Replace($"local{m.Groups[1].Value}", loc);
            else 
                s2 = s2.Replace($"local{m.Groups[1].Value};", "");
        }

        s += s2;

        // Advance to next instruction
        if (index + 1 < SubOpcodes.Length)
            s += "InstrPoint = InstrPoint + 1;Inst = Instr[InstrPoint];";
    }

    // Hoist all locals to the top
    foreach (string l in locals)
        s = "local " + l + ';' + s;
        
    return s;
}

Result: Instead of 10 separate opcode dispatches, one dispatch executes all 10 inline.

String Encryption

XOR-Based Encryption

String encryption uses XOR with a random table (ConstantEncryption.cs:17-27):

public string Encrypt(byte[] bytes)
{
    List<byte> encrypted = new List<byte>();
    int L = Table.Length;
    
    for (var index = 0; index < bytes.Length; index++)
        encrypted.Add((byte)(bytes[index] ^ Table[index % L]));
    
    // Returns inline Lua decryption code
    return $"((function(b)...xor decryption...end)(\"...encrypted data...\"))";
}

Decryption Table Generation

Each string gets a random XOR table (ConstantEncryption.cs:29-36):

public Decryptor(string name, int maxLen)
{
    Random r = new Random();
    Name = name;
    Table = Enumerable.Repeat(0, maxLen)
        .Select(i => r.Next(0, 256))
        .ToArray();
}

String Encryption Modes

Mode 1: Encrypt All Strings

When EncryptStrings = true (ConstantEncryption.cs:140-165):

Regex r = new Regex(encRegex, RegexOptions.Singleline);
var matches = r.Matches(_src);

Decryptor dec = GenerateGenericDecryptor(matches);

foreach (Match m in matches)
{
    string captured = m.Groups[2].Value + m.Groups[4].Value;
    string nStr = before + dec.Encrypt(UnescapeLuaString(captured));
    // Replace string with encrypted version
}

All strings use the same decryptor for efficiency.

Mode 2: Encrypt Marked Strings

When EncryptStrings = false but EncryptImportantStrings = true (ConstantEncryption.cs:167-196):

foreach (Match m in matches)
{
    string captured = m.Groups[2].Value + m.Groups[4].Value;
    
    if (!captured.StartsWith("[STR_ENCRYPT]"))
        continue;  // Only encrypt marked strings

    captured = captured.Substring(13);
    Decryptor dec = new Decryptor("IRONBREW_STR_ENCRYPT" + n++, m.Length);
    // Each marked string gets its own decryptor
}

Mark strings with "[STR_ENCRYPT]your text" to encrypt them.

Mode 3: Important String Detection

When EncryptImportantStrings = true (ConstantEncryption.cs:198-239):

List<string> sTerms = new List<string>() {"http", "function", "metatable", "local"};

foreach (Match m in matches)
{
    string captured = m.Groups[2].Value + m.Groups[4].Value;
    
    bool cont = false;
    foreach (string search in sTerms)
    {
        if (captured.ToLower().Contains(search.ToLower()))
            cont = true;
    }

    if (!cont)
        continue;  // Only encrypt important strings
    
    // Encrypt with unique decryptor
}

Automatically encrypts strings containing sensitive keywords.

Bytecode Compression

LZW Compression

Bytecode can be compressed using LZW (Generator.cs:41-72):

public static List<int> Compress(byte[] uncompressed)
{
    Dictionary<string, int> dictionary = new Dictionary<string, int>();
    for (int i = 0; i < 256; i++)
        dictionary.Add(((char)i).ToString(), i);
 
    string w = string.Empty;
    List<int> compressed = new List<int>();
 
    foreach (byte b in uncompressed)
    {
        string wc = w + (char)b;
        if (dictionary.ContainsKey(wc))
            w = wc;
        else
        {
            compressed.Add(dictionary[w]);
            dictionary.Add(wc, dictionary.Count);
            w = ((char)b).ToString();
        }
    }
 
    return compressed;
}

Base-36 Encoding

Compressed data is encoded in base-36 to embed in Lua (Generator.cs:74-98):

public static string ToBase36(ulong value)
{
    const string base36 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
    var sb = new StringBuilder(13);
    do
    {
        sb.Insert(0, base36[(byte)(value % 36)]);
        value /= 36;
    } while (value != 0);
    return sb.ToString();
}

public static string CompressedToString(List<int> compressed)
{
    StringBuilder sb = new StringBuilder();
    foreach (int i in compressed)
    {
        string n = ToBase36((ulong)i);
        sb.Append(ToBase36((ulong)n.Length));  // Length prefix
        sb.Append(n);                          // Value
    }
    return sb.ToString();
}

Result: Bytecode string is significantly smaller and harder to read.

Randomization Techniques

Chunk Step Shuffling

The order of chunk components is randomized (ObfuscationContext.cs:55-56):

ChunkSteps = Enumerable.Range(0, (int)ChunkStep.StepCount)
    .Select(i => (ChunkStep)i).ToArray();
ChunkSteps.Shuffle();

Possible orders:

Parameters → Instructions → Functions → LineInfo
Instructions → Functions → Parameters → LineInfo
Functions → LineInfo → Instructions → Parameters
… (24 permutations)

Constant Type Remapping

Constant type IDs are shuffled (ObfuscationContext.cs:64-65):

ConstantMapping = Enumerable.Range(0, 4).ToArray();
ConstantMapping.Shuffle();

Instead of standard mapping (0=nil, 1=bool, 3=number, 4=string):

Could be: 2=nil, 0=bool, 3=number, 1=string
Makes constant table harder to parse

Opcode Shuffling

Virtual opcodes are shuffled (Generator.cs:404-407):

virtuals.Shuffle();

for (int i = 0; i < virtuals.Count; i++)
    virtuals[i].VIndex = i;

Opcode indices are completely randomized each obfuscation.

XOR Key Randomization

New XOR keys for each obfuscation (ObfuscationContext.cs:67-71):

Random rand = new Random();
PrimaryXorKey = rand.Next(0, 256);
IXorKey1 = rand.Next(0, 256);
IXorKey2 = rand.Next(0, 256);

Layer Combination

All techniques work together:

Bytecode is deserialized from Lua 5.1 format
Control flow is obfuscated (see Control Flow)
Instructions are mapped to virtual opcodes
Mutations are generated (35-50 per opcode)
Super operators are created (up to 240 total)
Mutations are folded into instructions
Super operators are folded to combine sequences
Opcodes are shuffled for random indices
Bytecode is serialized with randomized chunk steps
Bytecode is XOR encrypted with random key
Bytecode is compressed (optional)
Strings are encrypted (optional)
VM is generated with binary search dispatch

Result: Every obfuscation is unique and extremely difficult to reverse engineer.

Performance Considerations

When to Use Each Technique

Always use:

VM-based obfuscation (core feature)
Bytecode XOR encryption (minimal overhead)
Chunk step shuffling (no runtime cost)

Use for strong protection:

Instruction mutations (moderate overhead)
Mini super operators (improves performance)
Bytecode compression (reduces size)

Use for maximum protection:

Mega super operators (can reduce dispatch overhead)
String encryption (adds decryption overhead)
Control flow obfuscation (significant complexity)

Avoid in performance-critical code:

Excessive mega super operators
Encrypting all strings
Maximum mutation count

VM Architecture - The VM foundation
Control Flow - Control flow transformations
Bytecode - Understanding the bytecode

Get Started

CLI Reference

Core Concepts

Configuration

Obfuscation Techniques

Overview

Obfuscation Settings

Instruction Mutations

What Are Mutations?

How Mutations Work

Mutation Folding

Super Operators

Concept

Types of Super Operators

Mini Super Operators

Mega Super Operators

Generation Process

Super Operator Code Generation

String Encryption

XOR-Based Encryption

Decryption Table Generation

String Encryption Modes

Mode 1: Encrypt All Strings

Mode 2: Encrypt Marked Strings

Mode 3: Important String Detection

Bytecode Compression

LZW Compression

Base-36 Encoding

Randomization Techniques

Chunk Step Shuffling

Constant Type Remapping

Opcode Shuffling

XOR Key Randomization

Layer Combination

Performance Considerations

When to Use Each Technique

Build docs developers (and LLMs) love

Get Started

CLI Reference

Core Concepts

Configuration

Documentation Index

​Overview

​Obfuscation Settings

​Instruction Mutations

​What Are Mutations?

​How Mutations Work

​Mutation Folding

​Super Operators

​Concept

​Types of Super Operators

​Mini Super Operators

​Mega Super Operators

​Generation Process

​Super Operator Code Generation

​String Encryption

​XOR-Based Encryption

​Decryption Table Generation

​String Encryption Modes

​Mode 1: Encrypt All Strings

​Mode 2: Encrypt Marked Strings

​Mode 3: Important String Detection

​Bytecode Compression

​LZW Compression

​Base-36 Encoding

​Randomization Techniques

​Chunk Step Shuffling

​Constant Type Remapping

​Opcode Shuffling

​XOR Key Randomization

​Layer Combination

​Performance Considerations

​When to Use Each Technique

​Related Topics

Build docs developers (and LLMs) love

Overview

Obfuscation Settings

Instruction Mutations

What Are Mutations?

How Mutations Work

Mutation Folding

Super Operators

Concept

Types of Super Operators

Mini Super Operators

Mega Super Operators

Generation Process

Super Operator Code Generation

String Encryption

XOR-Based Encryption

Decryption Table Generation

String Encryption Modes

Mode 1: Encrypt All Strings

Mode 2: Encrypt Marked Strings

Mode 3: Important String Detection

Bytecode Compression

LZW Compression

Base-36 Encoding

Randomization Techniques

Chunk Step Shuffling

Constant Type Remapping

Opcode Shuffling

XOR Key Randomization

Layer Combination

Performance Considerations

When to Use Each Technique

Related Topics