disassembly¶
Convert Binary Ninja disassembly tokens into the pretokenized instruction stream expected by the Undertale tokenizer (mnemonics, registers, and immediates space-separated; punctuation and formatting tokens dropped).
This module has no dependencies beyond Binary Ninja’s own API, this makes it safe to incorporate directly into contexts that cannot depend on the rest of Undertale’s (much heavier) dependency set. As an example, the Binary Ninja plugin at extras/binaryninja-plugin/undertale/utils/disassembly.py keeps a byte-for-byte copy of this file for exactly that reason. If you change this file, update that copy too (tests/unit.py checks that they match).
Functions
|
Convert basic blocks' disassembly text into pretokenized tokens. |
- undertale.pipeline.disassembly.pretokenize_disassembly(blocks, next_token: str = '[NEXT]') List[str]¶
Convert basic blocks’ disassembly text into pretokenized tokens.
Walks each block’s disassembly text in order and classifies every token, dropping punctuation and formatting tokens (commas, memory operand braces’ annotations, etc.) so the result is a flat sequence of mnemonics, registers, operators, and immediates - one token per element, ready to be joined with a single space per element.
- Parameters:
blocks – Binary Ninja basic blocks, in the order their instructions should appear.
next_token – The token to insert between instructions. Defaults to [NEXT] and must match the tokenizer’s special token for instruction boundaries (see
undertale.models.tokenizer.TOKEN_NEXT).
- Returns:
The pretokenized disassembly tokens.