EVM Toolkit (ETK)
The EVM Toolkit (or ETK) is a collection of tools for creating and analyzing smart contract programs on the Ethereum Virtual Machine. So far it consists of an assembler (eas) and a disassembler (disease).
ETK is new, and highly experimental. Use at your own risk.
Tools
Assembler: eas
The assembler lives in the etk-asm crate, and provides a command-line interface (eas) and a Rust library. The assembler has a couple notable features:
- Importing multiple files into the same scope, for organization.
- Including files in a separate scope, for constructors and initialization code.
- Automatic push size selection.
- Solidity-style function selectors.
Disassembler: disease
The disassembler lives in the etk-dasm crate, and also provides a command-line interface (disease) and a Rust library. The disassembler is much more experimental, and has a limited set of features:
- Disassemble hex or binary encoded instructions into their mnemonics.
- Identify basic blocks.
API Documentation
Alongside this book, you can also read the API docs generated by Rustdoc if you would like to use ETK as a library.
Dependencies
ecfg requires z3 to build
Ubuntu Installation Instructions (example):
sudo apt-get update -y
sudo apt-get install -y z3
Check the system logs to confirm that there are no related errors.
License
ETK, all the source code, is released under the Apache License Version 2.0 and under the MIT License.
Command-Line Tools
ETK provides a set of command-line tools, and can also be used as Rust crates. Let's dive into the command line tools first.
Install from Source
ETK and its tools can be installed by compiling the source code on your local machine.
Pre-requisite
ETK is written in Rust and needs to be compiled with cargo. The minimum supported Rust version 1.51. If you don't have Rust installed, install it now.
Install Development Version
The development version contains all the latest features, bugs, and maybe even some bug-fixes that will eventually be released in the next version. If you can't wait for the next release, you can install the development version from git yourself.
Open a terminal and use cargo to install ETK:
cargo install \
--git 'https://github.com/quilt/etk' \
--features cli \
etk-asm \
etk-dasm
Install Released Version
Once you have Rust and cargo installed, you just have to type this snippet in your terminal:
cargo install --features cli etk-asm etk-dasm
Install from Binaries
Precompiled binaries will be provided for select platforms on a best-effort basis. Visit the releases page to download the appropriate version, once we create one, for your platform.
Install Syntax Highlighting
Syntax highlighting for vim is available via vim-etk.
Assembler: eas
The assembler command converts human-readable mnemonics (ex. push2, caller) into the raw bytes the EVM interpreter expects, encoded in hexadecimal. In addition to this conversion, the assembler performs transformations on the source in the form of expression- and instruction- macros, which we'll get into later.
Invoking the assembler is pretty simple:
eas input.etk output.hex
The input argument (input.etk here) is the path to an assembly file, and is required. output.hex is the path where the assembled instructions will be written, encoded in hex. If the output path is omitted, the assembled instructions are written to the standard output.
A Note on Paths
The input argument determines the root of the project. If /home/user/foobar/main.etk is the input argument, the root would be /home/user/foobar. Only files within the root directory can be included or imported.
Disassembler: disease
The disassembler command disease is roughly the inverse of the assembler. It transforms a string of bytes into human-readable mnemonics.
The basic invocation of disease looks like:
disease --bin-file contract.bin # Disassemble a binary file
disease --hex-file contract.hex # Disassemble a hexadecimal file
disease --code 0x5b600056 # Disassemble the command line argument
Specifying Input
--bin-file, or -b
When you use the --bin-file argument, disease will read the code from the specified file, and interpret it as raw binary bytes. Few tools use this format.
--hex-file, or -x
With the --hex-file argument, the specified file is instead interpreted as hexadecimal.
--code, or -c
Great for short snippets, the --code argument instructs disease to disassemble the hexadecimal string given directly on the command line.
Specifying Output
--out-file, or -o
If provided, --out-file causes the disassembled source to be written to the given path. Without --out-file, the disassembly is written to the standard output.
Language & Syntax
The ETK assembly language takes inspiration from NASM and other similar assemblers, but has its own particular flavor.
Syntax
Friendly Example
This example should increment a value from 0 to 255 on the stack, then halt execution.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" push1 0x00 loop: jumpdest push1 0x01 add dup1 push1 0xFF gt push1 loop jumpi pop stop # This halts execution "#; let mut ingest = etk_asm::ingest::Ingest::new(Vec::new()); ingest.ingest(file!(), src).unwrap(); }
The first line—push1 0x00—describes a push instruction of length one, with a value of 0. When assembled, this line would become 0x6000.
Next, we have loop:, which introduces a label named loop. Labels can be used as arguments to push instructions, usually for jumps or subroutines.
Finally, we have # This halts execution, which is a comment. Comments are introduced with # and continue to the end of the line. Comments are ignored as far as the assembler is concerned.
There are a couple other features, like macros, which will be covered in later chapters.
Formal Syntax
For the language nerds, the ETK assembly language syntax is defined by the following Pest grammar:
///////////////////////
// program structure //
///////////////////////
program = _{ SOI ~ inner ~ EOI }
inner = _{ NEWLINE* ~ (stmt ~ (NEWLINE+|";"))* ~ stmt? }
stmt = _{ label_definition | builtin | local_macro | push | op }
//////////////////////
// opcode mnemonics //
//////////////////////
op = @{
"origin" | "stop" | "mulmod" | "mul" | "sub" | "div" | "sdiv" | "mod" | "smod" |
"addmod" | "exp" | "signextend" | "lt" | "gt" | "slt" |
"sgt" | "eq" | "iszero" | "and" | "or" | "xor" | "not" | "shl" | "shr" |
"sar" | "keccak256" | "address" | "add" | "balance" | "caller" |
"callvalue" | "calldataload" | "calldatasize" | "calldatacopy" |
"codesize" | "codecopy" | "gasprice" | "extcodesize" | "extcodecopy" |
"returndatasize" | "returndatacopy" | "extcodehash" | "blockhash" |
"coinbase" | "timestamp" | "number" | "difficulty" | "gaslimit" |
"pop" | "mload" | "mstore8" | "mstore" | "sload" | "sstore" | "jumpdest" |
"jumpi" | "jump" | "pc" | "msize" | "gas" | swap | dup | log |
"create2" | "callcode" | "call" | "return" | "delegatecall" | "create" |
"staticcall" | "revert" | "selfdestruct" | "byte" | "chainid" | "selfbalance" |
"basefee" | "invalid" | "push0" | "mcopy"
}
push = ${ "push" ~ word_size ~ WHITESPACE ~ expression }
swap = @{ "swap" ~ half_word_size }
dup = @{ "dup" ~ half_word_size }
log = @{ "log" ~ '0'..'4' }
word_size = @{ ('1'..'2' ~ '0'..'9') | ("3" ~ '0'..'2') | '1'..'9' }
half_word_size = @{ ("1" ~ '0'..'6') | '1'..'9' }
////////////////////////
// instruction macros //
////////////////////////
instruction_macro_definition = { "%macro" ~ function_declaration ~ NEWLINE* ~ (instruction_macro_stmt ~ NEWLINE+)* ~ "%end" }
instruction_macro_stmt = _{ label_definition | "%" ~ push_macro | local_macro | push | op }
instruction_macro_variable = @{ "$" ~ function_parameter }
instruction_macro = !{ "%" ~ function_invocation }
local_macro = { !builtin ~ (instruction_macro_definition | instruction_macro | expression_macro_definition) }
builtin = ${ "%" ~ ( import | include | include_hex | push_macro ) }
import = !{ "import" ~ arguments }
include = !{ "include" ~ arguments }
include_hex = !{ "include_hex" ~ arguments }
push_macro = !{ "push" ~ arguments }
arguments = _{ "(" ~ arguments_list? ~ ")" }
arguments_list = _{ ( argument ~ "," )* ~ argument? }
argument = _{ string | expression }
string = @{ "\"" ~ string_char* ~ "\"" }
string_char = _{ "\\\\" | "\\\"" | (!"\\" ~ !"\"" ~ ANY) }
///////////////////////
// expression macros //
///////////////////////
expression_macro_definition = !{ "%def" ~ function_declaration ~ NEWLINE ~ expression ~ NEWLINE ~ "%end" }
expression_macro = { function_invocation }
selector = ${ "selector(\"" ~ selector_function_declaration ~ "\")" }
topic = ${ "topic(\"" ~ selector_function_declaration ~ "\")" }
selector_function_declaration = @{ function_name ~ "(" ~ function_parameter* ~ ("," ~ function_parameter)* ~ ")" }
function_declaration = { function_name ~ "(" ~ function_parameter* ~ ("," ~ function_parameter)* ~ ")" }
function_invocation = _{ function_name ~ "(" ~ expression* ~ ("," ~ expression)* ~ ")" }
function_name = @{ ( ASCII_ALPHA | "_" ) ~ ( ASCII_ALPHANUMERIC | "_" )* }
function_parameter = @{ ASCII_ALPHA ~ ASCII_ALPHANUMERIC* }
//////////////
// operands //
//////////////
number = _{ binary | octal | hex | decimal }
binary = @{ "0b" ~ ASCII_BIN_DIGIT+ }
octal = @{ "0o" ~ ASCII_OCT_DIGIT+ }
decimal = @{ ASCII_DIGIT+ }
hex = @{ "0x" ~ ASCII_HEX_DIGIT ~ ASCII_HEX_DIGIT+ }
label = @{ ASCII_ALPHA ~ (ASCII_ALPHANUMERIC | "_")* }
label_definition = { label ~ ":" }
////////////////
// infix math //
////////////////
expression = !{ term ~ (operation ~ term)* }
term = _{ instruction_macro_variable | selector | topic | expression_macro | label | number | negative_decimal | "(" ~ expression ~ ")" }
negative_decimal = @{ "-" ~ ASCII_DIGIT+ }
operation = _{ plus | minus | times | divide }
plus = { "+" }
minus = { "-" }
times = { "*" }
divide = { "/" }
///////////////
// overrides //
///////////////
WHITESPACE = _{ " " | "\t" }
COMMENT = _{ "#" ~ (!NEWLINE ~ ANY)* }
Instructions
Instructions, also known as opcodes or Ops internally, are the building blocks of ETK smart contracts. Each instruction has a human-readable mnemonic (like dup3) and the machine readable equivalent (which would be 0x82). The push family of instructions also encode an immediate value (or argument.)
List of Instructions
stop
add
mul
sub
div
sdiv
mod
smod
addmod
mulmod
exp
signextend
lt
gt
slt
sgt
eq
iszero
and
or
xor
not
byte
shl
shr
sar
keccak256
address
balance
origin
caller
callvalue
calldataload
calldatasize
calldatacopy
codesize
codecopy
gasprice
extcodesize
extcodecopy
returndatasize
returndatacopy
extcodehash
blockhash
coinbase
timestamp
number
difficulty
gaslimit
chainid
selfbalance
basefee
pop
mload
mstore
mstore8
sload
sstore
jump
jumpi
pc
msize
gas
jumpdest
mcopy
push1 0xAA
push2 0xAABB
push3 0xAABBCC
push4 0xAABBCCDD
push5 0xAABBCCDDEE
push6 0xAABBCCDDEEFF
push7 0xAABBCCDDEEFF00
push8 0xAABBCCDDEEFF0011
push9 0xAABBCCDDEEFF001122
push10 0xAABBCCDDEEFF00112233
push11 0xAABBCCDDEEFF0011223344
push12 0xAABBCCDDEEFF001122334455
push13 0xAABBCCDDEEFF00112233445566
push14 0xAABBCCDDEEFF0011223344556677
push15 0xAABBCCDDEEFF001122334455667788
push16 0xAABBCCDDEEFF00112233445566778899
push17 0xAABBCCDDEEFF00112233445566778899AA
push18 0xAABBCCDDEEFF00112233445566778899AABB
push19 0xAABBCCDDEEFF00112233445566778899AABBCC
push20 0xAABBCCDDEEFF00112233445566778899AABBCCDD
push21 0xAABBCCDDEEFF00112233445566778899AABBCCDDEE
push22 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF
push23 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF00
push24 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF0011
push25 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF001122
push26 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF00112233
push27 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF0011223344
push28 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF001122334455
push29 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF00112233445566
push30 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF0011223344556677
push31 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF001122334455667788
push32 0xAABBCCDDEEFF00112233445566778899AABBCCDDEEFF00112233445566778899
dup1
dup2
dup3
dup4
dup5
dup6
dup7
dup8
dup9
dup10
dup11
dup12
dup13
dup14
dup15
dup16
swap1
swap2
swap3
swap4
swap5
swap6
swap7
swap8
swap9
swap10
swap11
swap12
swap13
swap14
swap15
swap16
log0
log1
log2
log3
log4
create
call
callcode
return
delegatecall
create2
staticcall
revert
invalid
selfdestruct
Expressions
push opcodes
The push family of instructions require an immediate argument. Unlike most arguments (which come from the stack), immediate arguments are encoded in the bytes immediately following the opcode.
For example, push2 258 would assemble to 0x610102. 0x61 is the instruction, and 0x0102 is the immediate argument. push2 instructs the EVM to use the next two bytes 0x01 and 0x02 as input for the op. That value is left-padded with zeros to 32 bytes (so 0x0000...000102) and placed on the stack.
While an assembled push must have a concrete value, it is often useful when developing a program to have the ability to manipulate the operand at compile time. For this reason, push opcodes take a single expression as their operand.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" push1 1+(2*3)/4 "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x02]); }
Definition
An expression is a standard infix mathematical expression that is evaluated during assembly. Its computed value must fit within the preceding push's size allowance (eg. less than 256 for push8).
Terms
Many different types of values are allowed as a term in an expression:
Integer Literals
Integer literals are described by the following Pest grammar:
number = _{ binary | octal | hex | decimal | negative }
decimal = @{ ASCII_DIGIT+ }
negative = @{ "-" ~ ASCII_DIGIT+ }
binary = @{ "0b" ~ ASCII_BIN_DIGIT+ }
hex = @{ "0x" ~ ASCII_HEX_DIGIT ~ ASCII_HEX_DIGIT+ }
octal = @{ "0o" ~ ASCII_OCT_DIGIT+ }
There is no limit for the length of integer literals. While expressions support
both signed and unsigned integers, the result of the expression must
non-negative and fit within the width of the corresponding push instruction.
Labels
A label may be used as a term in an expression.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" start: push1 start + 1 "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x01]); }
Macros
Expression macros may be used as a term in an expression.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" push4 selector("transfer(uint256,uint256)") "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x63, 12, 247, 158, 10]); }
Operators
Binary
Expressions support the following binary operators:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" push1 1+2 # addition push1 1*2 # multiplication push1 2-1 # subtraction push1 2/2 # division "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x03, 0x60, 0x02, 0x60, 0x01, 0x60, 0x01]); }
Labels
Manually counting out jump destination addresses would be a monumentally pointless task, so the assembler supports assigning names (or labels) to specific locations in code:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" label0: # <- This is a label called "label0", # and it has the value 0, since it is # before any instructions in scope. jumpdest push1 label0 # <- Here we push the value of "label0", # which is zero, onto the stack. jump # Now we jump to zero, which is a # `jumpdest` instruction, looping forever. "#; let mut ingest = etk_asm::ingest::Ingest::new(Vec::new()); ingest.ingest(file!(), src).unwrap(); }
Uses
The obvious (and only, currently) place to use a label is in a push instruction. That said, there are a couple interesting ways to use labels that might not be immediately obvious.
Jump Address
You can push a label, then jump to it like in the above example.
Length
That's not all! You can also use labels to calculate lengths:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" push1 start push1 end sub # <- Will leave a 3 on the stack. stop start: pc pc pc end: "#; let mut ingest = etk_asm::ingest::Ingest::new(Vec::new()); ingest.ingest(file!(), src).unwrap(); }
Calculating the length of a blob of instructions is very useful in contract initialization code (also known as constructors).
Macros
A macro is a rule or pattern that maps a given input to a replacement output. In other words, the assembler replaces a macro invocation with some other text (the macro expansion.)
Types
Macros in ETK take one of two forms: instruction macros, and expression macros. Both types of macros are written as a name followed by arguments in parentheses.
Instruction Macros
An instruction macro looks like this:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" %macro push_sum(a, b) push1 $a + $b %end %push_sum(4, 2) "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x06]); }
Instruction macros always begin with %, and expand to one or more instructions. In this case, %push_sum(4, 2) would expand to:
push1 0x06
Expression Macros
Expression macros do not begin with %, and cannot replace instructions. Instead, expression macros can be used in expressions. For example:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" %def add_one(num) $num+1 %end push1 add_one(41) "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x2a]); }
Here, add_one(...) is an expression macro that returns the num+1. The fully expanded source would look like:
push1 42
Built-In Macros
Built-in macros are implemented by the assembler, and provide additional features beyond basic instructions, constants, or labels.
Instruction Macros
%import("...")
The %import macro expands to the instructions read from another file as if they had been typed here. The path is resolved relative to the current file.
Source: main.etk
push1 some_label
jump
%import("other.etk")
Source: other.etk
some_label:
jumpdest
stop
After Expansion
push1 0x03
jump
jumpdest
stop
%include("...")
The %include macro expands to the instructions read from another file, but unlike %import, the included file is assembled independently from the current file:
- Labels from the included file are not available in the including file, and vise versa.
- The address of the first instruction in the included file will be zero.
The path is resolved relative to the current file.
Source: main.etk
some_label: # <- Not visible in `other.etk`.
push1 some_label # <- Pushes a zero on the stack.
%include("other.etk")
Source: other.etk
different_label: # <- Not visible in `main.etk`.
push1 different_label # <- ALSO pushes a zero on the stack.
After Expansion
push1 0x00
push1 0x00
%include_hex("...")
The %include_hex macro functions exactly like %include, except instead of assembling the given path, it includes the raw hexadecimal bytes.
%push(...)
The %push macro will expand to a reasonably sized push instruction for the given argument.
For example:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" %push(hello) hello: jumpdest "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x02, 0x5b]); }
Will look something like the following after expansion:
push1 0x02
jumpdest
Expression Macros
selector("...")
The selector macro is useful when writing contracts that adhere to the Solidity ABI. Specifically, the selector macro expands to the four byte selector of the given function signature.
For example:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" push4 selector("transfer(address,uint256)") # <- expands to 0x63a9059cbb "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x63, 0xa9, 0x05, 0x9c, 0xbb]); }
The fully expanded source would look like:
push4 0xa9059cbb
topic("...")
The topic macro is operates similiarly to selector, except it returns the entire 32 byte hash digest. This is useful for the log opcodes.
For example:
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" push32 topic("transfer(address,uint256)") "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x7f, 169, 5, 156, 187, 42, 176, 158, 178, 25, 88, 63, 74, 89, 165, 208, 98, 58, 222, 52, 109, 150, 43, 205, 78, 70, 177, 29, 160, 71, 201, 4, 155]); }
The fully expanded source would look like:
push32 0xa9059cbb2ab09eb219583f4a59a5d0623ade346d962bcd4e46b11da047c9049b
Expression Macros
An expression macro is a type of expression which is resolved during assembly. This is useful for defining constant values or constant functions on values, such as defining getters on memory pointers.
Defining an Expression Macro
Expression macros can accept an arbitrary number of parameters. Parameters are referenced within the macro definition by prepending $ to the parameter's name.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" %def my_macro() 42 %end %def sum(x, y, z) $x+$y+$z %end "#; let mut ingest = etk_asm::ingest::Ingest::new(Vec::new()); ingest.ingest(file!(), src).unwrap(); }
Using an Expression Macro
Expression macros can be invoked anywhere an expression is expected.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" %def my_macro() 42 %end %def sum(x, y, z) $x+$y+$z %end push1 my_macro() push1 sum(1, 2, my_macro()) "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x2a, 0x60, 0x2d]); }
Instruction Macros
An instruction macro is a type of macro that can expand to an arbitrary number of instructions. This is useful for defining routines that are repeated many times or routines with expressions that are parameterized.
Defining an Instruction Macro
Instruction macros can accept an arbitrary number of parameters. Parameters are referenced within the macro definition by prepending $ to the parameter's name.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" %macro my_macro() push1 42 %end %macro sum(x, y, z) push1 $x+$y+$z %end "#; let mut ingest = etk_asm::ingest::Ingest::new(Vec::new()); ingest.ingest(file!(), src).unwrap(); }
Using a Instruction Macro
Expression macros can be invoked anywhere an instruction is expected.
#![allow(unused)] fn main() { extern crate etk_asm; let src = r#" %macro my_macro() push1 42 %end %macro sum(x, y, z) push1 $x+$y+$z %end %my_macro() %sum(1, 2, 3) "#; let mut output = Vec::new(); let mut ingest = etk_asm::ingest::Ingest::new(&mut output); ingest.ingest(file!(), src).unwrap(); assert_eq!(output, &[0x60, 0x2a, 0x60, 0x06]); }