Base Parsers

This package contains the basic building blocks that the rest of the code is built on. The foundational element is the Parser class. Everything documented here can be imported from mtc.base.

Parser

class mtc.base.parser.Parser(value: Any)

Bases: object

The basic building block of the rest of the project. It provides a standard interface to serialize and deserialize an object to bytes (hence the name parser). Do not instantiate this class directly.

__init__(value: Any) → None

All subclasses initializers must have the same signature. This function must be idempotent due to how validation is handled.

to_bytes() → bytes

Serialize the object to bytes

classmethod parse(stream: BufferedIOBase) → Self

Deserialize the first object found from the stream. Raises ParsingError if the stream cannot be parsed.

classmethod skip(stream: BufferedIOBase) → None

skips the corresponding section in the stream, while doing minimum processing possible. This is especially useful when a large quantity of objects are serialized into a file.

__repr__() → str

__str__() → str

By default, returns the serialized bytes in hex format

__eq__(other: object) → bool

Compares two Parser object. They are equal if and only if their serialized bytes are equal

__len__() → int

Returns the length of the serialized bytes

__hash__() → int

Returns the hash of the serialized bytes

print() → str

Returns a string representation of the pretty-formatted byte structure of the object. Despite its name, this method does not write anything to stdout because it is sometimes recursively called in subclasses.

For example, calling

validate() → None

Performs basic validation on the data contained in the class and raises ValidationError if data is inconsistent. This function is a no-op on Parser and should be implemented in subclasses if needed

static disable_validation() → None

Disable validation for all Parser objects for the duration of the program. This operation cannot be reversed without restarting the program. Calling this method in subclasses is the same as calling it on Parser

exception mtc.base.parser.ParserError

Bases: Exception

exception mtc.base.parser.Parser.ParsingError(start: int, end: int, *args)

Bases: ParserError

exception mtc.base.parser.Parser.ValidationError

Bases: ParserError

Struct

The struct module defines the Struct class, which creates structs in accordance to the TLS presentation language and allows you to define them in a declarative way.

class mtc.base.struct.Struct(*values: Parser)

Bases: Parser

Implements a struct similar to how it works in C. With this class, you can define structs as simple as

class Assertion(Struct):
    subject_type: SubjectType
    subject_info: SubjectInfo
    claims: ClaimList

classmethod parse(stream: BufferedIOBase) → Self

classmethod skip(stream: BufferedIOBase) → None

to_bytes() → bytes

print() → str

validate() → None

Checks if all fields passed into the struct initializer are of the correct type in the correct order

class mtc.base.struct.StructMeta(name, bases, attrs, **kwargs)

Bases: type

The metaclass for Struct. This is what enables the dataclass-like behavior of Struct, but from inheritance instead of a class decorator. It reads the class annotation and instantiates the fields accordingly, in the order defined. It also defines __slots__ on the inherited classes to reduce access time and memory usage. All the metadata processed here is stored in the _fields attribute. For example, if you define a class like

class HashEmptyInput(Struct):
    hash_head: HashHead
    index: UInt64
    level: UInt8

Then HashEmptyInput._fields will be

HashEmptyInput._fields = [
    Field(name = 'hash_head', data_type = HashHead),
    Field(name = 'index', data_type = UInt64),
    Field(name = 'level', data_type = UInt8)
]

where Field is a named tuple.

Vector

The vector module defines 3 common types encountered in the TLS presentation language: Vector, OpaqueVector, and Array. Both Vector and OpaqueVector have a marker at the beginning of their serialized forms indicating how many bytes this vector contain, whereas Array does not because it is fixed size. All the lengths referred to in this module are in bytes.

To define a new vector, subclass Vector and define min_length, max_length, and data_type. For example,

class ClaimList(Vector):
    data_type = Claim
    min_length = 0
    max_length = 2 ** 16 - 1

To define a new opaque vector, subclass OpaqueVector and define min_length and max_length.

To define a new array, subclass Array and define length.

class mtc.base.vector.Vector(*values: Parser)

Bases: Parser

data_type: type[Parser]

max_length: int = 1

min_length: int = 1

marker_size: int = 1

to_bytes() → bytes

classmethod parse(data: BufferedIOBase) → Self

classmethod skip(stream: BufferedIOBase) → None

print() → str

validate() → None

class mtc.base.vector.OpaqueVector(value: bytes)

Bases: Parser

min_length: int = 0

max_length: int = 0

marker_size: int = 0

to_bytes() → bytes

classmethod parse(stream: BufferedIOBase) → Self

validate() → None

print() → str

classmethod skip(stream: BufferedIOBase) → None

class mtc.base.vector.Array(value: bytes)

Bases: Parser

length: int

to_bytes() → bytes

classmethod parse(stream: BufferedIOBase) → Self

classmethod skip(stream: BufferedIOBase) → None

validate() → None

print() → str

class mtc.base.vector.VectorMeta(name, bases, attrs, **kwargs)

Bases: type

Similar to StructMeta, this is what allows vectors to be defined in a declarative way behind the scene.

Numerical

Numerical classes handle common unsigned integer types. To define a new unsigned integer type, simply subclass Integer and define the size_in_bytes attribute. For example

class UInt32(Integer):
    size_in_bytes = 4

defines an unsigned 32-bit integer.

class mtc.base.numerical.Integer(value: int)

Bases: Parser

Base class for handling unsigned integers

size_in_bytes: int

to_bytes() → bytes

classmethod parse(stream: BufferedIOBase) → Self

classmethod skip(stream: BufferedIOBase) → None

print() → str

validate() → None

class mtc.base.numerical.UInt8(value: int)

Bases: Integer

class mtc.base.numerical.UInt16(value: int)

Bases: Integer

class mtc.base.numerical.UInt32(value: int)

Bases: Integer

class mtc.base.numerical.UInt64(value: int)

Bases: Integer

Enums

The Enum class is a thin wrapper around Python’s built-in enum type. To define a enum, you first define it as you normally would through enum.IntEnum. Then, define a new subclass of Enum and define both EnumClass and size_in_bytes. For example

class ClaimTypeEnum(enum.IntEnum):
    dns = 0
    dns_wildcard = 1
    ipv4 = 2
    ipv6 = 3

class ClaimType(Enum):
    EnumClass = ClaimTypeEnum
    size_in_bytes = 2

    dns: "ClaimType"
    dns_wildcard: "ClaimType"
    ipv4: "ClaimType"
    ipv6: "ClaimType"

Technically speaking, you don’t have to annotate anything on the Enum to access the attributes. In the example above, you can actually remove the last four lines of code and everything will still work just fine (e.g. you can still access ClaimType.ipv4). However, in order for IDE auto-completions and static type checkers to work correctly, you have to provide those annotations.

class mtc.base.enums.Enum(value: int)

Bases: Parser

Base class for handling enums

to_bytes() → bytes

classmethod parse(stream: BufferedIOBase) → Self

classmethod skip(stream: BufferedIOBase) → None

print() → str

class mtc.base.enums.EnumMeta(name, bases, attrs, **kwargs)

Bases: type

Similar to StructMeta, this is what allows enums to be proxied through a built-in enum and accessed by names.

Variant

Implements variant as defined in the TLS presentation language using Enum. To define a new variant, subclass mtc.base.variant.Variant and define vary_on_type and mapping. For example

class Claim(Variant):
vary_on_type = ClaimType
mapping = {
    ClaimType.dns: DNSNameList,
    ClaimType.dns_wildcard: DNSNameList,
    ClaimType.ipv4: IPv4AddressList,
    ClaimType.ipv6: IPv6AddressList
}

class mtc.base.variant.Variant(*args, **kwargs)

Bases: Parser

to_bytes() → bytes

classmethod parse(stream: BufferedIOBase) → Self

print() → str

Utils

This module contains some simple utility functions.

mtc.base.utils.bytes_needed(n: int) → int

calculates the minimum number of bytes needed to represent n (unsigned)

mtc.base.utils.bytes_to_int(b: bytes) → int

converts a bytes object to an int (unsigned)

mtc.base.utils.int_to_bytes(n: int, size: int) → bytes

converts n into its byte representation (unsigned) of size bytes.

mtc.base.utils.printable_bytes_truncate(b: bytes, limit: int) → str

Converts a bytes object into a string, with non-printable characters replaced by _. Similar to what you see in a hex-editor

Parameters:

• b – the bytes to be printed

• limit – the length limit

Returns:

the string with non-printable bytes replaced