schema

Dataset schema definition and enforcement.

Functions

validate_dataset(frame, schema)

Validate a dataset against a given schema.

Classes

BinaryDataset(*args, **kwargs)	Compiled binaries.
BinaryDatasetWithSource(*args, **kwargs)	Binaries with source code.
Dataset(*args, **kwargs)	Base class for all schema.
DisassembledFunctionDataset(*args, **kwargs)	Disassembled functions.
DisassembledFunctionDatasetWithSource(*args, ...)	Disassembled functions with source code.
FunctionDataset(*args, **kwargs)	Individual functions.
FunctionDatasetWithSource(*args, **kwargs)	Functions with source code.
GeneratedSummarizedDataset(*args, **kwargs)	A dataset including text summaries with generated predictions.
SourceDataset(*args, **kwargs)	Source code.
SummarizedDataset(*args, **kwargs)	A dataset including text summaries.
TokenizedClassificationDataset(*args, **kwargs)	A tokenized dataset with integer classification labels.
TokenizedDataset(*args, **kwargs)	A tokenized dataset.
TokenizedSummarizationDataset(*args, **kwargs)	A tokenized dataset with tokenized summaries.

class undertale.schema.Dataset(*args, **kwargs)

Bases: DataFrameModel

Base class for all schema.

id: Series[str] = 'id'

Row identifier.

class undertale.schema.SourceDataset(*args, **kwargs)

Bases: Dataset

Source code.

source: Series[str] = 'source'

Source code.

class undertale.schema.BinaryDataset(*args, **kwargs)

Bases: Dataset

Compiled binaries.

binary: Series[bytes] = 'binary'

Compiled binary code.

class undertale.schema.BinaryDatasetWithSource(*args, **kwargs)

Bases: BinaryDataset, SourceDataset

Binaries with source code.

class undertale.schema.FunctionDataset(*args, **kwargs)

Bases: Dataset

Individual functions.

name: Series[str] = 'name'

The name of the function.

class undertale.schema.FunctionDatasetWithSource(*args, **kwargs)

Bases: FunctionDataset, SourceDataset

Functions with source code.

class undertale.schema.DisassembledFunctionDataset(*args, **kwargs)

Bases: FunctionDataset

Disassembled functions.

class undertale.schema.DisassembledFunctionDatasetWithSource(*args, **kwargs)

Bases: DisassembledFunctionDataset, SourceDataset

Disassembled functions with source code.

class undertale.schema.SummarizedDataset(*args, **kwargs)

Bases: Dataset

A dataset including text summaries.

summary: Series[str] = 'summary'

Human-readable summary.

class undertale.schema.TokenizedDataset(*args, **kwargs)

Bases: Dataset

A tokenized dataset.

tokens: Series[object] = 'tokens'

The token IDs for the tokenized row.

mask: Series[object] = 'mask'

The attention mask for the tokenized row.

class undertale.schema.TokenizedClassificationDataset(*args, **kwargs)

Bases: TokenizedDataset

A tokenized dataset with integer classification labels.

label: Series[int] = 'label'

Integer class label for the sequence.

class undertale.schema.TokenizedSummarizationDataset(*args, **kwargs)

Bases: TokenizedDataset

A tokenized dataset with tokenized summaries.

summary_tokens: Series[object] = 'summary_tokens'

The token IDs for the summary.

summary_mask: Series[object] = 'summary_mask'

The attention mask for the summary.