Usage

Contents:

• Setup
  • Prerequisites
  • Setup
• Datasets
  • Build a Dataset
  • Explore a Dataset with a Shell
  • Use a Dataset in a Script
  • VLLM Server Integration
• Modeling
  • Pretoken Processing
  • Tokenizer Training
  • Tokenization
  • Pre-Training (Maked Language Modeling)
  • Fine-Tuning (Contrastive Embeddings)
  • Fine-Tuning (Multi-Modal Summarization)