Welcome to RLite’s documentation!

RLite is a lightweight and efficient framework for training and inference of large language models. It is designed to be simple, flexible, and scalable. To minimize the learning overhead, RLite is built on top of PyTorch and HuggingFace Transformers, keeping as similar interfaces as possible.

Contents

• Quick Start: Write an RL sketch in 10 minutes
  • 1. Import packages
  • 2. Define your torch.nn.Module
  • 3. Initialize rlite
  • 4. Initialize a vLLM actor for inference
  • 5. Initialize a FSDP2 actor for training
  • 6. Call the user-defined training logic
  • 7. Sync weight from FSDP2 actor to vLLM actor
• Introduction
  • Why RLite?
  • Programming Model
  • Key Interfaces
    • Global Configuration and Resource Management
    • Inference Interface
    • Training Interface
    • Device Management Interface
    • Resource Management Interface
  • Code Structure
  • Limitations
• Inference with RLite
  • Inference with Language Models
  • Inference with Vision-Language Models
• Training with RLite
  • Train with HuggingFace Language Models
  • Train with HuggingFace Vision-Language Models
  • Train with HuggingFace peft Models
  • Train with Custom Models
• Resource Management
  • Specify the Acceptable Node By IP
  • Specify Colocation Relationship
  • Specify Non-Colocation Relationship
  • Useful Interfaces Provided by Resource Manager
• Debugging
• Frequently Asked Questions
  • Training with Multiple Nodes