（五十）：COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

（五十）：COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

• Abstract
• 1. Introduction
• 2. Related Work
• 3. Method
• • 3.1. Preliminary
  • 3.2. Pretraining by Correcting and Contrasting
  • • 3.2.1. CORRECTIVE LANGUAGE MODELING
    • 3.2.2. SEQUENCE CONTRASTIVE LEARNING
    • 3.2.3. COCO-LM TRAINING
• 4. Experimental Setup
• 5. Evaluation Results