All tutorials
    NLP
    Advanced
    90 min

    Fine-tuning GPT-2

    Fine-tune GPT-2 on a custom text dataset to generate domain-specific text completions.

    Load and Tokenize

    python
    from transformers import GPT2LMHeadModel, GPT2Tokenizer, TextDataset, DataCollatorForLanguageModeling tokenizer = GPT2Tokenizer.from_pretrained('gpt2')model = GPT2LMHeadModel.from_pretrained('gpt2')

    Try it in DLWΛY

    Open the Studio and follow along in a real project. There is nothing to install.

    Open Studio