uds-lsv/bert-stable-fine-tuning
On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines
Provides empirical analysis of fine-tuning instability across BERT, RoBERTa, and ALBERT on GLUE tasks, identifying vanishing gradients and generalization variance as root causes rather than catastrophic forgetting. Implements stable fine-tuning baselines built on Huggingface Transformers (v2.5.1) with reproducible Docker-based evaluation scripts. The codebase includes diagnostic tools for analyzing optimization dynamics and gradient behavior during training.
138 stars. No commits in the last 6 months.
Stars
138
Forks
21
Language
Python
License
Apache-2.0
Category
Last pushed
Sep 06, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/uds-lsv/bert-stable-fine-tuning"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
VanekPetr/flan-t5-text-classifier
Fine-tuning of Flan-5T LLM for text classification 🤖 focuses on adapting a state-of-the-art...
MeryylleA/lunariscodex
A high-performance PyTorch toolkit for pre-training modern, Llama-style language models. Based...
kingTLE/literary-alpaca2
从词表到微调这就是你所需的一切
RunxinXu/ChildTuning
Source code for our EMNLP'21 paper 《Raise a Child in Large Language Model: Towards Effective and...
aymen-000/predict-reconstruct-language-models
"Predict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning"...