🔪 [LoRA] Low-Rank Adaptation of Large Language Models
LoRA Official Paper Review with Pytorch Implementation
🤖📖 Natural Language Process
🌆 [Linear Attention] Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
Linear Attention Official Paper Review with Pytorch Implementation
🗂️[SpanBERT] SpanBERT: Improving Pre-training by Representing and Predicting Spans
SpanBERT Official Paper Review with Pytorch Implementation
🎡 [Roformer] RoFormer: Enhanced Transformer with Rotary Position Embedding
Roformer Official Paper Review with Pytorch Implementation
👮 [ELECTRA] Pre-training Text Encoders as Discriminators Rather Than Generators
ELECTRA Official Paper Review with Pytorch Implementation
🧑🏫 [DistilBERT] DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
DistilBERT Official Paper Review with Pytorch Implementation
🪢 [DeBERTa-V3] DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing
DeBERTa-V3 Official Paper Review with Pytorch Implementation
🤖 [Transformer] Attention Is All You Need
Transformer Official Paper Review with Pytorch Implementation
🪢 [DeBERTa] DeBERTa: Decoding-Enhanced BERT with Disentangled-Attention
Transformer Official Paper Review with Pytorch Implementation