selected LLM notes

Hot topics in LLM

LLM - basic info and the source to other notes for deeper understanding
LLM inference basics and inference optimization
Paper Review - Llama 3.1 model - how is the one of the best open-source models designed?

Tokenizers, embedding models - corner stone of each LLM task and Byte Pair Encoding as one of popular tokenization ideas
Scaling Laws - how to distribute compute budget efficiently between dataset size and number of trainable parameters?

RLHF - how reinforcement learning finally found its application
Reward Modeling - how it is used for RLHF and LLM fine-tuning?
Direct Preference Optimization - is it a more stable and cheap alternative to RLHF?

What is RAG? How does it work and how to maximize its impact?

Deep Learning architecture blocks

Transformer architecture and how each of the following blocks fit together
Attention, self-attention. multi-headed attention
Skip connection
Batch normalization
Layer normalization
Dropout
Gated Linear Unit
Briefly about transformer’s evolution or why is softmax cool

Evaluation of LLMs

LIST
FROM outgoing([[model evaluation]]) OR [[model evaluation]] AND -[[]]
WHERE contains(parent,[[model evaluation]])
SORT file.name ASC

---