CTRL K

Home
- Overview
- - NV-EMBED: IMPROVED TECHNIQUES FOR TRAINING LLMS AS GENERALIST EMBEDDING MODELS
    Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
- Overview
- - Muon: An optimizer for hidden layers in neural networks
  - NorMuon: Making Muon more efficient and scalable
    Muon is Scalable for LLM Training

Overview
- - Scalable Diffusion Models with Transformers
- - FLOW MATCHING FOR GENERATIVE MODELING
- - Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- - DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
  - AUXILIARY-LOSS-FREE LOAD BALANCING STRATEGY FOR MIXTURE-OF-EXPERTS
- - mHC: Manifold-Constrained Hyper-Connections
- - AN IMAGE IS WORTH 16X16 WORDS: TRANSFORMERS FOR IMAGE RECOGNITION AT SCALE
  - Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution
- - KIMI LINEAR: AN EXPRESSIVE, EFFICIENT ATTENTION ARCHITECTURE
  - Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

On This Page

ArchitectureOverview

Architecture

This section contains paper notes related to models architecture.

Categories