Topic

On-Device AI

All digests tagged On-Device AI

Understand the Gemma 4 model family thumbnail

· 2:21

Understand the Gemma 4 model family

The Gemma 4 family is a set of multimodal, open-source large language models available in five sizes across four architectures. The models range from efficient, dense, on-device options (E2B, E4B) utilizing Per-Layer Embeddings (PLE), to advanced architectures like the encoder-free 12B model, the Mixture-of-Experts (MoE) 26B model, and the highly capable 31B dense model. Each size is optimized for specific use cases, including dedicated vision and audio processing.

Key takeaways

  1. E2B and E4B Models

    These smaller, dense models are optimized for on-device usage and utilize Per-Layer Embeddings (PLE), which are lookup tables for processing queries. They process audio and images using dedicated encoders.

  2. 12B Model Architecture

    This model is suitable for high-end laptops and employs an encoder-free method, removing dedicated encoders (e.g., audio encoder) and directly projecting audio to the LLM.

  3. 26B Model (MoE)

    This Mixture-of-Experts (MoE) model uses 26 billion parameters but only activates four billion at any given time ('A' for active). It features a larger vision encoder, making it ideal for difficult vision tasks.

  4. 31B Model

    This is the most capable dense model in the Gemma 4 family. It utilizes a larger vision encoder and is presented as the top-tier model.

Watch on YouTube Full article