Model Compression is the final engineering step before deployment, crucial for "Edge AI"—running intelligence on devices with limited battery, thermal, and memory constraints (like drones, wearables, or IoT sensors).

While Model Optimization (quantization) changes the precision of the math, Model Compression changes the architecture itself. It aims to fundamentally reduce the number of calculations required, effectively doing more with less.

Here is the breakdown of the three primary compression pillars: Pruning, Factorization, and Distillation, followed by the downloadable Word file.

1. Pruning: The "Sparse" Revolution

Neural networks are notoriously inefficient; deep networks are often 90% "empty space."

• The Concept: Pruning removes connections (weights) that contribute little to the final output. It is like synaptic pruning in the human brain: "Use it or lose it."
• Unstructured Pruning: Randomly zeroing out weights. This creates a "sparse" matrix. It compresses storage well (like a ZIP file), but standard GPUs struggle to speed it up because they prefer dense blocks of data.
• Structured Pruning: Removing entire channels or filters. Instead of zeroing 50% of a layer randomly, we physically shrink the layer size by 50%. This directly translates to faster inference on all hardware.

2. Low-Rank Factorization (The Matrix Trick)

• The Problem: Large convolutional or dense layers involve massive matrix multiplications. A layer might have a weight matrix of size $1000 \times 1000$ (1 million parameters).
• The Fix: We approximate this massive matrix by multiplying two tiny matrices: one $1000 \times 10$ and one $10 \times 1000$.
• The Gain: Total parameters drop from 1,000,000 to 20,000.
• Impact: This dramatically reduces Memory Bandwidth usage, which is often the primary bottleneck on mobile devices (rather than raw compute speed).

3. Knowledge Distillation: The Teacher & Student

This is the standard technique for compressing Transformers (LLMs).

• Mechanism:
1. Teacher: A massive, high-accuracy model (e.g., BERT-Large).
2. Student: A tiny, compact model (e.g., DistilBERT).
3. Training: Instead of training the Student on "Hard Labels" (Is this a cat? Yes/No), we train it on the Teacher's "Soft Logits" (Is this a cat? 90% Yes, 9% Dog, 1% Car).
• Why it works: The "Soft Logits" contain rich information about the relationships between classes (e.g., Dogs look more like Cats than Cars), allowing the Student to learn faster and generalize better with fewer parameters.

4. Key Applications & Tools

Category	Tool	Usage
Pruning	PyTorch Pruning API	Native tools to zero out weights iteratively during training.
	Neural Magic (DeepSparse)	A specialized inference engine designed to run unstructured sparse models on CPUs at GPU-like speeds.
AutoML	NetAdapt / AMC	Automated tools that search for the best compression ratio per layer (e.g., "Prune Layer 1 by 20%, Prune Layer 2 by 80%").
Framework	TensorFlow Model Optimization Toolkit	Google's suite for applying pruning and clustering (weight sharing) to Keras models.