Why Are L1 & L2 Regularization Used in Machine Learning?

In Machine Learning, building a model that performs well on training data is not enough. The real goal is to build a model that can make accurate predictions on new, unseen data.
🎯

Please click here for artificial intelligence training

But what happens when a model learns the training data too closely, including unnecessary details and noise?

This problem is called Overfitting. 🚨

To reduce overfitting and improve a model’s ability to generalize, we use a technique called Regularization.

Let’s understand L1 and L2 Regularization in a simple and practical way! 👇

🔹 1. What Is Regularization in Machine Learning?

Regularization is a technique used to control the complexity of a Machine Learning model.

It adds a penalty to the model’s loss function when the model uses large coefficient values. This encourages the model to learn meaningful patterns instead of relying too heavily on the training data.

📌 Simple Example:

Imagine you’re training a Machine Learning model to predict house prices.

You provide features such as:
🏠 House size
📍 Location
🛏️ Number of bedrooms
🏡 Age of the house
🎨 Wall color

The model might learn that some features are useful while others contribute very little to predicting the price.

If the model becomes too complex and learns unnecessary patterns, it may perform well on training data but poorly on new houses.

👉 Regularization helps control this complexity and can improve performance on unseen data.

There are two popular regularization techniques:

🔹 L1 Regularization (Lasso)
🔹 L2 Regularization (Ridge)

—

🔹 2. What Is L1 Regularization (Lasso)?

L1 Regularization stands for Least Absolute Shrinkage and Selection Operator (Lasso).

It adds a penalty based on the absolute values of the model’s coefficients.

The important feature of L1 Regularization is that it can make some coefficients exactly zero.

When a coefficient becomes zero, that feature no longer contributes to the model’s prediction through that coefficient.

📌 Simple Example:

Suppose you’re predicting house prices using five features:

✅ House size
✅ Location
✅ Number of bedrooms
✅ Age of the house
❌ Wall color

If wall color provides little useful information, L1 Regularization may reduce its coefficient to zero.

This means the model can effectively ignore that feature.

💡 Key Benefits of L1 Regularization:
✔️ Can perform feature selection
✔️ Can create simpler models
✔️ Can improve generalization
✔️ Can help when many features are unnecessary

📌 Easy way to remember:

L1 = Lasso = Feature Selection

Important note: L1 does not always remove unnecessary features, and the features it selects depend on the dataset and model.

—

🔹 3. What Is L2 Regularization (Ridge)?

L2 Regularization is also known as Ridge Regularization.

It adds a penalty based on the squared values of the model’s coefficients.

Unlike L1, L2 generally shrinks coefficients toward zero without making them exactly zero.

📌 Simple Example:

Imagine your house price prediction model assigns these coefficients to different features:

🏠 House size = 80
📍 Location = 60
🛏️ Bedrooms = 45
🏡 House age = 35

Suppose some coefficients are unnecessarily large, making the model overly sensitive to particular features.

L2 Regularization penalizes large coefficient values and encourages the model to distribute its reliance more moderately across features.

The coefficients might become smaller, such as 65, 48, 36 and 28. These numbers are only illustrative; actual values depend on the data, model and regularization strength.

💡 Key Benefits of L2 Regularization:
✔️ Controls large coefficient values
✔️ Helps reduce model complexity
✔️ Can improve prediction stability
✔️ Often works well when several features contribute useful information

📌 Easy way to remember:

L2 = Ridge = Coefficient Shrinkage

—

🔹 4. L1 vs L2 Regularization: What Is the Difference?

Feature| L1 Regularization| L2 Regularization
Common name| Lasso| Ridge
Penalty| Absolute coefficients| Squared coefficients
Coefficients| Some can become exactly zero| Usually shrink toward zero
Feature selection| Can perform feature selection| Does not usually eliminate features
Main purpose| Encourage sparsity| Control large coefficients

🧠 One-line difference:

👉 L1 can remove features by setting coefficients to zero.

👉 L2 reduces the magnitude of coefficients to control their influence.

—

🔹 5. How Does Regularization Reduce Overfitting?

Without regularization, a model may become unnecessarily complex and fit noise in the training data.

With regularization, the model is penalized for excessive coefficient values.

This encourages it to learn simpler patterns that may generalize better to new data.

🎯 The goal is not simply to achieve the lowest training error. It is to balance training performance with the ability to make accurate predictions on unseen data.

⚠️ Regularization does not guarantee that overfitting will disappear. The penalty strength must be selected appropriately.

—

🔹 6. What Is the Regularization Parameter (Lambda)?

Both L1 and L2 Regularization use a parameter commonly represented by λ (lambda), or by a parameter such as alpha in many Machine Learning libraries.

It controls how strongly the model is penalized.

🔸 Low regularization strength: The model has more freedom to fit the training data, which may increase overfitting.

🔸 High regularization strength: The model is constrained more strongly, which may lead to underfitting if the penalty is excessive.

The ideal value is generally selected using validation data or cross-validation.

—

🚀 Final Takeaway

Regularization is an important concept in Machine Learning because it helps control model complexity and can improve generalization.

✅ L1 (Lasso) → Can eliminate features by setting coefficients to zero.

✅ L2 (Ridge) → Shrinks coefficients to control their magnitude.

✅ Regularization → Helps manage overfitting and improve performance on unseen data.

If you’re learning Machine Learning, understanding L1 and L2 Regularization will help you build better models and understand how algorithms balance accuracy and complexity.

💬 Quick Question: Which regularization technique can set some feature coefficients exactly to zero?

A) L1 Regularization (Lasso)
B) L2 Regularization (Ridge)

Comment your answer below! 👇

🚀 Follow for more AI, Machine Learning, Deep Learning and Generative AI concepts explained in a simple and practical way.