Underfitting is a common problem in machine learning that occurs when a model is too simple to learn the underlying patterns and relationships in the training data.
In simple terms:
Underfitting happens when a machine learning model fails to learn enough from the data, causing it to make poor predictions on both training data and new unseen data.
An underfitted model cannot capture the complexity of the problem it is trying to solve, resulting in low accuracy and weak performance.
Why Does Underfitting Occur?
The primary goal of a machine learning model is to identify meaningful patterns within data and use those patterns to make accurate predictions.
However, if the model is too simple or does not have enough learning capacity, it may fail to understand important relationships in the dataset.
For example:
Imagine a student preparing for an exam but only reading a few pages of the textbook. Because the student has not learned enough material, they perform poorly not only on difficult questions but also on basic ones.
Similarly, an underfitted machine learning model has not learned enough from the training data to make reliable predictions.
Example of Underfitting
Suppose you want to predict house prices using information such as:
- House size
- Location
- Number of bedrooms
- Property age
- Nearby facilities
If you build a model that only considers house size and ignores all other factors, it may miss important relationships that influence pricing.
As a result:
- Predictions become inaccurate.
- Important patterns remain undiscovered.
- Overall model performance suffers.
This is a typical example of underfitting.
Characteristics of an Underfitted Model
1. Poor Training Performance
The model performs poorly even on the training dataset.
Since it cannot learn the underlying patterns, prediction errors remain high.
2. Poor Testing Performance
Because the model has not learned enough from the training data, it also performs poorly on unseen data.
3. High Bias
Underfitting is closely associated with high bias.
The model makes overly simple assumptions about the data and fails to capture important relationships.
4. Oversimplified Predictions
Predictions may appear too generalized and fail to reflect real-world complexity.
Common Causes of Underfitting
1. Model Is Too Simple
A model with insufficient complexity may not have the ability to learn complex patterns.
Examples include:
- Very simple linear models
- Shallow decision trees
- Small neural networks
2. Insufficient Training Time
If training is stopped too early, the model may not have enough opportunity to learn from the data.
3. Too Few Features
Important variables may be missing from the dataset.
Without relevant information, the model cannot make accurate predictions.
4. Excessive Regularization
Regularization helps prevent overfitting, but applying too much regularization can oversimplify the model and reduce its learning capability.
5. Poor Feature Engineering
Features that fail to represent important aspects of the problem can lead to underfitting.
How to Identify Underfitting
Several signs can indicate that a model is underfitting.
Low Training Accuracy
The model performs poorly on the training data itself.
For example:
- Training Accuracy: 60%
- Testing Accuracy: 58%
Since both values are low, the model has not learned enough.
High Error Rates
Metrics such as:
- Mean Squared Error (MSE)
- Mean Absolute Error (MAE)
- Classification Error
remain high during both training and testing.
Similar Training and Testing Performance
Unlike overfitting, where training performance is much higher than testing performance, underfitting usually results in similarly poor performance on both datasets.
Learning Curves Show Poor Performance
Learning curves often reveal that the model cannot achieve high accuracy even as more training data becomes available.
Underfitting vs Overfitting
Underfitting and overfitting are opposite problems.
Underfitting
- Model is too simple
- High bias
- Poor training performance
- Poor testing performance
- Fails to learn important patterns
Overfitting
- Model is too complex
- High variance
- Excellent training performance
- Poor testing performance
- Learns noise and irrelevant details
Ideal Model
The best model achieves a balance between underfitting and overfitting.
It learns meaningful patterns while maintaining strong generalization performance.
Techniques to Address Underfitting
1. Increase Model Complexity
A more sophisticated model can capture complex relationships more effectively.
Examples include:
- Deeper neural networks
- Larger decision trees
- Advanced machine learning algorithms
2. Add More Relevant Features
Including additional meaningful variables can improve the model's ability to learn.
For example:
A house price prediction model may benefit from features such as:
- Location
- School quality
- Property age
- Neighborhood information
3. Train for More Iterations
Allowing the model to train longer can help it learn additional patterns from the data.
4. Reduce Regularization
If regularization is too strong, reducing it may improve learning performance.
5. Improve Feature Engineering
Creating more informative features often leads to better model accuracy.
Examples include:
- Combining variables
- Creating interaction features
- Transforming existing data
6. Use More Advanced Algorithms
Switching from a simple model to a more powerful algorithm may improve performance.
Examples include:
- Random Forest
- Gradient Boosting
- Deep Learning Models
Real-World Examples of Underfitting
Healthcare
A disease prediction model that ignores important patient characteristics may fail to identify health risks accurately.
Finance
A credit scoring model using very limited information may incorrectly assess borrower risk.
Computer Vision
An image classification model with insufficient complexity may struggle to distinguish between similar objects.
Natural Language Processing
A text classification model trained on limited features may fail to understand language context properly.
Marketing Analytics
A customer behavior model may miss purchasing trends if important customer attributes are excluded.
Why Understanding Underfitting Is Important
Recognizing underfitting helps data scientists:
- Improve model accuracy
- Capture meaningful patterns
- Build reliable predictive systems
- Avoid poor business decisions
- Enhance real-world performance
Managing underfitting is essential for creating models that can learn effectively and make useful predictions.
Conclusion
Underfitting is a machine learning problem that occurs when a model is too simple to capture the underlying patterns and relationships within a dataset. As a result, the model performs poorly on both training data and unseen data because it has not learned enough from the available information. Underfitting is commonly caused by overly simple models, insufficient training, limited features, excessive regularization, or poor feature engineering. It can be identified through low training accuracy, high error rates, and weak performance across both training and testing datasets. By increasing model complexity, adding relevant features, improving feature engineering, reducing excessive regularization, and allowing more training, data scientists can address underfitting and build models that learn more effectively and deliver better predictive performance.