# Backgroud

bias-variance trade-off 在 Machine learning中蠻常被提及的一個名詞. 但研究所的時候老是搞不懂這兩個的差別. (可能是我沒好好的在聽老師上課). 直到最近有需要再重新自己好好研究一下. 並做個筆記, 記錄一下.

In this post, you will discover the Bias-Variance Trade-Off and how to use it to better understand machine learning algorithms and get better performance on your data.

The prediction error for any machine learning algorithm can be broken down into three parts:

• Bias Error
• Variance Error
• Irreducible Error

# Bias Error

Bias are the simplifying assumptions made by a model to make the target function easier to learn.

# Variance Error

Variance is the amount that the estimate of the target function will change if different training data was used.

Bias反映的是模型在樣本上的輸出與真實值之間的誤差，即模型本身的精準度，Variance反映的是模型每一次輸出結果與模型輸出期望之間的誤差，即模型的穩定性. 那在一個實際系統中，Bias與Variance往往是不能兼得的.

# Example

• The k-nearest neighbors algorithm has low bias and high variance, but the trade-off can be changed by increasing the value of k which increases the number of neighbors that contribute t the prediction and in turn increases the bias of the model.

• The support vector machine algorithm has low bias and high variance, but the trade-off can be changed by increasing the C parameter that influences the number of violations of the margin allowed in the training data which increases the bias but decreases the variance.

• Parametric or linear machine learning algorithms often have a high bias but a low variance.

• Non-parametric or non-linear machine learning algorithms often have a low bias but a high variance.