Very complex models (high degree polynomials) have low bias since you can get a descent approximation of the true function in expectation, but have high variance since they have the capabilities of fitting the noise in the data. Very simple models (constant functions) have high bias since your true function is usually not constant, but have low variance since they generally don't have the complexity to fit the noise of the specific dataset you got. There is a fundamental tradeoff between bias and variance that depends on how complex your model is. If slightly changing the dataset leads to a huge change in learned model, then variance is high.
Variance: How dependent the learned model is on the particular dataset it was learned on.Bias: How much our expected learned model (expectation is over all possible training sets) differs from the underlying model.In our model, y_i = f(x_i) + epsilon_i, and it is hopeless to learn the epsilon_i since its random and does not depend on the input Irreducible Error: The noise from the dataset.In lecture, we discussed how the source of error comes from one of three places
