In this section you have hopefully gained some insights into how we can evaluate model performance and choose between multiple models. It is not always the case that the model that adapts best to the training data also will perform well on new, previously unseen data. Therefore, model selection and evaluation is usually done through cross-validation or with a separate validation set. The latter approach is commonly used when the dataset is large or the model is highly complex, since training the model several times, as is done in cross-validation, is both computationally costly and time consuming.

We also covered the subject of model flexibility in relation to overfitting. So far, models with relatively few parameters have worked well on the examples that you have encountered. In the simple classification example in this section, the model overfitted to the training data already with only 15 model parameters. In this example, we avoided overfitting by keeping the parameter count low. However, there will be situations where we actually need a very high model flexibility, and a large number of parameters, in order to achieve good performance. In section 3, we discussed how we can increase model complexity by extending the linear regression model to a polynomial regression model and in this section, we extended the same idea to logistic regression. You might wonder if this is the only way to achieve a higher model complexity. If so, you will get your answer in the next section, where we will introduce another class of parametric models; namely neural networks.