In this section, you have seen how we can employ parametric models to predict house values as well as penguin species. However, we have still not covered the subject of model selection. How do we know if we should include more parameters in our models? When should we use a parametric model like logistic regression and when should we use a non-parameteric model like k-NN? How do we know which one is the best? You might also be curious about how the optimization methods that we use to find the parameters of a logistic regression model actually work.

In the next section, we will cover some concepts that can be helpful when answering the above-mentioned question. We will also learn about cross-validation as a tool for model selection. We will leave the optimization to a later section.