Logic behind the Statement on Non-Parametric models

Question

I am currently reading 'Mastering Machine Learning with scikit-learn', 2E, by Packt. In Lazy Learning and Non-Parametric models topic in Chapter 3- Classification and Regression with k-Nearest Neighbors, there is a paragraph stating-
Non-parametric models can be useful when training data is abundant and you have little prior knowledge about the relationship between the response and the explanatory variables. kNN makes only one assumption: instances that are near each other are likely to have similar values of the response variable. The flexibility provided by the non-parametric models is not always desirable; a model that makes assumptions about the relationship can be useful if training data is scarce or if you already know about the relationship.
My doubt is that I am able to reason out the logic behind the statement- "a model that makes assumptions about the relationship can be useful if training data is scarce or if you already know about the relationship." But, I am not able to see any logic for the first statement- "Non-parametric models can be useful when training data is abundant and you have little prior knowledge about the relationship between the response and the explanatory variables."
Please help me to find out the logic behind it other than for computation. Thanks for your time and consideration.

prashant0598 · Accepted Answer

Non-parametric machine learning algorithms try to make assumptions about the data given the patterns observed from similar instances.By not making assumptions, they are free to learn any functional form from the training data.

Unlike parametric approach, where the number of parameters are fixed,
in non-parametric approaches the number of parameters grow with
training data.

If your data set is too small or otherwise is a set that is not representative of the entire population, then your result will be biased in more ways than possible with parametric methods.
So,to get better results in Non-parametric machine learning algorithms we need large amount of data where relationship between features are not known.
While non-parametric machine learning algorithms are often slower and require large amounts of data, they are rather flexible as they minimize the assumptions they make about the data.

Logic behind the Statement on Non-Parametric models

One Answer

Add your own answers!

Ask a Question