In this article, we will see some examples of non-linear regression which are generally used in regression analysis the reason being that most of the real-world data follow highly complex and non-linear relationships between the dependent and the independent variables.
What is a Non-Linear Regression?
Non-Linear regression is a type of polynomial regression. It is a method to model a non-linear relationship between the dependent and independent variables. It is used in place when the data shows a curvy trend and linear regression would not produce very accurate results when compared to non-linear regression. This is because in linear regression it is pre-assumed that the data is linear.
There are many different regressions that exist and can be used to fit whatever the dataset looks like such as quadratic, cubic regression, and so on to infinite degrees according to our requirement.
How does a Non-Linear Regression work?
If we observe closely then we will realize that to evolve from linear regression to non-linear regression. We are just supposed to add the higher-order terms of the dependent features in the feature space. This is sometimes also known as feature engineering but not exactly.
The addition of non-linear terms is what allows us to fit a curvilinear model to the data at hand. Even though the non-linear regression is similar to the linear one but the different types of challenges are faced by the Machine Learning practitioner while training such a model. And hence several established methods, such as Levenberg-Marquardt and Gauss-Newton, are used to develop nonlinear models.
Python3
import numpy as npimport matplotlib.pyplot as plt%matplotlib inlinex = np.arange(-5.0, 5.0, 0.1)# You can adjust the slope and intercept# to verify the changes in the graphy = 2*(x) + 3y_noise = 2 * np.random.normal(size=x.size)ydata = y + y_noise# plt.figure(figsize =(8, 6))plt.plot(x, ydata, 'bo')plt.plot(x, y, 'r')plt.ylabel('Dependent Variable')plt.xlabel('Independent Variable')plt.show() |
Output:
Linear Regression
Now as we have seen an example of linear regression we will be able to appraise the non-linearity of the datasets and regressions. Let’s create quadratic regression data for instance.
Python3
import numpy as npimport matplotlib.pyplot as plt%matplotlib inlinex = np.arange(-5.0, 5.0, 0.1)# You can adjust the slope and intercept# to verify the changes in the graphy = np.power(x, 2)y_noise = 2 * np.random.normal(size=x.size)ydata = y + y_noiseplt.plot(x, ydata, 'bo')plt.plot(x, y, 'r')plt.ylabel('Dependent Variable')plt.xlabel('Independent Variable')plt.show() |
Output:
Quadratic Regression
Now let’s try to increase the degree of our polynomial by one more. This will help us get a feel for how non-linear data in the real world actually look.
Python3
import numpy as npimport matplotlib.pyplot as plt%matplotlib inlinex = np.arange(-5.0, 5.0, 0.1)# You can adjust the slope and intercept# to verify the changes in the graphy = 1*(x**3) + 1*(x**2) + 1 * x + 3y_noise = 20 * np.random.normal(size=x.size)ydata = y + y_noiseplt.plot(x, ydata, 'bo')plt.plot(x, y, 'r')plt.ylabel('Dependent Variable')plt.xlabel('Independent Variable')plt.show() |
Output:
Cubic Regression
For a model to be considered non-linear, Y hat must be a non-linear function of the parameters Theta, not necessarily the features X. When it comes to the non-linear equation, it can be the shape of exponential, logarithmic, logistic, or many other types.
Non-Linear Regression Equations
As you can see in all of these equations, the change of Y hat depends on changes in the parameters Theta, not necessarily on X only. That is, in non-linear regression, a model is non-linear by parameters.
What are the Applications of Non-Linear Regression?
As we know that most of the real-world data is non-linear and hence non-linear regression techniques are far better than linear regression techniques. Non-Linear regression techniques help to get a robust model whose predictions are reliable and as per the trend followed by the data in history. Tasks related to exponential growth or decay of a population, financial forecasting, and logistic pricing model were all successfully accomplished by the Non-Linear Regression techniques.
