polynomial regression torch

3 min read 14-11-2024

Introduction to Polynomial Regression

Polynomial regression is a form of regression analysis that models the relationship between a dependent variable and an independent variable as an nth degree polynomial. Unlike linear regression, which fits a straight line to the data, polynomial regression can capture more complex relationships by fitting a curved line. This guide will walk you through implementing polynomial regression using PyTorch, a powerful open-source machine learning library.

What is PyTorch?

PyTorch is a popular deep learning framework that provides flexibility and ease of use for developers. It allows for seamless integration of tensors and dynamic computation graphs, making it suitable for both research and production.

Why Use Polynomial Regression?

Polynomial regression is beneficial when:

The relationship between variables is nonlinear.
We want to capture complex patterns in data.
We need a simple model to explain relationships without making strong assumptions.

Setting Up Your Environment

To get started, ensure that you have Python and PyTorch installed. You can install PyTorch by running:

pip install torch torchvision

Importing Libraries

import numpy as np
import matplotlib.pyplot as plt
import torch
import torch.nn as nn
import torch.optim as optim

Generating Sample Data

Let’s create a sample dataset for our polynomial regression model.

# Generate synthetic data
np.random.seed(42)
X = np.random.rand(100, 1) * 10  # 100 random values in the range [0, 10)
y = 3 * X**2 + 2 * X + np.random.randn(100, 1) * 5  # Quadratic relationship with noise

# Convert to PyTorch tensors
X_tensor = torch.FloatTensor(X)
y_tensor = torch.FloatTensor(y)

Defining the Polynomial Regression Model

Next, we’ll define a polynomial regression model using PyTorch. We will use a simple feedforward neural network with one hidden layer.

class PolynomialRegressionModel(nn.Module):
    def __init__(self, degree):
        super(PolynomialRegressionModel, self).__init__()
        self.poly = nn.Sequential(
            nn.Linear(degree, 10),  # Hidden layer with 10 neurons
            nn.ReLU(),
            nn.Linear(10, 1)  # Output layer
        )

    def forward(self, x):
        return self.poly(x)

Preparing the Data for Polynomial Features

To perform polynomial regression, we need to generate polynomial features from our input data.

def polynomial_features(X, degree):
    return np.hstack([X**i for i in range(degree + 1)])

degree = 2
X_poly = polynomial_features(X, degree)
X_poly_tensor = torch.FloatTensor(X_poly)

Training the Model

Now, we’ll create an instance of our model, define the loss function and optimizer, and train the model.

# Initialize model, loss function, and optimizer
model = PolynomialRegressionModel(degree)
criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=0.01)

# Training loop
num_epochs = 5000
for epoch in range(num_epochs):
    model.train()
    
    # Forward pass
    outputs = model(X_poly_tensor)
    loss = criterion(outputs, y_tensor)

    # Backward pass and optimization
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (epoch + 1) % 500 == 0:
        print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {loss.item():.4f}')

Visualizing the Results

Finally, let's visualize the original data and the polynomial regression model's predictions.

# Plotting the results
model.eval()
with torch.no_grad():
    predicted = model(X_poly_tensor).detach().numpy()

plt.scatter(X, y, color='blue', label='Original data')
plt.plot(X, predicted, color='red', label='Polynomial regression fit')
plt.xlabel('X')
plt.ylabel('y')
plt.title('Polynomial Regression with PyTorch')
plt.legend()
plt.show()

Conclusion

In this article, we explored polynomial regression using PyTorch. We generated synthetic data, built a polynomial regression model, and visualized the results. By understanding and applying polynomial regression, you can capture nonlinear relationships in your data effectively.

Key Takeaways

Polynomial regression can model complex relationships between variables.
PyTorch provides a flexible framework for building and training models.
Generating polynomial features is crucial for polynomial regression.

By following these steps, you can implement polynomial regression for your datasets and enhance your machine learning repertoire. Happy coding!