California Housing Price Prediction: End-to-End PyTorch Deep Regression Pipeline
Regression with neural networks requires specialized preprocessing and loss metrics. Learn how to predict continuous home values using Huber Loss, custom PyTorch datasets, and learning rate scheduling.
PRACTICE THIS COMPLETE 7-STEP PIPELINE IN YOUR BROWSER
Write your code in the retro IDE, audit your tensor shapes, and receive sub-second AI diagnostics powered by Qwen 2.5 Coder.
California Housing Price Prediction: End-to-End PyTorch Deep Regression Pipeline
While classification is the most frequently taught deep learning task, continuous regression—predicting continuous real-valued targets such as home values, asset prices, temperature, and customer lifetime value—powers vast segments of industry applications.
Deep regression presents unique challenges: unconstrained output spaces, susceptibility to extreme outlier target values, and the risk of catastrophic loss scaling when errors square in Mean Squared Error (MSE).
In this tutorial, we build a 7-step Deep Tabular Regression Pipeline on the California Housing dataset using PyTorch, HuberLoss, and dynamic learning rate annealing.
#1. Regression vs Classification: Fundamental Differences
┌─────────────────────────────────┬──────────────────────────────────┐
│ Binary / Multiclass Tasks │ Continuous Regression Tasks │
├─────────────────────────────────┼──────────────────────────────────┤
│ Output layer: Logits + Softmax │ Output layer: Unbounded Linear │
│ Loss: BCEWithLogits / CrossEnt │ Loss: MSELoss, L1Loss, HuberLoss │
│ Metrics: Accuracy, ROC-AUC, F1 │ Metrics: RMSE, MAE, R² Score │
│ Target shape: [N] class indices │ Target shape: [N, 1] floats │
└─────────────────────────────────┴──────────────────────────────────┘#2. Step-by-Step Deep Regression Pipeline
Step 1: Data Ingestion & Target Distribution Analysis
import pandas as pd
import numpy as np
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
# Ingest California Housing feature matrix
housing = fetch_california_housing(as_frame=True)
df = housing.frame
print(df.head())
print(df.describe())
# Target: MedHouseVal (in $100,000s)
# Inspect target distribution skewness
print("Target Mean:", df['MedHouseVal'].mean(), "| Std:", df['MedHouseVal'].std())Step 2: Feature Transformation & Train/Val/Test Split
feature_cols = housing.feature_names
X = df[feature_cols].values
y = df['MedHouseVal'].values
# Stratified-style split by target quantiles or random split
X_train_raw, X_val_raw, y_train, y_val = train_test_split(
X, y, test_size=0.20, random_state=42
)
# Standardize inputs: zero-mean, unit variance
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train_raw)
X_val = scaler.transform(X_val_raw)Step 3: Tensor Datasets & Batched Loading
import torch
from torch.utils.data import TensorDataset, DataLoader
# Ensure y has shape [N, 1] to avoid silent broadcasting!
train_dataset = TensorDataset(
torch.tensor(X_train, dtype=torch.float32),
torch.tensor(y_train, dtype=torch.float32).unsqueeze(1)
)
val_dataset = TensorDataset(
torch.tensor(X_val, dtype=torch.float32),
torch.tensor(y_val, dtype=torch.float32).unsqueeze(1)
)
train_loader = DataLoader(train_dataset, batch_size=64, shuffle=True)
val_loader = DataLoader(val_dataset, batch_size=64, shuffle=False)Step 4: Regression MLP Architecture
import torch.nn as nn
class HousingRegressor(nn.Module):
def __init__(self, input_dim: int = 8, hidden_dim: int = 64):
super(HousingRegressor, self).__init__()
self.net = nn.Sequential(
nn.Linear(input_dim, hidden_dim),
nn.LayerNorm(hidden_dim),
nn.ReLU(),
nn.Dropout(0.1),
nn.Linear(hidden_dim, 32),
nn.LayerNorm(32),
nn.ReLU(),
nn.Linear(32, 16),
nn.ReLU(),
nn.Linear(16, 1) # Single continuous scalar output without activation
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
return self.net(x)
model = HousingRegressor(input_dim=8)Step 5: Why Huber Loss Outperforms MSE for Real Estate Data
Mean Squared Error (L_2) squares residuals (y - \hat{y})^2. In housing data with luxury multi-million-dollar outlier properties, a large residual produces huge gradient spikes that destabilize weights.
Huber Loss (Smooth L1 Loss) acts quadratic for small errors and linear for large errors (\delta = 1.0):
import torch.optim as optim
criterion = nn.HuberLoss(delta=1.0)
optimizer = optim.Adam(model.parameters(), lr=0.005, weight_decay=1e-4)
scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer, mode='min', factor=0.5, patience=5)Step 6: Epoch Training Loop with Dynamic Learning Rate Decay
EPOCHS = 40
for epoch in range(1, EPOCHS + 1):
model.train()
running_loss = 0.0
for bx, by in train_loader:
optimizer.zero_grad()
preds = model(bx)
loss = criterion(preds, by)
loss.backward()
optimizer.step()
running_loss += loss.item() * bx.size(0)
epoch_loss = running_loss / len(train_loader.dataset)
scheduler.step(epoch_loss)
if epoch % 10 == 0:
print(f"Epoch [{epoch:02d}/{EPOCHS}] | Huber Loss: {epoch_loss:.4f} | LR: {optimizer.param_groups[0]['lr']:.6f}")Step 7: Regression Diagnostics (RMSE, MAE, R²)
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
model.eval()
val_preds, val_targets = [], []
with torch.no_grad():
for bx, by in val_loader:
preds = model(bx)
val_preds.extend(preds.squeeze().tolist())
val_targets.extend(by.squeeze().tolist())
rmse = np.sqrt(mean_squared_error(val_targets, val_preds))
mae = mean_absolute_error(val_targets, val_preds)
r2 = r2_score(val_targets, val_preds)
print(f"--- REGRESSION EVALUATION ---")
print(f"Root Mean Squared Error (RMSE): ${rmse * 100_000:,.2f}")
print(f"Mean Absolute Error (MAE): ${mae * 100_000:,.2f}")
print(f"Coefficient of Determination (R²): {r2:.4f}")#Launch the Interactive Regression Lab
Practice continuous tabular regression in your browser with real-time AI validation.
READY TO AUDIT YOUR PYTORCH CODE LIVE?
Experience active deep learning with real-time feedback loops. No installation required—run directly in your browser.