Credit Card Fraud Detection with PyTorch: Handling Extreme Class Imbalance & Focal Loss
Classifying credit card transactions with a 99.83% negative class distribution creates an accuracy paradox. Master Weighted BCE, custom Focal Loss in PyTorch, and Precision-Recall AUC optimization.
PRACTICE THIS COMPLETE 7-STEP PIPELINE IN YOUR BROWSER
Write your code in the retro IDE, audit your tensor shapes, and receive sub-second AI diagnostics powered by Qwen 2.5 Coder.
Credit Card Fraud Detection with PyTorch: Handling Extreme Class Imbalance & Focal Loss
Financial fraud detection represents one of the most commercially critical applications of machine learning. In the canonical European Credit Card Fraud dataset (284,807 transactions), only 492 transactions are fraudulent (0.172%).
If a naive model predicts 0 (non-fraud) for every single transaction, it achieves a deceptive 99.83% raw accuracy while detecting exactly zero fraud cases.
In this deep-dive guide, we engineer a production-grade PyTorch Anomaly & Fraud Detection Pipeline that solves extreme class imbalance using Weighted Binary Cross-Entropy, custom Focal Loss, and Precision-Recall Area Under the Curve (PR-AUC) evaluation.
#1. The Imbalance Paradox: Why Standard Cross-Entropy Fails
When optimizing standard Binary Cross-Entropy with overwhelming negative samples (N_{neg} \gg N_{pos}):
The gradients from the millions of easy negative examples completely drown out the rare gradient signals from the few positive fraud examples during backpropagation. The network's weights update solely to minimize loss on the majority class.
#2. Mathematical Solutions to Severe Imbalance
Strategy A: Pos-Weight in BCEWithLogitsLoss
By assigning a positive class weight w_{pos} = \frac{N_{neg}}{N_{pos}} \approx 578.8, we scale the gradient penalty for false negatives:
Strategy B: Lin et al. Focal Loss
Focal Loss dynamically adds a modulating factor (1 - p_t)^\gamma to down-weight easy examples (p_t \to 1) and focus training on hard, ambiguous fraudulent transactions:
#3. Step-by-Step PyTorch Pipeline Implementation
Step 1 & 2: Ingestion, RobustScaler & Stratified Splitting
Features V1 to V28 are PCA-transformed components, but Amount and Time are unscaled continuous variables with heavy right-tail outliers.
import pandas as pd
import numpy as np
from sklearn.preprocessing import RobustScaler
from sklearn.model_selection import train_test_split
url = 'https://storage.googleapis.com/download.tensorflow.org/data/creditcard.csv'
df = pd.read_csv(url)
print(f"Total Transactions: {len(df)} | Fraud Count: {df['Class'].sum()} ({df['Class'].mean()*100:.3f}%)")
# RobustScaler uses median and IQR, resisting extreme outlier transactions
scaler = RobustScaler()
df['scaled_amount'] = scaler.fit_transform(df['Amount'].values.reshape(-1, 1))
df['scaled_time'] = scaler.fit_transform(df['Time'].values.reshape(-1, 1))
feature_cols = [f'V{i}' for i in range(1, 29)] + ['scaled_amount', 'scaled_time']
X = df[feature_cols].values
y = df['Class'].values
# Stratified Split guarantees both partitions have exact 0.172% fraud proportion
X_train, X_val, y_train, y_val = train_test_split(
X, y, test_size=0.20, random_state=42, stratify=y
)Step 3: PyTorch DataLoaders with Weighted Sampling (Optional) or Batching
import torch
from torch.utils.data import TensorDataset, DataLoader
train_dataset = TensorDataset(
torch.tensor(X_train, dtype=torch.float32),
torch.tensor(y_train, dtype=torch.float32).unsqueeze(1)
)
val_dataset = TensorDataset(
torch.tensor(X_val, dtype=torch.float32),
torch.tensor(y_val, dtype=torch.float32).unsqueeze(1)
)
BATCH_SIZE = 256
train_loader = DataLoader(train_dataset, batch_size=BATCH_SIZE, shuffle=True)
val_loader = DataLoader(val_dataset, batch_size=BATCH_SIZE, shuffle=False)Step 4: Custom Focal Loss Module in PyTorch
import torch.nn as nn
import torch.nn.functional as F
class BinaryFocalLoss(nn.Module):
def __init__(self, alpha: float = 0.25, gamma: float = 2.0, reduction: str = 'mean'):
super(BinaryFocalLoss, self).__init__()
self.alpha = alpha
self.gamma = gamma
self.reduction = reduction
def forward(self, logits: torch.Tensor, targets: torch.Tensor) -> torch.Tensor:
bce_loss = F.binary_cross_entropy_with_logits(logits, targets, reduction='none')
probs = torch.sigmoid(logits)
p_t = targets * probs + (1 - targets) * (1 - probs)
alpha_t = targets * self.alpha + (1 - targets) * (1 - self.alpha)
focal_weight = alpha_t * ((1.0 - p_t) ** self.gamma)
loss = focal_weight * bce_loss
if self.reduction == 'mean':
return loss.mean()
elif self.reduction == 'sum':
return loss.sum()
return lossStep 5: Fraud MLP Architecture with Dropout Regularization
class FraudClassifier(nn.Module):
def __init__(self, input_dim: int = 30, hidden_dim: int = 64):
super(FraudClassifier, self).__init__()
self.net = nn.Sequential(
nn.Linear(input_dim, hidden_dim),
nn.BatchNorm1d(hidden_dim),
nn.SiLU(), # Swish activation for smooth gradient backprop
nn.Dropout(0.3),
nn.Linear(hidden_dim, 32),
nn.BatchNorm1d(32),
nn.SiLU(),
nn.Dropout(0.2),
nn.Linear(32, 1)
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
return self.net(x)
model = FraudClassifier(input_dim=30)
criterion = BinaryFocalLoss(alpha=0.75, gamma=2.0)
optimizer = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-4)Step 6: Training Loop with Gradient Norm Clipping
EPOCHS = 15
for epoch in range(1, EPOCHS + 1):
model.train()
running_loss = 0.0
for bx, by in train_loader:
optimizer.zero_grad()
logits = model(bx)
loss = criterion(logits, by)
loss.backward()
# Clip gradient norm to avoid exploding gradients on rare fraud spikes
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)
optimizer.step()
running_loss += loss.item() * bx.size(0)
print(f"Epoch [{epoch:02d}/{EPOCHS}] | Focal Loss: {running_loss/len(train_loader.dataset):.6f}")Step 7: Precision-Recall Curve & Operating Threshold Tuning
In fraud detection, Precision-Recall AUC (PR-AUC) is vastly superior to ROC-AUC because ROC-AUC gives an over-optimistic score driven by large true negative counts.
from sklearn.metrics import precision_recall_curve, auc, classification_report, average_precision_score
model.eval()
all_probs, all_targets = [], []
with torch.no_grad():
for bx, by in val_loader:
logits = model(bx)
probs = torch.sigmoid(logits)
all_probs.extend(probs.squeeze().tolist())
all_targets.extend(by.squeeze().tolist())
# Calculate PR-AUC
precision, recall, thresholds = precision_recall_curve(all_targets, all_probs)
pr_auc = auc(recall, precision)
avg_prec = average_precision_score(all_targets, all_probs)
print(f"--- IMBALANCED FRAUD METRICS ---")
print(f"PR-AUC Score: {pr_auc:.4f}")
print(f"Average Precision: {avg_prec:.4f}")
# Find optimal threshold balancing F1 score for financial fraud
f1_scores = 2 * (precision * recall) / (precision + recall + 1e-8)
best_idx = np.argmax(f1_scores)
best_threshold = thresholds[best_idx] if best_idx < len(thresholds) else 0.5
print(f"Optimal Decision Threshold: {best_threshold:.4f} (Max F1: {f1_scores[best_idx]:.4f})")
binary_preds = [1 if p >= best_threshold else 0 for p in all_probs]
print(classification_report(all_targets, binary_preds, target_names=['Legit', 'Fraud']))#Summary & Live Interactive Lab
Mastering imbalanced learning separates entry-level data scientists from production engineers. Launch this full 7-step pipeline in our interactive browser lab with instant AI feedback.
READY TO AUDIT YOUR PYTORCH CODE LIVE?
Experience active deep learning with real-time feedback loops. No installation required—run directly in your browser.