Lab 5: Building Your First Neural Network with MLP

Bank Marketing Campaign Prediction

16 Pendahuluan

16.1 Gambaran Umum Lab

Selamat datang di Lab 5! Dalam praktikum ini, Anda akan membangun neural network pertama Anda menggunakan Multi-Layer Perceptron (MLP) untuk memprediksi apakah seorang nasabah bank akan berlangganan deposito berjangka atau tidak.

Mengapa MLP Penting?

Multi-Layer Perceptron adalah fondasi dari deep learning modern. Memahami MLP akan membantu Anda:

1. Memahami cara kerja neural networks
2. Menguasai konsep backpropagation dan gradient descent
3. Mempelajari teknik regularisasi (dropout, L2)
4. Mempersiapkan diri untuk arsitektur yang lebih kompleks (CNN, RNN)

16.2 Tujuan Pembelajaran

Setelah menyelesaikan lab ini, Anda diharapkan mampu:

1. Membangun model MLP menggunakan Keras (TensorFlow)
2. Mengimplementasikan MLP menggunakan PyTorch dari nol
3. Menerapkan teknik regularisasi (dropout, L2 regularization)
4. Melakukan hyperparameter tuning untuk optimasi performa
5. Mengevaluasi performa neural network dengan berbagai metrik
6. Menangani dataset tidak seimbang dalam konteks deep learning

16.3 Dataset: Bank Marketing

16.3.1 Deskripsi Dataset

Dataset Bank Marketing berasal dari UCI Machine Learning Repository dan berisi data kampanye pemasaran langsung dari sebuah lembaga perbankan Portugal. Kampanye pemasaran dilakukan melalui telepon untuk menawarkan deposito berjangka.

Informasi Dataset:

• Jumlah Sampel: 41,188
• Jumlah Fitur: 20 fitur (numerik dan kategorikal)
• Target: Binary classification (yes/no - apakah nasabah berlangganan?)
• Tantangan: Ketidakseimbangan kelas (~11% kelas positif)

16.3.2 Fitur Dataset

Fitur Demografis:

• age: Usia nasabah
• job: Jenis pekerjaan
• marital: Status pernikahan
• education: Tingkat pendidikan

Fitur Keuangan:

• balance: Saldo rata-rata tahunan (dalam euro)
• housing: Memiliki pinjaman rumah?
• loan: Memiliki pinjaman pribadi?

Fitur Kampanye:

• contact: Tipe komunikasi kontak
• day: Hari terakhir dihubungi
• month: Bulan terakhir dihubungi
• duration: Durasi kontak terakhir (detik)
• campaign: Jumlah kontak selama kampanye ini
• pdays: Hari sejak kontak terakhir dari kampanye sebelumnya
• previous: Jumlah kontak sebelum kampanye ini
• poutcome: Hasil kampanye pemasaran sebelumnya

Target Variable:

• y: Apakah klien berlangganan deposito berjangka? (yes/no)

Imbalanced Dataset

Dataset ini sangat tidak seimbang dengan hanya ~11% sampel kelas positif. Ini akan menjadi tantangan khusus yang akan kita tangani dengan berbagai teknik!

16.4 Struktur Lab

Lab ini dibagi menjadi 4 bagian utama dengan estimasi waktu 4-5 jam:

Bagian	Topik	Durasi
1	Data Preparation & EDA	60 menit
2	Keras Implementation	90 menit
3	PyTorch Implementation	90 menit
4	Advanced Techniques	60 menit

16.5 Persiapan Environment

16.5.1 Install Required Libraries

Jalankan kode berikut untuk memastikan semua library terinstall:

# Install TensorFlow/Keras
pip install tensorflow>=2.13.0

# Install PyTorch (CPU version)
pip install torch>=2.0.0

# Install supporting libraries
pip install scikit-learn pandas numpy matplotlib seaborn
pip install imbalanced-learn  # untuk SMOTE

GPU vs CPU

Untuk lab ini, CPU sudah cukup karena dataset relatif kecil. Namun, jika Anda memiliki GPU NVIDIA, PyTorch akan otomatis menggunakannya untuk mempercepat training.

16.5.2 Import Libraries

# Data manipulation
import pandas as pd
import numpy as np

# Visualization
import matplotlib.pyplot as plt
import seaborn as sns

# Scikit-learn utilities
from sklearn.model_selection import train_test_split, StratifiedKFold
from sklearn.preprocessing import StandardScaler, LabelEncoder
from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score,
    confusion_matrix, classification_report, roc_auc_score, roc_curve
)

# Keras/TensorFlow
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers, regularizers
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau

# PyTorch
import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader, TensorDataset

# Handling imbalanced data
from imblearn.over_sampling import SMOTE

# Utilities
import warnings
warnings.filterwarnings('ignore')

# Set random seeds for reproducibility
np.random.seed(42)
tf.random.set_seed(42)
torch.manual_seed(42)

# Set plot style
plt.style.use('seaborn-v0_8-darkgrid')
sns.set_palette("husl")

print("TensorFlow version:", tf.__version__)
print("PyTorch version:", torch.__version__)
print("GPU Available (TensorFlow):", tf.config.list_physical_devices('GPU'))
print("GPU Available (PyTorch):", torch.cuda.is_available())

---

17 Part 1: Data Preparation & EDA

17.1 Download Dataset

Dataset Bank Marketing dapat diunduh dari UCI ML Repository.

# Download dataset
import urllib.request
import zipfile
import os

# URL dataset
url = "https://archive.ics.uci.edu/ml/machine-learning-databases/00222/bank-additional.zip"

# Download
print("Downloading dataset...")
urllib.request.urlretrieve(url, "bank-additional.zip")

# Extract
print("Extracting dataset...")
with zipfile.ZipFile("bank-additional.zip", 'r') as zip_ref:
    zip_ref.extractall("bank_data")

print("Dataset downloaded successfully!")

Alternatif Download

Jika download gagal, Anda dapat mengunduh dataset secara manual dari:

https://archive.ics.uci.edu/ml/datasets/Bank+Marketing

Gunakan file bank-additional-full.csv untuk full dataset.

17.2 Load & Explore Dataset

# Load dataset
df = pd.read_csv('bank_data/bank-additional/bank-additional-full.csv', sep=';')

# Display basic info
print("=" * 60)
print("INFORMASI DATASET")
print("=" * 60)
print(f"Jumlah baris: {df.shape[0]:,}")
print(f"Jumlah kolom: {df.shape[1]}")
print(f"\nTipe data:")
print(df.dtypes)
print(f"\nMemori usage: {df.memory_usage(deep=True).sum() / 1024**2:.2f} MB")

# Display first few rows
print("\n" + "=" * 60)
print("PREVIEW DATA (5 baris pertama)")
print("=" * 60)
df.head()

Output Expected:

============================================================
INFORMASI DATASET
============================================================
Jumlah baris: 41,188
Jumlah kolom: 21

Tipe data:
age            int64
job           object
marital       object
education     object
default       object
...

Memori usage: 6.62 MB

17.3 Exploratory Data Analysis

17.3.1 Statistik Deskriptif

# Statistik deskriptif untuk fitur numerik
print("=" * 60)
print("STATISTIK DESKRIPTIF - FITUR NUMERIK")
print("=" * 60)
df.describe()

# Informasi fitur kategorikal
print("\n" + "=" * 60)
print("FITUR KATEGORIKAL")
print("=" * 60)

categorical_cols = df.select_dtypes(include=['object']).columns

for col in categorical_cols:
    print(f"\n{col.upper()}:")
    print(f"  Unique values: {df[col].nunique()}")
    print(f"  Values: {df[col].unique()[:10]}")  # Tampilkan max 10 values

17.3.2 Target Distribution

# Analisis target variable
print("=" * 60)
print("DISTRIBUSI TARGET VARIABLE")
print("=" * 60)

target_counts = df['y'].value_counts()
target_pct = df['y'].value_counts(normalize=True) * 100

print(f"\nCount:")
print(target_counts)
print(f"\nPercentage:")
print(target_pct)

# Visualisasi
fig, axes = plt.subplots(1, 2, figsize=(14, 5))

# Count plot
target_counts.plot(kind='bar', ax=axes[0], color=['#FF6B6B', '#4ECDC4'])
axes[0].set_title('Target Distribution (Count)', fontsize=14, fontweight='bold')
axes[0].set_xlabel('Subscribed', fontsize=12)
axes[0].set_ylabel('Count', fontsize=12)
axes[0].set_xticklabels(['No', 'Yes'], rotation=0)
axes[0].grid(axis='y', alpha=0.3)

# Add count labels
for i, v in enumerate(target_counts):
    axes[0].text(i, v + 500, f'{v:,}', ha='center', va='bottom', fontweight='bold')

# Pie chart
colors = ['#FF6B6B', '#4ECDC4']
target_counts.plot(kind='pie', ax=axes[1], autopct='%1.1f%%',
                   colors=colors, startangle=90)
axes[1].set_title('Target Distribution (Percentage)', fontsize=14, fontweight='bold')
axes[1].set_ylabel('')

plt.tight_layout()
plt.show()

# Calculate imbalance ratio
imbalance_ratio = target_counts['no'] / target_counts['yes']
print(f"\n⚠️  Imbalance Ratio: {imbalance_ratio:.2f}:1")
print(f"    (Untuk setiap 1 sampel positif, ada {imbalance_ratio:.2f} sampel negatif)")

Class Imbalance

Dataset ini sangat tidak seimbang dengan rasio ~8:1. Ini berarti:

• Model cenderung bias ke kelas mayoritas (no)
• Akurasi saja tidak cukup sebagai metrik evaluasi
• Perlu teknik khusus: class weights, SMOTE, atau threshold tuning

17.3.3 Analisis Fitur Numerik

# Pilih fitur numerik
numeric_cols = ['age', 'duration', 'campaign', 'pdays', 'previous',
                'emp.var.rate', 'cons.price.idx', 'cons.conf.idx',
                'euribor3m', 'nr.employed']

# Visualisasi distribusi fitur numerik
fig, axes = plt.subplots(5, 2, figsize=(15, 20))
axes = axes.ravel()

for idx, col in enumerate(numeric_cols):
    axes[idx].hist(df[col], bins=50, color='skyblue', edgecolor='black', alpha=0.7)
    axes[idx].set_title(f'Distribution of {col}', fontsize=12, fontweight='bold')
    axes[idx].set_xlabel(col, fontsize=10)
    axes[idx].set_ylabel('Frequency', fontsize=10)
    axes[idx].grid(axis='y', alpha=0.3)

    # Add statistics
    mean_val = df[col].mean()
    median_val = df[col].median()
    axes[idx].axvline(mean_val, color='red', linestyle='--', linewidth=2, label=f'Mean: {mean_val:.2f}')
    axes[idx].axvline(median_val, color='green', linestyle='--', linewidth=2, label=f'Median: {median_val:.2f}')
    axes[idx].legend(fontsize=8)

plt.tight_layout()
plt.show()

17.3.4 Korelasi Fitur Numerik

# Hitung korelasi
correlation_matrix = df[numeric_cols + ['y']].copy()
correlation_matrix['y'] = (correlation_matrix['y'] == 'yes').astype(int)
corr = correlation_matrix.corr()

# Visualisasi correlation heatmap
plt.figure(figsize=(14, 12))
mask = np.triu(np.ones_like(corr, dtype=bool))
sns.heatmap(corr, mask=mask, annot=True, fmt='.2f', cmap='coolwarm',
            center=0, square=True, linewidths=1, cbar_kws={"shrink": 0.8})
plt.title('Correlation Heatmap - Numeric Features', fontsize=16, fontweight='bold', pad=20)
plt.tight_layout()
plt.show()

# Korelasi dengan target
print("\n" + "=" * 60)
print("KORELASI DENGAN TARGET (y)")
print("=" * 60)
target_corr = corr['y'].sort_values(ascending=False)
print(target_corr)

Interpretasi Korelasi

Fitur dengan korelasi tinggi (positif/negatif) dengan target lebih penting:

• Positif: Duration, poutcome_success → meningkatkan peluang subscribe
• Negatif: Campaign, pdays → menurunkan peluang subscribe

17.3.5 Analisis Fitur Kategorikal

# Analisis hubungan fitur kategorikal dengan target
categorical_cols = ['job', 'marital', 'education', 'default', 'housing',
                   'loan', 'contact', 'month', 'day_of_week', 'poutcome']

fig, axes = plt.subplots(5, 2, figsize=(16, 25))
axes = axes.ravel()

for idx, col in enumerate(categorical_cols):
    # Buat crosstab
    ct = pd.crosstab(df[col], df['y'], normalize='index') * 100

    # Plot
    ct.plot(kind='bar', ax=axes[idx], color=['#FF6B6B', '#4ECDC4'])
    axes[idx].set_title(f'{col.upper()} vs Target', fontsize=12, fontweight='bold')
    axes[idx].set_xlabel(col, fontsize=10)
    axes[idx].set_ylabel('Percentage (%)', fontsize=10)
    axes[idx].legend(['No', 'Yes'], title='Subscribed')
    axes[idx].grid(axis='y', alpha=0.3)
    axes[idx].tick_params(axis='x', rotation=45)

plt.tight_layout()
plt.show()

17.4 Missing Values & Data Quality

# Check missing values
print("=" * 60)
print("MISSING VALUES CHECK")
print("=" * 60)

missing_counts = df.isnull().sum()
missing_pct = (df.isnull().sum() / len(df)) * 100
missing_df = pd.DataFrame({
    'Column': missing_counts.index,
    'Missing Count': missing_counts.values,
    'Missing %': missing_pct.values
})
missing_df = missing_df[missing_df['Missing Count'] > 0].sort_values('Missing Count', ascending=False)

if len(missing_df) == 0:
    print("✅ Tidak ada missing values!")
else:
    print(missing_df)

# Check for 'unknown' values (treated as missing)
print("\n" + "=" * 60)
print("'UNKNOWN' VALUES CHECK")
print("=" * 60)

for col in categorical_cols:
    unknown_count = (df[col] == 'unknown').sum()
    unknown_pct = (unknown_count / len(df)) * 100
    if unknown_count > 0:
        print(f"{col:20s}: {unknown_count:5d} ({unknown_pct:5.2f}%)")

17.5 Data Preprocessing

17.5.1 Handle Missing/Unknown Values

# Buat copy dataframe untuk preprocessing
df_processed = df.copy()

# Replace 'unknown' dengan mode untuk setiap kolom kategorikal
print("Mengganti 'unknown' values dengan mode...")

for col in categorical_cols:
    if 'unknown' in df_processed[col].values:
        mode_value = df_processed[df_processed[col] != 'unknown'][col].mode()[0]
        df_processed[col] = df_processed[col].replace('unknown', mode_value)
        print(f"  {col}: replaced with '{mode_value}'")

print("✅ Selesai menangani unknown values!")

17.5.2 Feature Engineering

# Tambahkan fitur baru yang mungkin berguna

# 1. Balance category
df_processed['balance_category'] = pd.cut(df_processed['balance'],
                                           bins=[-np.inf, 0, 1000, 5000, np.inf],
                                           labels=['negative', 'low', 'medium', 'high'])

# 2. Age group
df_processed['age_group'] = pd.cut(df_processed['age'],
                                   bins=[0, 30, 40, 50, 60, 100],
                                   labels=['young', 'adult', 'middle', 'senior', 'elderly'])

# 3. Contact frequency (campaign indicator)
df_processed['high_contact_freq'] = (df_processed['campaign'] > 3).astype(int)

# 4. Previous success indicator
df_processed['prev_success'] = (df_processed['poutcome'] == 'success').astype(int)

print("✅ Feature engineering selesai!")
print(f"Jumlah fitur baru: {df_processed.shape[1] - df.shape[1]}")
print(f"Total fitur sekarang: {df_processed.shape[1]}")

17.5.3 Encode Categorical Variables

# Separate target variable
X = df_processed.drop('y', axis=1)
y = df_processed['y'].map({'no': 0, 'yes': 1})

print("=" * 60)
print("ENCODING CATEGORICAL VARIABLES")
print("=" * 60)

# Identifikasi kolom kategorikal
categorical_columns = X.select_dtypes(include=['object']).columns.tolist()
print(f"\nKolom kategorikal yang akan di-encode: {len(categorical_columns)}")
print(categorical_columns)

# Binary encoding untuk kolom dengan 2 kategori
binary_cols = []
for col in categorical_columns:
    if X[col].nunique() == 2:
        binary_cols.append(col)
        # Label encoding untuk binary
        le = LabelEncoder()
        X[col] = le.fit_transform(X[col])
        print(f"  Binary encoding: {col}")

# One-hot encoding untuk kolom dengan >2 kategori
multi_category_cols = [col for col in categorical_columns if col not in binary_cols]
if multi_category_cols:
    print(f"\nOne-hot encoding untuk {len(multi_category_cols)} kolom:")
    X = pd.get_dummies(X, columns=multi_category_cols, drop_first=True)
    print(f"  Jumlah fitur setelah encoding: {X.shape[1]}")

print(f"\n✅ Encoding selesai!")
print(f"   Shape akhir: X {X.shape}, y {y.shape}")

17.5.4 Train-Test Split dengan Stratification

# Split data dengan stratification untuk menjaga proporsi kelas
X_train, X_test, y_train, y_test = train_test_split(
    X, y,
    test_size=0.2,
    random_state=42,
    stratify=y
)

print("=" * 60)
print("TRAIN-TEST SPLIT")
print("=" * 60)
print(f"\nTraining set:")
print(f"  X_train: {X_train.shape}")
print(f"  y_train: {y_train.shape}")
print(f"  Distribusi kelas: {y_train.value_counts().to_dict()}")

print(f"\nTest set:")
print(f"  X_test: {X_test.shape}")
print(f"  y_test: {y_test.shape}")
print(f"  Distribusi kelas: {y_test.value_counts().to_dict()}")

# Verifikasi stratification
print("\n" + "=" * 60)
print("VERIFIKASI STRATIFICATION")
print("=" * 60)
print(f"Original positive class ratio: {y.mean():.4f}")
print(f"Train positive class ratio: {y_train.mean():.4f}")
print(f"Test positive class ratio: {y_test.mean():.4f}")

17.5.5 Feature Scaling

# Standardize features menggunakan StandardScaler
scaler = StandardScaler()

# Fit pada training set dan transform keduanya
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

print("=" * 60)
print("FEATURE SCALING")
print("=" * 60)
print(f"Scaler: StandardScaler")
print(f"Shape X_train_scaled: {X_train_scaled.shape}")
print(f"Shape X_test_scaled: {X_test_scaled.shape}")

# Lihat statistik sebelum dan sesudah scaling
print("\nStatistik fitur pertama SEBELUM scaling:")
print(f"  Mean: {X_train.iloc[:, 0].mean():.2f}")
print(f"  Std: {X_train.iloc[:, 0].std():.2f}")

print("\nStatistik fitur pertama SESUDAH scaling:")
print(f"  Mean: {X_train_scaled[:, 0].mean():.2e}")
print(f"  Std: {X_train_scaled[:, 0].std():.4f}")

print("\n✅ Feature scaling selesai!")

Mengapa Feature Scaling Penting?

Neural networks sangat sensitif terhadap skala fitur karena:

1. Gradient descent lebih stabil dengan fitur yang ter-standardisasi
2. Learning rate optimal lebih mudah ditemukan
3. Konvergensi lebih cepat karena loss landscape lebih smooth
4. Mencegah fitur dengan skala besar mendominasi learning process

---

18 Part 2: Keras Implementation

18.1 Baseline MLP Model

18.1.1 Arsitektur Model Baseline

Mari kita mulai dengan model MLP sederhana:

# Tentukan input dimension
input_dim = X_train_scaled.shape[1]

print("=" * 60)
print("BASELINE MLP ARCHITECTURE")
print("=" * 60)

# Build model menggunakan Sequential API
model_baseline = keras.Sequential([
    # Input layer
    layers.Input(shape=(input_dim,)),

    # Hidden layer 1
    layers.Dense(64, activation='relu', name='hidden1'),

    # Hidden layer 2
    layers.Dense(32, activation='relu', name='hidden2'),

    # Output layer
    layers.Dense(1, activation='sigmoid', name='output')
], name='Baseline_MLP')

# Compile model
model_baseline.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy', tf.keras.metrics.AUC(name='auc')]
)

# Display model summary
print(model_baseline.summary())

# Visualisasi arsitektur (optional)
print("\n📊 Arsitektur Model:")
print(f"   Input: {input_dim} neurons")
print(f"   Hidden Layer 1: 64 neurons (ReLU)")
print(f"   Hidden Layer 2: 32 neurons (ReLU)")
print(f"   Output Layer: 1 neuron (Sigmoid)")

Arsitektur Baseline

Model baseline menggunakan:

• 2 hidden layers (64 dan 32 neurons)
• ReLU activation untuk hidden layers
• Sigmoid activation untuk output layer (binary classification)
• Adam optimizer dengan learning rate default (0.001)
• Binary crossentropy loss untuk binary classification

18.1.2 Training Baseline Model

print("=" * 60)
print("TRAINING BASELINE MODEL")
print("=" * 60)

# Train model
history_baseline = model_baseline.fit(
    X_train_scaled, y_train,
    epochs=50,
    batch_size=32,
    validation_split=0.2,
    verbose=1
)

print("\n✅ Training selesai!")

18.1.3 Visualisasi Training History

# Function untuk visualisasi training history
def plot_training_history(history, title='Training History'):
    """
    Visualisasi loss dan metrics selama training
    """
    fig, axes = plt.subplots(1, 3, figsize=(18, 5))

    # Loss
    axes[0].plot(history.history['loss'], label='Train Loss', linewidth=2)
    axes[0].plot(history.history['val_loss'], label='Val Loss', linewidth=2)
    axes[0].set_title('Model Loss', fontsize=14, fontweight='bold')
    axes[0].set_xlabel('Epoch', fontsize=12)
    axes[0].set_ylabel('Loss', fontsize=12)
    axes[0].legend()
    axes[0].grid(alpha=0.3)

    # Accuracy
    axes[1].plot(history.history['accuracy'], label='Train Accuracy', linewidth=2)
    axes[1].plot(history.history['val_accuracy'], label='Val Accuracy', linewidth=2)
    axes[1].set_title('Model Accuracy', fontsize=14, fontweight='bold')
    axes[1].set_xlabel('Epoch', fontsize=12)
    axes[1].set_ylabel('Accuracy', fontsize=12)
    axes[1].legend()
    axes[1].grid(alpha=0.3)

    # AUC
    if 'auc' in history.history:
        axes[2].plot(history.history['auc'], label='Train AUC', linewidth=2)
        axes[2].plot(history.history['val_auc'], label='Val AUC', linewidth=2)
        axes[2].set_title('Model AUC', fontsize=14, fontweight='bold')
        axes[2].set_xlabel('Epoch', fontsize=12)
        axes[2].set_ylabel('AUC', fontsize=12)
        axes[2].legend()
        axes[2].grid(alpha=0.3)

    plt.suptitle(title, fontsize=16, fontweight='bold', y=1.02)
    plt.tight_layout()
    plt.show()

# Plot training history
plot_training_history(history_baseline, 'Baseline Model - Training History')

18.1.4 Evaluasi Baseline Model

# Function untuk evaluasi model
def evaluate_model(model, X_test, y_test, model_name='Model'):
    """
    Evaluasi model dengan berbagai metrik
    """
    print("=" * 60)
    print(f"EVALUASI {model_name.upper()}")
    print("=" * 60)

    # Predictions
    y_pred_proba = model.predict(X_test).flatten()
    y_pred = (y_pred_proba > 0.5).astype(int)

    # Metrics
    accuracy = accuracy_score(y_test, y_pred)
    precision = precision_score(y_test, y_pred)
    recall = recall_score(y_test, y_pred)
    f1 = f1_score(y_test, y_pred)
    auc = roc_auc_score(y_test, y_pred_proba)

    print(f"\n📊 Classification Metrics:")
    print(f"   Accuracy:  {accuracy:.4f}")
    print(f"   Precision: {precision:.4f}")
    print(f"   Recall:    {recall:.4f}")
    print(f"   F1-Score:  {f1:.4f}")
    print(f"   AUC-ROC:   {auc:.4f}")

    # Confusion Matrix
    cm = confusion_matrix(y_test, y_pred)

    # Visualizations
    fig, axes = plt.subplots(1, 2, figsize=(14, 5))

    # Confusion Matrix
    sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', ax=axes[0],
                xticklabels=['No', 'Yes'], yticklabels=['No', 'Yes'])
    axes[0].set_title(f'Confusion Matrix - {model_name}', fontsize=14, fontweight='bold')
    axes[0].set_xlabel('Predicted', fontsize=12)
    axes[0].set_ylabel('Actual', fontsize=12)

    # ROC Curve
    fpr, tpr, thresholds = roc_curve(y_test, y_pred_proba)
    axes[1].plot(fpr, tpr, linewidth=2, label=f'ROC Curve (AUC = {auc:.4f})')
    axes[1].plot([0, 1], [0, 1], 'k--', linewidth=2, label='Random Classifier')
    axes[1].set_title(f'ROC Curve - {model_name}', fontsize=14, fontweight='bold')
    axes[1].set_xlabel('False Positive Rate', fontsize=12)
    axes[1].set_ylabel('True Positive Rate', fontsize=12)
    axes[1].legend()
    axes[1].grid(alpha=0.3)

    plt.tight_layout()
    plt.show()

    # Classification Report
    print("\n" + "=" * 60)
    print("CLASSIFICATION REPORT")
    print("=" * 60)
    print(classification_report(y_test, y_pred, target_names=['No', 'Yes']))

    return {
        'accuracy': accuracy,
        'precision': precision,
        'recall': recall,
        'f1': f1,
        'auc': auc,
        'y_pred': y_pred,
        'y_pred_proba': y_pred_proba
    }

# Evaluate baseline model
baseline_results = evaluate_model(model_baseline, X_test_scaled, y_test,
                                  'Baseline Model')

Overfitting Check

Perhatikan gap antara training dan validation loss/accuracy. Jika validation metrics jauh lebih buruk dari training metrics, model kemungkinan overfitting!

18.2 Model dengan Regularization

18.2.1 Dropout Regularization

print("=" * 60)
print("MLP WITH DROPOUT REGULARIZATION")
print("=" * 60)

# Build model dengan dropout
model_dropout = keras.Sequential([
    layers.Input(shape=(input_dim,)),

    # Hidden layer 1 dengan dropout
    layers.Dense(128, activation='relu', name='hidden1'),
    layers.Dropout(0.3, name='dropout1'),

    # Hidden layer 2 dengan dropout
    layers.Dense(64, activation='relu', name='hidden2'),
    layers.Dropout(0.3, name='dropout2'),

    # Hidden layer 3 dengan dropout
    layers.Dense(32, activation='relu', name='hidden3'),
    layers.Dropout(0.2, name='dropout3'),

    # Output layer
    layers.Dense(1, activation='sigmoid', name='output')
], name='MLP_Dropout')

# Compile
model_dropout.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy', tf.keras.metrics.AUC(name='auc')]
)

print(model_dropout.summary())

Dropout Regularization

Dropout adalah teknik regularisasi yang:

• Secara random “mematikan” beberapa neurons selama training
• Rate 0.3 = 30% neurons di-dropout
• Memaksa network belajar representasi yang lebih robust
• Mengurangi overfitting dengan mencegah co-adaptation

Best Practices:

• Dropout 0.2-0.5 untuk hidden layers
• Tidak perlu dropout di input/output layer
• Dropout lebih tinggi untuk layer lebih awal

18.2.2 Training dengan Callbacks

# Setup callbacks untuk training yang lebih optimal
callbacks = [
    # Early stopping: stop jika val_loss tidak improve
    EarlyStopping(
        monitor='val_loss',
        patience=10,
        restore_best_weights=True,
        verbose=1
    ),

    # Reduce learning rate jika val_loss plateau
    ReduceLROnPlateau(
        monitor='val_loss',
        factor=0.5,
        patience=5,
        min_lr=1e-7,
        verbose=1
    ),

    # Save best model
    ModelCheckpoint(
        'best_model_dropout.h5',
        monitor='val_auc',
        save_best_only=True,
        mode='max',
        verbose=1
    )
]

print("🚀 Training MLP with Dropout...")
print("=" * 60)

# Train model
history_dropout = model_dropout.fit(
    X_train_scaled, y_train,
    epochs=100,
    batch_size=64,
    validation_split=0.2,
    callbacks=callbacks,
    verbose=1
)

print("\n✅ Training selesai!")

Callbacks untuk Training Optimal

EarlyStopping: Menghentikan training jika tidak ada improvement, mencegah overfitting

ReduceLROnPlateau: Mengurangi learning rate saat stuck di plateau, membantu fine-tuning

ModelCheckpoint: Menyimpan model terbaik berdasarkan validation metrics

18.2.3 Evaluasi Model dengan Dropout

# Plot training history
plot_training_history(history_dropout, 'Dropout Model - Training History')

# Evaluate model
dropout_results = evaluate_model(model_dropout, X_test_scaled, y_test,
                                 'Dropout Model')

# Bandingkan dengan baseline
print("\n" + "=" * 60)
print("COMPARISON: BASELINE vs DROPOUT")
print("=" * 60)
print(f"{'Metric':<15} {'Baseline':<12} {'Dropout':<12} {'Improvement':<12}")
print("-" * 60)
for metric in ['accuracy', 'precision', 'recall', 'f1', 'auc']:
    baseline_val = baseline_results[metric]
    dropout_val = dropout_results[metric]
    improvement = ((dropout_val - baseline_val) / baseline_val) * 100
    print(f"{metric.capitalize():<15} {baseline_val:<12.4f} {dropout_val:<12.4f} {improvement:+.2f}%")

18.2.4 L2 Regularization

print("=" * 60)
print("MLP WITH L2 REGULARIZATION")
print("=" * 60)

# Build model dengan L2 regularization
l2_reg = 0.01

model_l2 = keras.Sequential([
    layers.Input(shape=(input_dim,)),

    # Hidden layers dengan L2 regularization
    layers.Dense(128, activation='relu',
                kernel_regularizer=regularizers.l2(l2_reg),
                name='hidden1'),

    layers.Dense(64, activation='relu',
                kernel_regularizer=regularizers.l2(l2_reg),
                name='hidden2'),

    layers.Dense(32, activation='relu',
                kernel_regularizer=regularizers.l2(l2_reg),
                name='hidden3'),

    # Output layer
    layers.Dense(1, activation='sigmoid', name='output')
], name='MLP_L2')

# Compile
model_l2.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy', tf.keras.metrics.AUC(name='auc')]
)

print(model_l2.summary())

# Train
print("\n🚀 Training MLP with L2 Regularization...")
history_l2 = model_l2.fit(
    X_train_scaled, y_train,
    epochs=100,
    batch_size=64,
    validation_split=0.2,
    callbacks=callbacks,
    verbose=1
)

# Evaluate
plot_training_history(history_l2, 'L2 Regularization Model - Training History')
l2_results = evaluate_model(model_l2, X_test_scaled, y_test, 'L2 Model')

L2 vs Dropout Regularization

L2 Regularization (Weight Decay):

• Menambahkan penalty term pada loss function
• Mendorong weights tetap kecil
• Good untuk: Dataset kecil, banyak fitur

Dropout:

• Random dropout neurons saat training
• Ensemble effect
• Good untuk: Deep networks, mengurangi co-adaptation

Kombinasi keduanya sering memberikan hasil terbaik!

18.2.5 Combined: Dropout + L2

print("=" * 60)
print("MLP WITH COMBINED REGULARIZATION (DROPOUT + L2)")
print("=" * 60)

# Build model dengan dropout dan L2
model_combined = keras.Sequential([
    layers.Input(shape=(input_dim,)),

    # Layer 1
    layers.Dense(128, activation='relu',
                kernel_regularizer=regularizers.l2(0.01),
                name='hidden1'),
    layers.Dropout(0.3),

    # Layer 2
    layers.Dense(64, activation='relu',
                kernel_regularizer=regularizers.l2(0.01),
                name='hidden2'),
    layers.Dropout(0.3),

    # Layer 3
    layers.Dense(32, activation='relu',
                kernel_regularizer=regularizers.l2(0.01),
                name='hidden3'),
    layers.Dropout(0.2),

    # Output
    layers.Dense(1, activation='sigmoid', name='output')
], name='MLP_Combined')

model_combined.compile(
    optimizer=keras.optimizers.Adam(learning_rate=0.001),
    loss='binary_crossentropy',
    metrics=['accuracy', tf.keras.metrics.AUC(name='auc')]
)

print(model_combined.summary())

# Train
print("\n🚀 Training MLP with Combined Regularization...")
history_combined = model_combined.fit(
    X_train_scaled, y_train,
    epochs=100,
    batch_size=64,
    validation_split=0.2,
    callbacks=callbacks,
    verbose=1
)

# Evaluate
plot_training_history(history_combined, 'Combined Regularization - Training History')
combined_results = evaluate_model(model_combined, X_test_scaled, y_test,
                                  'Combined Model')

18.3 Hyperparameter Tuning

18.3.1 Manual Hyperparameter Search

print("=" * 60)
print("HYPERPARAMETER TUNING - MANUAL SEARCH")
print("=" * 60)

# Define hyperparameter configurations
configs = [
    {'layers': [64, 32], 'dropout': 0.2, 'lr': 0.001, 'batch_size': 32},
    {'layers': [128, 64, 32], 'dropout': 0.3, 'lr': 0.001, 'batch_size': 64},
    {'layers': [256, 128, 64], 'dropout': 0.4, 'lr': 0.0005, 'batch_size': 64},
    {'layers': [128, 64], 'dropout': 0.25, 'lr': 0.001, 'batch_size': 128},
]

results_tuning = []

for idx, config in enumerate(configs, 1):
    print(f"\n{'='*60}")
    print(f"Configuration {idx}/{len(configs)}")
    print(f"{'='*60}")
    print(f"Layers: {config['layers']}")
    print(f"Dropout: {config['dropout']}")
    print(f"Learning Rate: {config['lr']}")
    print(f"Batch Size: {config['batch_size']}")

    # Build model
    model = keras.Sequential([layers.Input(shape=(input_dim,))])

    for i, units in enumerate(config['layers']):
        model.add(layers.Dense(units, activation='relu',
                              kernel_regularizer=regularizers.l2(0.01)))
        model.add(layers.Dropout(config['dropout']))

    model.add(layers.Dense(1, activation='sigmoid'))

    # Compile
    model.compile(
        optimizer=keras.optimizers.Adam(learning_rate=config['lr']),
        loss='binary_crossentropy',
        metrics=['accuracy', tf.keras.metrics.AUC(name='auc')]
    )

    # Train
    history = model.fit(
        X_train_scaled, y_train,
        epochs=50,
        batch_size=config['batch_size'],
        validation_split=0.2,
        callbacks=[EarlyStopping(patience=10, restore_best_weights=True)],
        verbose=0
    )

    # Evaluate
    y_pred_proba = model.predict(X_test_scaled, verbose=0).flatten()
    y_pred = (y_pred_proba > 0.5).astype(int)

    auc = roc_auc_score(y_test, y_pred_proba)
    f1 = f1_score(y_test, y_pred)
    accuracy = accuracy_score(y_test, y_pred)

    results_tuning.append({
        'config': idx,
        'layers': config['layers'],
        'dropout': config['dropout'],
        'lr': config['lr'],
        'batch_size': config['batch_size'],
        'accuracy': accuracy,
        'f1': f1,
        'auc': auc
    })

    print(f"Results: Accuracy={accuracy:.4f}, F1={f1:.4f}, AUC={auc:.4f}")

# Display results
print("\n" + "=" * 60)
print("TUNING RESULTS SUMMARY")
print("=" * 60)
results_df = pd.DataFrame(results_tuning)
print(results_df.to_string(index=False))

# Best configuration
best_idx = results_df['auc'].idxmax()
best_config = results_df.iloc[best_idx]
print(f"\n🏆 Best Configuration (by AUC):")
print(f"   Config: {best_config['config']}")
print(f"   Layers: {best_config['layers']}")
print(f"   Dropout: {best_config['dropout']}")
print(f"   Learning Rate: {best_config['lr']}")
print(f"   Batch Size: {best_config['batch_size']}")
print(f"   AUC: {best_config['auc']:.4f}")

18.3.2 Comparison Summary - All Keras Models

# Compile all results
all_keras_results = pd.DataFrame({
    'Model': ['Baseline', 'Dropout', 'L2', 'Combined'],
    'Accuracy': [baseline_results['accuracy'],
                dropout_results['accuracy'],
                l2_results['accuracy'],
                combined_results['accuracy']],
    'Precision': [baseline_results['precision'],
                 dropout_results['precision'],
                 l2_results['precision'],
                 combined_results['precision']],
    'Recall': [baseline_results['recall'],
              dropout_results['recall'],
              l2_results['recall'],
              combined_results['recall']],
    'F1-Score': [baseline_results['f1'],
                dropout_results['f1'],
                l2_results['f1'],
                combined_results['f1']],
    'AUC': [baseline_results['auc'],
           dropout_results['auc'],
           l2_results['auc'],
           combined_results['auc']]
})

print("=" * 80)
print("KERAS MODELS COMPARISON SUMMARY")
print("=" * 80)
print(all_keras_results.to_string(index=False))

# Visualize comparison
fig, axes = plt.subplots(1, 2, figsize=(16, 6))

# Bar plot
all_keras_results.set_index('Model')[['Accuracy', 'Precision', 'Recall', 'F1-Score', 'AUC']].plot(
    kind='bar', ax=axes[0], width=0.8
)
axes[0].set_title('Keras Models Performance Comparison', fontsize=14, fontweight='bold')
axes[0].set_xlabel('Model', fontsize=12)
axes[0].set_ylabel('Score', fontsize=12)
axes[0].legend(loc='lower right')
axes[0].set_xticklabels(all_keras_results['Model'], rotation=0)
axes[0].grid(axis='y', alpha=0.3)
axes[0].set_ylim([0.7, 1.0])

# Radar chart untuk best model
categories = ['Accuracy', 'Precision', 'Recall', 'F1-Score', 'AUC']
best_model_idx = all_keras_results['AUC'].idxmax()
values = all_keras_results.iloc[best_model_idx][categories].values

angles = np.linspace(0, 2 * np.pi, len(categories), endpoint=False).tolist()
values = np.concatenate((values, [values[0]]))
angles += angles[:1]

axes[1] = plt.subplot(122, projection='polar')
axes[1].plot(angles, values, 'o-', linewidth=2, label=all_keras_results.iloc[best_model_idx]['Model'])
axes[1].fill(angles, values, alpha=0.25)
axes[1].set_xticks(angles[:-1])
axes[1].set_xticklabels(categories)
axes[1].set_ylim(0, 1)
axes[1].set_title(f'Best Model Performance Profile', fontsize=14, fontweight='bold', pad=20)
axes[1].legend(loc='upper right')
axes[1].grid(True)

plt.tight_layout()
plt.show()

print(f"\n🏆 Best Keras Model: {all_keras_results.iloc[best_model_idx]['Model']}")
print(f"   AUC Score: {all_keras_results.iloc[best_model_idx]['AUC']:.4f}")

---

19 Part 3: PyTorch Implementation

19.1 Custom MLP Class dengan PyTorch

19.1.1 Define MLP Architecture

# Set device
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")

# Define MLP class
class MLPClassifier(nn.Module):
    """
    Multi-Layer Perceptron untuk Binary Classification
    """
    def __init__(self, input_dim, hidden_dims, dropout_rate=0.3):
        """
        Args:
            input_dim: Dimensi input features
            hidden_dims: List dimensi hidden layers, e.g., [128, 64, 32]
            dropout_rate: Dropout probability
        """
        super(MLPClassifier, self).__init__()

        # Build layers
        layers = []
        prev_dim = input_dim

        for hidden_dim in hidden_dims:
            layers.append(nn.Linear(prev_dim, hidden_dim))
            layers.append(nn.ReLU())
            layers.append(nn.Dropout(dropout_rate))
            prev_dim = hidden_dim

        # Output layer
        layers.append(nn.Linear(prev_dim, 1))
        layers.append(nn.Sigmoid())

        # Combine all layers
        self.network = nn.Sequential(*layers)

    def forward(self, x):
        """Forward pass"""
        return self.network(x)

# Instantiate model
input_dim_torch = X_train_scaled.shape[1]
hidden_dims = [128, 64, 32]
dropout_rate = 0.3

model_pytorch = MLPClassifier(input_dim_torch, hidden_dims, dropout_rate).to(device)

print("=" * 60)
print("PYTORCH MLP ARCHITECTURE")
print("=" * 60)
print(model_pytorch)
print(f"\nTotal parameters: {sum(p.numel() for p in model_pytorch.parameters()):,}")
print(f"Trainable parameters: {sum(p.numel() for p in model_pytorch.parameters() if p.requires_grad):,}")

PyTorch vs Keras

PyTorch Advantages:

• Lebih fleksibel dan Pythonic
• Lebih baik untuk research dan eksperimen
• Dynamic computation graph
• Lebih eksplisit (lebih kontrol)

Keras Advantages:

• Lebih simple dan high-level
• Lebih cepat untuk prototyping
• Built-in callbacks dan utilities
• Terintegrasi dengan TensorFlow ecosystem

19.1.2 Prepare DataLoaders

# Convert numpy arrays ke PyTorch tensors
X_train_tensor = torch.FloatTensor(X_train_scaled)
y_train_tensor = torch.FloatTensor(y_train.values).unsqueeze(1)
X_test_tensor = torch.FloatTensor(X_test_scaled)
y_test_tensor = torch.FloatTensor(y_test.values).unsqueeze(1)

# Create datasets
train_dataset = TensorDataset(X_train_tensor, y_train_tensor)
test_dataset = TensorDataset(X_test_tensor, y_test_tensor)

# Create dataloaders
batch_size = 64
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

print("=" * 60)
print("PYTORCH DATALOADERS")
print("=" * 60)
print(f"Train DataLoader:")
print(f"  Batch size: {batch_size}")
print(f"  Number of batches: {len(train_loader)}")
print(f"  Total samples: {len(train_dataset)}")

print(f"\nTest DataLoader:")
print(f"  Batch size: {batch_size}")
print(f"  Number of batches: {len(test_loader)}")
print(f"  Total samples: {len(test_dataset)}")

19.1.3 Training Loop Implementation

def train_pytorch_model(model, train_loader, val_loader, criterion, optimizer,
                       num_epochs=50, device='cpu', patience=10):
    """
    Training loop untuk PyTorch model

    Args:
        model: PyTorch model
        train_loader: DataLoader untuk training
        val_loader: DataLoader untuk validation
        criterion: Loss function
        optimizer: Optimizer
        num_epochs: Jumlah epochs
        device: Device (cpu/cuda)
        patience: Early stopping patience

    Returns:
        history: Dictionary berisi training history
        best_model_state: State dict dari best model
    """
    history = {
        'train_loss': [],
        'train_acc': [],
        'val_loss': [],
        'val_acc': []
    }

    best_val_loss = float('inf')
    patience_counter = 0
    best_model_state = None

    for epoch in range(num_epochs):
        # ============ TRAINING ============
        model.train()
        train_loss = 0.0
        train_correct = 0
        train_total = 0

        for batch_X, batch_y in train_loader:
            # Move to device
            batch_X = batch_X.to(device)
            batch_y = batch_y.to(device)

            # Zero gradients
            optimizer.zero_grad()

            # Forward pass
            outputs = model(batch_X)
            loss = criterion(outputs, batch_y)

            # Backward pass
            loss.backward()
            optimizer.step()

            # Statistics
            train_loss += loss.item() * batch_X.size(0)
            predictions = (outputs > 0.5).float()
            train_correct += (predictions == batch_y).sum().item()
            train_total += batch_y.size(0)

        # Calculate epoch training metrics
        epoch_train_loss = train_loss / train_total
        epoch_train_acc = train_correct / train_total

        # ============ VALIDATION ============
        model.eval()
        val_loss = 0.0
        val_correct = 0
        val_total = 0

        with torch.no_grad():
            for batch_X, batch_y in val_loader:
                batch_X = batch_X.to(device)
                batch_y = batch_y.to(device)

                outputs = model(batch_X)
                loss = criterion(outputs, batch_y)

                val_loss += loss.item() * batch_X.size(0)
                predictions = (outputs > 0.5).float()
                val_correct += (predictions == batch_y).sum().item()
                val_total += batch_y.size(0)

        # Calculate epoch validation metrics
        epoch_val_loss = val_loss / val_total
        epoch_val_acc = val_correct / val_total

        # Save history
        history['train_loss'].append(epoch_train_loss)
        history['train_acc'].append(epoch_train_acc)
        history['val_loss'].append(epoch_val_loss)
        history['val_acc'].append(epoch_val_acc)

        # Print progress
        if (epoch + 1) % 5 == 0 or epoch == 0:
            print(f"Epoch [{epoch+1}/{num_epochs}] - "
                  f"Train Loss: {epoch_train_loss:.4f}, Train Acc: {epoch_train_acc:.4f}, "
                  f"Val Loss: {epoch_val_loss:.4f}, Val Acc: {epoch_val_acc:.4f}")

        # Early stopping check
        if epoch_val_loss < best_val_loss:
            best_val_loss = epoch_val_loss
            patience_counter = 0
            best_model_state = model.state_dict().copy()
        else:
            patience_counter += 1
            if patience_counter >= patience:
                print(f"\n⚠️  Early stopping triggered at epoch {epoch+1}")
                break

    # Restore best model
    if best_model_state is not None:
        model.load_state_dict(best_model_state)

    return history, best_model_state

print("✅ Training function defined!")

Training Loop Components

Sebuah PyTorch training loop harus memiliki:

1. model.train(): Set model ke training mode (enable dropout)
2. optimizer.zero_grad(): Reset gradients
3. loss.backward(): Compute gradients
4. optimizer.step(): Update weights
5. model.eval(): Set model ke eval mode (disable dropout)
6. torch.no_grad(): Disable gradient calculation untuk validation

19.1.4 Train PyTorch Model

# Split train into train/val
val_size = int(0.2 * len(train_dataset))
train_size = len(train_dataset) - val_size
train_subset, val_subset = torch.utils.data.random_split(
    train_dataset, [train_size, val_size]
)

train_loader_pytorch = DataLoader(train_subset, batch_size=64, shuffle=True)
val_loader_pytorch = DataLoader(val_subset, batch_size=64, shuffle=False)

# Define loss and optimizer
criterion = nn.BCELoss()
optimizer = optim.Adam(model_pytorch.parameters(), lr=0.001)

print("=" * 60)
print("TRAINING PYTORCH MODEL")
print("=" * 60)
print(f"Optimizer: Adam (lr=0.001)")
print(f"Loss Function: Binary Cross Entropy")
print(f"Epochs: 100 (with early stopping)")
print(f"Batch Size: 64")
print(f"Device: {device}")
print("=" * 60)

# Train
history_pytorch, best_model_state = train_pytorch_model(
    model_pytorch,
    train_loader_pytorch,
    val_loader_pytorch,
    criterion,
    optimizer,
    num_epochs=100,
    device=device,
    patience=15
)

print("\n✅ Training selesai!")

19.1.5 Visualize PyTorch Training History

# Plot training history
fig, axes = plt.subplots(1, 2, figsize=(14, 5))

# Loss
axes[0].plot(history_pytorch['train_loss'], label='Train Loss', linewidth=2)
axes[0].plot(history_pytorch['val_loss'], label='Val Loss', linewidth=2)
axes[0].set_title('PyTorch Model - Loss', fontsize=14, fontweight='bold')
axes[0].set_xlabel('Epoch', fontsize=12)
axes[0].set_ylabel('Loss', fontsize=12)
axes[0].legend()
axes[0].grid(alpha=0.3)

# Accuracy
axes[1].plot(history_pytorch['train_acc'], label='Train Accuracy', linewidth=2)
axes[1].plot(history_pytorch['val_acc'], label='Val Accuracy', linewidth=2)
axes[1].set_title('PyTorch Model - Accuracy', fontsize=14, fontweight='bold')
axes[1].set_xlabel('Epoch', fontsize=12)
axes[1].set_ylabel('Accuracy', fontsize=12)
axes[1].legend()
axes[1].grid(alpha=0.3)

plt.tight_layout()
plt.show()

19.1.6 Evaluate PyTorch Model

def evaluate_pytorch_model(model, test_loader, device='cpu'):
    """
    Evaluasi PyTorch model
    """
    model.eval()
    all_predictions = []
    all_probabilities = []
    all_labels = []

    with torch.no_grad():
        for batch_X, batch_y in test_loader:
            batch_X = batch_X.to(device)
            batch_y = batch_y.to(device)

            outputs = model(batch_X)
            predictions = (outputs > 0.5).float()

            all_predictions.extend(predictions.cpu().numpy())
            all_probabilities.extend(outputs.cpu().numpy())
            all_labels.extend(batch_y.cpu().numpy())

    all_predictions = np.array(all_predictions).flatten()
    all_probabilities = np.array(all_probabilities).flatten()
    all_labels = np.array(all_labels).flatten()

    return all_labels, all_predictions, all_probabilities

# Evaluate
y_true, y_pred, y_pred_proba = evaluate_pytorch_model(model_pytorch, test_loader, device)

# Calculate metrics
accuracy = accuracy_score(y_true, y_pred)
precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)
auc = roc_auc_score(y_true, y_pred_proba)

print("=" * 60)
print("PYTORCH MODEL EVALUATION")
print("=" * 60)
print(f"\n📊 Classification Metrics:")
print(f"   Accuracy:  {accuracy:.4f}")
print(f"   Precision: {precision:.4f}")
print(f"   Recall:    {recall:.4f}")
print(f"   F1-Score:  {f1:.4f}")
print(f"   AUC-ROC:   {auc:.4f}")

# Confusion Matrix & ROC Curve
cm = confusion_matrix(y_true, y_pred)

fig, axes = plt.subplots(1, 2, figsize=(14, 5))

# Confusion Matrix
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', ax=axes[0],
            xticklabels=['No', 'Yes'], yticklabels=['No', 'Yes'])
axes[0].set_title('Confusion Matrix - PyTorch Model', fontsize=14, fontweight='bold')
axes[0].set_xlabel('Predicted', fontsize=12)
axes[0].set_ylabel('Actual', fontsize=12)

# ROC Curve
fpr, tpr, _ = roc_curve(y_true, y_pred_proba)
axes[1].plot(fpr, tpr, linewidth=2, label=f'ROC Curve (AUC = {auc:.4f})')
axes[1].plot([0, 1], [0, 1], 'k--', linewidth=2, label='Random Classifier')
axes[1].set_title('ROC Curve - PyTorch Model', fontsize=14, fontweight='bold')
axes[1].set_xlabel('False Positive Rate', fontsize=12)
axes[1].set_ylabel('True Positive Rate', fontsize=12)
axes[1].legend()
axes[1].grid(alpha=0.3)

plt.tight_layout()
plt.show()

# Classification Report
print("\n" + "=" * 60)
print("CLASSIFICATION REPORT")
print("=" * 60)
print(classification_report(y_true, y_pred, target_names=['No', 'Yes']))

pytorch_results = {
    'accuracy': accuracy,
    'precision': precision,
    'recall': recall,
    'f1': f1,
    'auc': auc
}

19.2 Save & Load PyTorch Models

19.2.1 Save Model

# Save model
model_save_path = 'pytorch_mlp_best.pth'

# Save complete model
torch.save({
    'model_state_dict': model_pytorch.state_dict(),
    'optimizer_state_dict': optimizer.state_dict(),
    'input_dim': input_dim_torch,
    'hidden_dims': hidden_dims,
    'dropout_rate': dropout_rate,
    'history': history_pytorch,
    'test_metrics': pytorch_results
}, model_save_path)

print(f"✅ Model saved to: {model_save_path}")

19.2.2 Load Model

# Load model
checkpoint = torch.load(model_save_path)

# Recreate model architecture
loaded_model = MLPClassifier(
    checkpoint['input_dim'],
    checkpoint['hidden_dims'],
    checkpoint['dropout_rate']
).to(device)

# Load state dict
loaded_model.load_state_dict(checkpoint['model_state_dict'])

print("✅ Model loaded successfully!")
print(f"   Input dim: {checkpoint['input_dim']}")
print(f"   Hidden dims: {checkpoint['hidden_dims']}")
print(f"   Test AUC: {checkpoint['test_metrics']['auc']:.4f}")

19.3 PyTorch dengan Learning Rate Scheduler

# Create new model instance
model_pytorch_scheduled = MLPClassifier(input_dim_torch, [128, 64, 32], 0.3).to(device)

# Optimizer
optimizer_scheduled = optim.Adam(model_pytorch_scheduled.parameters(), lr=0.01)

# Learning rate scheduler
scheduler = optim.lr_scheduler.ReduceLROnPlateau(
    optimizer_scheduled,
    mode='min',
    factor=0.5,
    patience=5,
    verbose=True
)

print("=" * 60)
print("TRAINING DENGAN LEARNING RATE SCHEDULER")
print("=" * 60)

# Modified training loop dengan scheduler
history_scheduled = {
    'train_loss': [],
    'train_acc': [],
    'val_loss': [],
    'val_acc': [],
    'lr': []
}

criterion = nn.BCELoss()
num_epochs = 100
best_val_loss = float('inf')
patience_counter = 0
patience = 15

for epoch in range(num_epochs):
    # Training phase
    model_pytorch_scheduled.train()
    train_loss = 0.0
    train_correct = 0
    train_total = 0

    for batch_X, batch_y in train_loader_pytorch:
        batch_X = batch_X.to(device)
        batch_y = batch_y.to(device)

        optimizer_scheduled.zero_grad()
        outputs = model_pytorch_scheduled(batch_X)
        loss = criterion(outputs, batch_y)
        loss.backward()
        optimizer_scheduled.step()

        train_loss += loss.item() * batch_X.size(0)
        predictions = (outputs > 0.5).float()
        train_correct += (predictions == batch_y).sum().item()
        train_total += batch_y.size(0)

    epoch_train_loss = train_loss / train_total
    epoch_train_acc = train_correct / train_total

    # Validation phase
    model_pytorch_scheduled.eval()
    val_loss = 0.0
    val_correct = 0
    val_total = 0

    with torch.no_grad():
        for batch_X, batch_y in val_loader_pytorch:
            batch_X = batch_X.to(device)
            batch_y = batch_y.to(device)

            outputs = model_pytorch_scheduled(batch_X)
            loss = criterion(outputs, batch_y)

            val_loss += loss.item() * batch_X.size(0)
            predictions = (outputs > 0.5).float()
            val_correct += (predictions == batch_y).sum().item()
            val_total += batch_y.size(0)

    epoch_val_loss = val_loss / val_total
    epoch_val_acc = val_correct / val_total

    # Update learning rate
    scheduler.step(epoch_val_loss)
    current_lr = optimizer_scheduled.param_groups[0]['lr']

    # Save history
    history_scheduled['train_loss'].append(epoch_train_loss)
    history_scheduled['train_acc'].append(epoch_train_acc)
    history_scheduled['val_loss'].append(epoch_val_loss)
    history_scheduled['val_acc'].append(epoch_val_acc)
    history_scheduled['lr'].append(current_lr)

    if (epoch + 1) % 5 == 0 or epoch == 0:
        print(f"Epoch [{epoch+1}/{num_epochs}] - "
              f"Train Loss: {epoch_train_loss:.4f}, Val Loss: {epoch_val_loss:.4f}, "
              f"LR: {current_lr:.6f}")

    # Early stopping
    if epoch_val_loss < best_val_loss:
        best_val_loss = epoch_val_loss
        patience_counter = 0
    else:
        patience_counter += 1
        if patience_counter >= patience:
            print(f"\n⚠️  Early stopping at epoch {epoch+1}")
            break

print("\n✅ Training dengan scheduler selesai!")

# Plot learning rate history
plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.plot(history_scheduled['val_loss'], linewidth=2)
plt.title('Validation Loss', fontsize=14, fontweight='bold')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.grid(alpha=0.3)

plt.subplot(1, 2, 2)
plt.plot(history_scheduled['lr'], linewidth=2, color='orange')
plt.title('Learning Rate Schedule', fontsize=14, fontweight='bold')
plt.xlabel('Epoch')
plt.ylabel('Learning Rate')
plt.yscale('log')
plt.grid(alpha=0.3)

plt.tight_layout()
plt.show()

Learning Rate Scheduling

Learning rate scheduler secara otomatis menyesuaikan learning rate:

• ReduceLROnPlateau: Reduce LR ketika metric tidak improve
• StepLR: Reduce LR setiap N epochs
• CosineAnnealingLR: Cosine annealing schedule

Benefit: Konvergensi lebih baik dan menghindari stuck di local minima

---

20 Part 4: Advanced Techniques

20.1 Handling Class Imbalance

20.1.1 Class Weights

from sklearn.utils.class_weight import compute_class_weight

# Compute class weights
class_weights_array = compute_class_weight(
    'balanced',
    classes=np.unique(y_train),
    y=y_train
)

# Convert to dictionary
class_weights_dict = {0: class_weights_array[0], 1: class_weights_array[1]}

print("=" * 60)
print("CLASS WEIGHTS")
print("=" * 60)
print(f"Class 0 (No): {class_weights_dict[0]:.4f}")
print(f"Class 1 (Yes): {class_weights_dict[1]:.4f}")
print(f"Weight Ratio: {class_weights_dict[1] / class_weights_dict[0]:.2f}:1")

# Train Keras model dengan class weights
model_weighted = keras.Sequential([
    layers.Input(shape=(input_dim,)),
    layers.Dense(128, activation='relu', kernel_regularizer=regularizers.l2(0.01)),
    layers.Dropout(0.3),
    layers.Dense(64, activation='relu', kernel_regularizer=regularizers.l2(0.01)),
    layers.Dropout(0.3),
    layers.Dense(32, activation='relu', kernel_regularizer=regularizers.l2(0.01)),
    layers.Dropout(0.2),
    layers.Dense(1, activation='sigmoid')
])

model_weighted.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy', tf.keras.metrics.AUC(name='auc')]
)

print("\n🚀 Training dengan class weights...")
history_weighted = model_weighted.fit(
    X_train_scaled, y_train,
    epochs=100,
    batch_size=64,
    validation_split=0.2,
    class_weight=class_weights_dict,  # ← Class weights di sini
    callbacks=[EarlyStopping(patience=10, restore_best_weights=True)],
    verbose=0
)

# Evaluate
weighted_results = evaluate_model(model_weighted, X_test_scaled, y_test,
                                  'Weighted Model')

Class Weights Explained

Class weights memberikan penalty lebih besar pada misclassification kelas minoritas:

• Kelas mayoritas (No): weight rendah
• Kelas minoritas (Yes): weight tinggi

Formula: weight = n_samples / (n_classes * n_samples_class)

Efek: Model lebih fokus belajar dari kelas minoritas

20.1.2 SMOTE (Synthetic Minority Over-sampling)

from imblearn.over_sampling import SMOTE

print("=" * 60)
print("SMOTE - SYNTHETIC MINORITY OVERSAMPLING")
print("=" * 60)

# Apply SMOTE
smote = SMOTE(random_state=42)
X_train_smote, y_train_smote = smote.fit_resample(X_train_scaled, y_train)

print(f"\nSebelum SMOTE:")
print(f"  Total samples: {len(y_train)}")
print(f"  Class 0: {(y_train == 0).sum()} ({(y_train == 0).mean() * 100:.1f}%)")
print(f"  Class 1: {(y_train == 1).sum()} ({(y_train == 1).mean() * 100:.1f}%)")

print(f"\nSesudah SMOTE:")
print(f"  Total samples: {len(y_train_smote)}")
print(f"  Class 0: {(y_train_smote == 0).sum()} ({(y_train_smote == 0).mean() * 100:.1f}%)")
print(f"  Class 1: {(y_train_smote == 1).sum()} ({(y_train_smote == 1).mean() * 100:.1f}%)")

# Train model dengan SMOTE data
model_smote = keras.Sequential([
    layers.Input(shape=(input_dim,)),
    layers.Dense(128, activation='relu', kernel_regularizer=regularizers.l2(0.01)),
    layers.Dropout(0.3),
    layers.Dense(64, activation='relu', kernel_regularizer=regularizers.l2(0.01)),
    layers.Dropout(0.3),
    layers.Dense(32, activation='relu', kernel_regularizer=regularizers.l2(0.01)),
    layers.Dropout(0.2),
    layers.Dense(1, activation='sigmoid')
])

model_smote.compile(
    optimizer='adam',
    loss='binary_crossentropy',
    metrics=['accuracy', tf.keras.metrics.AUC(name='auc')]
)

print("\n🚀 Training dengan SMOTE data...")
history_smote = model_smote.fit(
    X_train_smote, y_train_smote,
    epochs=100,
    batch_size=64,
    validation_split=0.2,
    callbacks=[EarlyStopping(patience=10, restore_best_weights=True)],
    verbose=0
)

# Evaluate
smote_results = evaluate_model(model_smote, X_test_scaled, y_test, 'SMOTE Model')

SMOTE vs Class Weights

SMOTE:

• Membuat synthetic samples dari kelas minoritas
• Meningkatkan jumlah training samples
• Risk: Overfitting pada synthetic data

Class Weights:

• Tidak mengubah data
• Hanya mengubah loss calculation
• Lebih efisien secara komputasi

Recommendation: Coba keduanya, pilih yang performa lebih baik!

20.1.3 Comparison: Imbalance Handling Techniques

# Compile results
imbalance_results = pd.DataFrame({
    'Method': ['No Handling', 'Class Weights', 'SMOTE'],
    'Accuracy': [combined_results['accuracy'],
                weighted_results['accuracy'],
                smote_results['accuracy']],
    'Precision': [combined_results['precision'],
                 weighted_results['precision'],
                 smote_results['precision']],
    'Recall': [combined_results['recall'],
              weighted_results['recall'],
              smote_results['recall']],
    'F1-Score': [combined_results['f1'],
                weighted_results['f1'],
                smote_results['f1']],
    'AUC': [combined_results['auc'],
           weighted_results['auc'],
           smote_results['auc']]
})

print("=" * 80)
print("IMBALANCE HANDLING COMPARISON")
print("=" * 80)
print(imbalance_results.to_string(index=False))

# Visualize
imbalance_results.set_index('Method')[['Precision', 'Recall', 'F1-Score']].plot(
    kind='bar', figsize=(12, 6), width=0.8
)
plt.title('Impact of Imbalance Handling Techniques', fontsize=14, fontweight='bold')
plt.xlabel('Method', fontsize=12)
plt.ylabel('Score', fontsize=12)
plt.xticks(rotation=0)
plt.legend(loc='lower right')
plt.grid(axis='y', alpha=0.3)
plt.ylim([0, 1])
plt.tight_layout()
plt.show()

print(f"\n💡 Best method by Recall: {imbalance_results.loc[imbalance_results['Recall'].idxmax(), 'Method']}")
print(f"   (Recall penting untuk mendeteksi lebih banyak kelas positif)")

20.2 Model Ensembling (Optional)

print("=" * 60)
print("MODEL ENSEMBLE")
print("=" * 60)

# Collect predictions from all best models
predictions_ensemble = []

# Model 1: Combined (Keras)
pred1 = model_combined.predict(X_test_scaled, verbose=0).flatten()
predictions_ensemble.append(pred1)

# Model 2: Weighted (Keras)
pred2 = model_weighted.predict(X_test_scaled, verbose=0).flatten()
predictions_ensemble.append(pred2)

# Model 3: PyTorch
model_pytorch.eval()
with torch.no_grad():
    pred3 = model_pytorch(torch.FloatTensor(X_test_scaled).to(device)).cpu().numpy().flatten()
predictions_ensemble.append(pred3)

# Average ensemble
ensemble_pred_proba = np.mean(predictions_ensemble, axis=0)
ensemble_pred = (ensemble_pred_proba > 0.5).astype(int)

# Evaluate ensemble
ensemble_accuracy = accuracy_score(y_test, ensemble_pred)
ensemble_precision = precision_score(y_test, ensemble_pred)
ensemble_recall = recall_score(y_test, ensemble_pred)
ensemble_f1 = f1_score(y_test, ensemble_pred)
ensemble_auc = roc_auc_score(y_test, ensemble_pred_proba)

print(f"\n📊 Ensemble Performance:")
print(f"   Accuracy:  {ensemble_accuracy:.4f}")
print(f"   Precision: {ensemble_precision:.4f}")
print(f"   Recall:    {ensemble_recall:.4f}")
print(f"   F1-Score:  {ensemble_f1:.4f}")
print(f"   AUC-ROC:   {ensemble_auc:.4f}")

# Compare with individual models
print(f"\n📈 Improvement over best single model:")
best_single_auc = max([combined_results['auc'], weighted_results['auc'], pytorch_results['auc']])
improvement = ((ensemble_auc - best_single_auc) / best_single_auc) * 100
print(f"   AUC improvement: {improvement:+.2f}%")

# Confusion matrix
cm_ensemble = confusion_matrix(y_test, ensemble_pred)

plt.figure(figsize=(8, 6))
sns.heatmap(cm_ensemble, annot=True, fmt='d', cmap='Greens',
            xticklabels=['No', 'Yes'], yticklabels=['No', 'Yes'])
plt.title('Confusion Matrix - Ensemble Model', fontsize=14, fontweight='bold')
plt.xlabel('Predicted', fontsize=12)
plt.ylabel('Actual', fontsize=12)
plt.tight_layout()
plt.show()

Model Ensembling

Ensemble menggabungkan prediksi dari multiple models untuk performa lebih baik:

Metode:

• Averaging: Mean dari probabilities
• Voting: Majority vote dari predictions
• Stacking: Train meta-model pada predictions

Benefit: Mengurangi variance dan bias, lebih robust

---

21 Summary & Key Takeaways

21.1 Lab Summary

Selamat! Anda telah menyelesaikan Lab 5 dan mempelajari:

21.1.1 1. Data Preparation

• Loading dan eksplorasi Bank Marketing dataset
• Handling missing values dan feature engineering
• Encoding categorical variables
• Feature scaling untuk neural networks
• Train-test split dengan stratification

21.1.2 2. Keras Implementation

• Membangun baseline MLP model
• Menerapkan regularization (Dropout, L2)
• Training dengan callbacks (EarlyStopping, ReduceLROnPlateau)
• Hyperparameter tuning
• Model evaluation dengan multiple metrics

21.1.3 3. PyTorch Implementation

• Membuat custom MLP class dengan nn.Module
• Implementing training loop dari scratch
• Using DataLoaders untuk batch processing
• Learning rate scheduling
• Saving dan loading models

21.1.4 4. Advanced Techniques

• Handling imbalanced datasets (class weights, SMOTE)
• Model ensembling untuk improved performance
• Comprehensive model comparison

21.2 Key Takeaways

Poin-Poin Penting

1. Feature Scaling is Critical: Neural networks membutuhkan scaled features untuk training yang stabil

2. Regularization Prevents Overfitting: Dropout dan L2 regularization penting untuk generalization

3. Class Imbalance Matters: Dataset tidak seimbang memerlukan teknik khusus (weights, SMOTE, metrics)

4. Multiple Metrics: Jangan hanya lihat accuracy - gunakan precision, recall, F1, AUC

5. Keras vs PyTorch: Keras lebih simple, PyTorch lebih flexible - pilih sesuai kebutuhan

6. Callbacks are Powerful: EarlyStopping dan LR scheduling menghemat waktu dan improve results

7. Ensemble for Best Results: Combining multiple models sering memberikan performa terbaik

21.3 Best Practices

1. Always split with stratification untuk dataset tidak seimbang
2. Use validation set untuk hyperparameter tuning
3. Monitor both training & validation metrics untuk detect overfitting
4. Save best models menggunakan callbacks/checkpoints
5. Compare multiple architectures sebelum memilih final model
6. Use appropriate metrics untuk problem domain (recall untuk medical, precision untuk spam)

21.4 Next Steps

Untuk pengembangan lebih lanjut, Anda bisa:

1. Mencoba arsitektur yang lebih deep (4-5 hidden layers)
2. Experiment dengan activation functions (LeakyReLU, ELU, SELU)
3. Implement batch normalization
4. Try different optimizers (RMSprop, AdaGrad)
5. Explore advanced ensemble methods (stacking, boosting)
6. Deploy model ke production (TensorFlow Serving, TorchServe)

21.5 Resources

• Keras Documentation
• PyTorch Tutorials
• Deep Learning Book
• Fast.ai Course

---

Terima kasih telah menyelesaikan Lab 5! 🎉

Next Lab: CNN untuk Image Classification