Ames Housing Price Prediction — XGBoost Regression

An end-to-end machine learning pipeline for the Kaggle Housing Prices Competition, predicting residential home sale prices in Ames, Iowa from 79 explanatory features. The project follows a full, iterative workflow: exploratory data analysis, systematic feature engineering, model comparison, and hyperparameter tuning, with every design decision validated against cross-validation scores before being kept or discarded. Exploratory analysis showed SalePrice is right-skewed, with OverallQual (0.79 correlation) and GrLivArea (0.71) as the strongest individual predictors, and several highly collinear feature pairs (GarageCars/GarageArea: 0.88) that motivated combining them into engineered features. A handful of large-living-area homes with unexpectedly low prices were identified as outliers and removed from training. Preprocessing treats missing values carefully: columns where NaN genuinely means "feature does not exist" (no pool, no garage) are filled with the literal string "None" rather than imputed, preserving their meaning. Quality-rating columns follow a natural order and are ordinal-encoded rather than one-hot encoded, letting the model learn the quality gradient directly. Eight candidate engineered features were tested individually via 5-fold cross-validation against a no-engineering baseline; only the five that measurably improved the CV score were kept (TotalSF, TotalBath, QualCond, an OverallQual x GrLivArea interaction term, and GarageAge). XGBoost was compared against Random Forest, LightGBM, and CatBoost, consistently winning; ensembling was tested but did not improve on XGBoost alone. Two rounds of RandomizedSearchCV tuned the final hyperparameters. The project went through nine iterations, moving from a Random Forest baseline (16,452 MAE) down to the final XGBoost model (13,384.82 MAE) through categorical features, outlier removal, validated feature engineering, hyperparameter tuning, and ordinal encoding for quality columns — each step measured against the leaderboard, not just cross-validation.