Predicting Singapore Condo Prices with Machine Learning: An End-to-End MLOps Journey
From problem framing and metric Cartesian feature engineering to walk-forward validation, Optuna Bayesian tuning, split conformal uncertainty bounds, and a 233,000-lease quantile rental engine.
Welcome
Welcome! I built this platform as an end-to-end personal capstone project after my study in NTU SCTP to demonstrate how modern data science and machine learning can demystify one of the largest financial commitments in Singapore: residential property.
1 Problem Framing & Quantitative Objectives
In Singapore's high-stakes residential real estate market—where median private condominium transactions exceed $1.8M SGD—property valuation has historically suffered from a difficult dilemma between slow manual surveyor appraisals (3–7 business days, $800+ fees) and crude district medians that completely ignore micro-unit realities.
We framed valuation through Hedonic Pricing Theory: an asset's observed price is the composite sum of its constituent utility-bearing characteristics:
2 Data Collection & Ingestion Pipeline
The platform integrates four authoritative, public and statutory data sources into an Extract, Load, Transform (ELT) architecture powered by DuckDB:
Official secondary resale caveat records capturing transacted prices, contract dates, floor level bands, and floor areas.
32-month rolling window of private residential leasing agreements recording monthly rental quantum and bedroom configuration.
High-precision geocoding yielding SVY21 metric coordinates for 2,500+ condominiums, 160+ MRT stations, and 180+ primary schools.
URA Private Property Price Index (PPI) and Singapore Overnight Rate Average 3-Month Compounded (MAS SORA) matched by quarter.
3 Exploratory Data Analysis (EDA) & Domain Insights
Exploratory analysis of the 28,119 caveat population revealed three structural phenomena that dictated our feature engineering decisions:
CCR exhibits high median pricing ($2,318 PSF) with wide variance (σ = $675) driven by prime luxury architecture. RCR ($1,792 PSF, σ = $381) acts as city-fringe transit equilibrium. OCR ($1,448 PSF, σ = $249) exhibits tight mass-market affordability boundaries.
Vertical pricing is non-linear: rapid appreciation from floors 1 to 12 (+1.2% PSF per floor) as road acoustic noise drops, tapering above floor 20 once unblocked horizon views are secured, followed by a discrete jump for penthouses.
99-year leasehold assets trade at near-parity with Freehold during their first 20 years, but experience accelerated structural decay after year 30 in alignment with the Singapore Land Authority's official Bala's table.
4 Data Cleaning & Regulatory Anonymisation
Real estate filings contain non-arm's-length anomalies that mislead regression models: nominal transfers ($1 gifts, probate distribution), commercial shophouses misclassified as residential, and extreme penthouses (>10,000 sqft).
Under Singapore's Personal Data Protection Act, publishing raw caveat unit numbers (e.g. #04-12) allows re-identification via SLA title searches. Our pipeline parses the string at ingestion to extract floor_num = 4 and stack = "12", and permanently masks the database record to #04-XX or #XX-XX.
5 Preprocessing & Spatial Feature Engineering
We engineered a production feature store consisting of 23 explanatory covariates across 4 tiers:
2,500+ condos and 11,882 stacks encoded via 5-fold cross-validation with empirical Bayes smoothing to prevent data leakage.
Random train/test splitting was eliminated. Strictly trained on historical quarters (2013–2023) and tested blindly on 2024–2026.
6 Feature Correlation Matrix & Real Estate Signals
Spearman and Pearson correlation analysis against secondary resale PSF revealed the primary economic drivers:
| Feature Covariate | Correlation | Impact Magnitude | Physical / Economic Signal |
|---|---|---|---|
| project_name (TargetEnc) | +0.82 | Condo prestige, developer brand & micro-location | |
| dist_to_cbd_km (SVY21) | -0.68 | Prime central location elasticity | |
| ura_ppi (Macro Index) | +0.54 | Macro capital momentum overlay | |
| floor_ratio (Normalized) | +0.34 | Unblocked view & breeze elevation premium | |
| is_freehold (Tenure) | +0.29 | Freehold title scarcity premium | |
| dist_to_mrt_m (SVY21) | -0.27 | Walking transit accessibility | |
| area_sqft (Unit Size) | -0.21 | Quantum elasticity (larger units trade at lower PSF) | |
| sora_3m (Mortgage Cost) | -0.18 | Cost of debt / mortgage rate headwind |
7 Model Development & Architecture Evolution
Tracking model generalization across development sprints revealed why random train/test splits are deeply deceptive:
| Model Iteration | Out-of-Time MAPE | Test R² | Test MAE | Key Evolution & Breakthrough |
|---|---|---|---|---|
| V1: Naive Grouped Median | 14.82% | 0.5810 | $168.04 | Unweighted historical medians |
| V2: Ridge Linear Baseline | 9.83% | 0.7709 | $124.50 | Standardized linear baseline with WGS84 |
| V2.5: GBDT (Random Split) | 3.65%* | 0.9613 | $67.24 | *Look-ahead leakage trap (shuffled 2024 into train) |
| V3.0: Walk-Forward GBDT | 7.12% | 0.8840 | $98.15 | Strict out-of-time future test fold (2013–23 vs 2024+) |
| V3.1: GBDT + Macro Overlays | 5.92% | 0.9215 | $81.20 | URA PPI & MAS SORA interest rate overlays |
| V3.2: Optuna Champion GBDT | 5.04% | 0.9412 | $68.45 | Bayesian tuned + 90% Conformal Coverage |
Quantile rental engine trained on 233,507 contracts yielding P10 (conservative floor), P50 (median fair rent), and P90 (peak) projections with 6.36% test MAPE.
Distribution-free uncertainty intervals guaranteeing 90% finite-sample coverage (±$215 PSF) without relying on normal distribution heuristics.
8 Hyperparameter Tuning & Optimization
We ran a 50-trial Bayesian optimization study using Optuna with the Tree-structured Parzen Estimator (TPE). Objective: minimize out-of-time MAPE on future test folds:
Optuna Bayesian tuning lowered out-of-time forecasting error from 7.12% to 5.04% MAPE (a 29.2% relative error reduction).
9 Deployment, Serving Architecture & Governance
Production serving decouples heavy analytical operations from high-concurrency API reads:
DuckDB OLAP analytical warehouse exports pre-computed scores into an indexed SQLite store for sub-10ms query responses.
Zero math executed in browser JavaScript. All amortizations, conformal margins, and valuations originate strictly from FastAPI Python endpoints.
554 automated pytests run on every CI build. Quarterly macro drift detection and MAS FAA zero-normative compliance.
Top 5 Student Takeaways (For Peer Data Scientists)
- Time-Series Leakage: A random shuffle split on temporal transactions will deceive you with inflated R² scores. Always test out-of-time.
- Cartesian Projections: Project spherical Lat/Long to local metric grids (SVY21) before feeding to decision trees.
- High Cardinality: Use K-Fold Target Encoding with Bayes smoothing over One-Hot Encoding for 2,500+ categories.
- Uncertainty Quantification: A point estimate without bounds breeds false confidence. Use Split Conformal Prediction for distribution-free guarantees.
- Decouple Analytics & Serving: Never run spatial joins in web handlers. Build with DuckDB for training and SQLite for serving.
Ready to Explore Singapore Condos?
Put the machine learning models and quantile rental estimates to work on real transactions.