BTK Datathon 2026: A Deep Dive into Predicting Career Success with ML and NLP

This is a full technical walkthrough of my project for BTK Akademi's Datathon 2026 competition (run in collaboration with Google and the Turkish Entrepreneurship Foundation): predicting a continuous career_success_score (0-100) from student academic, technical, project, portfolio, interview, and social profile data - including a natural language mentor feedback field. The competition metric was Mean Squared Error (MSE), lower is better.

Iterative Methodology

Instead of building one large model upfront, I worked in six measurable steps - each version adding exactly one idea, each submitted to Kaggle to see its real effect.

Exploratory Data Analysis

EDA surfaced several findings that shaped every decision downstream.

Missing values: seven columns had gaps, the worst being internship_duration_months (~16.6% missing). 82% of those missing rows belonged to students with zero internships - meaning the value wasn't truly "missing," it was logically zero. This insight led to smart imputation rather than blind median-filling.

Target distribution: mean 76.9, slightly left-skewed (skewness -0.45), with 773 students scoring exactly 100 - a deliberately engineered ceiling effect in the dataset that would later require special handling.

Correlations: project_quality_score was the dominant signal (0.54), followed by technical_interview_score (0.34). Counterintuitively, cgpa alone showed almost zero correlation (-0.01) - a finding that later proved important once interaction features were introduced.

Noise columns: hobby and preferred_social_media_platform showed no meaningful difference in average score across categories - these were effectively noise columns, likely deliberate traps in the competition data, and were dropped.

Feature Engineering

Key derived features included avg_technical_score and avg_soft_skill_score (group averages across skill categories), role_weighted_score (technical skills weighted by the student's target role), experience_score (a weighted blend of internship, real-client-project, freelance, and hackathon experience), interaction terms (e.g. technical_interview_score × problem_solving_score, and cgpa interactions to surface its otherwise-hidden signal), and log1p transforms for right-skewed count features like github_avg_stars.

NLP on Mentor Feedback

The mentor_feedback_text field - a short Turkish evaluation written by each student's mentor - was processed with a three-layer strategy: sentiment word counting using hand-built Turkish positive/negative/potential word lists, a derived sentiment score and ratio, and TF-IDF with Turkish stopword removal to automatically surface the 100 most distinctive terms.

sentiment_score reached a 0.40 correlation with the target - the second-strongest signal in the entire dataset after project_quality_score. TF-IDF independently confirmed the hand-built word lists: "ancak" (Turkish for "however") emerged as the strongest negative signal (-0.24), since in Turkish it typically introduces criticism after praise.

Modeling and Ensemble

Three gradient boosting models - XGBoost, LightGBM, and CatBoost - formed the base layer, tuned via Optuna over 5-fold cross-validation. Hyperparameter tuning alone delivered the single largest improvement in the entire project (~6 points of Kaggle MSE).

Rather than a simple average, the final model used stacking: each base model's out-of-fold predictions fed into a Ridge regression meta-model, which learned how much to trust each one from the data itself (CatBoost received the highest weight, ~0.55).

V6 Refinements

Two concrete improvements were identified and applied after V5's 90.33 score, each validated in cross-validation before being confirmed on Kaggle.

OOF Target Encoding: department and target_role were switched from One-Hot Encoding to Out-of-Fold Target Encoding - each category assigned the mean score of students within it, computed using only out-of-fold data to avoid leakage. This delivered a -0.64 MSE gain in CV.

Soft-Blend Ceiling Correction: the 773 students with a perfect 100 were being systematically under-predicted at ~95 by the tree models. A "hard threshold" fix (round anything above 94 up to 100) was tested and failed - false positives cost more than the gains. Instead, a learned classifier drove a soft blend: final = P(100) × 100 + (1 − P(100)) × stacking_prediction.

The more confident the model, the closer the prediction moves toward 100; when unsure, it barely adjusts. This delivered a -0.71 MSE gain in CV. Combined, the two refinements improved the Kaggle score by 1.39 points (90.33 → 88.94).

Error Analysis

The model captures the 50-95 score band well. The largest single error was a student with every feature above average but a true score of 0 - a clear case of label noise rather than model failure.

Residuals are centered near zero and symmetric (mean -0.26), confirming the model is unbiased. The remaining long tails trace back to structural noise deliberately built into the competition dataset.

Results and Lessons

Kaggle public MSE dropped from 99.09 to 88.94 - a 10.15-point improvement through disciplined, measured iteration. Hyperparameter tuning delivered the single biggest win; NLP genuinely mattered rather than being a checkbox feature; and small-looking refinements like target encoding and ceiling correction compounded into a meaningful 1.39-point gain. Not every idea worked - early stopping, native CatBoost categorical handling, and hard-threshold ceiling correction were all tested and rejected based on evidence, and documenting the failures was as valuable as documenting the wins.

The full report, with additional hypothesis tests and rejected approaches, is available in the PDF below alongside the GitHub repository and Kaggle profile.