Auditing Gender and Geographic Disparities in Machine Learning Models of KIP Kuliah Award Decisions: A Two-Cohort Temporal Evaluation

Universities increasingly consider machine learning (ML) to help screen applications for need-based scholarships. Such models learn from past award decisions and may carry forward the group differences they contain, yet they are usually tested on the same intake used to build them. In practice, a model built on one year is applied to the next. This study examines whether a fairness audit of one intake still holds for the following intake. All applications for the Kartu Indonesia Pintar Kuliah (KIP Kuliah) scholarship at a private university in Indonesia in 2023 (n = 708) and 2024 (n = 829) were analysed. Logistic regression, random forest, and XGBoost models were trained on the 2023 cohort and tested once on the 2024 cohort; gender and geographic origin were withheld from the models and used only to compare their selections with past decisions. The models predicted past decisions only slightly better than chance, and their apparent loss of accuracy in 2024 was largely due to a lower acceptance rate rather than to poorer applicant rankings. No consistent gender gap was found. When the models filled a fixed quota, applicants from municipal (city) schools were selected 19–23 percentage points more often than those from regency schools, about twice the gap in the 2024 decisions themselves, because parental occupation acted as an indirect signal of school location. This gap depended on the selection rule and did not hold under every setting. The proposed contribution is a two-intake audit protocol that separates disparities inherited from past decisions from those added by a model. Institutions using such models should re-audit them before every intake.