Performance Summary

This is a summary view of model performance on the ML10M data set.

Top-N Recommendation Accuracy

Rating Prediction Accuracy

Resource Consumption

Time

Memory

Power

Leaderboard Table

This provides quick numeric access to the model results by mean metric. Note, however, that simple means can be misleading!

Top-N Recommendation

    RBP NDCG RecipRank
model variant      
flexmf-bpr random-best 0.253 0.176 0.445
flexmf-logistic hyperopt-best 0.252 0.175 0.445
flexmf-logistic optuna-best 0.245 0.171 0.431
flexmf-bpr optuna-best 0.244 0.169 0.430
flexmf-bpr hyperopt-best 0.243 0.170 0.435
flexmf-logistic random-best 0.238 0.168 0.417
flexmf-warp optuna-best 0.237 0.165 0.417
uknn-implicit hyperopt-best 0.235 0.167 0.416
uknn-implicit optuna-best 0.235 0.167 0.416
uknn-implicit random-best 0.234 0.167 0.415
flexmf-warp hyperopt-best 0.232 0.161 0.410
flexmf-warp random-best 0.221 0.153 0.400
uknn-implicit default 0.219 0.161 0.392
als-implicit optuna-best 0.215 0.154 0.391
als-implicit hyperopt-best 0.214 0.159 0.386
als-implicit random-best 0.213 0.157 0.387
iknn-implicit hyperopt-best 0.200 0.144 0.365
iknn-implicit random-best 0.200 0.144 0.364
iknn-implicit optuna-best 0.200 0.144 0.364
flexmf-warp default 0.200 0.140 0.376
iknn-implicit default 0.185 0.141 0.339
flexmf-explicit optuna-best 0.183 0.121 0.379
flexmf-bpr default 0.183 0.126 0.334
flexmf-logistic default 0.180 0.124 0.337
popular default 0.175 0.119 0.337
als-implicit default 0.136 0.123 0.256
flexmf-explicit random-best 0.117 0.085 0.259
bias random-best 0.094 0.068 0.264
bias optuna-best 0.094 0.068 0.264
bias hyperopt-best 0.094 0.068 0.264
als-biased default 0.085 0.067 0.210
als-biased optuna-best 0.081 0.066 0.190
als-biased hyperopt-best 0.064 0.058 0.153
iknn-explicit default 0.040 0.046 0.094
iknn-explicit random-best 0.038 0.046 0.087
iknn-explicit hyperopt-best 0.038 0.046 0.087
iknn-explicit optuna-best 0.038 0.046 0.087
flexmf-explicit hyperopt-best 0.037 0.046 0.102
als-biased random-best 0.025 0.042 0.065
flexmf-explicit default 0.021 0.029 0.062
uknn-explicit default 0.016 0.032 0.045
uknn-explicit hyperopt-best 0.014 0.033 0.043
uknn-explicit random-best 0.014 0.033 0.042
uknn-explicit optuna-best 0.013 0.033 0.042
bias default 0.005 0.030 0.030

Rating Prediction

    RMSE MAE
model variant    
als-biased hyperopt-best 0.749 0.596
als-biased optuna-best 0.753 0.598
als-biased random-best 0.755 0.600
flexmf-explicit optuna-best 0.757 0.602
iknn-explicit hyperopt-best 0.759 0.604
iknn-explicit random-best 0.759 0.604
iknn-explicit optuna-best 0.759 0.604
flexmf-explicit hyperopt-best 0.759 0.603
iknn-explicit default 0.760 0.605
flexmf-explicit random-best 0.761 0.605
als-biased default 0.762 0.608
flexmf-explicit default 0.777 0.618
uknn-explicit random-best 0.778 0.617
uknn-explicit hyperopt-best 0.778 0.618
uknn-explicit optuna-best 0.778 0.617
bias optuna-best 0.783 0.630
bias hyperopt-best 0.783 0.630
bias random-best 0.783 0.630
uknn-explicit default 0.784 0.622
bias default 0.791 0.635