Performance Summary

This is a summary view of model performance on the ML1M data set.

Top-N Recommendation Accuracy

Rating Prediction Accuracy

Resource Consumption

Time

Memory

Power

Leaderboard Table

This provides quick numeric access to the model results by mean metric. Note, however, that simple means can be misleading!

Top-N Recommendation

    RBP NDCG RecipRank
model variant      
uknn-implicit hyperopt-best 0.109 0.289 0.367
uknn-implicit optuna-best 0.109 0.289 0.367
uknn-implicit random-best 0.109 0.289 0.367
uknn-implicit default 0.105 0.282 0.345
als-implicit optuna-best 0.105 0.295 0.336
als-implicit hyperopt-best 0.103 0.292 0.324
als-implicit random-best 0.101 0.289 0.320
flexmf-logistic optuna-best 0.098 0.273 0.332
flexmf-logistic hyperopt-best 0.097 0.274 0.335
flexmf-bpr hyperopt-best 0.097 0.270 0.334
flexmf-bpr optuna-best 0.096 0.271 0.333
flexmf-logistic random-best 0.092 0.260 0.310
implicit-bpr optuna-best 0.092 0.253 0.319
implicit-bpr hyperopt-best 0.091 0.250 0.323
flexmf-warp hyperopt-best 0.090 0.257 0.306
implicit-bpr random-best 0.089 0.244 0.315
flexmf-warp optuna-best 0.084 0.246 0.294
implicit-bpr default 0.081 0.231 0.277
flexmf-bpr random-best 0.080 0.233 0.279
iknn-implicit hyperopt-best 0.075 0.205 0.271
iknn-implicit random-best 0.075 0.205 0.271
iknn-implicit optuna-best 0.075 0.205 0.271
flexmf-warp random-best 0.072 0.215 0.260
iknn-implicit default 0.071 0.198 0.260
als-implicit default 0.069 0.231 0.208
flexmf-logistic default 0.046 0.145 0.172
flexmf-bpr default 0.045 0.151 0.162
popular default 0.042 0.134 0.152
flexmf-warp default 0.033 0.119 0.127
als-biased default 0.014 0.056 0.057
flexmf-explicit hyperopt-best 0.014 0.055 0.054
bias optuna-best 0.013 0.052 0.043
flexmf-explicit random-best 0.013 0.050 0.050
bias hyperopt-best 0.013 0.051 0.043
bias random-best 0.012 0.050 0.042
uknn-explicit random-best 0.011 0.051 0.043
uknn-explicit hyperopt-best 0.011 0.051 0.043
flexmf-explicit optuna-best 0.008 0.041 0.032
iknn-explicit optuna-best 0.007 0.042 0.028
iknn-explicit random-best 0.007 0.041 0.027
iknn-explicit hyperopt-best 0.007 0.041 0.027
iknn-explicit default 0.007 0.040 0.026
flexmf-explicit default 0.005 0.032 0.026
uknn-explicit default 0.004 0.034 0.018
uknn-explicit optuna-best 0.003 0.034 0.017
als-biased random-best 0.003 0.033 0.017
als-biased optuna-best 0.002 0.033 0.014
als-biased hyperopt-best 0.002 0.033 0.014
bias default 0.001 0.032 0.012

Rating Prediction

    RMSE MAE
model variant    
iknn-explicit hyperopt-best 0.806 0.674
iknn-explicit optuna-best 0.806 0.674
iknn-explicit random-best 0.806 0.674
iknn-explicit default 0.807 0.675
als-biased optuna-best 0.812 0.679
als-biased hyperopt-best 0.812 0.679
flexmf-explicit hyperopt-best 0.812 0.682
als-biased random-best 0.815 0.682
als-biased default 0.817 0.686
flexmf-explicit random-best 0.821 0.688
flexmf-explicit optuna-best 0.824 0.692
uknn-explicit optuna-best 0.852 0.709
uknn-explicit hyperopt-best 0.852 0.709
uknn-explicit random-best 0.852 0.709
bias optuna-best 0.857 0.724
bias hyperopt-best 0.857 0.724
bias random-best 0.857 0.724
uknn-explicit default 0.858 0.715
bias default 0.859 0.724
flexmf-explicit default 0.860 0.720