Performance Summary

This is a summary view of model performance on the ML20M data set.

Top-N Recommendation Accuracy

Rating Prediction Accuracy

Resource Consumption

Time

Memory

Power

Leaderboard Table

This provides quick numeric access to the model results by mean metric. Note, however, that simple means can be misleading!

Top-N Recommendation

    RBP NDCG RecipRank
model variant      
flexmf-logistic optuna-best 0.235 0.155 0.419
flexmf-bpr optuna-best 0.230 0.151 0.415
flexmf-bpr random-best 0.228 0.150 0.407
flexmf-warp hyperopt-best 0.228 0.150 0.404
flexmf-logistic hyperopt-best 0.225 0.148 0.406
flexmf-bpr hyperopt-best 0.225 0.148 0.409
flexmf-logistic random-best 0.222 0.146 0.403
flexmf-warp optuna-best 0.222 0.147 0.395
flexmf-warp random-best 0.217 0.143 0.385
als-implicit random-best 0.198 0.136 0.364
iknn-implicit random-best 0.196 0.132 0.351
iknn-implicit hyperopt-best 0.196 0.132 0.351
iknn-implicit optuna-best 0.196 0.132 0.351
als-implicit hyperopt-best 0.194 0.133 0.354
als-implicit optuna-best 0.191 0.131 0.351
iknn-implicit default 0.180 0.128 0.317
popular default 0.168 0.111 0.330
flexmf-warp default 0.163 0.110 0.315
flexmf-logistic default 0.147 0.100 0.287
flexmf-explicit hyperopt-best 0.139 0.095 0.302
als-implicit default 0.130 0.109 0.247
flexmf-bpr default 0.129 0.092 0.233
flexmf-explicit random-best 0.129 0.093 0.278
flexmf-explicit optuna-best 0.120 0.086 0.269
als-biased default 0.098 0.068 0.238
bias hyperopt-best 0.097 0.064 0.271
bias random-best 0.097 0.064 0.271
bias optuna-best 0.097 0.064 0.270
als-biased hyperopt-best 0.088 0.062 0.219
iknn-explicit default 0.065 0.059 0.122
iknn-explicit random-best 0.065 0.061 0.114
iknn-explicit hyperopt-best 0.065 0.061 0.114
iknn-explicit optuna-best 0.065 0.061 0.114
als-biased random-best 0.055 0.055 0.103
als-biased optuna-best 0.055 0.056 0.099
bias default 0.037 0.048 0.063
flexmf-explicit default 0.035 0.035 0.092

Rating Prediction

    RMSE MAE
model variant    
als-biased optuna-best 0.765 0.613
als-biased random-best 0.772 0.619
als-biased hyperopt-best 0.774 0.621
iknn-explicit hyperopt-best 0.777 0.624
iknn-explicit optuna-best 0.777 0.624
iknn-explicit random-best 0.777 0.624
flexmf-explicit optuna-best 0.777 0.622
iknn-explicit default 0.778 0.624
als-biased default 0.781 0.628
flexmf-explicit random-best 0.782 0.626
flexmf-explicit hyperopt-best 0.783 0.627
flexmf-explicit default 0.794 0.636
bias hyperopt-best 0.799 0.647
bias random-best 0.799 0.647
bias optuna-best 0.799 0.647
bias default 0.806 0.653