Performance Summary

This is a summary view of model performance on the ML100K data set.

Top-N Recommendation Accuracy

Rating Prediction Accuracy

Resource Consumption

Time

Memory

Power

Leaderboard Table

This provides quick numeric access to the model results by mean metric. Note, however, that simple means can be misleading!

Top-N Recommendation

    RBP NDCG RecipRank
model variant      
als-implicit random-best 0.143 0.371 0.407
uknn-implicit hyperopt-best 0.139 0.363 0.455
uknn-implicit optuna-best 0.139 0.363 0.451
uknn-implicit default 0.139 0.363 0.437
als-implicit optuna-best 0.139 0.370 0.417
uknn-implicit random-best 0.139 0.364 0.434
flexmf-logistic optuna-best 0.138 0.361 0.423
als-implicit hyperopt-best 0.137 0.366 0.421
flexmf-bpr optuna-best 0.134 0.356 0.439
flexmf-logistic hyperopt-best 0.129 0.345 0.411
flexmf-bpr random-best 0.120 0.333 0.380
flexmf-warp optuna-best 0.120 0.326 0.398
flexmf-warp hyperopt-best 0.120 0.329 0.394
iknn-implicit random-best 0.110 0.287 0.387
iknn-implicit optuna-best 0.110 0.287 0.387
iknn-implicit hyperopt-best 0.110 0.287 0.387
iknn-implicit default 0.104 0.275 0.349
flexmf-warp random-best 0.100 0.278 0.349
flexmf-logistic random-best 0.100 0.277 0.342
als-implicit default 0.098 0.295 0.283
flexmf-bpr hyperopt-best 0.098 0.270 0.330
popular default 0.067 0.194 0.231
flexmf-explicit random-best 0.028 0.095 0.104
flexmf-explicit optuna-best 0.027 0.095 0.112
flexmf-explicit hyperopt-best 0.025 0.085 0.104
als-biased hyperopt-best 0.024 0.093 0.099
bias hyperopt-best 0.022 0.081 0.083
bias optuna-best 0.021 0.080 0.079
bias random-best 0.021 0.079 0.079
als-biased default 0.017 0.072 0.071
flexmf-explicit default 0.015 0.065 0.061
iknn-explicit random-best 0.010 0.055 0.043
iknn-explicit hyperopt-best 0.010 0.055 0.040
iknn-explicit default 0.010 0.056 0.041
iknn-explicit optuna-best 0.009 0.054 0.041
flexmf-warp default 0.004 0.019 0.019
flexmf-logistic default 0.004 0.022 0.016
als-biased optuna-best 0.003 0.047 0.022
uknn-explicit optuna-best 0.003 0.042 0.020
uknn-explicit hyperopt-best 0.003 0.042 0.020
flexmf-bpr default 0.003 0.015 0.014
als-biased random-best 0.002 0.042 0.017
uknn-explicit default 0.002 0.033 0.014
bias default 0.001 0.042 0.016
uknn-explicit random-best 0.001 0.029 0.011

Rating Prediction

    RMSE MAE
model variant    
iknn-explicit optuna-best 0.916 0.770
iknn-explicit hyperopt-best 0.917 0.771
iknn-explicit random-best 0.917 0.771
iknn-explicit default 0.920 0.774
als-biased hyperopt-best 0.921 0.776
als-biased default 0.923 0.779
als-biased optuna-best 0.930 0.784
flexmf-explicit hyperopt-best 0.931 0.786
als-biased random-best 0.933 0.785
flexmf-explicit optuna-best 0.939 0.791
flexmf-explicit default 0.942 0.788
flexmf-explicit random-best 0.944 0.797
uknn-explicit optuna-best 0.944 0.795
uknn-explicit hyperopt-best 0.944 0.795
uknn-explicit random-best 0.944 0.796
uknn-explicit default 0.950 0.800
bias optuna-best 0.958 0.812
bias random-best 0.958 0.812
bias hyperopt-best 0.958 0.812
bias default 0.970 0.821