This page analyzes the hyperparameter tuning results for the FlexMF scorer in implicit-feedback mode with logistic loss (Logistic Matrix Factorization).
Parameter Search Space
/home/mde48/lenskit/lenskit-codex/.venv/lib/python3.12/site-packages/ray/tune/search/sample.py:700: RayDeprecationWarning: The `base` argument is deprecated. Please remove it as it is not actually needed in this method.
embedding_size |
Integer |
LogUniform |
4 ≤ \(x\) ≤ 512 |
130 |
regularization |
Float |
LogUniform |
0.0001 ≤ \(x\) ≤ 10 |
0.0625 |
learning_rate |
Float |
LogUniform |
0.001 ≤ \(x\) ≤ 0.1 |
0.00235 |
reg_method |
Categorical |
Uniform |
L2, AdamW |
AdamW |
negative_count |
Integer |
Uniform |
1 ≤ \(x\) ≤ 5 |
3 |
positive_weight |
Float |
Uniform |
1 ≤ \(x\) ≤ 10 |
5.55 |
user_bias |
Categorical |
Uniform |
True, False |
False |
item_bias |
Categorical |
Uniform |
True, False |
True |
Final Result
Searching selected the following configuration:
{
'embedding_size': 130,
'regularization': 0.06254722563846132,
'learning_rate': 0.002348262652167187,
'reg_method': 'AdamW',
'negative_count': 3,
'positive_weight': 5.551829597025691,
'user_bias': False,
'item_bias': True,
'epochs': 10
}
With these metrics:
{
'RBP': 0.1763513697213253,
'DCG': 11.466147341874354,
'NDCG': 0.42159244735845863,
'RecipRank': 0.33826189606414603,
'Hit10': 0.5797829036635007,
'max_epochs': 50,
'epoch_train_s': 29.1736630230007,
'epoch_measure_s': 14.4465691660007,
'done': True,
'training_iteration': 10,
'trial_id': 'accd2f14',
'date': '2025-07-29_01-18-40',
'timestamp': 1753766320,
'time_this_iter_s': 43.62479782104492,
'time_total_s': 881.3234508037567,
'pid': 210957,
'hostname': 'CCI-ws21',
'node_ip': '10.248.127.152',
'config': {
'embedding_size': 130,
'regularization': 0.06254722563846132,
'learning_rate': 0.002348262652167187,
'reg_method': 'AdamW',
'negative_count': 3,
'positive_weight': 5.551829597025691,
'user_bias': False,
'item_bias': True,
'epochs': 10
},
'time_since_restore': 43.62479782104492,
'iterations_since_restore': 1
}
Parameter Analysis
Embedding Size
The embedding size is the hyperparameter that most affects the model’s fundamental logic, so let’s look at performance as a fufnction of it:
Iteration Completion
How many iterations, on average, did we complete?
How did the metric progress in the best result?
How did the metric progress in the longest results?