FlexMF Explicit

This page analyzes the hyperparameter tuning results for the FlexMF scorer in explicit-feedback mode (a biased matrix factorization model trained with PyTorch).

Parameter Search Space

/home/mde48/lenskit/lenskit-codex/.venv/lib/python3.12/site-packages/ray/tune/search/sample.py:700: RayDeprecationWarning: The `base` argument is deprecated. Please remove it as it is not actually needed in this method.

Parameter	Type	Distribution	Values	Selected
embedding_size	Integer	LogUniform	4 ≤ \(x\) ≤ 512	14
regularization	Float	LogUniform	0.0001 ≤ \(x\) ≤ 10	0.00098
learning_rate	Float	LogUniform	0.001 ≤ \(x\) ≤ 0.1	0.00139
reg_method	Categorical	Uniform	L2, AdamW	AdamW

Final Result

Searching selected the following configuration:

{
    'embedding_size': 14,
    'regularization': 0.0009802629975817626,
    'learning_rate': 0.001386984443349164,
    'reg_method': 'AdamW',
    'epochs': 6
}

With these metrics:

{
    'RBP': 0.09563383798537865,
    'LogRBP': 1.4470114020420564,
    'NDCG': 0.3501002231575794,
    'RecipRank': 0.22524821087033406,
    'RMSE': 0.8001071436411419,
    'TrainTask': 'f7917e43-3801-4a21-9fbf-28808050a7c2',
    'TrainTime': None,
    'TrainCPU': None,
    'max_epochs': 50,
    'done': True,
    'training_iteration': 6,
    'trial_id': '2dfdb308',
    'date': '2025-05-06_00-23-29',
    'timestamp': 1746505409,
    'time_this_iter_s': 22.26735281944275,
    'time_total_s': 142.54408049583435,
    'pid': 463935,
    'hostname': 'CCI-ws21',
    'node_ip': '10.248.127.152',
    'config': {
        'embedding_size': 14,
        'regularization': 0.0009802629975817626,
        'learning_rate': 0.001386984443349164,
        'reg_method': 'AdamW',
        'epochs': 6
    },
    'time_since_restore': 142.54408049583435,
    'iterations_since_restore': 6
}

Parameter Analysis

Embedding Size

The embedding size is the hyperparameter that most affects the model’s fundamental logic, so let’s look at performance as a fufnction of it:

Iteration Completion

How many iterations, on average, did we complete?

How did the metric progress in the best result?

How did the metric progress in the longest results?