FlexMF Explicit on ML1M

This page analyzes the hyperparameter tuning results for the FlexMF scorer in explicit-feedback mode (a biased matrix factorization model trained with PyTorch).

Parameter Search Space

Parameter	Type	Distribution	Values
embedding_size	Integer	LogUniform	4 ≤ \(x\) ≤ 512
regularization	Float	LogUniform	0.0001 ≤ \(x\) ≤ 10
learning_rate	Float	LogUniform	0.001 ≤ \(x\) ≤ 0.1
reg_method	Categorical	Uniform	L2, AdamW

Final Result

Searching selected the following configuration:

{
    'embedding_size': 8,
    'regularization': 0.04348568763678297,
    'learning_rate': 0.007860577021310463,
    'reg_method': 'L2',
    'epochs': 7
}

With these metrics:

{
    'RBP': 0.0159899526060798,
    'NDCG': 0.1816292386546767,
    'RecipRank': 0.06256693398264787,
    'RMSE': 0.8224932070056729,
    'TrainTask': 'caf514e0-bac0-486a-a261-fe588be878c4',
    'TrainTime': None,
    'TrainCPU': None,
    'max_epochs': 50,
    'done': False,
    'training_iteration': 7,
    'trial_id': 'cdb8e_00091',
    'date': '2025-04-03_22-16-25',
    'timestamp': 1743732985,
    'time_this_iter_s': 2.600450277328491,
    'time_total_s': 17.758015394210815,
    'pid': 1297952,
    'hostname': 'CCI-ws21',
    'node_ip': '10.248.127.152',
    'config': {
        'embedding_size': 8,
        'regularization': 0.04348568763678297,
        'learning_rate': 0.007860577021310463,
        'reg_method': 'L2',
        'epochs': 7
    },
    'time_since_restore': 17.758015394210815,
    'iterations_since_restore': 7
}

Parameter Analysis

Embedding Size

The embedding size is the hyperparameter that most affects the model’s fundamental logic, so let’s look at performance as a fufnction of it:

Learning Parameters

Learning Rate

Regularization

Iteration Completion

How many iterations, on average, did we complete?

How did the metric progress in the best result?

How did the metric progress in the longest results?