FlexMF WARP

This page analyzes the hyperparameter tuning results for the FlexMF scorer in implicit-feedback mode with WARP loss.

Parameter Search Space

Parameter	Type	Distribution	Values	Selected
embedding_size	Integer	LogUniform	4 ≤ \(x\) ≤ 512	64
regularization	Float	LogUniform	0.0001 ≤ \(x\) ≤ 10	0.152
learning_rate	Float	LogUniform	0.001 ≤ \(x\) ≤ 0.1	0.0278
reg_method	Categorical	Uniform	L2, AdamW	AdamW
item_bias	Categorical	Uniform	True, False	True

Final Result

Searching selected the following configuration:

{
    'embedding_size': 64,
    'regularization': 0.15190199878621286,
    'learning_rate': 0.0278038096931685,
    'reg_method': 'AdamW',
    'item_bias': True,
    'epochs': 11
}

With these metrics:

{
    'RBP': 0.08698373228130635,
    'LogRBP': 1.3522058066879419,
    'NDCG': 0.32800770158911924,
    'RecipRank': 0.306070705739976,
    'TrainTask': 'ee551db9-e5a1-4535-921c-3230ab07f1b0',
    'TrainTime': None,
    'TrainCPU': None,
    'max_epochs': 50,
    'done': False,
    'training_iteration': 11,
    'trial_id': '31f16361',
    'date': '2025-05-05_07-23-15',
    'timestamp': 1746444195,
    'time_this_iter_s': 4.359487056732178,
    'time_total_s': 42.627496004104614,
    'pid': 3860051,
    'hostname': 'CCI-ws21',
    'node_ip': '10.248.127.152',
    'config': {
        'embedding_size': 64,
        'regularization': 0.15190199878621286,
        'learning_rate': 0.0278038096931685,
        'reg_method': 'AdamW',
        'item_bias': True,
        'epochs': 11
    },
    'time_since_restore': 42.627496004104614,
    'iterations_since_restore': 11
}

Parameter Analysis

Embedding Size

The embedding size is the hyperparameter that most affects the model’s fundamental logic, so let’s look at performance as a fufnction of it:

Learning Parameters

Learning Rate

Regularization

Iteration Completion

How many iterations, on average, did we complete?

How did the metric progress in the best result?

How did the metric progress in the longest results?