FlexMF BPR

This page analyzes the hyperparameter tuning results for the FlexMF scorer in implicit-feedback mode with pairwise loss (Bayesian Personalized Ranking).

Parameter Search Space

<frozen abc>:106: UserWarning: component class Placeholder does not define a config attribute type
Parameter Type Distribution Values Selected
embedding_size_exp Integer Uniform 3 ≤ \(x\) ≤ 10 5
regularization Float LogUniform 0.0001 ≤ \(x\) ≤ 10 0.0664
learning_rate Float LogUniform 0.001 ≤ \(x\) ≤ 0.1 0.00718
reg_method Categorical Uniform L2, AdamW AdamW
negative_count Integer Uniform 1 ≤ \(x\) ≤ 5 4
item_bias Categorical Uniform True, False False

Final Result

Searching selected the following configuration:

{
    'embedding_size_exp': 5,
    'regularization': 0.0663856149154072,
    'learning_rate': 0.007184176491001479,
    'reg_method': 'AdamW',
    'negative_count': 4,
    'item_bias': False,
    'epochs': 11
}

With these metrics:

{
    'RBP': 0.24754969398670904,
    'DCG': 11.834603286748468,
    'NDCG': 0.4826160071848388,
    'RecipRank': 0.4374838234297654,
    'Hit10': 0.6854413702239789,
    'max_epochs': 50,
    'epoch_train_s': 1.9734547059051692,
    'epoch_measure_s': 5.043461619876325,
    'done': False,
    'training_iteration': 11,
    'trial_id': '49350fd4',
    'date': '2025-09-29_23-29-32',
    'timestamp': 1759202972,
    'time_this_iter_s': 7.0205841064453125,
    'time_total_s': 71.02409791946411,
    'pid': 3910351,
    'hostname': 'CCI-ws21',
    'node_ip': '10.248.127.152',
    'config': {
        'embedding_size_exp': 5,
        'regularization': 0.0663856149154072,
        'learning_rate': 0.007184176491001479,
        'reg_method': 'AdamW',
        'negative_count': 4,
        'item_bias': False,
        'epochs': 11
    },
    'time_since_restore': 71.02409791946411,
    'iterations_since_restore': 11
}

Parameter Analysis

Embedding Size

The embedding size is the hyperparameter that most affects the model’s fundamental logic, so let’s look at performance as a fufnction of it:

Data Handling

Learning Parameters

Iteration Completion

How many iterations, on average, did we complete?

How did the metric progress in the best result?

How did the metric progress in the longest results?