| Parameter | Type | Distribution | Values | Selected |
|---|---|---|---|---|
| embedding_size | Integer | LogUniform | 4 ≤ \(x\) ≤ 512 | 24 |
| regularization | Float | LogUniform | 0.0001 ≤ \(x\) ≤ 10 | 0.143 |
| learning_rate | Float | LogUniform | 0.001 ≤ \(x\) ≤ 0.1 | 0.0133 |
| reg_method | Categorical | Uniform | L2, AdamW | AdamW |
| negative_count | Integer | Uniform | 1 ≤ \(x\) ≤ 5 | 2 |
| item_bias | Categorical | Uniform | True, False | False |
FlexMF BPR
This page analyzes the hyperparameter tuning results for the FlexMF scorer in implicit-feedback mode with pairwise loss (Bayesian Personalized Ranking).
Parameter Search Space
Final Result
Searching selected the following configuration:
{ 'embedding_size': 24, 'regularization': 0.14262140027938575, 'learning_rate': 0.013342910754451731, 'reg_method': 'AdamW', 'negative_count': 2, 'item_bias': False, 'epochs': 18 }
With these metrics:
{ 'RBP': 0.09800347170685311, 'NDCG': 0.34548183418989437, 'RecipRank': 0.3302095187258008, 'TrainTask': '433c21bb-f27b-4b68-ba04-b142e39b0dfc', 'TrainTime': None, 'TrainCPU': None, 'max_epochs': 50, 'done': False, 'training_iteration': 18, 'trial_id': 'b9699_00067', 'date': '2025-04-03_22-08-23', 'timestamp': 1743732503, 'time_this_iter_s': 1.3059678077697754, 'time_total_s': 28.048662900924683, 'pid': 1276741, 'hostname': 'CCI-ws21', 'node_ip': '10.248.127.152', 'config': { 'embedding_size': 24, 'regularization': 0.14262140027938575, 'learning_rate': 0.013342910754451731, 'reg_method': 'AdamW', 'negative_count': 2, 'item_bias': False, 'epochs': 18 }, 'time_since_restore': 2.8064963817596436, 'iterations_since_restore': 2 }
Parameter Analysis
Embedding Size
The embedding size is the hyperparameter that most affects the model’s fundamental logic, so let’s look at performance as a fufnction of it:
Data Handling
--------------------------------------------------------------------------- NameError Traceback (most recent call last) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/mapping/evaluation.py:223, in evaluate(aesthetics, data, env) 222 try: --> 223 new_val = env.eval(col, inner_namespace=data) 224 except Exception as e: File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/mapping/_env.py:69, in Environment.eval(self, expr, inner_namespace) 68 code = _compile_eval(expr) ---> 69 return eval( 70 code, {}, StackedLookup([inner_namespace] + self.namespaces) 71 ) File <string-expression>:1 NameError: name 'config' is not defined The above exception was the direct cause of the following exception: PlotnineError Traceback (most recent call last) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/IPython/core/formatters.py:984, in IPythonDisplayFormatter.__call__(self, obj) 982 method = get_real_method(obj, self.print_method) 983 if method is not None: --> 984 method() 985 return True File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/ggplot.py:141, in ggplot._ipython_display_(self) 134 def _ipython_display_(self): 135 """ 136 Display plot in the output of the cell 137 138 This method will always be called when a ggplot object is the 139 last in the cell. 140 """ --> 141 self._display() File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/ggplot.py:181, in ggplot._display(self) 179 figure_size_px = self.theme._figure_size_px 180 buf = BytesIO() --> 181 self.save(buf, format=save_format, verbose=False) 182 display_func = get_display_function(format, figure_size_px) 183 display_func(buf.getvalue()) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/ggplot.py:673, in ggplot.save(self, filename, format, path, width, height, units, dpi, limitsize, verbose, **kwargs) 624 def save( 625 self, 626 filename: Optional[str | Path | BytesIO] = None, (...) 635 **kwargs: Any, 636 ): 637 """ 638 Save a ggplot object as an image file 639 (...) 671 Additional arguments to pass to matplotlib `savefig()`. 672 """ --> 673 sv = self.save_helper( 674 filename=filename, 675 format=format, 676 path=path, 677 width=width, 678 height=height, 679 units=units, 680 dpi=dpi, 681 limitsize=limitsize, 682 verbose=verbose, 683 **kwargs, 684 ) 686 with plot_context(self).rc_context: 687 sv.figure.savefig(**sv.kwargs) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/ggplot.py:621, in ggplot.save_helper(self, filename, format, path, width, height, units, dpi, limitsize, verbose, **kwargs) 618 if dpi is not None: 619 self.theme = self.theme + theme(dpi=dpi) --> 621 figure = self.draw(show=False) 622 return mpl_save_view(figure, fig_kwargs) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/ggplot.py:278, in ggplot.draw(self, show) 276 self = deepcopy(self) 277 with plot_context(self, show=show): --> 278 self._build() 280 # setup 281 self.figure, self.axs = self.facet.setup(self) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/ggplot.py:368, in ggplot._build(self) 364 layout.setup(layers, self) 366 # Compute aesthetics to produce data with generalised 367 # variable names --> 368 layers.compute_aesthetics(self) 370 # Transform data using all scales 371 layers.transform(scales) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/layer.py:468, in Layers.compute_aesthetics(self, plot) 466 def compute_aesthetics(self, plot: ggplot): 467 for l in self: --> 468 l.compute_aesthetics(plot) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/layer.py:260, in layer.compute_aesthetics(self, plot) 253 def compute_aesthetics(self, plot: ggplot): 254 """ 255 Return a dataframe where the columns match the aesthetic mappings 256 257 Transformations like 'factor(cyl)' and other 258 expression evaluation are made in here 259 """ --> 260 evaled = evaluate(self.mapping._starting, self.data, plot.environment) 261 evaled_aes = aes(**{str(col): col for col in evaled}) 262 plot.scales.add_defaults(evaled, evaled_aes) File ~/lenskit/lenskit-codex/.pixi/envs/dev/lib/python3.12/site-packages/plotnine/mapping/evaluation.py:226, in evaluate(aesthetics, data, env) 224 except Exception as e: 225 msg = _TPL_EVAL_FAIL.format(ae, col, str(e)) --> 226 raise PlotnineError(msg) from e 228 try: 229 evaled[ae] = new_val PlotnineError: "Could not evaluate the 'x' mapping: 'config.positive_weight' (original error: name 'config' is not defined)"
<plotnine.ggplot.ggplot at 0x7d5776322810>Learning Parameters
Iteration Completion
How many iterations, on average, did we complete?
How did the metric progress in the best result?
How did the metric progress in the longest results?
