<frozen importlib._bootstrap>:491: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module '_duckdb', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.Data Description
Rating Statistics
| Ratings | 24,999,912 |
| Users | 162,540 |
| Items | 59,046 |
| Density | 0.260% |
| Item Gini | 0.942 |
| Start Date | 1995-01-09 11:46:49 |
| End Date | 2019-11-21 09:15:03 |
Item Statistics
This section describes the distribution of various item statistics from the data set.
Item Popularity
What is the distribution of popularity?
Let’s also look at this as a Lorenz curve, for clarity:
Item Average Rating
What is the distribution of average ratings?
User Statistics
We now turn to the distribution of various user statistics.
User Average Ratings
How are user averages distributed?
User Activity Level
And what is the distribution of user activity levels (# of ratings)?
Ratings over Time
The MovieLens ratings have timestamps, so we’ll also look at a temporal view of the data.
Data Volume
How did the data grow over time?
How many ratings are we getting each month through the life of the data set?
User Activity
Monthly unique users is a good measure of user activity.
How long do users usually stick around?
Parametric Activity Distributions
Some downstream uses benefit from parametric distributions of user/item activity levels.
Item Activity Distributions
This section models the item popularity distribution with various parametric distributions.
| Distribution | Params | Location | Scale | D(KL) | Δ(JS) |
|---|---|---|---|---|---|
| Log-normal | s=13.629 |
1 | 0.102 | 0.966 | 0.42 |
| Pareto | b=0.341 |
0.61 | 0.39 | 0.157 | 0.179 |
| Power law | a=2.069 |
0.657 | 999.74 | inf | 0.741 |
| Geometric | p=0.002 |
0 | 2.672 | 0.583 |
User Activity Distributions
Now the same, for user activity distributions.
| Distribution | Params | Location | Scale | D(KL) | Δ(JS) |
|---|---|---|---|---|---|
| Log-normal | s=1.593 |
19.019 | 46.377 | 0.031 | 0.09 |
| Pareto | b=0.704 |
0.144 | 19.854 | 0.098 | 0.162 |
| Power law | a=0.460 |
19.756 | 999.292 | inf | 0.318 |
| NegBinom | n=1.000, p=0.007 |
9 | 0.288 | 0.195 | |
| BetaNegBinom | n=197.000, a=2.050, b=0.756 |
20 | 0.015 | 0.064 |
