While reading the article Matrix Factorization Techniques for Recommender Systems I came across the following description:

"Matrix factorization models map both users and items to a joint latent factor space of dimensionality f, such that user-item interactions are modeled as inner products in that space."

Could someone explain to me --or guide me to an article -- what is meant by a "joint latent factor space of dimensionality f".

The articles is available here.

Thank you


closed as off-topic by Ricardo Andrade, Chris Godsil, Yemon Choi, Ian Morris, Dima Pasechnik Jan 3 '15 at 18:19

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question does not appear to be about research level mathematics within the scope defined in the help center." – Ricardo Andrade, Chris Godsil, Yemon Choi, Ian Morris, Dima Pasechnik
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ I'd read the next sentence in the paper you linked. Specifically: "Accordingly, each item $i$ is associated with a vector $q_i \in R^f$, and each user $u$ is associated with a vector $p_u \in R^f$. For a given item $i$, the elements of $q_i$ measure the extent to which the item possesses those factors, positive or negative. For a given user $u$, the elements of $pu$ measure the extent of interest the user has in items that are high on the corresponding factors, again, positive or negative. $\endgroup$ – user3035 Feb 6 '10 at 19:51
  • $\begingroup$ The resulting dot product, $q_i^T p_u$, captures the interaction between user $u$ and item $i$---the user's overall interest in the item's characteristics. This approximates user $u$'s rating of item $i$, which is denoted by $r_{ui}$, leading to the estimate $r_{ui} = q_i^T p_u$." $\endgroup$ – user3035 Feb 6 '10 at 19:52
  • $\begingroup$ Thanks, I read the rest of the article and understand what is meant. But in general if confronted with a "joint latent factor space of dimensionality f" does it mean the dot-product of two vectors of dimension f. $\endgroup$ – Francois Feb 6 '10 at 19:58
  • 1
    $\begingroup$ In general, if confronted with a "joint latent factor space of dimensionality f" then I jump into the nearest Tardis and run to the next universe because this one is clearly beyond saving. $\endgroup$ – Loop Space Feb 6 '10 at 20:03
  • 1
    $\begingroup$ No. In general, "latent" means "hidden", or "not directly observable", something that needs to be inferred. For example, in medical diagnosis, symptoms are observable, and the disease that causes them is a latent variable to be inferred. In collaborative filtering, the ratings are observed, and the properties of items and preferences of users that cause them to like or dislike those properties are latent variables. These latent variables can in principle be anything (e.g. diagnosis is just an integer variable referring to a particular disease). $\endgroup$ – user3035 Feb 6 '10 at 20:07

I'll give an example from politics. Let's say you have a legislative body, such as the House of Representatives in the U. S. Congress. Over a period of time, the members of the House will vote on many bills and thereby accrue a voting history. Let's encode votes as numbers: a vote for a bill is 1, a vote against a bill is -1, and an abstention (no vote) is 0. Also, let's label the representatives $R_1,\ldots,R_m$, and label the bills $B_1,\ldots,B_n$. We thus have, for each pair $i,j$ of numbers with $0 <i\leq m$, $0 < j \leq n$, a vote $V_{ij}\in \{-1,0,1\}$, namely how congressperson $R_i$ voted on bill $B_j$. This gives us a big $m\times n$ "vote matrix" $V$ whose entries are the votes $V_{ij}$.

Now, it's true that each congressperson has an opinion on every bill, or dually, that each bill appeals to different congresspeople in different amounts (possibly negative). However, that description misses an important aspect of the situation: a congressperson's voting behavior can be approximated using much fewer parameters than a list of all his votes. Also, a bill's tendency to appeal to different people can be approximated using much fewer parameters than a list of all the people who voted for and against it. Indeed, it's not really bills that congresspeople have opinions on; it's issues and policies. On the flip side, a given bill will implement various types of policies and address various issues, and that's really what determines who will like it and how much.

Thus, to describe voting behaviors, what you really need is (1) a list of policies $P_1,\ldots,P_f$ (the latent factors), (2) for each congressperson $R_i$, a degree of preference $S_{ik}$ for each policy $P_k$, and (3) for each bill $B_j$, a value $C_{kj}$ describing to what extent it implements policy $P_k$. Let's continue the convention that positive values for $S_{ik}$ or $C_{kj}$ indicate accordance and negative values indicate opposition. To each congressperson $R_i$, we can assign the vector $S_i = (S_{i1},\ldots,S_{if})$ (which we might call the "policy vector" for that congressperson), and to each bill $B_j$, we can assign the vector $C_j = (C_{1j},\ldots,C_{fj})$ (which we might also call the "policy vector" for that bill).

For what follows, I'm going to use a very simplistic (but somewhat plausible) mathematical model. (Your article uses a more complicated and more realistic model, taking bias into account, for example.) Also, the model makes a lot more sense if congresspeople are asked to state their degree of preference for each bill rather than simply voting "yes" or "no," so that the "votes" $V_{ik}$ take values in $\mathbb{R}$. When deciding how to vote on a bill, a congressperson may consider how well it correlates with her opinions and make a decision based on that. With a lot of vigorous hand waving and wishful thinking, the outcome of this process can be described very simply in terms of policy vectors: the vote $V_{ij}$ is simply the dot product $S_i\cdot C_j = \sum_k S_{ik} C_{kj}$. (I'll leave it as an exercise to show that's not completely ridiculous, even if unlikely to be exactly true.) Another way of saying this is that if $S$ is the matrix with entries $S_{ik}$ and $C$ is the matrix with entries $C_{kj}$, then $V = SC$. In other words, our knowledge about legislative policies as latent factors induces a factorization of the matrix $V$.

One merit of the above approach is that although it's a bit too simple, it leads to well understood mathematics. For it to be useful, the number of policies $f$ should be much smaller than $m$ and $n$, the numbers of congresspeople and bills. In that case, the factorization $V = SC$ means that $V$ has rank $f$, which is small compared to its dimensions. In practice, admitting that the description in terms of policies as latent factors can only be a good approximation, not exact, this means that $V$ is well approximated by a low-rank matrix. Factorizations can be obtained from that observation alone using standard matrix tools like the singular-value decomposition. In particular, the policy vectors can be found even before you have any idea what the "policies" should be. (In other words, you don't have to sit down and make a list of policies you think are important and figure out what the policy vectors must be from that; you can use a standard algorithm which will determine the policies for you. Of course, it won't name the policies, but if you need to, you can compare policy vectors to figure out what real-world policies the algorithmically extracted policies approximate.)

  • $\begingroup$ can you mention some of the "standard algorithm which will determine the policies"? $\endgroup$ – graphtheory92 Dec 31 '14 at 18:38
  • 1
    $\begingroup$ @graphtheory92 - Singular Value Decomposition is the obvious one. $\endgroup$ – Darsh Ranjan Feb 12 '15 at 5:19

Not the answer you're looking for? Browse other questions tagged or ask your own question.