« Voting up or down is dead| Main | Lookin' for Love in all the Wrong Places »
Recommendation-Consumption Impedance Mismatch
There's been a terrific discussion around social sites both here and other other parts of the blogosphere. It seems to me that content selection systems swing out of whack after achieving a certain level of growth, as I pointed out yesterday.
You can't have a wagon hitched to two horses pulling in different directions. Likewise, if people will do what you reward them for doing, what you reward them for has to be in the best interests of the user community.
A diagram of the famous Object-Relational Impedance Mismatch
People creating object hierarchies are doing something fundamentally different than
people creating databases. A similar mismatch applies between
people participating in many sites and people consuming them
When I'm on a site that has vote-up.vote-down, and I vote, I am doing it for one of two reasons:
- I'm voting based on how I feel - This is the standard reason most sites want you to think the voting system is in place, to provide your feedback on the article to the other users.
- I'm voting based on how I want the system to perform - This is the actual reason most people vote. Whether to pay off a debt, stay in a clan, get more hits, or change people's minds, I'm voting to make the system perform in some certain way
Now if people are voting on a system to make it perform a certain way, that is, "out-thinking' the system designers, then the system will not perform the way it is designed, only the way the mob wants it to. This means, that as much as the creators of Digg would like it, the front page is not the best content for the most users to consume. Instead, it is the content that a plurality of the voting participants want the readers to read. That's a totally different concept. Heck, the front page stories may actually not have been read by anybody -- they're just the ones that people want on the front page. This is akin to the old saw that the smartest person does not always get to be president, only the person who is capable of being elected.
Let's give this phenomenon a name; recommendation-consumption impedance mismatch, which is kind of reminiscent of object-relational impedance mismatch in that there's a difference in what each participant is trying to do. For me to come back to a site, it has to present me with material I want to consume, not material other people want me to consume.
But like every other design decision, there are a pair of opposite forces at play here. On one side, people love participating in sites where they can have a score, where some folks (or articles) are publicly rewarded and acknowledged as being better than others. This "gaming of the system" has to be a huge part of the meteoric rise in popularity of some of these sites: they're fun to play. On the other side, people don't want to be manipulated. Just show me a page with all the stuff I want to read. I mean seriously, it's not like I have time to read 30 articles anyway -- couldn't one page just about cover the things I'd like to consume for today? But the game dynamic forces me to consume more and more pages for a decreasing return (while page stats and ad revenue go through the roof). This is obviously non-sustainable.
A system I was working on a couple of years ago.
While the voting box was kept out of the way until users clicked on it,
they still found the choices too confusing and the User Interface too much
I've seen three solutions to this problem. Each has benefits and drawbacks:
- Use a set of editors and weigh the votes - Paul Graham puts it like this:
"We think we have an answer to that. We're going to have a group of human editors who train the system in what counts as a good story. Each user's voting power will then be scaled based on whether they vote for good stories or bad ones. This should protect us against the arrival of users who vote up dumb stories. The worse stuff a user upvotes, the less effect their future votes will have. And vice versa: someone who consistently recommends interesting stories will be rewarded with a louder voice."
On the plus side, this is easy to implement and doesn't involve any UI changes or added complexity at all. It also guarantees a certain level of consistency for material on the site.On the minus side, this just switches the recommendation-consumption impedance mismatch from user-to-user to user-to-editor, that is, the system now will display things the editor likes, instead of things the individual user might like. That's great if you want to be an online magazine, but not if you want to be an online portal. (BTW, FaceBook beats the crowd here by providing new information each day about stuff your friends are doing. If you really like these people, the "lock is in" and presumably you will want to hear about them, thereby reducing the RCIM)
- Making smaller communities - as waleedka said:
It's not a voting problem, it's a scaling problem; I believe. Simply put: social news sites don't scale very well.
As communities grow:
- You and I can't get things on the home page any more. It becomes only for the well connected.
- Subjects lose focus.
- It becomes a gold mine for spammers. They'd do anything to game the system if they can get 234,000 visitors from one post
The solution, I believe: Keep communities smaller and focused. If need be, create many small communitiesOn the plus side, this keeps the community smaller, which (I think) will lead to more self-enforcement. Smaller groups tend to be more "social". And isn't the problem that many "social" sites aren't really social at all?
On the minus side, I think joining, forming, and knowing which groups the user should be in are non-trivial. People have "favorite" topics, but it's a loose and free-flowing thing, not a rigid structure. Computers are really good at rigid definitions, while people may be amenable to all kinds of fuzziness. Perhaps my group is better defined as "people who like science, astronomy, politics, and startups" -- but that wouldn't be the topics of the articles! The topics of the articles I like could be anything. And over time, I might add or subtract the stuff I like, even without being consciously aware of it. Heck, I'm not even sure I'm able to give you a precise list of topics I'm interested in, much less self-select against a preconceived ontology.
- Get rid of explicit information altogether - As Udi points out on his blog:
The massively important, and often overlooked, thing about implicit metadata is that it’s generally trustworthy. It’s like the results of a double-blind scientific study. Explicit metadata on the other hand, while often useful, is always in doubt. It’s like the results of an exit poll during an election. People lie. People are stupid. People are remarkably un-self-aware. Going the explicit route exposes you to all of these problems. It’s icky.
On the pro side, this has a long track record of actually working.
On the minus side, there is still an impedance mismatch between structure and quality to the user, but this mismatch is reduced by making the bar "taking the time to make a link". Sure, the system can still be gamed, just ask Google. But it's tougher to do. I'm just not sure that the "authoritative" quality that Google is pulling out from the links, or the "good photographer" quality Flickr is using, is what I want. But maybe I'm too picky. Are we looking for good editors or good content?
When I communicate with you in a certain language, there are all
kinds of implicit information in my content -- information that's
just part of the medium, not the message.
Likewise, HTML and various Web 2.0 sites have implicit
knowledge about the content that appears on them
Is there a fourth option? Or is one of these options enough?
this already has a name; misaligned incentives.
Are you sure about that? There's no Wiki page for "misaligned incentives", therefore it must not exist.
And I'd like to point out that "misaligned incentives" isn't very catchy. You could take my moniker and make CRIM out of it. Or RIMC.
So you could have the old clunky phrase, or you could have the new, shiny, Web 2.0 phrase. BTW, misaligned incentives would apply to the ORIM as well, but that's not what they used. Actually "misaligned incentives" can apply to all kinds of stuff, which makes it much less cool than "Recommendation-Consumption Impedance Mismatch."
You say to-mae-to, I'll say to-mah-to.
Im sure, google it . That it is and old phrase that applies to all sort of other problems and not just recommendation systems is a feature not a bug: it means we can use the same tool (game theory) to think about all these problems and advances in one often shed light on others.
Thanks.
Of course I was aware it was a concept, however I am concerned that it is an ancilliary concept and not a prime area of research, hence the lack of the wiki page. It just sounds too generic -- I know that's not much of a criticism. Sorry. Here I came up with a cool name (modeled after another well-known concept) and you go and pop my balloon. May a thousand camels spit into your nose! (grin)
But I'm with you on using it as a feature. Whatever information is out there should be correlated and integrated into whatever the solutions are. In a way, not only is this game theory, but it's also economics, with various actors involved in the production and distribution and consumption of goods and services: content, links, ratings, and comments. And we can't be sure the actors are always acting rationally either. I'd add that quite a bit of psychology is also at work.
I would like to make the case further for "impedance mismatch" if you'll permit me. I do not think it is only the incentivization that is the problem. It's that the 4 players (site, content creators, content promoters, and users) all are evolving in different directions. Content consumers want easier content quicker and without distractions. Sites want revenue. Content creators want all kinds of things. Content promoters have a lot of goals as well. Perhaps misaligned incentives cover a multi-dimenstional mismatch among several parties where all of the systems are evolving. I'd defer to you for the answer to that question.
Great comments!