« Multiple Choice Quiz: a) be | Main | idea database »

Saturday, 23 August 2003

Comments

I was actually going to point you towards Advogato and it's trust metric (http://www.advogato.org/trust-metric.html) but a little time on Google revealed you'd mentioned it last year (http://erikbenson.com/index.cgi?node=web%20of%20trust).

So instead take a look at the following discussions:
http://www.advogato.org/article/554.html
http://www.advogato.org/person/danf/diary.html?start=2
http://sccm.stanford.edu/pub/sccm/sccm02-16.pdf
http://www.advogato.org/person/ade/diary.html?start=6

You should also consider that our brains tend to calculate trust with ourselves as the seed. Everybody else's trustworthiness then gets calculated by traversing the graph from ourselves to them.

The non tech point
Less feedback on implementation and more on what a reputation system needs to work. Google's pagerank works because it requires no participation from users. They just click around and Google uses this data to create meaningful search results. Amazon does the same thing with personalization. They just watch what people do and then look for patterns. What you're describing involves friction. People have to pysically do something to tell you something about a person. This introduces all sorts of bias in your reputation rank. Of course you could say that a reputation is subjective anyway. But still, the big question for success to me would be: 1) why would people us it? and 2) would it be friction less enough for people to use regularily so that you'll get enough data for a meaningful rank? Quick thoughts :-).

Yeah, I completely agree. I was using the software metaphor more as a way to try to imagine what the ideal system would be. In order to build a truly compelling website that would encourage usage of it and eventually be a success, however, a lot of other factors would need to be taken into consideration.

One interesting point here, though, is that I think a lot of this can be approximated in HTML, as Google has done with PageRank. Also, there are a lot of pseudo-reputation ranks out there already... all of them specific to the site which they're providing reputation for. Amazon has one, eBay has one, slashdot has one, etc. People seem to know what reputation is and are willing to invest themselves in preserving their reputation within these contexts. If someone could create a system that was then adopted by these specific websites, and the reputation was independant from the site, I think that would be one way to bootstrap usage of the system. It might require that someone build the reputation system and create one compelling usage for it... while also offering it up for others to build off of and contribute to. I haven't given it much more thought than that though.

Perhaps a first version could some how come up with a reputation score for folks by summing pseudo-reputation ranks from Amazon, Ebay and the like into some overall score. Call it their Internet Rank or iRank (or maybe iRep). This might be your baseline rank and the rest of the stuff you discuss could be optional ways of building a more complete score for users that care to "play." I could see this sort of concept interesting to sites like Amazon/Ebay because they get easy access to competitor data that could add legitimate value to their sites. Just an idea.

cory doctorow wrote about a system very similar, "whuffie", in his second newest book, down and out in the magic kindom

Reality Check
If the end goal of a reputation system is not just to establish a personalized trust metric for people but for statements as well (as you describe) then I think it is useful to differentiate between object, verifiable statements and statements of opinion or taste. The former falls more squarely in the purvue of a reputation system, while the latter is more like collaborative filtering.

Interestingly, if you subscribe to the viewpoint that most opinions about the world (say, political opinions) are determined as much or more by the belief in a given subset of the underlying facts surrounding it as by ones values, then a reputation system relating to belief in objective facts can serve to strengthen the confidence of collaborative filtering predictions of opinion.

But be careful not to conflate subjective statements and objective facts as you seem to have done, or else you sound like one of those wishy-washy extreme subjectivist. :)

I am a wishy-washy extreme subjectivist (if I understand the label correctly). I don't believe there is a concrete difference (practically, maybe if you had a real behind-the-scenes knowledge of statements) between subjective statements and objective facts. In other words, I can say "my cat is green" and that might be an opinion or a statement of truth, but you have no way of knowing other than by relying on your level of trust for me.

Good point. Okay then, to ground the reputation rankings, instead of the Page Rank-like snapshot, how about a list of "statements of fact" that you believe to be true? Collaborative filtering could then be used to compare your ground with others, and another metric can relate these statements of fact with correlated statements of opinion. Back when people actually had iterated interactions with one another in a close-knit community environment, reputation was definetely a "behind-the-scenes" verifiable attribute. This was because you could actually witness the results of their handiwork, after-the-fact. Without something akin to this eye-witness accounting, a reputation system would simply magnify people's ignorance and bias along with everything else.

I guess I have always thought some sort of objective grounding was necessary because of my observation that people are easily misled, in that they have a tendency to be swayed by the personality, popularity, and charisma of the individual while glossing over the content of what that person says.

The comments to this entry are closed.