Truth with trust and reputation

Lately, I have been developing a model for some project at my institute. The idea was the crowdsource water point quality assessment in some country in Africa. So users would vote on water quality using SMS. Very basic technology but could make the life of the people that much better by directing them to drinkable water points across vast plains where water and resources are scarce.

 

I have checked the blog post on the meeting results of the swift river team. The whole approach is indeed very good. I think an addition to this approach would be the integration of trust and reputation models into the architecture of the crowdsourced filter. I will explain briefly some working of the model without going into much details here.

 

The model I’m working on is time sensitive which makes it distinct from many other trust and reputation models out there (I will explain what this means later). Also, the model rewards good contributors by a slow build up of user reputations and a fast decline of reputations for those who commit mistakes often. This allows the model to be more resilient to attacks compared to other standard models. 

 

The main idea is that as users make information contributions about some event which are proved to be valuable to the community, the reputation of the user will grow. This is some form of a rewarding system. Research shows that a major factor of encouraging users to contribute in crowdsourcing is to acknowledge the people’s contributions and provide some sort of reward that does not have to be material, but in the majority of cases the rewards are in the form of gratitude. The reputation system rewards users while giving them the opportunity to build AND maintain their reputations in their community. Their reputations are then used to filter the information contributions they make in the future, making it possible to develop a crowd sourced filter that does not on the longer run require much intervention from information assessors as opposed to information contributors. This is one aspect I think is lacking in the current Swift River approach. The system is always dependant on having a crowd responsible for assessing information, while the system I propose is self learning and self sustaining which makes for a good compliment to the current Swift River filter architecture.

 

The diagram below shows the performance of the model for rewarding good contributions. It basically shows how the reputation of the users changes after good contributions to the system. The higher the reputation of some users the higher the veracity of future contributions from these users. It is safe to make that claim, since trusted media sources (blogs, tweets, etc of people) are simply assumed to consistently provide trusted information.

 

 

           

 

On the other hand fraudulent or lousy folks will be subject to the behavior in the diagram below:

 

 

 

 

I

I

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Information triage using a reputation and trust based system is proven to be robust, and if the model is resilient to attacks then the information filtered using the model is certainly more trusted.

 

another interesting feature of the model is that the reports made by the users (say a tweet for e.g.) are given trust ratings. Such trust ratings are subject to time decay themselves. So that a report that is 5 hours old loses trustworthiness as time passes in favor of newer more trust worthy reports. Such behavior, of course, can be turned on or off. But it comes in handy in cases of high volume high intensity information situations where the information value decays over time. This is the case with the water point assessment because in this African country, a water point that is good today might no be good tomorrow. The diagram below shows how the model deals with this problem to provide final status values for water points over time (or any other information entity for that matter).

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The model, of course, can and will be extended to include spatial aspects of the information, such that proximity of the source to the events can be taken into account as well. I have also developed a separate model which just deals with the spatial aspect, which is a simple model, but the foundation of it remain unchanged, and the model can certainly be further improved.

 

I think integration of a time sensitive trust and reputation model in the Swift River architecture can lead to a more self sustaining system and more automated filtering as the system learns more about its users.

 

any comments, ideas?