No “One-Size Fits all”, Towards a principled approach for incentives in mobile crowdsourcing

Motivation:

Despite the rich potential of mobile croudsourcing services, their effectiveness depends on the whims of their participants., impacting everything from user engagement to their compliance with the crowdsource tasks. In response of this, a lot of systems have started to incorporate different incentive features and want to enhancing the quality of the collected data. This incentive method can improve the performance of their applications.

Main points:

In this paper, there is a large scale experiment, which runs over 2 days with 96 participants. In this experiment, the author comes up with 2 incentive structure. He try to compare these two different structure and find some relative effect between them. These two different structures include significant different behavior of participants.

In this paper, the author identify three key performance metrics: recruitment, compliance and user-effort. The result of the experiment is showing according to these three aspects.

Recruitment: (the number of users who choose to participate in this application)

weighted lottery has achieved greater total recruitment as well as larger active recruitment than micro-payment. However, the author also points out that this result could not explain the whole story of the incentive selection.

Compliance:

In this paper, the author measured the total number of tasks completed by a participant. The result is that there is no correlation between booth popularity and task compliance.

User-effort:

this refer to the amount of the interaction spent by each user within the mobile crowdsourcing system. There are 3 aspects to measure the user-effort. One is the check-out time, one is the active time and the other one is the time between consecutive tasks to look at the eveness of temporal coverage of participant. The result is that these participants of the micro-payment spent more time traversing the conference area than those of the weighted lottery.

The author shows us that both incentive mechanisms provide sufficient motivation for participants to visit the booths with low popularity.

Trade-off:

1. In this paper, it only mentions 3 performance metrics. However, as we know, there must has some more other factors which will definitely affect the performance. This will cause a problem of the experiment result.
2. When comes to the user-effort, the author only mentions 3 ways to compare. But besides these aspects, there will be some other factors which will lead to a different performance of the users. For example, when the one user has something wrong with the phone, so the check-out time is not correct. So when doing this experiment, consider some special cases is needed. Otherwise, the result may not be correct.
3. The author points out that the recruitment could not explain the incentive selection and try to combine other metrics. However, when combine these metrics, there will be a probability that each aspect will affect others.