Ethical Considerations in Recruiting Microworkers for Academic Studies

The Ethics of Recruiting Microworkers in Academic Studies Dr Andy Evans Chair, AREA Ethics Committee Acknowledgements: comments and additions from Jennifer Blaikie, Senior Research Ethics Administrator, the AREA Faculty Research Ethics Committee, along with colleagues in Psychology and the Leeds University Business School.

Citizen science/crowdsourcing data Generally regarded as win:win trade. Improved scale of data collection. Improved public understanding of science. Cost savings generally offset by engagement costs.

Microwork Payment of a small amount (a micropayment) for a discrete task: Fill in survey for a few cents. Translate materials for a dollar. Usually organised through a microwork broker.

Example Brokers Amazon s Mechanical Turk ( MTurk ) TaskRabbit Uber BitCoin

Issues Allows for access to a very large community (thousands; MTurk claims 500,000 from 190 countries). The population sampled is often more representative than general internet samples and certainly more representative than student-body samples (Paolacci et al., 2010; Buhrmester et al., 2011; Berinsky et al., 2011). Researchers can set criteria when registering with the microwork sites, e.g. to restrict participants to degree-holders over 30. But issues, both traditional in participant recruitment, and new.

Traditional concerns Payment generally regarded as ethically dubious because: 1) The Nuremberg Code states that participants should act under free power of choice, without the intervention of any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion . It is often assumed that payment encourages involvement by the poor in research that the rich would avoid. 2) Payment generally regarded as perverting the response towards something the participant feels the researcher expects; e.g. the Acquiescence Bias.

Continuum Non-named researchers paid as full members of staff Researchers drawn internally and contracted for work Researchers drawn from external organisations contracted for work Freelancers contracted for work Researchers drawn from community groups and paid a per-hour rate Researchers drawn from community groups and paid expenses Participants drawn from community groups and paid expenses Big data resources we have purchased from companies paying for individual data Big data resources we have purchased from service companies Participants drawn from community groups and not paid Big data resources we have been given Participants drawn from community groups who pay to be involved Students who pay for their own educational projects Participant-ness level Payment formality

Payment The Nuremberg Code centres on experiment where there is risk. We could argue that where there is no risk, there is no issue. Why shouldn t we pay the poor for their time the rich can afford to volunteer? We may regard payment as fairer than volunteering or pay-to-participate. Nevertheless, something about the poor being encouraged into research still leaves us queasy . Is it because we class this as coercion rather than offering an opportunity? Focus is: Our agency; Our responsibility. But We could argue that where there is no risk, there is no issue

Payment Generally microworkers are likely to do so many tasks that bias to perceived expectations is unlikely to impact. Evidence from testing suggests little difference with other sample techniques (Paolacci et al., 2010; Berinsky et al., 2012). Of course, tasks need designing well. Payment can be withheld for poor quality, so we rely on individual encouragement to give overall quality. While it is possible with most systems to set skills benchmarks, there is more complexity involved in withholding payment for unsatisfactory work.

Ethics Employment law International data transfers Global wage labour New issues largely play out in the field of neo-liberal vs socialist welfare economics

Employment Many people still complete microwork as a socially worthy activity, regarding the payment, especially payment in kind (e.g. vouchers), as a small bonus beyond this. It potentially involves payment of individuals not identified on grant applications or registered with University HR. Scott (2008), for the National Institute for Health Research, suggests that small gift money for participation less than a day is not regarded as payment for tax or benefit purposes. Nevertheless, longer participation needs care.

Global tax/benefit systems Broader social issue: Workers are contracted, and as such are usually (though not always) expected to engage with any local taxation and welfare systems if necessary themselves. It is unlikely most do so, and, indeed, payment is often in kind .

Global wage labour Many workers will complete micropayment work in addition to other work to supplement incomes in a manner that is time-flexible. The dominant demographic for Amazon in the USA is young white women (Berinsky et al., 2012), and it is not clear if this is as a replacement for work otherwise unavailable or to fit in around young families. TaskRabbit, which is set up more for local physical tasks, anecdotally draws from income supplementers, students, homecarers, and the retired in local communities (Hoshaw, 2011).

Global wage labour Techlist Mturk workers http://techlist.com/mturk/global-mturk-worker-map.php

Global wage labour The requester usually sets the price paid for the work. This is usually arbitrary. Costs, for example for surveys, compare very favourably (for the requester) with expense payments ( optimal MTurk payment of ~$0.50 per survey Berinsky et al., 2012). However, more often than not this is below minimum wages for countries in the developed world, undermining legislation and potentially driving down local wages. While some systems have limits on the countries they may provide payment to, many do not.

Global wage labour There is an increasing use of external microwork contracting as an employment model in overseas call centres (Tsaplin et al., 2013), centres essentially acting as brokers and infrastructure suppliers. Workers are offered little protection, there is little opportunity for organised wage action, medical cover, or health and safety checks. Workers can, after the fact, give institutions poor reviews if payment is withheld or conflicts are unresolved but that is largely the limit of their power.

Global wage labour In the developing world, micropayments could represent a significant income (Gawade et al., 2012), and, in a sense represent an, albeit problematic, adjustment to a global wage rate. Indeed, although there is a considerable level of apparent cause-washing of cheap and unprotected labour practices with statements that microwork alleviates poverty, at least one company (Samasource) has been set up with the aim of specifically helping in this area. In addition, as workers can refuse work for a set pay and this impacts on requester reputation, institutions may have to consider regulating price asked. For a discussion of pricing issues, see Ipeirotis and Horton (2011).

International data Data may be stored outside of the EU. With some systems surveys etc. are actually stored outside of the recruitment system, so these can be organised and regulated through a separate contract and within EU boundaries. However use of such systems complicates both user anonymity, data protection, data ownership and data storage times (we don t know who participants are, but an organisation outside of the EU knows, possibly permanently, they have been involved).

Support Overall, the technique has strong support, both internationally and locally, in the fields of academic Psychology and Business Studies, though not without contention. There is a movement to utilise these systems within the social sciences, especially within the States where some companies limit requesters (employers) to. Universities may need to consider whether they want to engage in collaborations with other institutions utilising the technique. While the movement towards such systems seems strong, it should be noted that there was a 2014 debate about refusing publications involving such systems within journal editorial circles in the US, and there have been debates about their use outside of academia (Harris, 2014; Caballero, 2015).

Leeds The University currently permits this approach, but with a watching brief for future developments. Researchers are asked to think through the use of such services and, in particular, the data protection elements of the application and the ethical policies of the services involved.

Recommendations The storage of data: that it is appropriate and that arrangements are in place to ensure against the unanonymisation of data or its inappropriate reuse. Ensuring that the personal data of citizens within the EU is not exported illegally. The worker/company relationships of the organisations organising the microwork; for example, whether workers are under coercion to take on work, and whether microwork is being used to implement zero hour contracts where alternatives would be possible. The legal relationships entailed by the contract for microwork. The level of appropriate payment, with due regard to local and community standards.

Recommendations Researchers in areas where such techniques are common are encouraged to taking the lead on community discussion of these issues, for example, through conference panels. Where appropriate, support Microworker collective action through nascent unions such as Dynamo (see also MTurkGrind). http://www.wearedynamo.org/ http://www.mturkgrind.com/

Questions See notes below this slide for a broad picture of the post-seminar points raised.