Strategies for Efficient Crowd Labor Management
Strategies outlined by Lance West, including hiring workers in advance, implementing on-demand crowds, and optimizing response times, focus on reducing latency, increasing reliability, and improving task completion speed in crowd labor management. The use of Puppeteer and Adrenaline tools further aids in narrowing down search space and achieving worker consensus efficiently.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Solve complex problems Crowd Latency >= 1 Minute Fastest workers start task 10 seconds after submission This is not real time. This is nearly realtime
Create On-demand, synchronous crowds Retainer Model Rapid Refinement
Hires crowd workers in advance and pays them a low stand-by rate until a task is ready. With this method, a majority of workers will begin work within 2 seconds. Over 75% in 3 seconds.
Puppeteer: First responses in 2.1 seconds A|B First 5 responses in 5-7 seconds
As the first responders begin to show a consensus, the search space is narrowed for later responders. This decreases latency, increases reliability, and allows the possibility of having an initial guess in the first couple seconds of task submission, followed by a verified answer a few seconds later.
When at least 33% of the workers have been in the same 25% of the timeline for at least 2 seconds, Adrenaline declares agreement. These values can be adjusted to trade off delay for false positives. With these values, Adrenaline converges in 3 4 phases per video. The first phase is the slowest, and agreement accelerates as workers attention is focused on one area.
Bernstein, M. S., Brandt, J., Miller, R. C., & Karger, D. R. (n.d.). Retrieved November 13, 2014, from http://hci.stanford.edu/publications/2011/realtime/adrenal ine-uist2011.pdf
