Encouraging a Household Panel to Transition Online: Insights and Strategies

 
T
i
m
i
n
g
 
I
s
n
t
E
v
e
r
y
t
h
i
n
g
,
 
B
u
t
M
o
n
e
y
 
T
a
l
k
s
 
How to encourage a face-to-face
household panel to go online?
 
University of Essex, July 2013
 
G
e
r
r
y
 
N
i
c
o
l
a
a
s
C
a
r
l
 
C
u
l
l
i
n
a
n
e
 
1
 
Contents
 
Background
Design of Experiment
Results
Summary of Results and Conclusions
 
B
a
c
k
g
r
o
u
n
d
 
1.
 
3
 
Background
 
Switch to mixed mode data collection at
wave 7 of Understanding Society
Potential for cost savings
Potential for reducing attrition
 
 
4
 
Context
 
Longitudinal household panel
Face-to-face interviewing of all household members at
waves 1 to 6
Greatest potential for reducing data collection costs
when an interviewer does not have to visit the
household
Previous experiment mixing telephone & face-to-face:
Costs can be reduced
BUT response rates suffer
 
5
 
Innovation Panel – wave 5
 
Vehicle for methodological development & testing
About 2,500 individuals in 1,500 households
Main objective of IP5 =
Determine whether it is possible to reduce costs by
mixing web questionnaires and face-to-face
interviews without sacrificing data quality
Sequential mixed mode design starting with web
 
6
 
6
 
IP5 
Design
E
x
p
e
r
i
m
e
n
t
a
l
g
r
o
u
p
F
2
F
 
p
h
a
s
e
,
w
e
b
 
o
p
e
n
W
e
b
 
o
n
l
y
 
p
h
a
s
e
C
o
n
t
r
o
l
 
g
r
o
u
p
F
2
F
 
o
n
l
y
 
p
h
a
s
e
N
o
 
w
e
b
 
7
 
Research question
 
   
Is it possible to boost the proportion of 
whole
households completing web questionnaires by
Timing the arrival of the invitation to go online?
Offering a web bonus?
 
D
e
s
i
g
n
 
o
f
 
e
x
p
e
r
i
m
e
n
t
w
i
t
h
i
n
 
m
i
x
e
d
-
m
o
d
e
 
s
a
m
p
l
e
 
2.
 
9
 
Design of experiment within mixed-mode
sample
(1) Timing of invitation to go online
Random allocation of households to:
Letter (+ email) to arrive on Friday
Letter (+ email) to arrive on Monday
Reminder letters (+ emails) sent 2 and 4 days later
(2) Web bonus
Random allocation of households to:
No web bonus
£5 per household member conditional on
   all completing online questionnaire
 
10
 
Design of experiment within mixed-mode
sample
2
7
0
h
o
u
s
e
h
o
l
d
s
2
6
5
h
o
u
s
e
h
o
l
d
s
2
6
6
h
o
u
s
e
h
o
l
d
s
2
7
6
h
o
u
s
e
h
o
l
d
s
N
o
 
w
e
b
 
b
o
n
u
s
C
o
n
d
.
 
£
5
 
p
e
r
h
h
l
d
 
m
e
m
b
e
r
I
n
v
i
t
e
 
a
r
r
i
v
a
l
o
n
 
F
r
i
d
a
y
I
n
v
i
t
e
 
a
r
r
i
v
a
l
o
n
 
M
o
n
d
a
y
 
R
e
s
u
l
t
s
 
3.
 
12
 
Experimental effects on Web
Response
 
Base: IP5 WEB Sample (n=1077)
 
13
 
Experimental effects on Web
Response
 
Base: IP5 WEB Sample (n=1077)
 
14
 
Effect of Bonus by Sample Type
 
Base: IP5 WEB Sample (n=1077)
 
15
 
Effect of Bonus by Upfront
Incentives
 
Base: IP5 WEB Sample (n=1077)
 
16
 
16
 
Unconditional Incentives
O
r
i
g
i
n
a
l
 
S
a
m
p
l
e
R
e
f
r
e
s
h
m
e
n
t
 
S
a
m
p
l
e
£
5
£
1
0
£
1
0
£
3
0
£
2
0
 
17
 
Effect of Bonus by Children in
Household
 
Base: IP5 WEB Sample (n=1077)
 
18
 
Effect of Bonus by Advance
Mailing
 
Base: IP5 WEB Sample (n=1077)
 
19
 
Effect of timing by Advance
Mailing
 
Base: IP5 WEB Sample (n=1077)
 
20
 
Experimental effects on Final
Response
 
Base: IP5 WEB Sample (n=1077)
 
7
5
.
6
 
7
2
.
8
 
 
21
 
Experimental effects on Final
Response
 
Base: IP5 WEB Sample (n=1077)
 
7
5
.
6
 
7
2
.
8
 
7
4
 
7
4
.
4
 
22
 
Estimation of cost savings
 
Web bonus of £5 compared to no web bonus:
Small reduction in travel and mileage costs
15% reduction in interviewer fees
Offsetting the cost of web bonuses reduces the
saving in interviewer fees to less than 5%
 
 
23
 
S
u
m
m
a
r
y
 
o
f
 
r
e
s
u
l
t
s
a
n
d
 
c
o
n
c
l
u
s
i
o
n
s
 
4.
 
24
 
Summary of results
 
Friday mailing had a small but diluted effect
Web bonus increased full household web response
Effect of bonus varied by a number of factors, e.g.
Diminishing returns with larger upfront incentives
Greater effect for households with children
Greater effect when household contacted by email
Web bonus did not increase final response rate
But potential for cost savings
 
 
 
25
 
Limitations
 
Small sample sizes
Confounding of sample type and value of
unconditional incentives sent with advance letter
Estimation of costs
 
 
26
 
Points for discussion
 
Timing to be looked at further?
Potential for greater cost savings through targeting
Large upfront incentives vs conditional web bonus- costs
 
 
 
 
If you want further information or
would like to contact the author,
C
a
r
l
 
C
u
l
l
i
n
a
n
e
T
.
 
0
2
0
 
7
5
4
9
 
7
1
5
8
E
.
 
c
a
r
l
.
c
u
l
l
i
n
a
n
e
@
n
a
t
c
e
n
.
a
c
.
u
k
V
i
s
i
t
 
u
s
 
o
n
l
i
n
e
,
 
n
a
t
c
e
n
.
a
c
.
u
k
 
T
h
a
n
k
 
y
o
u
Slide Note
Embed
Share

The study explores methods to incentivize a face-to-face household panel to participate online, focusing on cost savings and minimizing attrition rates. By mixing online and face-to-face data collection methods, the researchers aim to maintain data quality while reducing expenses. The experiment involves timing invitations and offering web bonuses to enhance household engagement in completing web questionnaires. The mixed-mode design seeks to find the optimal approach for transitioning the panel online effectively.


Uploaded on Sep 10, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. Timing Isnt Everything, But Money Talks How to encourage a face-to-face household panel to go online? Gerry Nicolaas Carl Cullinane University of Essex, July 2013

  2. Contents Background Design of Experiment Results Summary of Results and Conclusions 1

  3. Background

  4. Background Switch to mixed mode data collection at wave 7 of Understanding Society Potential for cost savings Potential for reducing attrition 3

  5. Context Longitudinal household panel Face-to-face interviewing of all household members at waves 1 to 6 Greatest potential for reducing data collection costs when an interviewer does not have to visit the household Previous experiment mixing telephone & face-to-face: Costs can be reduced BUT response rates suffer 4

  6. Innovation Panel wave 5 Vehicle for methodological development & testing About 2,500 individuals in 1,500 households Main objective of IP5 = Determine whether it is possible to reduce costs by mixing web questionnaires and face-to-face interviews without sacrificing data quality Sequential mixed mode design starting with web 5

  7. IP5 Design Experimental group Web only phase F2F phase, web open Control group No web F2F only phase 6 6

  8. Research question Is it possible to boost the proportion of whole households completing web questionnaires by Timing the arrival of the invitation to go online? Offering a web bonus? 7

  9. Design of experiment within mixed-mode sample

  10. Design of experiment within mixed-mode sample (1) Timing of invitation to go online Random allocation of households to: Letter (+ email) to arrive on Friday Letter (+ email) to arrive on Monday Reminder letters (+ emails) sent 2 and 4 days later (2) Web bonus Random allocation of households to: No web bonus 5 per household member conditional on all completing online questionnaire 9

  11. Design of experiment within mixed-mode sample Invite arrival on Friday Invite arrival on Monday No web bonus 270 265 households households Cond. 5 per hhld member 266 276 households households 10

  12. Results

  13. Experimental effects on Web Response Full Household Web Response (%) 100 80 60 40 23.7 21.4 20 0 Monday Friday Base: IP5 WEB Sample (n=1077) 12

  14. Experimental effects on Web Response Full Household Web Response (%) 100 80 60 40 24.7 23.7 21.4 20.4 20 0 Monday Friday No Bonus Bonus Base: IP5 WEB Sample (n=1077) 13

  15. Effect of Bonus by Sample Type No bonus Bonus 100 80 60 34 33 40 30.5 18 20 9.4 8.6 0 W4 Responding W4 Non responding Refreshment Base: IP5 WEB Sample (n=1077) 14

  16. Effect of Bonus by Upfront Incentives No bonus Bonus 100 80 60 40.2 37.4 40 22.6 18.8 17.9 13.5 20 0 5 10 20+ Base: IP5 WEB Sample (n=1077) 15

  17. Unconditional Incentives Original Sample 5 10 Refreshment Sample 10 20 30 16 16

  18. Effect of Bonus by Children in Household No bonus Bonus 100 80 60 35.4 33.3 40 22.8 22.8 20.6 18.2 16.4 20 3.7 0 None One Two Three + Base: IP5 WEB Sample (n=1077) 17

  19. Effect of Bonus by Advance Mailing No bonus Bonus 100 80 60 43.8 36.3 40 20 6.5 5.1 0 Postal Valid email Base: IP5 WEB Sample (n=1077) 18

  20. Effect of timing by Advance Mailing Monday Friday 100 80 60 41.9 38.3 40 20 6.8 4.8 0 Postal Valid email Base: IP5 WEB Sample (n=1077) 19

  21. Experimental effects on Final Response Full household Partial household 100 75.6 72.8 80 23.7 18.1 60 40 54.7 51.9 20 0 Monday Friday Base: IP5 WEB Sample (n=1077) 20

  22. Experimental effects on Final Response Full household Partial household 100 75.6 74 74.4 72.8 80 23.7 18.1 20.4 21.4 60 40 54.7 53.6 53 51.9 20 0 Monday Friday No Bonus Bonus Base: IP5 WEB Sample (n=1077) 21

  23. Estimation of cost savings Web bonus of 5 compared to no web bonus: Small reduction in travel and mileage costs 15% reduction in interviewer fees Offsetting the cost of web bonuses reduces the saving in interviewer fees to less than 5% 22

  24. Summary of results and conclusions 23

  25. Summary of results Friday mailing had a small but diluted effect Web bonus increased full household web response Effect of bonus varied by a number of factors, e.g. Diminishing returns with larger upfront incentives Greater effect for households with children Greater effect when household contacted by email Web bonus did not increase final response rate But potential for cost savings 24

  26. Limitations Small sample sizes Confounding of sample type and value of unconditional incentives sent with advance letter Estimation of costs 25

  27. Points for discussion Timing to be looked at further? Potential for greater cost savings through targeting Large upfront incentives vs conditional web bonus- costs 26

  28. Thank you If you want further information or would like to contact the author, Carl Cullinane T. 020 7549 7158 E. carl.cullinane@natcen.ac.uk Visit us online, natcen.ac.uk

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#