Key Issues in Longitudinal Research Design: Insights from the Growing Up in Scotland Study by Paul Bradshaw
Key issues for longitudinal
research design:
Lessons from the Growing
Up in Scotland study
Paul Bradshaw
Definitions of longitudinal research
•
In the broadest sense longitudinal research involves
the follow up of any set of entities in which changes
can be observed over time.
•
Individuals
•
Households
•
Institutions such as hospitals or schools
•
Nations
•
Most commonly focussed on individuals
•
No recognised definition of what period or what
number of follow-ups constitutes
‘
longitudinal
’
•
Aim is usually to determine causes and processes
which lead to change and/or to particular outcomes
The UK experience
•
The UK has a long, and highly regarded, history of
(particularly quantitative) longitudinal social enquiry
•
A number of ongoing internationally renowned
longitudinal studies providing information on the life
histories of people as they move from birth to old age
•
Current QNLR being undertaken in Britain includes:
•
Birth cohort studies
•
Age cohort studies
•
Family/Household panel studies
•
Area studies
•
Census-based studies
The challenges of longitudinal
research
“
The success of a longitudinal study depends on stable
leadership from a committed principal investigator and a team of
highly skilled researchers
”
(Bynner et al, 2006)
Sample
Data collection
Duration
(& relevance)
Ethics
Cost
Data analysis
The Sample
Two main issues:
•
Sample design
•
Non-response and attrition
Sample design considerations
•
What is the population of interest?
•
All vulnerable children?
•
Sub-groups of interest? E.g. with particular characteristics –
age, family circumstances, area where they live,
interventions received
•
What do you want to be able to say about them?
•
Use sample to generalise to population?
•
Compare outcomes and experiences between children in
different groups?
•
How will you find/recruit them?
•
Is there a sampling frame with the info you need?
•
Will you need the help of an agency/organisation to recruit?
Sampling precision (1)
•
When using a sample, the data produces
‘
estimates
’
of what the
real value may be in the population. The precision of these
estimates is affected by:
•
Sample size
•
Sample clustering
•
Measuring sample precision:
•
Use
‘
confidence intervals
’
and
‘
standard errors
’
•
Confidence interval (CI)
•
Typically 95% meaning 95 times out of 100 this interval will capture the
true population value that we are trying to estimate
•
Expressed as e.g. 64% (+/- 6%) suggesting the true value is
somewhere between 58% and 70%
•
Differences will usually have to be of the magnitude of the CI for it to be
statistically significant – e.g. in example above, an increase or decrease
of around 6% would be necessary.
•
Smaller sample sizes produce larger CIs requiring greater change for
statistically significant differences to be detected
Sampling precision (2)
S
a
m
p
l
e
n
u
m
b
e
r
s
r
e
q
u
i
r
e
d
i
n
e
a
c
h
g
r
o
u
p
t
o
d
e
m
o
n
s
t
r
a
t
e
s
i
g
n
i
f
i
c
a
n
t
d
i
f
f
e
r
e
n
c
e
s
(
w
i
t
h
a
p
o
w
e
r
o
f
0
.
8
)
GUS response and attrition
rates
GUS: Sweep 5 response by age
of mother
N
o
n
-
c
o
n
t
a
c
t
a
n
d
r
e
f
u
s
a
l
r
a
t
e
s
Sample maintenance strategies
•
Keeping in touch
•
Tracing
•
Keeping respondents informed
•
Valuing and appreciating respondents (including, but
not restricted to, use of incentives)
•
Boosts of key sub-groups of interest and on-going
sample refreshment
•
Making reasonable demands
Sample maintenance: keeping in
touch
•
Potentially willing respondents can be lost by virtue of
moving home.
•
Important to establish effective procedures for
obtaining updated contact information.
•
Techniques include:
•
Collecting as much contact information as possible at first
contact including telephone/mobile numbers and e-mail
•
Information about
‘
stable contacts
’
– someone who knows
the respondent and would know where they had moved to
•
Regular mailings with return address – undelivered items act
as address checks ahead of fieldwork
•
Updates from administrative and service databases
Sample maintenance: keeping
respondents informed
•
In any research project, it is considered important to
provide good information to respondents about the
purpose and nature of research
•
Respondents must understand the longitudinal nature
of the study so that they also recognise why they are
repeatedly visited and why they are irreplaceable
•
Many studies adopt a
‘
branding
’
approach – a name
and logo that clearly identifies the study and which is
easily recognisable by respondents
•
Websites can be extremely useful in providing more
detailed information
Sample maintenance: valuing and
appreciating respondents
•
If respondents are going to continue to take part in a
longitudinal study they need to feel that their time and
effort is valued and is worthwhile
.
•
This can be demonstrated through:
•
The interviewer
•
Thank you letters sent after each interview
•
Providing evidence that survey findings have been used
•
Illustrating the impact they have had on government policy
•
Use of incentives
•
Non-financial rewards – gifts such as pens, fridge
magnets, calendars
Being part of NCDS
T
h
e
r
e
’
s
o
n
e
t
h
i
n
g
I
’
m
g
u
a
r
a
n
t
e
e
d
,
a
l
l
o
f
m
y
e
n
t
i
r
e
l
i
f
e
,
i
s
o
n
e
b
i
r
t
h
d
a
y
c
a
r
d
.
I
’
v
e
a
l
w
a
y
s
l
i
k
e
d
b
e
i
n
g
p
a
r
t
o
f
i
t
,
I
’
v
e
a
l
w
a
y
s
e
n
j
o
y
e
d
b
e
i
n
g
p
a
r
t
o
f
i
t
b
e
c
a
u
s
e
i
t
’
s
d
i
f
f
e
r
e
n
t
a
n
d
I
’
m
q
u
i
t
e
p
r
o
u
d
o
f
i
t
r
e
a
l
l
y
.
I think I could almost put this in as a landmark in my life.
I
h
a
v
e
n
’
t
k
i
d
d
e
d
o
n
a
b
o
u
t
a
n
y
t
h
i
n
g
i
n
m
y
l
i
f
e
,
w
a
r
t
s
a
n
d
a
l
l
,
I
’
v
e
b
e
e
n
h
o
n
e
s
t
a
n
d
I
’
v
e
s
a
i
d
i
t
a
l
l
.
….. I think, you know, it does make you feel special in certain
ways, like really…..
I
h
a
v
e
n
’
t
e
v
e
r
s
e
e
n
a
n
y
t
h
i
n
g
n
e
g
a
t
i
v
e
i
n
i
t
a
c
t
u
a
l
l
y
,
n
o
t
a
n
y
t
h
i
n
g
n
e
g
a
t
i
v
e
a
t
a
l
l
…
.
.
I
’
v
e
a
l
w
a
y
s
f
e
l
t
c
o
m
f
o
r
t
a
b
l
e
a
n
d
I
a
l
w
a
y
s
k
n
o
w
t
h
a
t
i
f
I
d
o
n
’
t
w
a
n
t
t
o
a
n
s
w
e
r
a
q
u
e
s
t
i
o
n
I
d
o
n
’
t
h
a
v
e
t
o
…
…
…
…
I
t
h
i
n
k
i
t
’
s
j
u
s
t
b
e
e
n
f
a
s
c
i
n
a
t
i
n
g
Recognising the importance of
NCDS
I
c
a
n
r
e
m
e
m
b
e
r
b
e
c
o
m
i
n
g
p
a
r
t
o
f
t
h
e
N
C
D
S
,
I
d
i
d
n
’
t
u
n
d
e
r
s
t
a
n
d
i
t
.
.
.
.
I
t
h
i
n
k
I
’
v
e
j
u
s
t
g
r
o
w
n
t
o
u
n
d
e
r
s
t
a
n
d
i
t
m
o
r
e
a
s
I
’
v
e
g
o
t
o
l
d
e
r
a
n
d
t
h
e
i
m
p
o
r
t
a
n
c
e
o
f
i
t
a
n
d
h
o
w
i
t
’
s
h
e
l
p
i
n
g
e
v
e
r
y
b
o
d
y
r
e
a
l
l
y
,
t
h
a
t
’
s
w
h
a
t
I
t
h
i
n
k
.
W
e
l
l
,
I
t
h
i
n
k
t
o
a
c
e
r
t
a
i
n
e
x
t
e
n
t
y
o
u
’
v
e
g
o
t
t
o
s
a
y
i
t
’
s
m
a
i
n
l
y
f
o
r
h
e
l
p
i
n
g
o
t
h
e
r
s
,
y
o
u
k
n
o
w
.
L
i
k
e
y
o
u
s
a
y
i
t
’
s
n
o
b
e
n
e
f
i
t
t
o
m
e
t
o
d
o
i
t
,
b
u
t
t
h
e
n
a
g
a
i
n
i
t
’
s
n
o
s
k
i
n
o
f
f
m
y
n
o
s
e
n
o
t
t
o
d
o
i
t
,
s
o
.
I
t
’
s
o
n
e
o
f
t
h
o
s
e
t
h
i
n
g
s
l
i
k
e
,
y
o
u
k
n
o
w
…
.
I
d
o
n
’
t
s
e
e
a
r
e
a
s
o
n
[
n
o
t
t
o
]
…
…
I
t
h
i
n
k
i
t
’
s
i
n
t
e
r
e
s
t
i
n
g
t
h
a
t
t
h
e
y
’
r
e
a
c
t
u
a
l
l
y
f
o
l
l
o
w
i
n
g
a
l
l
t
h
e
s
e
p
e
o
p
l
e
,
r
i
g
h
t
t
h
r
o
u
g
h
t
h
e
i
r
l
i
f
e
a
n
d
…
…
t
h
e
y
f
i
n
d
o
u
t
c
o
m
p
a
r
i
s
o
n
s
.
A
y
e
,
I
t
h
i
n
k
i
t
’
s
r
e
a
l
l
y
…
.
t
h
a
t
’
s
w
h
y
I
t
a
k
e
t
h
e
b
o
t
h
e
r
t
o
,
a
y
e
,
c
o
m
e
.
I
w
a
n
t
t
o
b
e
p
a
r
t
o
f
i
t
b
e
c
a
u
s
e
I
’
v
e
b
e
e
n
i
n
i
t
a
l
l
m
y
l
i
f
e
.
Data collection: what to ask and
when
•
Decisions on content of data collection require good
planning – important to ask the right questions at the
right time
•
Take a
‘
longitudinal
’
view:
–
What
‘
stage
’
are your subjects at?
–
What stage will they reach?
–
What situations, characteristics or contexts might you want to
compare between those two stages?
–
What might be significant at stage 1 when you are comparing
outcomes at later stages?
•
Decisions on content will usually be theory or
hypothesis-based
•
What if you miss something?
Data collection: intervals between
fieldwork
•
How often do you follow-up your sample? This is
dependent upon:
•
Respondent burden
•
Developmental stages, processes or transitions that you are
interested in
•
Budget
•
There is no recognised nor legitimate pattern – it
’
s
largely dependent upon the objectives and focus of
the research
Follow-up intervals of selected UK
child cohort studies
G
U
S
:
S
o
u
r
c
e
s
o
f
d
a
t
a
Data analysis
•
Benefits of longitudinal data
•
Measuring change over time:
–
Flows into and out of
‘
states
’
(e.g. poverty, unemployment,
being looked after)
–
The effects of change, or of state durations, on outcomes
–
Impact of interventions
–
‘
Individual
’
development
•
Temporal ordering of events
•
Improved control for omitted explanatory variables
•
Improved control for the effects of previous states
•
Exploring the effects of ageing and cohort membership
Data analysis
•
Drawbacks of longitudinal data
•
Data management
–
Complex structure/relations
–
Complex variable/samples
–
Resultant file and variable management requires training and
skills of good practice
•
Software issues
–
Complexity of methods
–
Some methods only available via specific software packages
Longitudinal models
•
Two main modelling approaches in social science
research:
•
Event history analysis, time to an event
(Also known as: duration analysis; survival analysis; failure time;
duration economics; hazard modelling)
•
Panel data analysis
–
Regression models suitable for repeated observations
–
Time generally conceptualised as being discrete
–
Extension of standard regression models
–
Closely related to multilevel modelling
–
(Simple methods can also be used)
Ethical issues (1)
•
Confidentiality, data security and data access
•
Safeguarding confidentiality through
–
Restricting the data that are released
–
Controlling the arrangements under which potentially disclosive
data is released
•
Data access/release
–
Providing data for researchers to analyse on their desktop with
the level of sensitive detail restricted
–
Providing more sensitive data under a special licence
–
Remote access where raw data is never released and analyses
are run in house on behalf of researchers who receive edited
outputs
–
Safe settings where researchers are required to visit a
protected site where access to the data is carefully managed.
Ethical issues (2)
•
Informed consent
•
What is
‘
informed consent
’
?
–
Respondents must understand that the project is longitudinal in
nature and what that means for them
–
They are free to withdraw at any time from the project as a
whole or any aspect of it
•
One-off or repeated consent?
–
No universal practice, but
–
Some form of repeated consent would normally be involved
Summary
•
Longitudinal research in any methodological
discipline presents a set of core challenges
•
The key issue is to consider, in detail, how the
fundamental temporal nature of the project affects the
basic aspects of its design including:
•
The sample
•
Data collection
•
Data analysis
•
Ethics
•
Requires a more complex design, but a good design
will produce tremendously valuable data
Longitudinal research involves observing changes over time in various entities like individuals, households, and institutions. The UK has a rich history of conducting longitudinal studies, including birth cohort studies and family panel studies. Challenges in longitudinal research include stable leadership, data collection, analysis, sample considerations, and ethical issues. Key aspects to consider include sample design, non-response, attrition, and sampling precision.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Key issues for longitudinal research design: Lessons from the Growing Up in Scotland study Paul Bradshaw
Definitions of longitudinal research In the broadest sense longitudinal research involves the follow up of any set of entities in which changes can be observed over time. Individuals Households Institutions such as hospitals or schools Nations Most commonly focussed on individuals No recognised definition of what period or what number of follow-ups constitutes longitudinal Aim is usually to determine causes and processes which lead to change and/or to particular outcomes
The UK experience The UK has a long, and highly regarded, history of (particularly quantitative) longitudinal social enquiry A number of ongoing internationally renowned longitudinal studies providing information on the life histories of people as they move from birth to old age Current QNLR being undertaken in Britain includes: Birth cohort studies Age cohort studies Family/Household panel studies Area studies Census-based studies
The challenges of longitudinal research The success of a longitudinal study depends on stable leadership from a committed principal investigator and a team of highly skilled researchers (Bynner et al, 2006) Data collection Data analysis Sample Cost Ethics Duration (& relevance)
The Sample Two main issues: Sample design Non-response and attrition
Sample design considerations What is the population of interest? All vulnerable children? Sub-groups of interest? E.g. with particular characteristics age, family circumstances, area where they live, interventions received What do you want to be able to say about them? Use sample to generalise to population? Compare outcomes and experiences between children in different groups? How will you find/recruit them? Is there a sampling frame with the info you need? Will you need the help of an agency/organisation to recruit?
Sampling precision (1) When using a sample, the data produces estimates of what the real value may be in the population. The precision of these estimates is affected by: Sample size Sample clustering Measuring sample precision: Use confidence intervals and standard errors Confidence interval (CI) Typically 95% meaning 95 times out of 100 this interval will capture the true population value that we are trying to estimate Expressed as e.g. 64% (+/- 6%) suggesting the true value is somewhere between 58% and 70% Differences will usually have to be of the magnitude of the CI for it to be statistically significant e.g. in example above, an increase or decrease of around 6% would be necessary. Smaller sample sizes produce larger CIs requiring greater change for statistically significant differences to be detected
Sampling precision (2) Sample numbers required in each group to demonstrate significantdifferences (with a power of 0.8) Size of difference between groups (percentage points) 5 7 Lower of the two percentages: 10 10 15 687 374 200 101 1252 653 330 153 25 1566 797 389 170 50 1095 541 251 101 75 436 195 75 - 90
GUS response and attrition rates No. cases issued Birth cohort Sweep 1 6583 Sweep 2 5217 Sweep 3 4665 Sweep 4 4394 Child cohort Sweep 1 3605 Sweep 2 2859 Sweep 3 2599 Sweep 4 2460 No. cases achieved Response rate As % of sw1 achieved 5217 4512 4193 3994 79% 86% 90% 91% 100% 86% 80% 77% 2859 2500 2332 2200 79% 87% 90% 89% 100% 87% 82% 77%
GUS: Sweep 5 response by age of mother 95 90 % achieved 85 80 75 70 Under 25 25-29 Mother's age at child's birth 30-34 35-39 40+
Non-contact and refusal rates No. cases issued No. non- contact % non- contact No. refused % refused Birth cohort Sweep 2 Sweep 3 Sweep 4 Child cohort Sweep 2 Sweep 3 Sweep 4 5217 4665 4394 162 102 93 3% 2% 2% 316 268 203 6% 6% 5% 2859 2599 2460 100 51 51 3% 2% 2% 153 142 136 5% 5% 6%
Sample maintenance strategies Keeping in touch Tracing Keeping respondents informed Valuing and appreciating respondents (including, but not restricted to, use of incentives) Boosts of key sub-groups of interest and on-going sample refreshment Making reasonable demands
Sample maintenance: keeping in touch Potentially willing respondents can be lost by virtue of moving home. Important to establish effective procedures for obtaining updated contact information. Techniques include: Collecting as much contact information as possible at first contact including telephone/mobile numbers and e-mail Information about stable contacts someone who knows the respondent and would know where they had moved to Regular mailings with return address undelivered items act as address checks ahead of fieldwork Updates from administrative and service databases
Sample maintenance: keeping respondents informed In any research project, it is considered important to provide good information to respondents about the purpose and nature of research Respondents must understand the longitudinal nature of the study so that they also recognise why they are repeatedly visited and why they are irreplaceable Many studies adopt a branding approach a name and logo that clearly identifies the study and which is easily recognisable by respondents Websites can be extremely useful in providing more detailed information
Sample maintenance: valuing and appreciating respondents If respondents are going to continue to take part in a longitudinal study they need to feel that their time and effort is valued and is worthwhile. This can be demonstrated through: The interviewer Thank you letters sent after each interview Providing evidence that survey findings have been used Illustrating the impact they have had on government policy Use of incentives Non-financial rewards gifts such as pens, fridge magnets, calendars
Being part of NCDS There There s one thing I s one thing I m guaranteed, all of my entire life, is one m guaranteed, all of my entire life, is one birthday card. birthday card. I I ve always liked being part of it, I ve always liked being part of it, I ve always enjoyed being part of it because it of it because it s different and I s different and I m quite proud of it really. I think I could almost put this in as a landmark in my life. I think I could almost put this in as a landmark in my life. I haven I haven t kidded on about anything in my life, warts and all, t kidded on about anything in my life, warts and all, I I ve been honest and I ve been honest and I ve said it all. .. I think, you know, it does make you feel special in certain .. I think, you know, it does make you feel special in certain ways, like really .. ways, like really .. I haven I haven t ever seen anything negative in it actually, not t ever seen anything negative in it actually, not anything negative at all ..I anything negative at all ..I ve always felt comfortable and I ve always felt comfortable and I always know that if I don always know that if I don t want to answer a question I don t want to answer a question I don t t have to I think it have to I think it s just been fascinating ve always enjoyed being part m quite proud of it really. ve said it all. s just been fascinating
Recognising the importance of NCDS I can remember becoming part of the NCDS, I didn I can remember becoming part of the NCDS, I didn t understand it.... I think I it.... I think I ve just grown to understand it more as I ve just grown to understand it more as I ve got older and the importance of it and how it older and the importance of it and how it s helping everybody really, that really, that s what I think. s what I think. Well, I think to a certain extent you Well, I think to a certain extent you ve got to say it for helping others, you know. Like you say it for helping others, you know. Like you say it s no benefit to me to do it, but then again it to do it, but then again it s no skin off my nose not to do it, so. s no skin off my nose not to do it, so. It It s one of those things like, you know .I don s one of those things like, you know .I don t see a reason [not to] [not to] I think it I think it s interesting that they s interesting that they re actually following all these people, right through their life and they find out people, right through their life and they find out comparisons. Aye, I think it comparisons. Aye, I think it s really . that s really . that s why I take the bother to, aye, come. I want to be part of it because I bother to, aye, come. I want to be part of it because I ve been in it all my life. all my life. t understand ve got s helping everybody ve got to say it s mainly s no benefit to me s mainly t see a reason re actually following all these s why I take the ve been in it
Data collection: what to ask and when Decisions on content of data collection require good planning important to ask the right questions at the right time Take a longitudinal view: What stage are your subjects at? What stage will they reach? What situations, characteristics or contexts might you want to compare between those two stages? What might be significant at stage 1 when you are comparing outcomes at later stages? Decisions on content will usually be theory or hypothesis-based What if you miss something?
Data collection: intervals between fieldwork How often do you follow-up your sample? This is dependent upon: Respondent burden Developmental stages, processes or transitions that you are interested in Budget There is no recognised nor legitimate pattern it s largely dependent upon the objectives and focus of the research
Follow-up intervals of selected UK child cohort studies Child s age in years 4 5 Study 0 1 2 3 6 7 8 10 11 16 NCDS (1958) BCS (1970) MCS (2000) GUS BC (2005) GUS CC (2005) ALSPAC (1991)
GUS: Sources of data Sw1 (2005/6) Sw2 (2006/7) Sw3 (2007/8) Sw4 (2008/9) Sw5 (2009/10) BC only Main carer Sw6 (2010/11) BC only Main carer Main carer Main carer Main carer Main carer Partner Child height & weight Child height & weight Child height & weight Cognitive assessmts Health records Cognitive assessmts Health records Health records Health records Health records Health records School records School records
Data analysis Benefits of longitudinal data Measuring change over time: Flows into and out of states (e.g. poverty, unemployment, being looked after) The effects of change, or of state durations, on outcomes Impact of interventions Individual development Temporal ordering of events Improved control for omitted explanatory variables Improved control for the effects of previous states Exploring the effects of ageing and cohort membership
Data analysis Drawbacks of longitudinal data Data management Complex structure/relations Complex variable/samples Resultant file and variable management requires training and skills of good practice Software issues Complexity of methods Some methods only available via specific software packages
Longitudinal models Two main modelling approaches in social science research: Event history analysis, time to an event (Also known as: duration analysis; survival analysis; failure time; duration economics; hazard modelling) Panel data analysis Regression models suitable for repeated observations Time generally conceptualised as being discrete Extension of standard regression models Closely related to multilevel modelling (Simple methods can also be used)
Ethical issues (1) Confidentiality, data security and data access Safeguarding confidentiality through Restricting the data that are released Controlling the arrangements under which potentially disclosive data is released Data access/release Providing data for researchers to analyse on their desktop with the level of sensitive detail restricted Providing more sensitive data under a special licence Remote access where raw data is never released and analyses are run in house on behalf of researchers who receive edited outputs Safe settings where researchers are required to visit a protected site where access to the data is carefully managed.
Ethical issues (2) Informed consent What is informed consent ? Respondents must understand that the project is longitudinal in nature and what that means for them They are free to withdraw at any time from the project as a whole or any aspect of it One-off or repeated consent? No universal practice, but Some form of repeated consent would normally be involved
Summary Longitudinal research in any methodological discipline presents a set of core challenges The key issue is to consider, in detail, how the fundamental temporal nature of the project affects the basic aspects of its design including: The sample Data collection Data analysis Ethics Requires a more complex design, but a good design will produce tremendously valuable data
