Paired Comparisons Testing Dimensions in Cultural Domains
Paired Comparisons
Testing Dimensions
Studying attributes of items in a
cultural domain
Free listing is a way to define the contents of a
domain.
Pile sorts and triad tests are ways to measure
similarity.
B
ut we want to do more. We want to describe
the attributes of the relations among the
elements of a domain.
To evaluate the items on an attribute, we can use
several techniques: ratings, rankings, and paired
comparisons.
Rating scales vs. paired
comparisons
Ratings are the most common way to measure attributes in
the social sciences
How dangerous is [this illness]:
1
2
3
4
5
This produces many tied scores
How many points should we allow?
An even number or an odd number of choices?
In some research, you don
’
t want to allow people to equivocate
Scales are often the best we can do for measuring attributes,
but for short lists of items, rank-ordered data is better.
Paired comparisons
Paired comparisons produce perfect rank-ordered lists.
There are N(N-1)/2 pairs of items, so there are 6 pairs in 4
items:
AB
AC
AD
BC
BD
CD
Each element occurs N – 1 times and each element is
paired with each other element.
You can count how many times each element “wins” in a
paired comparison of items.
How to do PC’s in Anthopac
Produce a person-by-pair matrix.
Convert this matrix into a person-by-item
matrix where cells indicate the number of
times, for each person, each item “won” in its
contest.
This matrix can be summed by columns to
produce a vector, indicating, for each item, over
all informants, how many times the item won.
Anthropac does all this when it imports paired
comparison data.
The method of paired comparisons is an
alternative way to get rank orderings of a list of
items in a domain. There are
n
(
n
–1)/2
pairs of
things in a list of things.
Suppose 5 colors: red, green, yellow, blue, and
brown.
The next slide shows the paired comparison
test to find an informant’s rank-ordered
preference for these five colors.
In each of the following pairs of colors, please
circle the one you like best:
RED
GREEN
RED
YELLOW
RED
BLUE
RED
BROWN
GREEN
YELLOW
GREEN
BLUE
GREEN
BROWN
YELLOW
BLUE
YELLOW
BROWN
BLUE
BROWN
Order effects
The pairs of colors are listed in such a way that you
can see how the 10 of them exhausts the
possibilities for five items.
For a real test, scramble the order of the pairs to
ensure that the order of the items in a list doesn’t
influence the choices that informants make.
Use
Anthropac
to do this.
Some more examples of paired
comparisons
Choose the animal in each pair that is more
[vicious, exotic, expensive]
Choose the illness in each pair that is more
[scary, hard to treat, life threatening]
Choose the food in each pair that is [better
for you, harder to find, best for children]
Choose the crime in pair that [you’re most
afraid of, deserves more punishment, harder
to stop]
How Anthropac calculates the rank
order in paired comparisons
For each informant, count up how many times each
item in a list ‘‘wins’’—that is, how many times it
was circled.
For a list of illnesses you expect cancer to win over
most other illnesses.
It gets interesting when compare the average rank
ordering across ethnic groups of, say, high blood
pressure and diabetes.
Plusses and minuses of paired
comparisons
Plus: People make one judgment at a time --
easier than rank ordering a list of items by
staring at all the items at once.
Plus: You can read a list of pairs to
nonliterate informants.
Minus: With 20 items, informants make
separate 190 judgments.
With 60 pile-sort cards, informants make 1770
judgments – but it’s all in their heads.
Studying attributes in cultural domains involves various methods like free listing, pile sorts, and triad tests. Rating scales and paired comparisons are used to evaluate items based on attributes. Paired comparisons produce rank-ordered lists and can be done effectively with tools like Anthopac, enhancing understanding of relations among elements in a domain.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Paired Comparisons Testing Dimensions
Studying attributes of items in a cultural domain Free listing is a way to define the contents of a domain. Pile sorts and triad tests are ways to measure similarity. But we want to do more. We want to describe the attributes of the relations among the elements of a domain. To evaluate the items on an attribute, we can use several techniques: ratings, rankings, and paired comparisons.
Rating scales vs. paired comparisons Ratings are the most common way to measure attributes in the social sciences How dangerous is [this illness]: 1 2 3 4 5 This produces many tied scores How many points should we allow? An even number or an odd number of choices? In some research, you don t want to allow people to equivocate Scales are often the best we can do for measuring attributes, but for short lists of items, rank-ordered data is better.
Paired comparisons Paired comparisons produce perfect rank-ordered lists. There are N(N-1)/2 pairs of items, so there are 6 pairs in 4 items: AB AC AD BC BD CD Each element occurs N 1 times and each element is paired with each other element. You can count how many times each element wins in a paired comparison of items.
How to do PCs in Anthopac Produce a person-by-pair matrix. Convert this matrix into a person-by-item matrix where cells indicate the number of times, for each person, each item won in its contest. This matrix can be summed by columns to produce a vector, indicating, for each item, over all informants, how many times the item won. Anthropac does all this when it imports paired comparison data.
The method of paired comparisons is an alternative way to get rank orderings of a list of items in a domain. There are n(n 1)/2 pairs of things in a list of things. Suppose 5 colors: red, green, yellow, blue, and brown. The next slide shows the paired comparison test to find an informant s rank-ordered preference for these five colors.
In each of the following pairs of colors, please circle the one you like best: RED RED RED RED GREEN GREEN GREEN YELLOW YELLOW BLUE GREEN YELLOW BLUE BROWN YELLOW BLUE BROWN BLUE BROWN BROWN
Order effects The pairs of colors are listed in such a way that you can see how the 10 of them exhausts the possibilities for five items. For a real test, scramble the order of the pairs to ensure that the order of the items in a list doesn t influence the choices that informants make. Use Anthropac to do this.
Some more examples of paired comparisons Choose the animal in each pair that is more [vicious, exotic, expensive] Choose the illness in each pair that is more [scary, hard to treat, life threatening] Choose the food in each pair that is [better for you, harder to find, best for children] Choose the crime in pair that [you re most afraid of, deserves more punishment, harder to stop]
How Anthropac calculates the rank order in paired comparisons For each informant, count up how many times each item in a list wins that is, how many times it was circled. For a list of illnesses you expect cancer to win over most other illnesses. It gets interesting when compare the average rank ordering across ethnic groups of, say, high blood pressure and diabetes.
Plusses and minuses of paired comparisons Plus: People make one judgment at a time -- easier than rank ordering a list of items by staring at all the items at once. Plus: You can read a list of pairs to nonliterate informants. Minus: With 20 items, informants make separate 190 judgments. With 60 pile-sort cards, informants make 1770 judgments but it s all in their heads.
