Word Order and Information Density in Language
Information Density and Word
Order
Why are some word orders more
common than others?
•
In the majority of languages (with dominant
word order) subjects precede objects
•
(SOV,SVO) > VSO > (VOS, OVS) > OSV
•
Genetically encoded bias?
•
Single common ancestor (SOV)?
•
General linguistic principles
–
Theme-first
–
Verb-object bodning
–
Animate-first
•
Great, but why do these principles work?
Why are some word orders more
common than others?
•
Constant information transmission rate
–
Slower for unexpected, high entropy content
–
Faster for predictable, low entropy content
•
The basic word order of a language influences
the average transmission rate
•
Thus languages that are closer to the UID ideal
will be more common compared to others
further away from it
Uniform information density hypothesis
Word-order model
•
Simple world with
–
13 objects (O)
•
5 people
•
8 food/drink items
–
2 relations (R)
•
eat/drink
•
Events in this world consist of one relation and
two objects
–
(o
1
, r, o
2
)
•
And appear with a certain probability
P
•
Base entropy (the initial state of the observer
before words are spoken)
•
After each word, observers adjust their
expectations for the following ones, reaching
an entropy of zero after the third word of the
event
Word-order model
•
Each event has an information profile
I
1
= H
0
− H
1
, I
2
= H
2
− H
1
, I
3
= H
2
•
Where
H
n
are entropy trajectories of each
word
•
UID suggests a straight line from base entropy
to zero entropy such that each word conveys
1/3 of the total information
Word-order model
Word-order model
•
UID deviation score
•
Deviation of toy-world events from the “ideal
information profile” according to UID
VSO > VOS > SVO > OVS > SOV > OSV
Corpus study
•
Child-directed speech (English and Japanese
corpora)
•
Utterances involving singly transitive verbs
•
Ignored adjectives, plurality, tense etc
•
English: VSO (0.38), SVO (0.41), VOS (0.48),
SOV (0.64), OSV (0.78), OVS (0.79)
•
Japanese:
SVO (0.66), VSO (0.71), SOV (0.72),
VOS (0.72), OSV (0.82), OVS (0.83)
Experiment
•
Languages must be optimal with respect to the
frequencies of events in the real world
•
Judgement tasks for pairs of sentences (which
one is more probable?)
•
VSO (0.17), SVO (0.18), VOS (0.20), SOV (0.23),
OVS (0.23), OVS (0.24).
Discussion
•
Object-first word orders are rare
•
Object-first word orders have least uniform
information density (first word carries too much
information)
•
SOV is not as compatible with the UID as it is
frequent in real languages – perhaps due to other
important factors beside UID
•
TFP and AFP favor SOV, SVO (highest ranked in
the results) and VSO – perhaps UID provides
some justification at least for some word order
rankings
Conclusion
•
Findings consistent with a
weaker
hypothesis
that word order is optimal wrt the frequency
speakers choose to discuss events (not wrt to
how often these events really occur)
•
UID may not provide explanation for all of the
word order rankings, but does explain several
aspects of the empirical distribution of word
orders
A Noisy Channel Account of
Crosslinguistic Word Order Variation
•
In 96.3% of studied languages S precede O
•
SVO (English) and SOV (Japanese) are more
prevalent than VSO
•
People construct sentences from and agent
perspective – why SVO/SOV then?
•
Innate universal grammar – independent of
communicative or performance factors
Why SOV/SVO
•
Communicative-based explanation
•
SOV default for the human language
–
Preference for S to precede O
–
Preference for the V to appear in the end of the
clause
•
SVO arises from SOV as a result of
communication/memory pressures that
sometimes outweigh the second preference
Shanon’s communication theory
•
Comprehension and production operate via a
noisy channel
•
Speakers are under constraints to chose
utterances that will ensure maximal meaning
recoverability by the listener
•
When does word order affect how easily meaning
can be recovered?
–
The girl kicks the ball.
(people should adhere to SOV)
–
The girl kicks the boy.
(potential confusion resolved perhaps by the position of
the noun wrt to the verb)
Method
•
Study investigates whether gestured word order
across languages (English-SVO, Japanese, Korean-
SOV) is depending on semantic reversibility of the
event
–
Initial bias to SOV
–
Initial bias to native language
–
Communicative or memory pressures
•
English
–
Shift to SVO (second and third factors)
•
Japanese&Korean
–
Shift to SVO (only due to the third factor)
Method
•
Brief silent animations of
intransitive/transitive events
–
First verbally described the animations
–
Then hand-gestured the meanings of the events
•
Verbal and gesture responses were coded for
the relative position of the agent, action, and
patient
Experiment 1
•
Animate/inanimate patients (reversible or
non-reversible sentences)
•
More SVO word orders should be produced if
reversible
•
Results – uniformly SVO for verbal responses
–
Gestured S before O for animate patients
–
Gestured V before O for human patients (as
expected)
–
Overwhelmingly gestured SOV for non-reversible
events
Experiment 1&2 – Japanese/Korean
•
English participants’ results can be explained
without resorting to noisy-channel hypothesis
–
Participants may shift from SOV to native (SVO)
due to increased ambiguity in reversible events
•
Thus, tested participants with a SOV native
language
–
Expected shift to SVO in reversible events
•
Experiment 2 – used more complex structures
The old woman says that the fireman kicks the girl
•
If participants use native word-order (SOV)
–
Then they should gesture both levels of
embedded events with the same order:
S
1
[S
2
O
2
V
2
] V
1
•
In case of reversible events SOV creates
maximal potential confusion
–
Then they should gesture using SVO:
S
1
V
1
[S
2
V
2
O
2
]
Experiment 1&2 – Japanese/Korean
•
Exp 1 results – native language word-order
–
J&K speakers verbalized patient before action (100%)
–
Gestured patient before action in both animate and
inanimate patients
•
Exp 2 results – shift to SVO
–
J speakers never verbalized SVO; K speakers rarely
–
Both J&K speakers almost always gestured top-level verb in
2
nd
position between the top-level subject and the
embedded subject
–
In the embedded clause patients were gestured before the
action almost always, but more often in non-reversible
events (both for J&K speakers)
•
Results predicted by noisy-channel but not by the
combination of SOV default and native-language order
Experiment 1&2 – Japanese/Korean
Experiment 3
•
Alternative explanation of previous results
–
Minimizing syntactic dependency distances
–
Number of words between a syntactic head (verb)
and its dependents (subject and object)
–
Shorter dependencies are easier
•
Shift from SOV to SVO given that SVO allows
for shorter dependency distances
Experiment 3 - method
•
Animations of a boy and a girl interacting with one of a set
of objects:
–
Circle/star/heart which was either
–
Spotted/striped (surface); in a box/pail (container);
wearing a top/witch’s hat (headwear)
–
Giving/putting/intransitive event
•
Participants were to gesture each event and the features of
the object
•
If sensitive to distance b/n agent and verb, then higher SVO
gesture order for longer patient descriptions
•
No such shift predicted by noisy channel – patient is not a
possible agent of the verb, adding modifiers will not affect
the recoverability of who is doing what to whom
•
Gestured patient before action for most of
events
•
Verbalized action before patient for most of
events
•
Even with long productions still gestured
patient before action, consistently with the
noisy-channel hypothesis and not with the
dependency-distance hypothesis
Experiment 3 - results
Discussion
•
English speakers have a strong SOV preference for non-
reversible events even when the inanimate patient has
up to 3 features to be gestured
•
SOV seems to be the preferred word order in human
communication
•
For reversible events the preference for SOV
disappears in favor of SVO
•
Although SOV-natives gesture SOV in simple events,
they revert to SVO for more complex ones
•
This shift to SVO occurs in order to maximize meaning
recoverability
Discussion
•
Case marking is often used in SOV
–
Mitigates the confusability of subject and object, helping to
retain the default SOV
•
If no case marking is used, then SVO shift
•
Large majority of SOV languages are case marked, whereas few of
SVO are
•
Used
location in space
as possible case marking in the experiments
–
Of the case-marked gestures most had SOV order
•
Animacy-dependent case marking
–
Many languages mark only animate direct objects
•
Non SVO languages have more word-order flexibility than SVO
–
Contain other mechanisms for disambiguation
–
So fixed word orders mostly SVO
Conclusion
•
No need for sophisticated innate machinery to
explain word-order variation
•
Many aspects of crosslinguistic word-order
variance are easily explained by
communicative or memory pressures
Explore the reasons behind the commonality of certain word orders in languages, from genetically encoded biases to linguistic principles. Discover the Uniform Information Density Hypothesis and its impact on language evolution, as well as insights from a Word-Order Model that examines entropy and information profiles in communication.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Why are some word orders more common than others? In the majority of languages (with dominant word order) subjects precede objects (SOV,SVO) > VSO > (VOS, OVS) > OSV
Why are some word orders more common than others? Genetically encoded bias? Single common ancestor (SOV)? General linguistic principles Theme-first Verb-object bodning Animate-first Great, but why do these principles work?
Uniform information density hypothesis Constant information transmission rate Slower for unexpected, high entropy content Faster for predictable, low entropy content The basic word order of a language influences the average transmission rate Thus languages that are closer to the UID ideal will be more common compared to others further away from it
Word-order model Simple world with 13 objects (O) 5 people 8 food/drink items 2 relations (R) eat/drink Events in this world consist of one relation and two objects (o1, r, o2) And appear with a certain probability P
Word-order model Base entropy (the initial state of the observer before words are spoken) After each word, observers adjust their expectations for the following ones, reaching an entropy of zero after the third word of the event
Word-order model Each event has an information profile I1= H0 H1, I2= H2 H1, I3= H2 Where Hnare entropy trajectories of each word UID suggests a straight line from base entropy to zero entropy such that each word conveys 1/3 of the total information
Word-order model UID deviation score Deviation of toy-world events from the ideal information profile according to UID VSO > VOS > SVO > OVS > SOV > OSV
Corpus study Child-directed speech (English and Japanese corpora) Utterances involving singly transitive verbs Ignored adjectives, plurality, tense etc English: VSO (0.38), SVO (0.41), VOS (0.48), SOV (0.64), OSV (0.78), OVS (0.79) Japanese: SVO (0.66), VSO (0.71), SOV (0.72), VOS (0.72), OSV (0.82), OVS (0.83)
Experiment Languages must be optimal with respect to the frequencies of events in the real world Judgement tasks for pairs of sentences (which one is more probable?) VSO (0.17), SVO (0.18), VOS (0.20), SOV (0.23), OVS (0.23), OVS (0.24).
Discussion Object-first word orders are rare Object-first word orders have least uniform information density (first word carries too much information) SOV is not as compatible with the UID as it is frequent in real languages perhaps due to other important factors beside UID TFP and AFP favor SOV, SVO (highest ranked in the results) and VSO perhaps UID provides some justification at least for some word order rankings
Conclusion Findings consistent with a weaker hypothesis that word order is optimal wrt the frequency speakers choose to discuss events (not wrt to how often these events really occur) UID may not provide explanation for all of the word order rankings, but does explain several aspects of the empirical distribution of word orders
A Noisy Channel Account of Crosslinguistic Word Order Variation In 96.3% of studied languages S precede O SVO (English) and SOV (Japanese) are more prevalent than VSO People construct sentences from and agent perspective why SVO/SOV then? Innate universal grammar independent of communicative or performance factors
Why SOV/SVO Communicative-based explanation SOV default for the human language Preference for S to precede O Preference for the V to appear in the end of the clause SVO arises from SOV as a result of communication/memory pressures that sometimes outweigh the second preference
Shanons communication theory Comprehension and production operate via a noisy channel Speakers are under constraints to chose utterances that will ensure maximal meaning recoverability by the listener When does word order affect how easily meaning can be recovered? The girl kicks the ball. (people should adhere to SOV) The girl kicks the boy. (potential confusion resolved perhaps by the position of the noun wrt to the verb)
Method Study investigates whether gestured word order across languages (English-SVO, Japanese, Korean- SOV) is depending on semantic reversibility of the event Initial bias to SOV Initial bias to native language Communicative or memory pressures English Shift to SVO (second and third factors) Japanese&Korean Shift to SVO (only due to the third factor)
Method Brief silent animations of intransitive/transitive events First verbally described the animations Then hand-gestured the meanings of the events Verbal and gesture responses were coded for the relative position of the agent, action, and patient
Experiment 1 Animate/inanimate patients (reversible or non-reversible sentences) More SVO word orders should be produced if reversible Results uniformly SVO for verbal responses Gestured S before O for animate patients Gestured V before O for human patients (as expected) Overwhelmingly gestured SOV for non-reversible events
Experiment 1&2 Japanese/Korean English participants results can be explained without resorting to noisy-channel hypothesis Participants may shift from SOV to native (SVO) due to increased ambiguity in reversible events Thus, tested participants with a SOV native language Expected shift to SVO in reversible events Experiment 2 used more complex structures The old woman says that the fireman kicks the girl
Experiment 1&2 Japanese/Korean If participants use native word-order (SOV) Then they should gesture both levels of embedded events with the same order: S1 [S2O2V2] V1 In case of reversible events SOV creates maximal potential confusion Then they should gesture using SVO: S1 V1 [S2V2O2]
Experiment 1&2 Japanese/Korean Exp 1 results native language word-order J&K speakers verbalized patient before action (100%) Gestured patient before action in both animate and inanimate patients Exp 2 results shift to SVO J speakers never verbalized SVO; K speakers rarely Both J&K speakers almost always gestured top-level verb in 2nd position between the top-level subject and the embedded subject In the embedded clause patients were gestured before the action almost always, but more often in non-reversible events (both for J&K speakers) Results predicted by noisy-channel but not by the combination of SOV default and native-language order
Experiment 3 Alternative explanation of previous results Minimizing syntactic dependency distances Number of words between a syntactic head (verb) and its dependents (subject and object) Shorter dependencies are easier Shift from SOV to SVO given that SVO allows for shorter dependency distances
Experiment 3 - method Animations of a boy and a girl interacting with one of a set of objects: Circle/star/heart which was either Spotted/striped (surface); in a box/pail (container); wearing a top/witch s hat (headwear) Giving/putting/intransitive event Participants were to gesture each event and the features of the object If sensitive to distance b/n agent and verb, then higher SVO gesture order for longer patient descriptions No such shift predicted by noisy channel patient is not a possible agent of the verb, adding modifiers will not affect the recoverability of who is doing what to whom
Experiment 3 - results Gestured patient before action for most of events Verbalized action before patient for most of events Even with long productions still gestured patient before action, consistently with the noisy-channel hypothesis and not with the dependency-distance hypothesis
Discussion English speakers have a strong SOV preference for non- reversible events even when the inanimate patient has up to 3 features to be gestured SOV seems to be the preferred word order in human communication For reversible events the preference for SOV disappears in favor of SVO Although SOV-natives gesture SOV in simple events, they revert to SVO for more complex ones This shift to SVO occurs in order to maximize meaning recoverability
Discussion Case marking is often used in SOV Mitigates the confusability of subject and object, helping to retain the default SOV If no case marking is used, then SVO shift Large majority of SOV languages are case marked, whereas few of SVO are Used location in space as possible case marking in the experiments Of the case-marked gestures most had SOV order Animacy-dependent case marking Many languages mark only animate direct objects Non SVO languages have more word-order flexibility than SVO Contain other mechanisms for disambiguation So fixed word orders mostly SVO
Conclusion No need for sophisticated innate machinery to explain word-order variation Many aspects of crosslinguistic word-order variance are easily explained by communicative or memory pressures
