Exploring Discretized Interpretation of Continuous Prompts

 
On Discretized Interpretation of
Continuous Prompts
 
Joint work w/ Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sameer Singh,
Sean Welleck,  Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, Yejin Choi
 
Allen Institute for AI     University of Washington    University of California-Irvine
 
1
LM
pre-trained
language models (LM)
2
[Peters et al. ’18 , Radford et al. ’19, Brown et al. ’20, …. ]
 
3
LM
 
Language prompt
 
[Peters et al. ’18 , Radford et al. ’19, Brown et al. ’20, …. ]
4
LM
In today's presentation at
Allen Institute for AI,
 
 
w
e
 
s
h
a
r
e
d
 
s
o
m
e
 
o
f
 
t
h
e
 
m
o
s
t
e
x
c
i
t
i
n
g
 
r
e
c
e
n
t
 
d
e
v
e
l
o
p
m
e
n
t
s
i
n
 
t
h
e
 
f
i
e
l
d
 
o
f
 
A
I
,
 
i
n
c
l
u
d
i
n
g
o
u
r
 
r
e
c
e
n
t
 
b
r
e
a
k
t
h
r
o
u
g
h
s
 
i
n
s
p
a
t
i
a
l
 
r
e
a
s
o
n
i
n
g
.
Language prompt
[Peters et al. ’18 , Radford et al. ’19, Brown et al. ’20, …. ]
5
LM
What is the sentiment of the
following review? (positive
or negative)
 
 
p
o
s
i
t
i
v
e
 
Sentence: That was a great
fantasy movie.
discrete (text) 
prompts:
easy to interpret, 
but not easy to optimize
[Peters et al. ’18 , Radford et al. ’19, Brown et al. ’20, …. ]
 
6
LM
 
What is the sentiment of the
following review? (positive
or negative)
 
positive
 
Sentence: That was a great
fantasy movie.
discrete (text) 
prompts:
easy to interpret, 
but not easy to optimize
 
[Peters et al. ’18 , Radford et al. ’19, Brown et al. ’20, …. ]
7
[Li and Liang’21; Lester et al.’21]
LM
 
 
p
o
s
i
t
i
v
e
 
Sentence: That was a great
fantasy movie.
continuous 
prompts: 
unclear how to interpret, 
but easy to optimize
LM
What is the sentiment of the
following review? (positive
or negative)
positive
Sentence: That was a great
fantasy movie.
discrete (text) 
prompts:
easy to interpret, 
but not easy to optimize
Something related to
sentiment analysis?
8
LM
positive
Sentence: That was a great
fantasy movie.
Something related to
sentiment analysis?
Research question: 
are there any meaningful discrete
(textual) interpretations to continuous prompts?
Opposite: 
how 
unfaithful
can their 
interpretation
be to what they do?
 
9
LM
 
positive
 
Sentence: That was a great
fantasy movie.
 
Flip the sentiment of the sentence
Research question: 
are there any meaningful discrete
(textual) interpretations to continuous prompts?
Opposite: 
how 
unfaithful
can their 
interpretation
be to what they do?
 
any arbitrary text:
 
Proj(.)
 
10
 
[Khashabi et al.’22]
LM
 
positive
 
Sentence: That was a great
fantasy movie.
 
Flip the sentiment of the sentence
Opposite: 
how 
unfaithful
can their 
interpretation
be to what they do?
 
Proj(.)
 
any arbitrary text:
Waywardness hypothesis
 (informal):
One can find “accurate” continuous prompts such
that they can be “projected” to 
any 
arbitrary text.
11
LM
positive
Sentence: That was a great
fantasy movie.
 
an 
arbitrary
 text
Write down the conclusion you can
reach by combining the given
Fact 1 and Fact 2.
Waywardness hypothesis
 (informal):
One can find “accurate” continuous prompts such
that they can be “projected” to 
any 
arbitrary text.
 
Proj(.)
[Khashabi et al.’22]
 
12
LM
 
positive
 
Sentence: That was a great
fantasy movie.
 
an 
arbitrary
 text
int clamp(int val, int min_val) {
    return std::max(min_val, val);
}
Waywardness hypothesis
 (informal):
One can find “accurate” continuous prompts such
that they can be “projected” to 
any 
arbitrary text.
 
Proj(.)
 
[Khashabi et al.’22]
13
LM
positive
Sentence: That was a great
fantasy movie.
Waywardness hypothesis
 (informal):
One can find “accurate” continuous prompts such
that they can be “projected” to 
any 
arbitrary text.
an 
arbitrary
 text
accuracy
85
 
90
 
95
 
100
Proj(.)
[Khashabi et al.’22]
 
14
LM
 
positive
 
Sentence: That was a great
fantasy movie.
Waywardness hypothesis
 (informal):
One can find “accurate” continuous prompts such
that they can be “projected” to 
any 
arbitrary text.
 
an 
arbitrary
 text
 
Proj(.)
 
[Khashabi et al.’22]
 
(1) The mapping between continuous and
discrete space is not one-to-one.
It is true for a many choices of 
Proj(.)
 
15
Making Sense of “Waywardness”
Making Sense of “Waywardness”
 
(1) The mapping between continuous and
discrete space is not one-to-one.
It is true for a many choices of 
Proj(.)
 
 
(2) Deep models give a lot of expressivity
power to the earlier layers.
 
16
 
[Telgarsky ’16; Raghu et al. ’17]
 
Implications of Waywardness (1)
 
Faithful interpretation of 
continuous
 prompts is difficult.
 
17
LM
 
positive
 
Sentence: That was a great
fantasy movie.
continuous 
prompts:
unclear how to interpret, 
but easy to optimize
Something related to
sentiment analysis?
Implications of Waywardness (2)
Risk of interpreting continuous prompts:
concealed adversarial attacks.
18
LM
continuous prompt
 
Rank the candidates ignoring
their race or gender.
 
😇 benign
 projection
 
Proj(.)
Implications of Waywardness (3)
19
discrete (text) 
prompts:
easy to interpret, 
but not easy to optimize
LM
positive
Sentence: That was a great
fantasy movie.
 
An optimization in search of
discrete 
(human-readable) prompts:
Summary
 
Waywardness Hypothesis — a surprising difficulty in interpreting
continuous prompts.
We provided empirical evidence and intuitions for this hypothesis.
 
Concluded with implications of this hypothesis.
 
We need algorithmic or architectural innovations for automatic
discovery of human-readable prompts.
20
 
Experiment: effect of prompt length
 
21
 
The relative accuracy drop
is marginal when the
prompt length is not too
small (e.g. 7 or larger).
Slide Note

Hi, I am Daniel Khashabi, and I am a post-doc with the Mosaic team.

Today I am going to talk about ….

This is based on a paper that will appear in NAACL 2022 (in Seattle)

And it is a joint work with my wonderful colleagues here at AI2, UW and UC Irvine.

Embed
Share

Delve into the analysis of discrete text prompts and their interpretation of continuous prompts in AI research. The work explores sentiment analysis using pre-trained language models along with recent breakthroughs in spatial reasoning. Discover the challenges in interpreting and optimizing text prompts for sentiment analysis and explore the meaningful interpretations of continuous prompts.


Uploaded on Jul 18, 2024 | 0 Views


Download Presentation

Please find below an Image/Link to download the presentation.

The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.

E N D

Presentation Transcript


  1. On Discretized Interpretation of Continuous Prompts Joint work w/ Shane Lyu, SewonMin, LianhuiQin, Kyle Richardson, Sameer Singh, Sean Welleck, HannanehHajishirzi, Tushar Khot, Ashish Sabharwal, YejinChoi Allen Institute for AI University of Washington University of California-Irvine 1

  2. pre-trained language models (LM) LM 2 [Peters et al. 18 , Radford et al. 19, Brown et al. 20, . ]

  3. LM Language prompt 3 [Peters et al. 18 , Radford et al. 19, Brown et al. 20, . ]

  4. we shared some of the most exciting recent developments in the field of AI, including our recent breakthroughs in spatial reasoning. In today's presentation at Allen Institute for AI, LM Language prompt 4 [Peters et al. 18 , Radford et al. 19, Brown et al. 20, . ]

  5. discrete (text) prompts: easy to interpret, but not easy to optimize What is the sentiment of the following review? (positive or negative) LM Sentence: That was a great fantasy movie. positive 5 [Peters et al. 18 , Radford et al. 19, Brown et al. 20, . ]

  6. discrete (text) prompts: easy to interpret, but not easy to optimize What is the sentiment of the following review? (positive or negative) LM Sentence: That was a great fantasy movie. positive 6 [Peters et al. 18 , Radford et al. 19, Brown et al. 20, . ]

  7. discrete (text) prompts: easy to interpret, but not easy to optimize What is the sentiment of the following review? (positive or negative) LM Sentence: That was a great fantasy movie. positive Something related to sentiment analysis? continuous prompts: 0.9 0.1 -2.1 0.0 unclear how to interpret, but easy to optimize LM Sentence: That was a great fantasy movie. positive 7 [Li and Liang 21; Lester et al. 21]

  8. Research question: are there any meaningful discrete (textual) interpretations to continuous prompts? Opposite: how unfaithful can their interpretation be to what they do? Something related to sentiment analysis? 0.9 0.1 -2.1 0.0 LM Sentence: That was a great fantasy movie. positive 8

  9. Research question: are there any meaningful discrete (textual) interpretations to continuous prompts? Opposite: how unfaithful can their interpretation be to what they do? any arbitrary text: Flip the sentiment of the sentence Proj(.) 0.9 0.1 -2.1 0.0 LM Sentence: That was a great fantasy movie. positive 9

  10. Waywardness hypothesis (informal): One can find accurate continuous prompts such that they can be projected to any arbitrary text. Opposite: how unfaithful can their interpretation be to what they do? any arbitrary text: Flip the sentiment of the sentence Proj(.) 0.9 0.1 -2.1 0.0 LM Sentence: That was a great fantasy movie. positive 10 [Khashabi et al. 22]

  11. Waywardness hypothesis (informal): One can find accurate continuous prompts such that they can be projected to any arbitrary text. ?: optimized for the task + projecting to a given text Proj(.) Write down the conclusion you can reach by combining the given Fact 1 and Fact 2. an arbitrary text ? : optimized for the task LM Sentence: That was a great fantasy movie. positive 11 [Khashabi et al. 22]

  12. Waywardness hypothesis (informal): One can find accurate continuous prompts such that they can be projected to any arbitrary text. ?: optimized for the task + projecting to a given text Proj(.) int clamp(int val, int min_val) { return std::max(min_val, val); } an arbitrary text ? : optimized for the task LM Sentence: That was a great fantasy movie. positive 12 [Khashabi et al. 22]

  13. Waywardness hypothesis (informal): One can find accurate continuous prompts such that they can be projected to any arbitrary text. ?: optimized for the task + projecting to a given text Proj(.) 85 90 95 100 accuracy an arbitrary text 91.8 ~0.6% ? ? : optimized for the task ? 92.4 LM Sentence: That was a great fantasy movie. positive 13 [Khashabi et al. 22]

  14. Waywardness hypothesis (informal): One can find accurate continuous prompts such that they can be projected to any arbitrary text. ?: optimized for the task + projecting to a given text Proj(.) an arbitrary text ? : optimized for the task LM Sentence: That was a great fantasy movie. positive 14 [Khashabi et al. 22]

  15. Making Sense of Waywardness (1) The mapping between continuous and discrete space is not one-to-one. It is true for a many choices of Proj(.) 15

  16. Making Sense of Waywardness (1) The mapping between continuous and discrete space is not one-to-one. It is true for a many choices of Proj(.) (2) Deep models give a lot of expressivity power to the earlier layers. [Telgarsky 16; Raghu et al. 17] ? 16

  17. Implications of Waywardness (1) Faithful interpretation of continuous prompts is difficult. Something related to sentiment analysis? continuous prompts: 0.9 0.1 -2.1 0.0 unclear how to interpret, but easy to optimize LM Sentence: That was a great fantasy movie. positive 17

  18. Implications of Waywardness (2) Risk of interpreting continuous prompts: concealed adversarial attacks. continuous prompt benign projection Proj(.) Rank the candidates ignoring their race or gender. malicious behavior LM < < 18

  19. Implications of Waywardness (3) p= What is the sentiment of the following review? (positive or negative) discrete (text) prompts: easy to interpret, but not easy to optimize LM Sentence: That was a great fantasy movie. positive maximize?Readability(p) Utility(p) An optimization in search of discrete (human-readable) prompts: There are many ? s that maximize both utility and readability , though ? s interpretation is not faithful to its effect degenerate problem. 19

  20. Summary Waywardness Hypothesis a surprising difficulty in interpreting continuous prompts. We provided empirical evidence and intuitions for this hypothesis. Concluded with implications of this hypothesis. We need algorithmic or architectural innovations for automatic discovery of human-readable prompts. 20

  21. Experiment: effect of prompt length The relative accuracy drop is marginal when the prompt length is not too small (e.g. 7 or larger). 21

Related


More Related Content

giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#giItT1WQy@!-/#