Enhancing Usability and Accessibility in Next-Generation Elections: NIST Workshop Insights
U
s
a
b
i
l
i
t
y
&
a
c
c
e
s
s
i
b
i
l
i
t
y
o
f
n
e
x
t
g
e
n
e
r
a
t
i
o
n
e
l
e
c
t
i
o
n
s
N
I
S
T
R
o
a
d
m
a
p
Notes from the October 8-9, 2014 workshop
Whitney Quesenbery and Dana Chisnell
Center for Civic Design
Kathryn Summers and graduate students
University of Baltimore
Sharon Laskowski and Shaneé Dawkins
NIST
A
p
r
e
l
i
m
i
n
a
r
y
r
e
p
o
r
t
o
n
t
h
e
w
o
r
k
s
h
o
p
,
O
c
t
.
8
-
9
,
2
0
1
4
This was the first of two workshops in a process to create a roadmap for
developing usability and accessibility guidance, best practices, and
standards for next generation voting systems that will help election
officials, manufacturers, and other stakeholders to ensure that all voters
can vote independently and privately.
The roadmap, when completed, will outline steps needed to produce this
guidance for election officials, manufacturers, and other stakeholders. It
will identify issues, gaps, new technology, and processes, how to develop
guidance, as well as relevant research and best practices that can be
used to improve voting systems given next generation technology.
In this first workshop, we:
Explored uses of current and future technology in elections,
Identified gaps in the research, and
Brainstormed new ideas to develop useful guidance.
The goal of these activities was to identify the topics that new guidance
must consider and explore the issues that shape current thought on
these topics.
W
o
r
k
s
h
o
p
p
a
r
t
i
c
i
p
a
n
t
s
Andrew Baranak, GTRI
David Bjerke, Falls Church, VA
Stephen Blosser, MSU RCPD
Steven Booth, NFB
Mike Byrne Rice University
McDermott Coutts, Unisyn Voting Solutions
Jim Dickson Nat'l Council on Ind. Living
Jeremy Epstein, NSF
Josh Franklin, NIST
Bob Giles NJ Division of Elections
Thomas Hicks, House Admin. Committee
Merle King, Kennesaw Center for Election Systems
Ben Long, NIST
Christy McCormick U.S. DOJ
Alysoun McLaughlin, Montgomery County, MD
Whitney May, ELECTricity
Tammy Patrick, Bipartisan Policy Committee
Sarah Swierenga MSU UARC
EAC
Megan Dillon
Monica Evans
Brian Hancock
Patrick Leahy
Alice Miller
Jessica Myers
Robin Sargent
Brian Whitener
University of Baltimore GAs
Jaime Lee
Kathryn Locke
Emily Rhodes
Caitlin Rinn
Joel Stevenson
B
a
c
k
g
r
o
u
n
d
NIST has worked on voting system standards since the Help
America Vote Act of 2002, both establishing requirements for
certification test labs and creating the Voluntary Voting System
Guidelines (VVSG). The VVSG 2005 included the first
comprehensive usability and accessibility standards for voting
systems.
Elections are changing. There are new technologies, new
research, new laws, and new elections procedures since the
2005 Voluntary Voting System Guidelines 1.0 were published.
Keeping up with these changes requires a new approach to
usability and accessibility guidance for election systems.
Recent years have brought changes to the state of the art and
technology for voting systems, as well as public expectations
about how voters will participate in elections.
B
a
c
k
g
r
o
u
n
d
(
2
)
Despite 12 years of work within elections on standards for
usability and accessibility, the reality is that there are still many
barriers.
Even newer systems show poor accessibility and usability,
suggesting lack of knowledge of best practices and existing
standards and guidelines. This is true of both voting systems
and related technology.
As more jurisdictions have switched to paper ballots, there is
even more isolation of the "accessible" voting system.
The accessible systems may go unused through the entire
day, further reducing the likelihood that they will be set up
and ready to use.
Systems for UOCAVA voters under the MOVE Act allow for
online ballot marking.
Disability rights groups advocate for
making these systems available to voters with disabilities (or
all voters). Security experts point out many pitfalls.
W
h
a
t
i
s
a
r
o
a
d
m
a
p
?
A NIST roadmap is an outline for future work.
A roadmap:
Identifies gaps in knowledge to be filled
Identifies issues to be resolved
Looks at technology, processes, standards & guidelines
Recommend approaches to the work
It does not:
Prescribe solutions
Recommend specific guidelines
Rather, it shows how to structure work to accomplish the goals
This roadmap will cover future guidance to ensure the usability and
accessibility of election systems.
P
o
s
s
i
b
l
e
g
o
a
l
s
f
o
r
t
h
e
r
o
a
d
m
a
p
Increase the level of knowledge for how to design, develop,
deploy, and use of usable and accessible elections systems.
Promote consistent levels of usability and accessibility
across technology in all parts of the elections process.
Make systems more usable for everyone in the elections
process, including voters, poll workers, elections staff, and third-
parties like election interest and advocacy groups or technology
developers.
Shift from single focus on standards and certification to
identifying the appropriate guidance and how to implement the
guidance, including:
Guidelines for best practices
Procedural support
Training
A
b
o
u
t
t
h
e
w
o
r
k
s
h
o
p
The group started from this focus question:
W
h
a
t
w
i
l
l
t
h
e
v
o
t
e
r
e
x
p
e
r
i
e
n
c
e
o
f
e
l
e
c
t
i
o
n
s
b
e
l
i
k
e
i
n
t
h
e
f
u
t
u
r
e
?
Through a KJ* activity, the group identified 4 priority areas for
breakout topics:
C
o
n
v
e
n
i
e
n
c
e
v
o
t
i
n
g
a
n
d
"
V
o
t
e
A
n
y
w
h
e
r
e
"
A
c
c
e
s
s
i
b
i
l
i
t
y
a
n
d
u
n
i
v
e
r
s
a
l
u
s
a
b
i
l
i
t
y
T
r
u
s
t
,
s
e
c
u
r
i
t
y
a
n
d
v
e
r
i
f
i
c
a
t
i
o
n
D
e
s
i
g
n
a
n
d
e
v
a
l
u
a
t
i
o
n
o
f
t
h
e
u
s
e
r
i
n
t
e
r
f
a
c
e
The groups rotated through the breakout topics during the first
afternoon.
* See
How to KJ: Setting Priorities Quickly
http://uxpamagazine.org/how-to-kj/
A
b
o
u
t
t
h
e
w
o
r
k
s
h
o
p
(
2
)
The discussions of the focus topics identified:
Current and possible future scenarios for usable and
accessible elections
Conditions required for these scenarios for future elections
Strategies for supporting voters in navigating across the
voter journey
Conditions, challenges, or limits that could constrain these
scenarios
T
h
i
s
r
e
s
u
l
t
e
d
i
n
h
i
g
h
e
r
q
u
a
l
i
t
y
n
o
t
e
s
t
h
a
n
j
u
s
t
l
i
s
t
i
n
g
i
s
s
u
e
s
i
n
e
a
c
h
c
a
t
e
g
o
r
y
.
H
o
w
e
v
e
r
,
i
t
c
a
n
a
l
s
o
b
e
h
a
r
d
t
o
c
a
p
t
u
r
e
f
r
e
e
-
r
a
n
g
i
n
g
d
i
s
c
u
s
s
i
o
n
,
s
o
t
h
e
o
u
t
c
o
m
e
s
o
f
t
h
e
s
e
d
i
s
c
u
s
s
i
o
n
s
a
r
e
r
e
a
l
l
y
i
n
p
u
t
t
o
a
m
o
r
e
s
t
r
u
c
t
u
r
e
d
d
i
s
c
u
s
s
i
o
n
i
n
t
h
e
s
e
c
o
n
d
w
o
r
k
s
h
o
p
.
T
h
e
v
o
t
e
r
j
o
u
r
n
e
y
(
1
)
As the group worked on the 4 priority topics, we used portraits of voters (called 'personas') as
a reminder of the range of people who use election systems.
We also looked at the user experience across the entire process of voting, not just marking
and casting a ballot. The goal was to be able to think about the context in which the systems
are used, not just the equipment.
A simple timeline helped organized the notes by both stages in the voter journey and type of
notes.
Getting the
results
T
h
e
v
o
t
e
r
j
o
u
r
n
e
y
(
2
)
The election process and voter journey is a useful organizing principle, to ensure that the
scope is grounded in the voter's process rather than specific technology.
Elections are a service design, requiring coordination of people, procedures, policy,
information, and systems.
The voter journey includes learning and making choices as well as the core activities
of participation.
There is a wide array of technology systems in use. They include specialized
systems, systems used throughout the journey, and general systems used as part of
elections
Technology and procedures are used in the context of the voter journey. Guidance for
them must take context into account.
Inclusion of a process or technology on the voter journey map does not mean that
NIST or the EAC will automatically write standards for it.
The next slide shows a summary of the notes collected on the journey map during the
workshop.
T
h
e
V
o
t
e
r
J
o
u
r
n
e
y
Overall Thoughts
Plain language
Universal ID
Universal WIFI for all
devices
Your choice is your
choice, regardless of
disability
Blue Sky Ideas
Elections know me
Use the cloud
Virtual voter
representative the
"knows" voter
configuration
SMS and texting for
voting
Vote from home
Vote on a smartphone
Vote on home PC
Build in "negotiation"
vs. using personalized
config and PII
Preferences for voters
match to choices (OK
Cupid)
UberVote (car service)
Absentee voters can
vote anywhere
Pushing ballot to
people the way they
want
Online ballot marking
tool and backup
support
Designed and built by
states and voting
jurisdictions
A well-designed ballot
should be shared via
internet or by NIST
Photos of candidates
on ballots
Common interactions
patterns layout
template (so good,
you'd be foolish not to
use it)
A pilot project with
small elections would
usability test a ballot
design
Ballot designed to work
on standard computers
Paper based system
then PDF'd and goes to
the cloud
Take picture of your
ballot and mail or
upload it
Audio version of the
ballot
One "time zone" for
elections
Other Notes
Voters need the ability to
"rehearse" to prepare
Identify preferences, not
abilities
Personal settings "card"
My details have
changed - how to
update
Bringing voting (iPad)
tech to you
Ballot marking saves $?
Support phone lines for
voters
Marking/reading
anywhere give syou
time
Online ballot marking
prevents error
All systems must be
subject to the same
standards - not like
current double-
standard for paper and
DRE
Minimum standards vs.
goals
Votes don't get
announced until a
specific time
T
h
e
V
o
t
e
r
J
o
u
r
n
e
y
(
c
o
n
t
i
n
u
e
d
)
Overall Thoughts
Time
Voting systems can't
touch internet
Costs: universal design is
expensive to implement
Problems
Coercion
How to deploy ballots
on many devices
I don't have a
smartphone
Current laws do not
allow for voting
anywhere
Big brother issues,
including PII that might
help voters
Options can be a
burden
USPS not reliable
Long wait times
degrade trust and
confidence
Changing for different
abilities, aging - like
vision
Supporting voters with
invisible disabilities
How to print out forms
from online
One system for
everyone? How to
match the right person
to the right ballot
QR codes scare people -
can we make the
machine read real text
Technology is a
problem not a full
solution
Poll workers and
procedural security
Racial and social issues
with pictures on ballots
Voters not educated
enough, lack of access
in general
Trust in the system to
count as cast
Trust in poll workers
until something goes
wrong
Trust that the
outcomes are as voters
voted
Tasks for voter self-
audie in conflict with
tools for preserving
privacy/vote selling
Compromising privacy
if auditing
Many kinds of voters
Other Notes
Voters need the ability to
"rehearse" to prepare
Identify preferences, not
abilities
Personal settings "card"
My details have
changed - how to
update
Bringing voting (iPad)
tech to you
Ballot marking saves $?
Support phone lines for
voters
Marking/reading
anywhere give syou
time
Online ballot marking
prevents error
All systems must be
subject to the same
standards - not like
current double-
standard for paper and
DRE
Minimum standards vs.
goals
Votes don't get
announced until a
specific time
T
h
e
V
o
t
e
r
J
o
u
r
n
e
y
(
t
e
c
h
n
o
l
o
g
y
)
Official Elections Web Sites and Social Media (Local and State)
Eligibility info
Sample ballots/voter
ballot info
How to vote info
Hours and dates
Voting locations
Accessibiity info
Language info
Voter ID requirements
Ballot delivery options
Online voter registration
'My Voter' portals
Polling place lookup
VBM request
Blank ballot access and
delivery
Marking instructions
cCasting instructions
Election results
Technology in the Polling Place
Pollbooks
Ballot activators
Online ballot marking
tools
Ballot marking tools
Ballot readers
Ballot printers
Phone/Voice
Assistive technology
Electronic ballot casting
Ballot scanners
Ballot readers for review
Ballot printing and return
tool
Interactive data
E2E verification
Other Organizations (Campaigns, Advocates, Good Government...)
Third party registration
apps
VIP-type information
app
Apps built on public
information
?? Blank ballot access and
delivery ??
?? Online ballot marking
tools ??
Personal AT
Support tools for
returning ballots
Personal AT
Citizen ballot review
E2E verification
Voter's Technology
Computer/Mobile
Social Media
Computer/Mobile
Passbook/Wallet (ID)
GPS
Computer/Mobile
Social Media
Computer/Mobile
Input/Output AT
Input/Output AT
Computer/Mobile
Social Media
Verification
& results
Preparing
to vote
Choosing how
to vote
Checking in/
getting ballot
Marking
the ballot
Casting
the ballot
R
e
s
o
u
r
c
e
s
i
d
e
n
t
i
f
i
e
d
i
n
t
h
e
w
o
r
k
s
h
o
p
E
M
L
o
r
C
o
m
m
o
n
D
a
t
a
F
o
r
m
a
t
–
a
l
l
o
w
s
m
u
l
t
i
p
l
e
d
e
v
i
c
e
s
t
o
s
h
a
r
e
e
l
e
c
t
i
o
n
d
a
t
a
.
C
r
i
t
i
c
a
l
f
o
r
c
o
m
p
o
n
e
n
t
a
r
c
h
i
t
e
c
t
u
r
e
s
.
Q
R
c
o
d
e
s
,
N
F
C
,
H
o
l
l
a
r
i
t
h
g
r
i
d
s
,
o
r
o
t
h
e
r
t
o
k
e
n
s
–
w
a
y
s
t
o
t
r
a
n
s
p
o
r
t
b
a
l
l
o
t
c
h
o
i
c
e
s
e
f
f
i
c
i
e
n
t
l
y
a
n
d
p
r
i
v
a
t
e
l
y
G
P
I
I
o
r
o
t
h
e
r
p
r
e
f
e
r
e
n
c
e
s
m
a
n
a
g
e
r
–
a
l
l
o
w
s
s
y
s
t
e
m
t
o
m
a
t
c
h
n
e
e
d
s
a
n
d
p
r
e
f
e
r
e
n
c
e
s
t
o
o
p
t
i
o
n
s
a
v
a
i
l
a
b
l
e
.
I
d
e
n
t
i
f
i
c
a
t
i
o
n
:
B
i
o
m
e
t
r
i
c
s
(
e
y
e
s
c
a
n
,
f
i
n
g
e
r
p
r
i
n
t
)
o
r
a
s
e
c
u
r
e
n
a
t
i
o
n
a
l
I
D
(
l
i
k
e
m
i
l
t
a
r
y
C
A
C
)
,
t
w
o
f
a
c
t
o
r
a
u
t
h
e
n
t
i
c
a
t
i
o
n
,
D
i
s
n
e
y
F
a
s
t
P
a
s
s
O
t
h
e
r
c
e
r
t
i
f
i
c
a
t
i
o
n
p
r
o
g
r
a
m
s
:
F
D
A
,
s
l
o
t
m
a
c
h
i
n
e
s
,
b
a
n
k
i
n
g
a
u
d
i
t
s
R
e
l
a
t
e
d
w
o
r
k
i
n
g
g
r
o
u
p
s
:
N
A
S
E
D
,
B
i
p
a
r
t
i
s
a
n
P
o
l
i
c
y
C
e
n
t
e
r
,
S
t
a
t
e
c
e
r
t
i
f
i
c
a
t
i
o
n
g
r
o
u
p
,
F
V
A
P
,
E
A
C
R
e
s
e
a
r
c
h
c
e
n
t
e
r
s
a
t
R
i
c
e
,
M
I
T
,
C
a
l
t
e
c
h
,
G
e
o
r
g
i
a
T
e
c
h
,
M
S
U
,
U
.
B
a
l
t
i
m
o
r
e
,
G
P
I
I
E
l
e
c
t
i
o
n
i
n
i
t
i
a
t
i
v
e
s
:
H
u
m
b
o
l
d
t
C
o
u
n
t
y
b
a
l
l
o
t
p
r
o
j
e
c
t
,
V
S
A
P
,
S
t
a
r
V
o
t
e
,
r
i
s
k
l
i
m
i
t
i
n
g
a
u
d
i
t
s
B
l
u
e
s
k
y
?
O
r
f
u
n
d
a
m
e
n
t
a
l
s
?
Many of the blue sky ideas are based on some fundamental concepts. All of them are in
use in some current context – the "blue sky" part of the idea is how to apply them to
elections effectively and consistently.
Vote anywhere, with options for casting
Well-designed ballot (and other) – so good, you'd be foolish not to use it
More use of COTS
In voting systems
Enabling use of voters' own systems
A way to transfer information easily between parts of the system, different devices,
and person technology
Universal ID
Easy personalization – either quickly set up, or recognized from ID/token
Plain language
T
h
e
V
o
t
e
r
J
o
u
r
n
e
y
(
a
p
o
s
s
i
b
l
e
s
t
r
u
c
t
u
r
e
)
Notes
on the priority topic discussions
Trust, security and verification
Convenience in voting and "Vote Anywhere"
Accessibility and universal usability
Design and evaluation of the user interface
Trust, security, and verification
Priority topic: Trust, security, and verification
The discussion of trust, security, and verification mapped trust
as an element in elections:
People
Procedures and processes
Systems
Policy and political issues
Polling places (and voting outside of them)
Information
Trust is based on each individual's perception of
the overall process
TRUST
Policy/Political
Your vote counts, no matter
how you vote.
The system is good. (Trust until
something bad goes wrong.)
Elections are transparent and
understandable.
Elections are convenient.
People
Rule makers
Congress/ Dept of Justice
State/Local Officials
Systems approvers
EAC/Certification labs
Manufacturers
Election participants
Local Election Office
Poll workers
Helpers
Voters
Systems
Reliable
Transparent
Accurate
Usable
Accessible
Consistent
(Standards)
How does voting compare
to other systems?
Banking
Motor Vehicles
Financial Aid forms
IRS
US Postal Service
Procedures
Easier to vote: Harder to cheat
Voter registration is accurate (your
registration is actually recorded)
Voters who should vote, can. And who
should not, don't.
Ballot marking, recording, tallying,
auditing, reporting are correct and
accurate
All procedural voting choices are equal
Polling Place
Access
- Transportation
- Accessibility
Orderly polling place
- Unknown area
- Long lines/wait time
Trustworthy people
- Willing to reveal disability?
- Give accurate information?
- Give unbiased assistance?
Information
Knowing your choices
Knowing where to go (in person or
online) and how to get there
(transportation or digitally
Knowing the procedures
Knowing how to use technology
Trust in the Voter Journey
Voter registration – was
it actually completed?
Is the information
official?
All choices are equal
and votes count equally
Vote centers are backup
for VoteByMail
Can I track my VBM
ballot?
Authentication:
•
Trust in who is
voting
•
Trust those who
sholdn't vote, don't
Signature verifiation is
fundamental to current
laws & procedures
Election officials trust in
voters
Challenges:
-
Unknown area
-
Long lines
-
Wait time
-
Lack of information
-
Use of
tech/Internet
Is it trustworthy?
Does it do what it is
supposed to do?
Does it break down?
Is the ballot accurate
and complete?
Do trends ike vote
early, then change our
minds suggest giving up
privacy?
Will my vote be cast as I
intend?
Will it be counted as
cast?
Which ballot counts?
(Last? First?)
Recording is complete
Tally procedures are
good.
Auditing/reporting is
correct
Blue Sky Ideas and Resources
Live pilots for testing as part of certification – see it
in use.
Use COTS scanning
devices (like grocery
scanner)
Voting equivalent to
direct deposit.
Eliminate the secret
ballot
Verification
& results
Preparing
to vote
Choosing how
to vote
Checking in/
getting ballot
Marking
the ballot
Casting
the ballot
Identity:
Witnesses for vouching
for identity
Every ballot gets
scanned to the cloud:
Humboldt project
Risk limiting audits
Questions to answer for better trust
How do we decide which systems or people to trust, and who has to trust them?
Every trust change ends with a human being.
Small problems add up to decreased trust.
How do we improve trust in the political and social aspecsts of elections?
What part do election procedures play in trusting elections.
Is privacy sacred?
Are we moving towards eliminating the secret ballot?
Many online options are difficult because of the identifiation issue
How do we deal with security problems?
There are time boundaries in elections when issues can be addressed.
Elections rarely allow a "
do-over."
Convenience in voting
Priority topic: Convenience in voting
The discussion of convenience in voting and the ability to "vote
anywhere" covered a wide range of issues, and what
"convenience" means in this context.
One answer was to allow more personal choice, including:
When to vote
Where to vote
What systems or assistance to use
Voting should be the most convenient government
service as voting is a right and not just a privilege.
Voters with disabilities use "convenience voting" options more than the general
population. These options include:
Early voting centers
Mobile early voting vans
Vote by mail and online ballot marking
More convenient voting can expand and improve procedures and equipment already in
use
Allow use of personal technology to mark ballots
Allow more flexibility in where and how to vote
Use online tools to mark and cast ballots
Convenience in the Voter Journey
Expand and improve procedures and systems already in use
Improve voters' access to
resources and
information:
•
Hours, dates,
locations
•
Information about
candidates and
measures
•
Finding the closest
polling place.
Allow for multiple ways
to vote (remote and in-
person) to
accommodate personal
preferences and needs
Allow voting at any
polling place (near
work, near hoem, etc)
Bring the polling place
to those who can't get
there
•
Long term care
facilities
•
Shut-ins
•
Disasters
Have voting systems
support all types of
personal assistive
technology.
Allow voters to use
their own systems at
the polling place
Allow voters to mark
their ballot online
Improve convenience
through technolgy
•
Scan a QR code or
other token on a
pre-marked ballot
•
Upload a picture of
a marked ballot
from a smart phone
Blue Sky Ideas
"Uber Vote" – a car
service that could bring
you to the polling place
This service could also
extend curb-side voting
to longer distances.
Voter Support Lines to help if those who have
difficulty marking/casting a ballot online.
Vote by Phone to allow voters to vote anywhere at
any time.
Vote via SMS – support voters who don't have
smartphones.
Polling place child care
SMS-based voter information for basics like election
dates, hours, polling place and early voting locations
Interactive app that uses voice search (like Siri) to
let voters ask for information about their ballot of
how to vote
Biometics (eyescan,
fingerprint) for voter
identification
GPII or other ID that
carries setup
preferences
Verification
& results
Preparing
to vote
Choosing how
to vote
Checking in/
getting ballot
Marking
the ballot
Casting
the ballot
Tensions to resolve in increased convenience
How do we increase convenience without sacrificing the voter's privacy or
security?
Remote voting may not be private, or may be coerced.
Serious security concerns for casting a ballot online.
How do we ensure that voters are provided with the resources and support they
need to vote from anywhere?
Do we need a better organization of voter outreach and support?
Especially support for using assistive technology
Is there a conflict between personalization and equality of experience for all?
Do all voters have equal access to choice and personalization?
What is the impact of the digital divide in what kinds of personal technology (like
mobile devices) people own?
How to we address differences in assistive technology
?
Accessibility and Universal Usability
Priority topic: Accessibility and Universal Usability
This topic overlapped with the discussion of convenience in
voting.
How far can the goal of universal usability work when
technology changes are inevitable?
How can personalization support voters in creating a more
usable and accessible voting experience?
How can we use systems and interfaces that voters have
already tailored for their own use?
Reaching universal design is a challenge when there
are so many different voter needs.
Allow more personalization for individual needs and preferences.
The digital divide is a real issue: some voters do not have smartphone
Access to information, resources – not just voting.
Mobile devices are already in use for notes to prepare for voting
Optimal usability is an important step
Stop creating a separate machine for people with disabilities
We need to address e-casting, not just e-marking.
Paper ballots introduce errors, are not environmentally sound, are not ADA compliant.
Why can't we accept electronically cast ballots that we count as a separate "stream"
like we do with overseas FWABS (minimize audit/pollworker complications)
Universal Usability in the Voter Journey (close-up view)
Ensure equal access to all parts of the voter journey
for all, even without personalization or individual technology
Resources
Existing standards –
WCAG 2.0, Section 508
Verification
& results
Preparing
to vote
Choosing how
to vote
Checking in/
getting ballot
Marking
the ballot
Casting
the ballot
NFC or QR codes
NFC or QR codes
Mobile phones are
important as a way to
bring a marked sample
ballot to the polling
place (but not always
allowed)
Need to include
technology like Braille
[preferred by some,
critical for deaf-blind)
Separate "stream" for
electronically cast
ballots
Thoughts about universal usability
Can standards and solutions for voting expand to all government related
interactions?
Capturing preferences could extend to all interactions
Consistency is more important for AT users and those with challenges
Can one size fit all?
We have to consider voters who arrive at the polls with no AT
Should the system be modular with alternatives for different needs?
Can the system be flexible with different interaction options?
How do we deal with differences between jurisdictions?
Every state has different rules – what is the common denominator?
Design and Evaluation
Priority topic: Design and evaluation
Discussion of design and evaluation included:
Design and development processes that encourage good
usability and accessibility.
Ways to write guidance for best practices, standards, and
test methods.
Different test approaches and how they might fit into the
certification process.
Design and Evaluations supports the voter journey
Resources
Challenges
Testing
Approaches
Certification
Process
Writing
Guidance
US Constitution
EAC and NIST limited to voting
systems. WIll the EAC be able
to change directions?
Standards are too absolute
•
A "testing standard" has
become a "design
standard"
•
High level vs. detailed
requirements
•
Cost of iterative
design/feedback loop+
certifiction
Until voters actually vote on
the machine, there is not way
to know it will work
We test voting systems, but
not other parts of the election
system
2018/2020 and the impending
crisis of out-of-date systems
Existing standards
•
WCAG 2.0, Section 508
•
IEEE common data format
for reporting/log files
Industry standards
•
Slot machine certifcation
•
Bank audits
•
FDA
•
FAA Declaration of
Conformance
Committees
•
NASED
•
State Certification Group
Guiding principles that
reach for an outcome
Usability standards based
on efficiency,
effectiveness, satisfaction
What technique tells you
what
Classify voters not
systems
Involve a broad group of
election officials in
generating standards
Benefits of certification:
license to use
Procurement intersects with
certification to force
procedures and
decisionmaking
EAC doing federal and state
certification at the same time
Piloting at different levels
Better feedback on the
outcomes of systems in use
•
Practice voting and
testing constantly
•
Consumer reports for
voting sysems
•
Vendor review website
(Yelp for voting systems)
•
Common/shared user
data to create better
voting systems
Testing with hardest,
not easiest, users
Create a voter expo for
combined testing days
to make it easier to
assemble a large and
diverse group of
voters as test
participants.
Design iterations and
testing
Attractive and simple
ballot that is used by
everyone
A concept for useful guidance in the right form
Goals
Clear statments of goals help everyone
understand the reason for any requirement
or guidelines.
The Core Requirements are testable
usability basics that apply to any interactive
system.
The system-specific guidlines extend the
core rules for types of systems, such as:
•
Informational websites
•
Interactive web features
•
ePoll books
•
Voting systems
•
Election management systems
Training
Testing and
Evaluation
Samples &
Examples
Voter
Scenarios
Voter scenarios illustrate the guidelines in
action, helping meet the goals.
Samples show design and code best
practices.
Testing and evaluation methods inform the
design of systems.
Training supports those new to the field and
continued learning.
Monitoring and feedback in use
Continued monitoring and feedback (from
the formal to informal) allows regular review
of both the goals and the guidance.
Preliminary notes from a workshop held by NIST focusing on developing guidance for improving usability and accessibility in next-generation voting systems. The workshop aimed to identify gaps in research, explore new ideas, and outline steps to ensure all voters can vote independently. Participants included key stakeholders from various organizations. NIST has been working on voting system standards since 2002, with the need for updated guidance highlighted due to evolving technologies and public expectations in elections.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Usability & accessibility of next generation elections NIST Roadmap Notes from the October 8-9, 2014 workshop Whitney Quesenbery and Dana Chisnell Center for Civic Design Kathryn Summers and graduate students University of Baltimore Sharon Laskowski and Shane Dawkins NIST 1 | Notes from NIST Usability and Accessibility Roadmap Workshop
A preliminary report on the workshop, Oct. 8-9, 2014 This was the first of two workshops in a process to create a roadmap for developing usability and accessibility guidance, best practices, and standards for next generation voting systems that will help election officials, manufacturers, and other stakeholders to ensure that all voters can vote independently and privately. The roadmap, when completed, will outline steps needed to produce this guidance for election officials, manufacturers, and other stakeholders. It will identify issues, gaps, new technology, and processes, how to develop guidance, as well as relevant research and best practices that can be used to improve voting systems given next generation technology. In this first workshop, we: Explored uses of current and future technology in elections, Identified gaps in the research, and Brainstormed new ideas to develop useful guidance. The goal of these activities was to identify the topics that new guidance must consider and explore the issues that shape current thought on these topics. 2 | Notes from NIST Usability and Accessibility Roadmap Workshop
Workshop participants Andrew Baranak, GTRI David Bjerke, Falls Church, VA Stephen Blosser, MSU RCPD Steven Booth, NFB Mike Byrne Rice University McDermott Coutts, Unisyn Voting Solutions Jim Dickson Nat'l Council on Ind. Living Jeremy Epstein, NSF Josh Franklin, NIST Bob Giles NJ Division of Elections Thomas Hicks, House Admin. Committee Merle King, Kennesaw Center for Election Systems Ben Long, NIST Christy McCormick U.S. DOJ Alysoun McLaughlin, Montgomery County, MD Whitney May, ELECTricity Tammy Patrick, Bipartisan Policy Committee Sarah Swierenga MSU UARC EAC Megan Dillon Monica Evans Brian Hancock Patrick Leahy Alice Miller Jessica Myers Robin Sargent Brian Whitener University of Baltimore GAs Jaime Lee Kathryn Locke Emily Rhodes Caitlin Rinn Joel Stevenson 3 | Notes from NIST Usability and Accessibility Roadmap Workshop
Background NIST has worked on voting system standards since the Help America Vote Act of 2002, both establishing requirements for certification test labs and creating the Voluntary Voting System Guidelines (VVSG). The VVSG 2005 included the first comprehensive usability and accessibility standards for voting systems. Elections are changing. There are new technologies, new research, new laws, and new elections procedures since the 2005 Voluntary Voting System Guidelines 1.0 were published. Keeping up with these changes requires a new approach to usability and accessibility guidance for election systems. Recent years have brought changes to the state of the art and technology for voting systems, as well as public expectations about how voters will participate in elections. 4 | Notes from NIST Usability and Accessibility Roadmap Workshop
Background (2) Despite 12 years of work within elections on standards for usability and accessibility, the reality is that there are still many barriers. Even newer systems show poor accessibility and usability, suggesting lack of knowledge of best practices and existing standards and guidelines. This is true of both voting systems and related technology. As more jurisdictions have switched to paper ballots, there is even more isolation of the "accessible" voting system. The accessible systems may go unused through the entire day, further reducing the likelihood that they will be set up and ready to use. Systems for UOCAVA voters under the MOVE Act allow for online ballot marking. Disability rights groups advocate for making these systems available to voters with disabilities (or all voters). Security experts point out many pitfalls. 5 | Notes from NIST Usability and Accessibility Roadmap Workshop
What is a roadmap? A NIST roadmap is an outline for future work. A roadmap: Identifies gaps in knowledge to be filled Identifies issues to be resolved Looks at technology, processes, standards & guidelines Recommend approaches to the work It does not: Prescribe solutions Recommend specific guidelines Rather, it shows how to structure work to accomplish the goals This roadmap will cover future guidance to ensure the usability and accessibility of election systems. 6 | Notes from NIST Usability and Accessibility Roadmap Workshop
Possible goals for the roadmap Increase the level of knowledge for how to design, develop, deploy, and use of usable and accessible elections systems. Promote consistent levels of usability and accessibility across technology in all parts of the elections process. Make systems more usable for everyone in the elections process, including voters, poll workers, elections staff, and third- parties like election interest and advocacy groups or technology developers. Shift from single focus on standards and certification to identifying the appropriate guidance and how to implement the guidance, including: Guidelines for best practices Procedural support Training 7 | Notes from NIST Usability and Accessibility Roadmap Workshop
About the workshop The group started from this focus question: What will the voter experience of elections be like in the future? Through a KJ* activity, the group identified 4 priority areas for breakout topics: Convenience voting and "Vote Anywhere" Accessibility and universal usability Trust, security and verification Design and evaluation of the user interface The groups rotated through the breakout topics during the first afternoon. * See How to KJ: Setting Priorities Quickly http://uxpamagazine.org/how-to-kj/ 8 | Notes from NIST Usability and Accessibility Roadmap Workshop
About the workshop (2) The discussions of the focus topics identified: Current and possible future scenarios for usable and accessible elections Conditions required for these scenarios for future elections Strategies for supporting voters in navigating across the voter journey Conditions, challenges, or limits that could constrain these scenarios This resulted in higher quality notes than just listing issues in each category. However, it can also be hard to capture free- ranging discussion, so the outcomes of these discussions are really input to a more structured discussion in the second workshop. 9 | Notes from NIST Usability and Accessibility Roadmap Workshop
The voter journey (1) As the group worked on the 4 priority topics, we used portraits of voters (called 'personas') as a reminder of the range of people who use election systems. We also looked at the user experience across the entire process of voting, not just marking and casting a ballot. The goal was to be able to think about the context in which the systems are used, not just the equipment. A simple timeline helped organized the notes by both stages in the voter journey and type of notes. Getting the results Marking the ballot Preparing to vote Getting to "the polls" Casting the ballot Choosing how to vote Blue sky / future tech ideas Problems, gaps and opportunities z Promising resources Other Notes 10 | Notes from NIST Usability and Accessibility Roadmap Workshop
The voter journey (2) The election process and voter journey is a useful organizing principle, to ensure that the scope is grounded in the voter's process rather than specific technology. Elections are a service design, requiring coordination of people, procedures, policy, information, and systems. The voter journey includes learning and making choices as well as the core activities of participation. There is a wide array of technology systems in use. They include specialized systems, systems used throughout the journey, and general systems used as part of elections Technology and procedures are used in the context of the voter journey. Guidance for them must take context into account. Inclusion of a process or technology on the voter journey map does not mean that NIST or the EAC will automatically write standards for it. The next slide shows a summary of the notes collected on the journey map during the workshop. 11 | Notes from NIST Usability and Accessibility Roadmap Workshop
The Voter Journey Preparing to vote Choosing how to vote Checking in/ getting ballot Marking the ballot Casting the ballot Verification & results Blue Sky Ideas SMS and texting for voting Elections know me UberVote (car service) Designed and built by states and voting jurisdictions Paper based system then PDF'd and goes to the cloud One "time zone" for elections Use the cloud Absentee voters can vote anywhere Vote from home Virtual voter representative the "knows" voter configuration A well-designed ballot should be shared via internet or by NIST Take picture of your ballot and mail or upload it Vote on a smartphone Pushing ballot to people the way they want Vote on home PC Build in "negotiation" vs. using personalized config and PII Photos of candidates on ballots Audio version of the ballot Online ballot marking tool and backup support Overall Thoughts Common interactions patterns layout template (so good, you'd be foolish not to use it) Preferences for voters match to choices (OK Cupid) Plain language Universal ID Universal WIFI for all devices A pilot project with small elections would usability test a ballot design Your choice is your choice, regardless of disability Ballot designed to work on standard computers Other Notes Votes don't get announced until a specific time Voters need the ability to "rehearse" to prepare My details have changed - how to update All systems must be subject to the same standards - not like current double- standard for paper and DRE Ballot marking saves $? Support phone lines for voters Identify preferences, not abilities Bringing voting (iPad) tech to you Marking/reading anywhere give syou time Personal settings "card" Minimum standards vs. goals Online ballot marking prevents error 12 | Notes from NIST Usability and Accessibility Roadmap Workshop
The Voter Journey (continued) Preparing to vote Choosing how to vote Checking in/ getting ballot Marking the ballot Casting the ballot Verification & results Problems Coercion USPS not reliable Changing for different abilities, aging - like vision QR codes scare people - can we make the machine read real text Trust that the outcomes are as voters voted How to deploy ballots on many devices Long wait times degrade trust and confidence Supporting voters with invisible disabilities Technology is a problem not a full solution Tasks for voter self- audie in conflict with tools for preserving privacy/vote selling I don't have a smartphone Overall Thoughts How to print out forms from online Current laws do not allow for voting anywhere Time Poll workers and procedural security Compromising privacy if auditing Voting systems can't touch internet One system for everyone? How to match the right person to the right ballot Racial and social issues with pictures on ballots Big brother issues, including PII that might help voters Many kinds of voters Costs: universal design is expensive to implement Voters not educated enough, lack of access in general Options can be a burden Trust in the system to count as cast Trust in poll workers until something goes wrong Other Notes Votes don't get announced until a specific time Voters need the ability to "rehearse" to prepare My details have changed - how to update All systems must be subject to the same standards - not like current double- standard for paper and DRE Ballot marking saves $? Support phone lines for voters Identify preferences, not abilities Bringing voting (iPad) tech to you Marking/reading anywhere give syou time Personal settings "card" Minimum standards vs. goals Online ballot marking prevents error 13 | Notes from NIST Usability and Accessibility Roadmap Workshop
The Voter Journey (technology) Preparing to vote Choosing how to vote Checking in/ getting ballot Marking the ballot Casting the ballot Verification & results Official Elections Web Sites and Social Media (Local and State) How to vote info Eligibility info Voter ID requirements Marking instructions cCasting instructions Election results Hours and dates Sample ballots/voter ballot info Ballot delivery options Voting locations Accessibiity info Language info Online voter registration Interactive data Polling place lookup Blank ballot access and delivery Online ballot marking tools Ballot printing and return tool 'My Voter' portals E2E verification VBM request Technology in the Polling Place Ballot marking tools Electronic ballot casting Pollbooks Ballot readers Ballot scanners Ballot activators Ballot printers Ballot readers for review Phone/Voice Assistive technology Voter's Technology Computer/Mobile Computer/Mobile Computer/Mobile Input/Output AT Computer/Mobile Computer/Mobile Social Media Social Media Passbook/Wallet (ID) Input/Output AT Social Media GPS Other Organizations (Campaigns, Advocates, Good Government...) Third party registration apps Citizen ballot review Apps built on public information ?? Blank ballot access and delivery ?? ?? Online ballot marking tools ?? Support tools for returning ballots E2E verification VIP-type information app Personal AT Personal AT 14 | Notes from NIST Usability and Accessibility Roadmap Workshop
Resources identified in the workshop EML or Common Data Format allows multiple devices to share election data. Critical for component architectures. QR codes, NFC, Hollarith grids, or other tokens ways to transport ballot choices efficiently and privately GPII or other preferences manager allows system to match needs and preferences to options available. Identification: Biometrics (eyescan, fingerprint) or a secure national ID (like miltary CAC), two factor authentication, Disney Fast Pass Other certification programs: FDA, slot machines, banking audits Related working groups: NASED, Bipartisan Policy Center, State certification group, FVAP, EAC Research centers at Rice, MIT, Caltech, Georgia Tech, MSU, U. Baltimore, GPII Election initiatives: Humboldt County ballot project, VSAP, Star Vote, risk limiting audits 15 | Notes from NIST Usability and Accessibility Roadmap Workshop
Blue sky? Or fundamentals? Many of the blue sky ideas are based on some fundamental concepts. All of them are in use in some current context the "blue sky" part of the idea is how to apply them to elections effectively and consistently. Vote anywhere, with options for casting Well-designed ballot (and other) so good, you'd be foolish not to use it More use of COTS In voting systems Enabling use of voters' own systems A way to transfer information easily between parts of the system, different devices, and person technology Universal ID Easy personalization either quickly set up, or recognized from ID/token Plain language 16 | Notes from NIST Usability and Accessibility Roadmap Workshop
The Voter Journey (a possible structure) Preparing to vote Choosing how to vote Checking in/ getting ballot Marking the ballot Casting the ballot Verification & results What is on the ballot? Where do I go to vote Who won? Am I/How do I register? How do I get my ballot How do I mark as I intend? What are my choices? How do I cast my ballot? Did my vote count? Learn Access to 'Polling Place' Review the ballot See election results Receive 'ballot' Mark the ballot Register to vote Request a VBM (or other) Authenticate/Si gn In Verify ballot was received Activate or open the ballot Cast the ballot Do Registration Forms/OVR Pollbook or Sign-in Online VBM SYstem Pre-Marked Ballot E2E Verification System Mail Ballot Return "My Voter" Portals Ballot Delivery System Ballot Marking System Electronic Casting "My Voter" Portals VBM/Ballot Tracking Elections Web/Phone Elections Web/Phone Transportation to Polls Elections Web/Phone Ballot Use Ballot Scanner Registrar People Elections office Poll workers Poll workers Poll workers Elections office Voter Ed Sample Ballots Ballot Access Voting Options Voter ID Counting Rules Eligibility Policy Canvass Helper Rules Provisional Helper Rules Hours/Places 17 | Notes from NIST Usability and Accessibility Roadmap Workshop
Notes on the priority topic discussions Trust, security and verification Convenience in voting and "Vote Anywhere" Accessibility and universal usability Design and evaluation of the user interface 18 | Notes from NIST Usability and Accessibility Roadmap Workshop
Trust, security, and verification 19 | Notes from NIST Usability and Accessibility Roadmap Workshop
Priority topic: Trust, security, and verification The discussion of trust, security, and verification mapped trust as an element in elections: People Procedures and processes Systems Policy and political issues Polling places (and voting outside of them) Information 20 | Notes from NIST Usability and Accessibility Roadmap Workshop
Trust is based on each individual's perception of the overall process Rule makers Congress/ Dept of Justice State/Local Officials Your vote counts, no matter how you vote. Reliable Transparent Accurate Usable Accessible Systems approvers EAC/Certification labs Manufacturers How does voting compare to other systems? Banking Motor Vehicles Financial Aid forms IRS US Postal Service The system is good. (Trust until something bad goes wrong.) Elections are transparent and understandable. Election participants Local Election Office Poll workers Helpers Voters Consistent (Standards) Elections are convenient. Systems Policy/Political People TRUST Procedures Information Polling Place Easier to vote: Harder to cheat Knowing your choices Access - Transportation - Accessibility Voter registration is accurate (your registration is actually recorded) Knowing where to go (in person or online) and how to get there (transportation or digitally Voters who should vote, can. And who should not, don't. Orderly polling place - Unknown area - Long lines/wait time Knowing the procedures Ballot marking, recording, tallying, auditing, reporting are correct and accurate Knowing how to use technology Trustworthy people - Willing to reveal disability? - Give accurate information? - Give unbiased assistance? All procedural voting choices are equal 21 | Notes from NIST Usability and Accessibility Roadmap Workshop
Trust in the Voter Journey Preparing to vote Choosing how to vote Checking in/ getting ballot Marking the ballot Casting the ballot Verification & results All choices are equal and votes count equally Voter registration was it actually completed? Is it trustworthy? Will my vote be cast as I intend? Recording is complete Authentication: Trust in who is voting Trust those who sholdn't vote, don't Does it do what it is supposed to do? Tally procedures are good. Is the information official? Will it be counted as cast? Vote centers are backup for VoteByMail Does it break down? Auditing/reporting is correct Is the ballot accurate and complete? Which ballot counts? (Last? First?) Signature verifiation is fundamental to current laws & procedures Can I track my VBM ballot? Do trends ike vote early, then change our minds suggest giving up privacy? Election officials trust in voters Challenges: - Unknown area - Long lines - Wait time - Lack of information - Use of tech/Internet Blue Sky Ideas and Resources Live pilots for testing as part of certification see it in use. Identity: Every ballot gets scanned to the cloud: Humboldt project Witnesses for vouching for identity Eliminate the secret ballot Use COTS scanning devices (like grocery scanner) Risk limiting audits Voting equivalent to direct deposit. 22 | Notes from NIST Usability and Accessibility Roadmap Workshop
Questions to answer for better trust How do we decide which systems or people to trust, and who has to trust them? Every trust change ends with a human being. Small problems add up to decreased trust. How do we improve trust in the political and social aspecsts of elections? What part do election procedures play in trusting elections. Is privacy sacred? Are we moving towards eliminating the secret ballot? Many online options are difficult because of the identifiation issue How do we deal with security problems? There are time boundaries in elections when issues can be addressed. Elections rarely allow a "do-over." 23 | Notes from NIST Usability and Accessibility Roadmap Workshop
Convenience in voting 24 | Notes from NIST Usability and Accessibility Roadmap Workshop
Priority topic: Convenience in voting The discussion of convenience in voting and the ability to "vote anywhere" covered a wide range of issues, and what "convenience" means in this context. One answer was to allow more personal choice, including: When to vote Where to vote What systems or assistance to use 25 | Notes from NIST Usability and Accessibility Roadmap Workshop
Voting should be the most convenient government service as voting is a right and not just a privilege. Voters with disabilities use "convenience voting" options more than the general population. These options include: Early voting centers Mobile early voting vans Vote by mail and online ballot marking More convenient voting can expand and improve procedures and equipment already in use Allow use of personal technology to mark ballots Allow more flexibility in where and how to vote Use online tools to mark and cast ballots 26 | Notes from NIST Usability and Accessibility Roadmap Workshop
Convenience in the Voter Journey Preparing to vote Choosing how to vote Checking in/ getting ballot Marking the ballot Casting the ballot Verification & results Expand and improve procedures and systems already in use Allow for multiple ways to vote (remote and in- person) to accommodate personal preferences and needs Improve voters' access to resources and information: Hours, dates, locations Information about candidates and measures Finding the closest polling place. Allow voting at any polling place (near work, near hoem, etc) Have voting systems support all types of personal assistive technology. Improve convenience through technolgy Scan a QR code or other token on a pre-marked ballot Upload a picture of a marked ballot from a smart phone Bring the polling place to those who can't get there Long term care facilities Shut-ins Disasters Allow voters to use their own systems at the polling place Allow voters to mark their ballot online Blue Sky Ideas "Uber Vote" a car service that could bring you to the polling place Voter Support Lines to help if those who have difficulty marking/casting a ballot online. SMS-based voter information for basics like election dates, hours, polling place and early voting locations This service could also extend curb-side voting to longer distances. Vote by Phone to allow voters to vote anywhere at any time. Interactive app that uses voice search (like Siri) to let voters ask for information about their ballot of how to vote Vote via SMS support voters who don't have smartphones. Polling place child care Biometics (eyescan, fingerprint) for voter identification GPII or other ID that carries setup preferences 27 | Notes from NIST Usability and Accessibility Roadmap Workshop
Tensions to resolve in increased convenience How do we increase convenience without sacrificing the voter's privacy or security? Remote voting may not be private, or may be coerced. Serious security concerns for casting a ballot online. How do we ensure that voters are provided with the resources and support they need to vote from anywhere? Do we need a better organization of voter outreach and support? Especially support for using assistive technology Is there a conflict between personalization and equality of experience for all? Do all voters have equal access to choice and personalization? What is the impact of the digital divide in what kinds of personal technology (like mobile devices) people own? How to we address differences in assistive technology? 28 | Notes from NIST Usability and Accessibility Roadmap Workshop
Accessibility and Universal Usability 29 | Notes from NIST Usability and Accessibility Roadmap Workshop
Priority topic: Accessibility and Universal Usability This topic overlapped with the discussion of convenience in voting. How far can the goal of universal usability work when technology changes are inevitable? How can personalization support voters in creating a more usable and accessible voting experience? How can we use systems and interfaces that voters have already tailored for their own use? 30 | Notes from NIST Usability and Accessibility Roadmap Workshop
Reaching universal design is a challenge when there are so many different voter needs. Allow more personalization for individual needs and preferences. The digital divide is a real issue: some voters do not have smartphone Access to information, resources not just voting. Mobile devices are already in use for notes to prepare for voting Optimal usability is an important step Stop creating a separate machine for people with disabilities We need to address e-casting, not just e-marking. Paper ballots introduce errors, are not environmentally sound, are not ADA compliant. Why can't we accept electronically cast ballots that we count as a separate "stream" like we do with overseas FWABS (minimize audit/pollworker complications) 31 | Notes from NIST Usability and Accessibility Roadmap Workshop
Universal Usability in the Voter Journey (close-up view) Preparing to vote Choosing how to vote Checking in/ getting ballot Marking the ballot Casting the ballot Verification & results Ensure equal access to all parts of the voter journey for all, even without personalization or individual technology Resources Existing standards WCAG 2.0, Section 508 NFC or QR codes NFC or QR codes Separate "stream" for electronically cast ballots Mobile phones are important as a way to bring a marked sample ballot to the polling place (but not always allowed) Need to include technology like Braille [preferred by some, critical for deaf-blind) 32 | Notes from NIST Usability and Accessibility Roadmap Workshop
Thoughts about universal usability Can standards and solutions for voting expand to all government related interactions? Capturing preferences could extend to all interactions Consistency is more important for AT users and those with challenges Can one size fit all? We have to consider voters who arrive at the polls with no AT Should the system be modular with alternatives for different needs? Can the system be flexible with different interaction options? How do we deal with differences between jurisdictions? Every state has different rules what is the common denominator? 33 | Notes from NIST Usability and Accessibility Roadmap Workshop
Design and Evaluation 34 | Notes from NIST Usability and Accessibility Roadmap Workshop
Priority topic: Design and evaluation Discussion of design and evaluation included: Design and development processes that encourage good usability and accessibility. Ways to write guidance for best practices, standards, and test methods. Different test approaches and how they might fit into the certification process. 35 | Notes from NIST Usability and Accessibility Roadmap Workshop
Design and Evaluations supports the voter journey Writing Guidance Testing Approaches Certification Process Challenges Resources US Constitution Existing standards WCAG 2.0, Section 508 IEEE common data format for reporting/log files Benefits of certification: license to use Guiding principles that reach for an outcome Testing with hardest, not easiest, users EAC and NIST limited to voting systems. WIll the EAC be able to change directions? Procurement intersects with certification to force procedures and decisionmaking Usability standards based on efficiency, effectiveness, satisfaction Create a voter expo for combined testing days to make it easier to assemble a large and diverse group of voters as test participants. Standards are too absolute A "testing standard" has become a "design standard" High level vs. detailed requirements Cost of iterative design/feedback loop+ certifiction What technique tells you what Industry standards Slot machine certifcation Bank audits FDA FAA Declaration of Conformance EAC doing federal and state certification at the same time Piloting at different levels Classify voters not systems Better feedback on the outcomes of systems in use Practice voting and testing constantly Consumer reports for voting sysems Vendor review website (Yelp for voting systems) Common/shared user data to create better voting systems Involve a broad group of election officials in generating standards Design iterations and testing Attractive and simple ballot that is used by everyone Committees NASED State Certification Group Until voters actually vote on the machine, there is not way to know it will work We test voting systems, but not other parts of the election system 2018/2020 and the impending crisis of out-of-date systems 36 | Notes from NIST Usability and Accessibility Roadmap Workshop
A concept for useful guidance in the right form Continued monitoring and feedback (from the formal to informal) allows regular review of both the goals and the guidance. Monitoring and feedback in use Voter scenarios illustrate the guidelines in action, helping meet the goals. Samples show design and code best practices. Testing and Evaluation Voter Scenarios Samples & Examples Training Testing and evaluation methods inform the design of systems. Training supports those new to the field and continued learning. System-Specific Guidelines by type of election system The system-specific guidlines extend the core rules for types of systems, such as: Informational websites Interactive web features ePoll books Voting systems Election management systems Test Core Usability & Accessibility Knowledge and Guidance The Core Requirements are testable usability basics that apply to any interactive system. Methods Clear statments of goals help everyone understand the reason for any requirement or guidelines. Goals 37 | Notes from NIST Usability and Accessibility Roadmap Workshop
