Observations on the Avante VOTE-TRAKKER™ Test

Wilton, Connecticut
November 4, 2003

Authors:
Marilyn Hutfilz Deputy Registrar of Voters, Wilton, CT, Moderator, Test Voting District
George Hutfilz Deputy Registrar of Voters, Wilton, CT, Assistant Registrar, Test Voting District

Summary

For the November 4, 2003 Local election, Wilton, Connecticut Voting District 3 served as a test site for the Avante VOTE-TRAKKER™ direct recording electronic (DRE) voting system with real-time paper audit trail.

About 3,000 voters are registered in this district. Nearly 1,100 voted on election day, a solid turn out for a local election. Because of software design flaws in the VOTE-TRAKKER™ system, many of which were immediately evident during initial testing and voter outreach demonstrations, a large number of voters problems were anticipated. These problems occurred. But with extensive advance preparation and training for the poll workers and significant additional poll staffing, the problems were managed, and the voters of District 3 exercised their franchise successfully and, we believe, for the most part happily.

We believe the Avante VOTE-TRAKKER™ shows great potential for improving the voting experience. The design goal to make the voter more aware of under voting is good. We question the necessity of the real-time, individual paper audit trail, but it did not cause problems during the voting. Except for two glitches discussed later, the hardware performed flawlessly. Insufficient attention has been paid by the company to human factors and ease of use in the software design; when the issues outlined below are addressed, we believe the system will be very effective.

Voter Outreach in Advance of the Election and on Election Day

We personally conducted two, two hour long demonstration sessions at the Wilton Library in the two week period before the election. The first, from noon until 2:00 p.m., had a continuous stream of visitors totaling about 45 for the session. The second, from 5:00 p.m. until 7:00 p.m., was sparsely attended; about 15 persons visited. Nearly all had come based on the invitation sent by the Registrars to all eligible voters in District 3, which invited them to the Library sessions or to drop by the office to see a demonstration of the test machines.

Reception to the machines was mixed. A few persons were comfortable with the technology and came simply to see it and have a chance to discuss it. Most of those who chose to visit were quite apprehensive about their own abilities to deal with the new technologies. Most of these people left feeling, if not completely comfortable, that they could survive on Election Day.

We also participated in demonstrations to the Wilton Republican Town Committee and to the Wilton Kiwanis Club before and after its luncheon meeting. Reactions from these two groups were also mixed.

Preparations Before Election Day

Set-Up the Night Before

Setting up the polling place took considerably longer than for the lever machines. The school custodial staff and the lever machine mechanics did the room set-up. Considerable extra furniture was needed to be moved into the gym for the voting including the ten tables on which the voting machines were placed.

Avante personnel handled all the physical set-up of the machines the night before including the UPSes and the stringing and securing of the electrical connections with hazard tape. Two demonstration machines with abbreviated practice ballots were prominently displayed just inside the entrance to the polling place along with sample ballots for the next days' voting prepared from screen shots of the twelve offices being contested. A continuous demonstration video was placed just outside the polling place entrance.

Great attention was paid to traffic patterns and crowd control in anticipation of problems on voting day, but there were no issues with crowd control during the voting. This attention to detail proved valuable when the press showed up in force; it was simple to set boundaries and provide the press access without interfering with the voters.

Walking Through the Day

Before the Polls Opened

The process of opening the machines took approximately four minutes per machine. By the time the polls were opened at 6:00 a.m., five machines were ready for voters, and this number was adequate to serve the initial voter stream without any voters queued for the machines. After the remaining five machines were opened, completion of all the opening paper work required approximately another forty minutes. This process was complicated and extended by the refusal of the Secretary of the State to redesign and adapt the official Moderator's Return for use with the electronic machines. Two full Moderator's Returns had to be completed for the district to record all ten machines, and duplicative paperwork was prepared between these lever-oriented state forms and the reports printed by the Avante system.

The polls could not have been opened successfully without the presence of our two additional Assistant Registrars. The normal complement of a Moderator and Assistant Registrar would have been overwhelmed.

During Voting Hours

Normally three old, familiar lever machines would have been available for voters with another one or two as back-up; for the test, ten VOTE-TRAKKER™ stations were available including one equipped with audio capability for the visually challenged. The audio capability was not used; all visually challenged voters who presented themselves at the polls chose to be assisted by another person rather than use the independent voting capabilities of the audio equipped machine. During the day one station was taken out of service because of a jam in the card reader. Only during brief peak times were all ten machines open for voting; normally only seven were in use. The queue of persons waiting to vote never exceeded twelve except during a brief twelve minute downtime of the machine encoding voter ballot activation cards.

The demonstration video was run all day, but the quality was poor to begin with and steadily degraded during the day. Voters viewing it were encouraged to step inside and view the actual demonstration machines before signing in to vote.

Most of the day a greeter was positioned at the entrance to encourage voters to familiarize themselves with the demonstration machines. The demonstrators were constantly busy.

Exit surveys were done by the League of Women Voters on behalf of the Wilton Registrars and by the University of CT on behalf of the Secretary of the State in an adjacent gym into which voters were directed after completing their ballots. We have not seen the results of those surveys, but anecdotal comments from the LWV canvassers indicated most people were quite satisfied with the experience.

Continuing confusion about polling places because of the recent re-districting and the fact that everyone votes at Middlebrook in the May budget referendum caused us to re-direct a large number of voters to other polling places. This continues to be an additional resource drain at this polling place, and it probably always will be. Only one voter who showed up in the last half hour before the polls closed was in the wrong place. We had an official placed at the outside door from 7:30 p.m. until 8:00 p.m. to assure voters they had the right polling place.

A major crises occurred shortly before 10:00 a.m. when the laptop computer used to encode voter cards lost contact with the attached card encoding device. Much scrambling ensued since this happened to be the time when two senior Avante executives were visiting the site. A re-boot of the laptop picked the encoding device back up after approximately a twelve minute outage. Approximately twenty-five voters were waiting for cards by the time the system was brought back up, and another twenty were waiting to check in. Once cards could be issued again, we opened all ten machines for voting, and within twentyfive minutes the backed up voters were all through the process.

Closing the Polls

Thanks to the Registrars recruiting some of our best poll workers to work in the district, the check-in books balanced to the machine public counters on the first try. Two more voters checked in than had voted officially. These were two fleeing voters not caught by our machine attendants whose ballots were ignored; a third fleeing voter was actually brought back (he was waiting for his wife to finish voting), and he was stepped through casting his ballot.

Closing the voting machines was complicated by the same paper work concerns mentioned in preparing Last the machines for voting. Approximately five minutes were required per machine to perform the closing software procedures and publicly to announce the vote recorded by the machine from the printed tally.

Completion of the Moderator's Returns was then done by two Assistant Registrars in parallel with a summarizing of the entire vote from the individual machine tallies performed by Avante. The summary tally was used to double check the numbers on the Moderator's Returns, and an error was found in the announced vote that was corrected.

Anything times ten rather than times three makes a big difference in elapsed time.

General Observations

On Reliability of the VOTE-TRAKKER™

The touch screens performed flawlessly throughout the day. Careful attention had been paid to the location of the screens with respect to the gym lighting, and no one complained of glare on the screens.

This was a frequent complaint during demonstrations where the location of the machine with respect to the lighting was not optimal, including in the Registrars' office.

The printers also performed flawlessly. There were a few complaints about the size of the print; magnifiers were available at each machine for voter use, but they were seldom used. A few voters complained it was difficult to see the first line of the paper record; perhaps a little more paper could be wasted at the top to ensure all of the record is visible.

One voting machine was taken out of service after the third time a voter had managed to insert a card so far into the reader slot that it could not be retrieved. The first two times this occurred, the machine was opened under supervision of members of both parties and the card retrieved. On the third occurrence the machine was taken out of service for fear that the card reader, needed to expedite the poll closing process, would be rendered unusable.

An unexplained failure as noted above occurred with the card encoding station. Avante personnel attempted after the fact to blame the poll worker operating the laptop, but we are not convinced that any user action caused the problem. It looked very much like a transient connection failure perhaps caused by a loose or slightly damaged cable or a bent pin.

On Ease of Use of the VOTE-TRAKKER™

In a word, the ease of use and human factors of the VOTE-TRAKKER ™ are appalling. Numerous issues have been apparent since the very first day we saw the system in Hartford, and these and others less obvious caused numerous problems for the voters and placed great stress on the poll workers acting as machine tenders. We had hoped that a machine tender might cover two machines, but this proved impossible. Each voter needed to be carefully monitored and most had to be coached to complete voting successfully.

We have characterized the following ease of use issues as critical, severe, and important. The interface is so poor that identifying things that would be nice to have is a wasted effort.

1. Pressing “Cast Ballot” twice to commit vote.
(Critical) To vote successfully, the user, on the review screen, must press the “Cast Ballot” button twice. The first time is obvious. The review screen is displayed, the user has completed his or her review, and the cast ballot button is the obvious choice. The instruction to press the button a second time, however, is contained in a non-obvious change to the direction line at the top of the screen. In Hartford and again in Wilton at poll worker training, we asked that this confirming directional message be converted to a pop-up window like the familiar commit prompts when doing actions such as file deletes in Windows. This request was ignored. The largest source of voter failures to complete ballots without intervention or coaching occurred because of this design flaw.

2. The “Skip Contest” button. (Critical)
To describe the wording of this button as misleading is generous. Again, on day one in Hartford, we stated that this button, which really means “Skip A Selection” would be a huge problem. Events proved us correct. Numerous voters timed out on the second ballot screen for Selectman where two votes were permitted. Voters wishing to vote for only one of the five possible candidates were convinced that pressing “Skip Contest” would erase the vote entered for their choice for that particular ballot page.

This problem was greatly exacerbated because only one race, the second on the ballot, was contested. Supporters of one of the candidates in this race had strongly encouraged voters to “bullet” vote the election, that is, to cast only a single vote on the entire ballot for their candidate, leaving all the others blank. This lead to a second issue for the voters, that in order to skip all the selections in a particular contest, they had to press “Skip Contest” multiple times. Talk about non-intuitive and user hostile!

And these intentional under voters were very unhappy about how long it took them to vote their one vote ballot. From the number of under votes apparent in the tallies, lots of voters managed to “bullet vote,” but the pain level for voters and especially machine tenders was very high.

3. Short timeout settings. (Critical)
Large numbers of voters timed out while reviewing their choices before casting their ballot. Part of this may have been caused by the poor design of the review screen identified below as an important issue. However, because of the large number of contests, the length of the review screen necessitated scrolling, and non-computer literate people took some time, usually too much, to figure out how to make scrolling work. Unfortunately, for the demonstrations we had shortened the sample ballot used, and thus the demonstrations did not show scrolling.

For all but a couple voters, timeouts at the review screen were not show stoppers. The voters were confident that they were prepared to cast their ballot, and, using the poll workers' “fleeing voter” Last Revised: intervention, we were able to allow them to cast their ballot.

A second large number of voters timed out on the second contest for Selectman. The reason for this was the confusion over the “Skip Contest” button covered in item 2 above. These voters chose to start the ballot process over again after reminder tutorials from the poll workers.

A third large number of voters timed out while reviewing the paper record and saw it disappear before their very eyes after being told on the screen to review it, then deposit it. For most voters, there simply was not adequate time to do the review. Since the voter's focus was on the printed record and not on the screen, few noticed that the screen had changed slightly asking whether more time was required for review. While there is no easy solution to the shift of focus issue, a pop-up window asking the voter if more time is needed would be more eye-catching.

In general, time outs of any kind, when the system is just being introduced, are a very bad idea. Attentive machine tenders ready to coach the voters is much to be preferred. Let the voter succeed, no matter how long it takes, rather than be confronted as a failure by a time out requiring intervention by a poll official.

4. Poor instruction areas on each screen (Severe)
The instruction areas at the top of the ballot screen are poorly done. Numerous problems were encountered. The most significant is that the instructions are not set off so as to make them obvious as instructions; the colorful candidate bars grab and hold the voter's attention, and he or she is then at loss as to what to do. A second problem is that the font size is small, and it does not change when the voter adjusts the character size on the initial screen. A third is the flashing direction line; conditioned by the flashing numbers on their VCRs and clocks when the power fails, voters were convinced that the flashing line indicated something had gone wrong. A fourth problem was the color choice for the flashing direction line; a significant number of people could not see the bright green line as anything but the faintest gray and were again convinced that they had erred or that the machine was dying.

5. Order of the first two screens. (Severe)
Having just been handed a card to activate the voting machine, the voters approached the machine to find no instructions on what to do with the card. Asking for the card on the second screen after the voter has pressed “Press Here to Begin Voting” on the first screen is poor design.

These screens should be reversed. The first thing the user should see on approaching the machine is what to do with the card. Then the user can be presented with a welcome screen that invites him or her to begin voting after tuning the screen display to his or her satisfaction.

6. Lack of status information for the machine tender. (Severe)
This issue was also identified at the first meeting in Hartford. The machine presents no information to the machine tender about the status of the machine, whether it is in use or available for another voter. To manage down the number of fleeing voters, it was necessary for the machine tenders to listen carefully for the printing of the paper record and for its deposit into the lock box to be sure the voter had completed the voting sequence. Fortunately in our large gym with machines widely separated and ambient noise levels low, this was possible.

Each machine should have a status light or lights indicating when it is in use, that is, when an part of the voting sequence is in process or when it is available for another voter. This addition would make viable supporting multiple machines with a single machine tender once voters have gained experience with the system. Fleeing voters could be readily intercepted and invited to complete their ballots.

7. User interface of the encoding station (Important for this election, critical if its feature set had been used—see below)
The user interface of the encoding station exposes the poll worker to a full-blown Windows application interface. This is unacceptable. The encoder user interface should only present the poll official the minimum information required to complete his or her task successfully. In practice at this election, we were able to work around this problem since only one ballot choice was available, and the encoding official could keep all but one finger away from the keyboard and press the “enter” key multiple times to encode a card.

We observe that if the feature set of the encoding station were in use (providing multiple ballot types for different languages and/or voting jurisdictions), this station and its operator would become the critical point of failure in the entire system. Wrongly encoded voter cards would become the bete noire of the election process. There are two important issues when the voter is given a card encoded with the incorrect ballot identifier. First, if the voter realizes the error while voting, he or she is immediately dissatisfied with the election process and will be compelled to start over with a correctly encoded card. Second, and more importantly, if the voter does not realize or report a ballot is in error, he or she will be unknowingly or knowingly voting for candidates for whom he or she is not eligible to vote. This is an election integrity show stopper.

Human error cannot be eliminated at this station, but it could be mitigated by providing better information at the voting machine. If the order of the first two screens were reversed (see the discussion above), then the voter could be presented on the second screen with information about his or her ballot's identification in plain English. And this welcome screen could be presented in the speaker's language of choice if that is not English. This would allow the voter an opportunity to verify the correctness of the ballot to be presented before beginning to vote.

This issue has the potential to be so critical we would recommend that, if multiple encoding choices for the activation card are to be used, a second station with the capability to read non-destructively the activation card be established where the ballot type can be verified with the voter's desires and eligibility before they approach the voting machines.

A more costly but safer alternative at a shared voting facility would be to segregate voter check-in and card encoding so that the feature to select a ballot would only identify language, not the ballot jurisdiction. The user interface should only present the choices available to the poll worker, not all possibilities generated for the election. In this case only the language might be wrong, and the verification station suggested above need not be installed since ballot integrity is not an issue.

8. Design of the review screen. (Important)
The presentation of the review screen was very confusing. In its current form, it is completely oriented to a federal election in which there is a single vote for a given contest. In our local election, where two, three, or as many as four votes could be cast for an contest, the stacking of voter choices two up where the first selection for an contest often appeared in the right hand column was difficult to decipher.

Voters would have been much better served by a review screen that had one or more rows for each contest where the first selection for the contest appears in the left hand column. While this would necessitate more scrolling, it would have eliminated many time outs.

9. Use of the review screen to change selections. (Important)
Voters got in trouble using the review screen to change selections not realizing that the selection they pressed on the review screen would be replaced by the first selection pressed when the contest screen was re-presented. Voters ended up in a ping-pong match between a contest screen and the review screen trying to figure out why their desired selections were being replaced.

A better design would allow the voter to press any selection on the review screen for a contest to go back to the ballot screen with choices displayed, make appropriate changes, then press a button to return to the review screen.

In general, a frequent source of user confusion was the “automatic” transitions being done on behalf of the user. From the questions asked of the machine tenders, many users seemingly would have preferred to manage their own transitions from screen to screen and to manage their own state in the voting process.

10. Design of the card disposition screen. (Important)
The card disposition screen would be much more effective if animated. Also, the choice of words is poor. The suggestion to “insert” the card in the slot caused many users to either try to press it in too far or quickly to insert and withdraw it like they would swiping a credit card at the gas pump or the grocery store. Asking the user to “drop” the card into the slot would have raised success rates and lowered blood pressures considerably.

11. Location of the public and protective counters. (Important)
The public and protective counters are on the wrong side of the machine. There is no reason for the voter to have access to them. They should be placed on the back side of the machine visible to the machine tender and other polling officials and in proximity to the machine serial number. This would allow observation of the number of ballots cast while the machines are in use. During busy periods, it took ten minutes or more to canvass voting totals to date because it was necessary to wait until each machine in turn was not in use to read the counters.

12. Integrity of the paper trail. (Important)
The original design of the printer and case, used on one of the demonstration machines, was unacceptable. The split viewing window seriously impaired readability of the paper record and presented a ready-made failure point for jams. The newer printer design and case performed perfectly during the voting; however there is an issue with the integrity of the so-called lock box.

On the new printers, there is a gap of about ½ inch at the bottom front of the lock box. When the cover is off the printer, paper records may be removed from the lock box by simply tilting the machine forward and gently shaking. The heavier, stiffer paper makes this simple. This puts to a lie the supposed integrity of the lock box. The keyed lock box access door with its protective seal at the back of the machine is rendered moot by this flaw. Anyone with the simple, “luggage lock” key for the printer covers can manipulate the paper records.

Fortunately there were no printer failures during the day necessitating removing the covers; otherwise voter attention would have been drawn to this flaw.

On Experiences of the Poll Workers

The over staffing of the polling place was absolutely essential to the success of this election as was the assignment of the town's most capable and most experienced poll workers to the district for this test. All of the workers seemed to have enjoyed the experience and were proud to be a part of the experiment.

The time investment made in advance to provide machine tenders with a brief checklist to ensue voters completed the voting process paid off in the low number of only two fleeing voters. While this was two more than we generally had with the lever machines, it was clearly acceptable and well below our fears going into the election.

The provision of a list of Frequently Asked Questions (FAQs) (and answers, of course) to the machine tenders was highly successful. The questions and answers were derived from our experience conducting demonstrations. Senior polling officials were infrequently called in to assist with voter questions. This was good since the human factors problems with the machines cited above kept the authorized officials (us) busy intervening to recover from time outs.

We were blessed with volunteers from the machine tender ranks to run the demonstrations in the morning who did a spectacular job. In the afternoon when we were short machine tenders, the assistant registrars and Avante support team chipped in to man the demonstrations.

The toll on the senior poll officials was high. The four of us had very little respite and were on our feet almost constantly from 5:00 am until nearly 10:00 pm.

On Experiences of the Voters (as we observed them)

The vast majority of voters seemed to have an acceptable experience. A large minority required some coaching to navigate the voting process successfully. A larger number of voters than usual either brought someone to assist them in voting or requested assistance from one of the poll officials.

Many elderly voters were very apprehensive about the change in the technology, but nearly all managed to complete the process with coaching. In retrospect, we should have had an “I Survived the New Voting Machine” sticker available for these stalwarts.

There was a limited number, perhaps a few dozen, voters who came in with a totally defeatist, antagonistic attitude toward the new machines. To the credit of our staff, these folks were dealt with patiently and charitably and exercised their franchise.

On Support from Avante

Support from Avante was adequate during the trial. We did not see the dedicated support for the entire process promised during the original Hartford meeting with the vendor and state officials, but we all survived. It was extremely disappointing that many of the problems discovered at the initial meeting months before the election were not dealt with and had to be worked around.

During set-up the night before and Election Day, the two on-site Avante persons for the election, Dave Alampi and Joe Tallone, worked tirelessly beside us doing whatever needed to be done for which they were qualified. They were active helpers, not spectators, and we needed their help to keep the lid on.

Conclusions

On balance, we believe this election was every bit as successful as previous ones we have managed. The key to this success was our exhaustive preparation. For diligent support in that effort we have our Registrars and project team from Avante to thank.

However, we worked much harder than in any previous election, as did our two assistants, Jackie Mineo and Dick Martin. Marilyn especially was occupied with managing the press during the day as the trial drew considerable attention. George majored in support for the machine tenders and intervening in support of timed out voters. Between 5:00 a.m. and 10:00 p.m., neither of us was off our feet for more than a total of an hour.

While we touted the security advantages of the standalone machines to those voters who are petrified of hackers, the sequential process of opening and closing ten machines simply took too long. We need better procedures for this, especially in a Presidential election year where probably fifteen machines would be needed in our district to avoid lengthy queues.

With correction of the software design flaws recounted above, the Avante VOTE-TRAKKER™ has the potential to be very successful without the extensive training and frequent interventions required of our team in this trial.

Member Benefits | About AAPD | Join | Disability Resources | News | Contact Us | Calendar | Home