Designing a TouchscreenWeb Browser for People withTremor

Chat Wacharamanothamchat@cs.rwth-aachen.de

Dennis Kehrigdennis.kehrig@rwth-aachen.de

Alexander Mertensa.mertens@iaw.rwth-aachen.de

Christopher Schlickc.schlick@iaw.rwth-aachen.de

Jan Borchersborchers@cs.rwth-aachen.de

RWTH Aachen University52056 Aachen, Germany

Figure 1: A web browser for users with tremor solely operatedby Swabbing: sliding towards a target.

Copyright is held by the author/owner(s). The 3rd Workshop

on Mobile Accessibility, CHI’13, April 28, 2013, Paris, France.

AbstractTypical touchscreen devices are not usable for users with apersistent hand tremor. In this paper, we discuss how theinvoluntary jittering movements of tremor influencesusers’ input on touchscreens. Drawing from these effects,we describe a preliminary design of a touchscreen webbrowser for users with hand tremor.

Author KeywordsTouchscreen, web browser, tremor, accessibility

IntroductionSince a chronic hand tremor is more prevalent in elderlyusers, the global increase of the older population couldlead to more people with tremor. In parallel, increasinglypopular touchscreen devices may lead to a future in whicha significant portion of computers is not accessible tothese users.

In this paper, we describe how tremor limits touchscreeninput and present a preliminary design of a web browserdesigned to within these limitations (Fig. 1). In theworkshop, we plan to share our experience from theon-going longitudinal evaluation of the Swabbing WebBrowser.

Effects of Tremor on Touchscreen UIsThis section summarized how tremor influencedtouchscreen input based on objective measures andobservations from several sessions of user studies. In total,20 participants (age 62–87; 5 female) with intention (12users) or parkinsonian tremor (8) used our custom-madeapplications on an iPad which records users’ input. Theinteractions include tapping and sliding, both by a singlefinger and multiple fingers.

Tapping: The jittering from tremor causes deviations fromthe target and inadvertent lift-and-land movements on thetarget, which results in unintentional activations of thetarget (Fig. 2). Consequently, it is difficult to distinguishdouble- or triple-taps from single taps. Furthermore, theirregular change of tremor frequency poses a difficulty infiltering with standard temporal filters.

10 mm

the center of the target

touch event

1

2

3

4

5

6

1 second

1 2 3 4 5 6 7

7

lift event

Figure 2: A trace from single tapof single finger plotted alongspace and time.

2 cm

0.2 s 0.4 s 0.6 s 0.8 s 1.0 s

Figure 3: The linear regression ofsliding trace stabilizes after 0.4 s.

309

0132639

5164

77

90

103

116129

141154167180193206

219231

244

257

270

283

296

321334347

0%10%20%30%40%50%

Error rate

Figure 4: Error rate of Swabbingfor different sliding angles.

Multiple-finger tapping: Even for able-bodied users, atemporal threshold is needed to group distinctsingle-touch events as one multiple-touch event. For userswith tremor, a multiple-finger tap recognizer needs anadditional spatial threshold to handle jittered andrepeated touches. Therefore, designers should maximizethe difference in number of touches between two differentgestures. E.g., we recommend using one-finger andfive-finger tapping when only two gestures are needed.

Sliding: The jitter from tremor causes fluctuation insliding; panning with tremor would result in a shakyscreen. The jitter also causes the finger to be liftedmomentarily from the screen, causing the user to lose thegrab of the virtual object in focus or accidentally interactwith a nearby object. Fortunately, a general direction ofsliding is rather stable and can be reliably determined witha linear regression. E.g., a trace from sliding in Fig. 3shows a stable direction after 0.4 seconds.

Some participants in our experiment had a problem whenthey slid the finger towards a target on the screen. Theirfinger jittered more intensely as it was closer to the targetdue to anxiety to stop at the target. Therefore, for inputtechniques that depend on the touch trace in the vicinityof the target to activate, e.g., [1, 4], the target has to beconsiderably large.

The performance of sliding is not equal in all directions.Error rate of upward sliding was outstandingly higher thanother directions (Fig. 4, from 1149 trials of sliding).Although the vertical upward motion had a remarkablyhigh error, the nearby angles did not have as high errorrates. We surmise that the user may place her fingerdifferently due to friction or comfort.

Multiple-finger sliding: Inadvertent finger lifting couldcause adjacent traces to exchange their associations withthe fingers. Nevertheless, the number of touches could bereliably determined by analyzing the sequence of touchsignals over a time window.

Swabbing InputSwabbing is an input method that allows the user toselect a target by sliding towards it [2]. The trajectory —calculated by a linear regression of the touch trace — isused to determine the targets, which are positioned alongthe screen edges. Using the trajectory instead of the rawtouch signals allows a more stable target determination.

A lab study showed that Swabbing improves error ratesand user satisfaction in participant with action tremor [3].To investigate Swabbing in a realistic setting, we designeda touchscreen web browser that uses Swabbing as inputmethod.

The visual layout of the web page

is preserved behind the Swabbing overlay.

Each hyperlink is indicated with anarrow with the same color andorientation as the correspondingSwabbing target.

An overlay menu allowing the user to select a target by sliding towards the direction of the target

Figure 5: An overview of an overlay for hyperlink selection.

Design ChallengesWe aim to preserve the visual appearance of the webpages to keep spatial information and the aesthetic intact,because users with hand tremor do not necessary have avisual disability. This goal, however, poses severalchallenges to the design:

Input/output multiplexing over the entire screen: WhereasSwabbing requires the space of the screen to maximizeinput stability, screen space is valuable for the contentdisplay and interactivity. This conflict necessitates aconvenient method to switch between content viewingand Swabbing input.

Densely positioned targets on a web page: A commonexample of this problem is a list of text links on anavigation menu. Since the number of different anglesthat the user can reliably produce is substantially smallerthan the possible number of links on a screen, we need tomap the links to the targets that are sparsely positionedfor Swabbing.

A Preliminary Design of Swabbing Web BrowserThe Swabbing Web Browser is presented as asemi-transparent layer overlaying a webpage. This layerallows the user to browse the web with Swabbing inputonly (Fig. 5). We avoid positioning the targets on the topto reduce errors from Swabbing upward. In this design,the Swabbing targets are not positioned on the edges ofthe screen for better visualization of the text labels.However, the activation of these targets is still tested withtheir projections onto the edges of the screen, and we planto add visual feedback to indicate this in a later iteration.

The user toggles the Swabbing Web Browser overlay bytapping on the screen with five fingers. We used a timethreshold to capture any five distinct touches occurring

within 0.5 seconds. The toggling gesture has a cool-downfor one second to prevent several inadvertent switchings inone action. If the overlay is not visible, single-touch

Figure 6: Each form field isindicated by a small arrow. Theuser activates the field bySwabbing on the target thatmatches the color and theorientation.

Figure 7: A long drop-down listis partitioned hierarchically.

Figure 8: Swabbing overlay fortext input.

sliding pans the web page. Holding a finger on the screenfor one second toggles zooming, allowing the user tonarrow down the area of interest, which reduces thenumber of hyperlinks on the screen. We used a timethreshold to tolerate noises from jittering from tremor.

Top-level menu: The first overlay, invoked by five-fingertapping, contains basic web browsing commands, e.g.,entering a web address, tab manipulations, and navigationto the previous and the next page in the browsing history.In addition, the user can activate the list of links or formfields from this menu.

Hyperlinks: We analyzed the DOM tree of the web pagefor the position of the links. A small arrow indicator isshown on top of each link. The tip of the arrow points tothe link, and the orientation of the arrow matches thecorresponding slice of the Swabbing targets (Fig. 5). Ouralgorithm tries to align the Swabbing targets such thatlinks on the left of the page are associated with theSwabbing targets on the left side. For pages with morelinks than the possible number of Swabbing targets, weshow the links grouped from top to bottom. Alternativelythe user can reduce the number of links shown on thescreen by zooming in to a region of interest beforeactivating the Swabbing Web Browser overlay.

Form fields: The user navigates and activates a form fieldusing the same method as the link selection. Once a fieldis activated, the overlay adapts according to field type,e.g., items in a drop-down list are partitionedhierarchically (Fig. 7). If the items in the list are notsorted alphabetically, a number is added in front of eachoption to help the user recognize the context of the list.

Text input: The user can enter text using an overlaidSwabbing keyboard (Fig. 8). Two-finger and three-fingersliding to the left and to the right navigates one characteror one word in the respective directions.

Future WorkAfter further design refinements, we will compare ourSwabbing Web Browser with a standard touchscreen webbrowser in a longitudinal study. We plan to analyze bothquantitative data, e.g., touch traces, target acquisitionerrors, text input accuracy, and quantitative data fromusage observations and semi-structured interviews.

AcknowledgementsThis work was funded in part by the German B-ITFoundation.

References[1] L. Findlater, A. Jansen, K. Shinohara, M. Dixon,

P. Kamb, J. Rakita, and J. O. Wobbrock. EnhancedArea Cursors: Reducing Fine Pointing Demands forPeople with Motor Impairments. In Proc. UIST ’10,153–162.

[2] A. Mertens, N. Jochems, C. Schlick, D. Dunnebacke,and J. Dornberg. Design Pattern TRABING:Touchscreen-based Input Technique for PeopleAffected by Intention Tremor. 267–272.

[3] C. Wacharamanotham, J. Hurtmanns, A. Mertens,M. Kronenbuerger, C. Schlick, and J. Borchers.Evaluating Swabbing: A Touchscreen Input Methodfor Elderly Users with Tremor. In Proc. CHI ’11,623–626.

[4] J. Wobbrock and K. Gajos. Goal Crossing with Miceand Trackballs for People with Motor Impairments:Performance, Submovements, and Design Directions.TACCESS, 1(1):4:1–4:37, 2008.