Paper Reading #2: Hands-on math

Hands-on math: a page-based multi-touch and pen desktop for technical work and problem solving

Robert Zeleznik, Andrew Bragdon, Ferdi Adeputra, Hsu-Sheng Ko
Brown University

Released: UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology

Summary:

In the past, mathematics were observed in two seemingly mutually exclusive techniques: using paper or using computer algebra systems (CAS). This is because CAS tends to limit creativity, but paper lacks computational assistance. This paper attempts to solve this by giving a user direct access to the underlying tools seen in CAS but in a way that resembles natural manipulation of algebra. 

Hypothesis:

Hands-on Math is designed to test whether or not a hybrid system is better than CAS individually if "CAS functionality were available in a paper-like environment."

Methods:

The program was built on Microsoft Surface and uses the StarPad SDK for handwriting recognition. The surface contains the workspace and virtual paper can be pulled out of the right bezel. After equations are entered, they can be manipulated in reasonable ways such as factoring, or simplification using gestures where the selected parts depend on where the finger touches the equation. For example, if a plus sign is selected, both the left and right parts are selected. If there is an addition, and the right section is pressed, it can be moved around without factoring. The same communitive property is implemented with multiplication, but there is no symbol used when variables are selected. A gesture for "folding" the previous equations using a vertical pinch gesture is also implemented. The main issue for designing gestures is to have them be reasonable and self evident. This is inherently difficult if the purpose is to expose hidden features of a CAS. This is partially solved by using transparent menus that appear near half finished gestures. This can separate variables into summations or factors. Nine participants were selected from Brown University to test the system and give opinions.  

Results:
Since some of the gestures were ambiguous, the test was not to see who could figure out the program without instruction. Some features that were not obvious seemed to be ergonomic after instruction. Also since this is a prototype, the students were asked to ignore bugs when possible. The paper interface was found to be natural even though the pen used was not accurate all the time. For deleting, the participants did not use the original single gesture to remove documents, but used a two step process that made more sense in a physical sense: by dragging it close to the edge (of a perceived desk) and then selecting the bin that appeared. In general the students enjoyed the program and therefore supported the original hypothesis

Discussion
The goal for touch based interfaces is to have a natural response from the computer that doesn't simply emulate normal actions in real life, but expands upon them. The researchers here could have made the best paper simulator ever but without the elements of CAS added, people would probably prefer paper. Also the gestures used to carry out tasks should be unambiguous but that is not always possible. Instruction on how to use the program is adequate sometimes, but it would be best if no explanation is necessary. This is why I think the iPhone has succeeded: a 5 year old without instruction can operate well. I am not convinced this is a commercializable product, but does prove its hypothesis decently since the information given by WolframAlpha is somewhat interactive already. For it to have widespread use, it would have to support more functions such as advanced calculus. If this was added though, even more ambiguous gestures or convoluted menus would have to be made. At this point I think it would be best to use specific programs to solve specific problems. It could be useful for young children to understand algebra though as a learning tool, which makes it interesting. In the future, many problems will have to be solved by assigning gestures in logical ways, but that is the main issue that will have to be solved.