Paper Reading #24

Gesture avatar: a technique for operating mobile user interfaces using gestures

Authors:	Hao Lü	University of Washington, Seattle, Washington, USA
	Yang Li	Google Research, Mountain View, California, USA

Proceeding  

CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems

Summary

• Hypothesis - The researchers believed that their system can be used to easily select small objects on a touch screen better than a previous system called Shift.
• Methods - In order to compare the usability of the Gesture Avatar to Shift in a non-biased way, 20 participants were asked to try each using different sized targets, different number of targets, and testing while either walking or sitting. 10 participants used the Gesture Avatar first, while the other 10 used Shift first. 
• Content - The point in this research is to resolve the "fat finger" and occlusion problems with touch gestures. The fat finger problem is the fact that a finger may be significantly larger than the object to be selected. The occlusion problem states that the user can not specifically see what they are selecting because the finger occludes the screen while interacting with it. Gesture Avatar works by drawing a letter to select a link on a page. If the link's first letter is the letter drawn, it is highlighted. To select the link, the user taps on the bounding box of the gesture they created. If they want to select another link that satisfies the letter, the user drags outside the bounding box to go to the next link. To undo the gesture, the user taps outside the bounding box.
• Results - GA was faster at selecting larger items, but slower at smaller items compared to Shift. GA had the upper hand to shift when walking was involved since walking did not effect the error rate for GA, while shift had a much higher error rate. Because of this, the majority of the participants preferred the Gesture Avatar.

Discussion
This is a pretty clever way of solving the fat finger and occlusion problems. However, I believe most of the issues this solves for web browsing have mostly been partially solved by having mobile versions of pages. Also, I'm not convinced that this is faster than simply zooming into the page to make the links larger. For future work, this application could be integrated into android as an option.

Paper Reading #23

User-defined motion gestures for mobile interaction

Authors:	Jaime Ruiz	University of Waterloo, Waterloo, Ontario, Canada
	Yang Li	Google Research, Mountain View, California, USA
	Edward Lank	University of Waterloo, Waterloo, Ontario, Canada

Proceeding 
CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems

Summary

• Hypothesis - The Researchers believe that some types of gestures will be more difficult than others
• Methods - Twenty participants were asked to make gestures on how to do simple tasks with mobile devices. Since learning about new concepts in smart phones may make the users less able to perform the tasks, only people who were previously used to smart phones were tested. The gestures to be made were supposed to be created to be as simple to the user as possible. Since the hardware in the mobile devices are not able to completely recognize all gestures perfectly, the users were told to treat them as "magic bricks" that could understand any gesture. Also the users were asked to describe why they chose such gestures while they were performing them in order to understand their reasoning. After the users created their gestures, they graded their own gestures on a Likert scale.
• Content - The tasks performed were as follows: Answer call, hang-up call, ignore call, voice search, place call, act on selection, home screen, app switch next, app switch previous, next (vertical), previous (vertical), next (horizontal), previous (horizontal), pan left, pan right, pan up, pan down, zoom in, and zoom out. The ways gestures are described are with the nature of the gesture (symbolic or abstract), whether it has a context, whether the gesture causes an action after or during the movement, how much impulse is applied to the gesture, the dimensionality of the gesture, and it's complexity.
• Results - The more direct mapping to real life examples for manipulation the gestures were, the more widely the participants liked the gestures. For example, the gesture for answering a call that most participants agreed upon was a motion that was similar to the motion that would be to answer a call by putting the phone to the ear. Hanging up the phone was done in another similar mapping to old-style phones by turning the screen around and parallel to the ground. A slightly unnatural mapping was found for moving the screen. On touch gestures, people "drag" the objects in the screen with a movement that is in the same direction as the motion of the object. However, for motion gestures, the participants moved the window in the opposite direction of the movement of the objects in the screen.

Discussion
It would be nice if every gesture had a very easy and simple direct mapping to normal movements in the physical world. The problem is that with the added complexity and functions that computer systems can offer, there sometimes is no other way to perform a task other than using a computational device. In this case it is probable that an unnatural gesture would have to be made. Then again, this theoretical situation would make the gesture the only way to perform the action in such a world, and in a sense, would be the "normal" action to do anyways. One can only hope that adequate gestures are mapped to actions when mobile device complexity explodes.

Paper Reading #22

Mid-air pan-and-zoom on wall-sized displays

Authors:
Mathieu Nancel - Univ. Paris-Sud & CNRS; INRIA, Orsay, France
Julie Wagner - INRIA; Univ. Paris-Sud & CNRS, Orsay, France
Emmanuel Pietriga - INRIA; Univ. Paris-Sud & CNRS, Orsay, France
Olivier Chapuis - Univ. Paris-Sud & CNRS; INRIA, Orsay, France
Wendy Mackay - INRIA; Univ. Paris-Sud & CNRS, Orsay, France

Proceeding CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems

Summary
Extremely large data sets are difficult to visualize. For example, the Spitzer's 4.7 billion pixel image of our galaxy can be visualized on multiple displays in different ways, but some are better than others. A user can sit at a desk in front of the wall-sized display and use physical input such as a mouse to manipulate the images, or use mid-air gestures in 1,2, or 3 dimensions.

Hypothesis
H1: Two handed gestures should be faster than one handed gestures.
H2: Two handed gestures should be easier and more accurate.
H3: Linear gestures for zooming should be better for zooming but eventually will be slower than circular gestures due to repositioning.
H4: Users will prefer gestures that do not require repositioning.
H5: Gestures using small muscles will be faster than larger ones.
H6: 1D-path gesures should be the fastest.
H7: 3D-free gestures will be more tiring.

Methods
There are several design factors to be determined. The user can use one or both hands for input. Using one hand allows the other hand to perform other actions, but using two hands can allow panning and zooming to be done at the same time. Gestures can be linear or circular. Linear gestures (moving a slider) are naturally mapped, but may require clutching. Circular gestures do not require clutching, but is not natural. Different degrees of freedom have different advantages as well. 1D path gestures have strong haptic feedback but are limited in dimension. 2D surface has more freedom but has less feedback. 3D free hand gestures have the most freedom, and does not require any device. The only issue is that there is no feedback at all.



Results
Two handed gestures were found to be faster than one hand gestures. One dimensional path gestures were the fastest of all the gestures as well. Even though for circular gestures, the participants did not have to move back to reposition, the linear gestures were faster because of less overshoot. The participants favored the faster gestures because they were the easiest.

Conclusion
There exists 3D stylus devices for 3D drawing programs. I can imagine these devices could be modified to allow better haptic feedback with servos and strain sensors to allow much better feedback. A more interesting task would be to allow for 3D-free manipulation of virtually projected 3D images. There exist several engineering problems to solve this issue however. Back to the display of large data sets, I don't think too many people will benefit directly from being able to see 100 million pixels at home.

Paper Reading #21

Human model evaluation in interactive supervised learning


Authors:

Rebecca Fiebrink Princeton University, Princeton, New Jersey, USA
Perry R. Cook Princeton University, Princeton, New Jersey, USA
Dan Trueman Princeton University, Princeton, New Jersey, USA


Proceeding
CHI '11 Proceedings of the 2011 annual conference on Human


Summary
Machine learning allows for algorithms to determine output from an input based on a trained model for the data. Not every input is specifically represented in the model, so the model needs to make generalizations. Interactive machine learning (IML) uses model evaluation so that users can assess how well a certain model is performing. Without IML, understanding these models can become a difficult debugging process.

Hypothesis /Motivation

The first goal of the paper is to learn more about how interactive machine learning based on user evaluation. The second goal is to determine how these evaluations effect what users do in utilizing IML, specifically what algorithms they use. The last goal is to describe the utility of supervised machine learning.

Methods
A tool called the Wekinator was designed to encompass several standard machine learning algorithms. It also allows for the training, modifying, and visualization of real time data used to create the model. Three studies were performed using the Wekinator.

• Study 'A' involved 6 PhD students and one faculty member at Princeton's Music Composition department. For 10 weeks, they met and discussed how they were using the tools in their work, and offered suggestions for the software. At the end, they completed a questionnaire about how they felt about the software.
• Study 'B' involved 21 undergraduate students. They were asked to design systems of input and train models using neural networks to create different sounds. The inputs needed to be continuously controlled.
• Study 'C' was a case study that involved a cellist for the purpose of building a gesture recognizing cello bow, called the "K-Bow." This had sensors for acceleration, tilt, and relative position. Also grip pressure is measured. The goal was to create a model to process the sensor data to create physically appropriate music.

Results

All 3 study groups used direct evaluation to determine the system is working properly in processing input data. However, in studies 'B' and 'C', the researchers noted that the participants used cross validation todetermine how well the models evaluated new data. Subjective evaluation was used in study C to find errors in the model that were not found using cross or direct evaluation.

Discussion
Machine learning is very interesting to me, primarily since I'm enrolled in CSCE 420. Creating good models is very good, but I personally would like to see a fully automated process for model creation and evaluation. This isn't to say that intelligence depends on having best model of the world, but it is a good start. 

Paper Reading #20

The aligned rank transform for nonparametric factorial analyses using only anova procedures

Authors:

Jacob O. Wobbrock - University of Washington, Seattle, Washington, USA

Leah Findlater - University of Washington, Seattle, Washington, USA

Darren Gergle - Northwestern University, Chicago, Illinois, USA

James J. Higgins - Kansas State University, Manhattan, Kansas, USA

Proceeding
CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems

Summary

ANOVA is short for ANalysis Of VAriance. Nonparametric data in experiments with multiple factors occur in the field of HCI. Analyzing interaction effects with this type of data is impossible with traditional analyses. The Aligned Rand Transform is able to analyze this type of data.

Hypothesis

The researchers propose an algorithm and tools for analyzing multi-factor non-parametric data.

Methods

ARTool and ARTweb are programs that utilize the ART algorithm. The algorithm starts by computing each cell's residual value. Next the estimated n-way effects are computed using alternating positive and negative summations. Then the residual is added to the estimated effects. Last the average ranks are assigned and a full factorial ANOVA is performed on the ranks.

Results

The ARTool was used to evaluate a satisfaction survey for how satisfied they were on a test interface. The data was analyzed with ANOVA and with ART. The ANOVA test did not show any interaction but the ART did. The researchers hypothesized the interaction would exist and the ARTool was able to prove it for them.

Conclusion

I have yet to take statistics, so much of this is quite confusing. I do know that determining correlations in data is important for finding unknown relations. The best example I know about this is the statistics Blogger and YouTube offer (both being from Google) for individual uploads. The paper notes that this algorithm is accessible to anyone "familiar with the F-test," which I am not. After skimming through what it is, it seems there is quite a bit of statistics I need to understand before fully understanding the ART algorithm.

Paper Reading #19

Reflexivity in digital anthropology

Author: Jennifer A. Rode - Drexel University, Philadelphia


Proceeding: CHI'11 Proceedings of the 2011 annual conference on Human factors in computing systems

Summary
There are three types of anthropological writings, consisting of realist, confessional, and impressionistic writing styles. Even though realist writing is most common, all have some good qualities. These types of writing have specific values that can affect the HCI design process.


Hypothesis
This paper’s purpose is to establish and characterize the contributions of anthropological ethnography.

Methods
Realist writing consists of creating a sense of authority from the author because of the fact they have collected data directly in the environment they are studying. This prolonged exposure to the environment gives enough data to determine an unambiguous result. The goal is to minimize the effect of the observer in the situation from the precision of the data.


Confessional writing instead directly uses the opinions of the observer. Since the observer now has a noted effect on how the perceived data is collected, the authority may be diminished, but sometimes allows for demystifying the data collecting process. Since ethnographers tend to avoid adding their own opinions in the process, this type of writing is not as common as realist writing.


Impressionist writing is done similarly to a narrative. Sometimes writers try to startle the audience through dramatic recall instead of dry facts.


One task the ethnographer should try to accomplish is to establish and maintain rapport with the subjects they are studying. This is to say that the subjects have a mutual understanding of how and why the ethnographer is doing a study. This allows access to the data being collected. However, lack of rapport sometimes allows understanding of the differences of cultures from the ethnographer and the subjects.


Participant observation is the core to reflexive studies. It is described as "deep hanging out" and is the key to cultural anthropology. 


If anthropology is purely experimental, it is essential for the opinions of the author to be left out. However, the relationships with the experimenter always causes actions that can not be properly explained without considering the writer's existence. This is the key to a formative design process.

Three different forms of ethnography can be used to HCI practices.

1. Formative ethnographies determine how technologies are currently being used for the purpose of improving the technology.
2. Summative ethnographies details how a certain group uses a technology, but does not help towards the HCI design process since it done after the fact.
3. Iteratively evaluated ethnographies use prototypes in practice and determine how well they are used. This data is used to improve another prototype design.

Results
There was no testing


Conclusion
It makes perfect sense to use ethnographic data for the HCI design process. If a design or product isn't working for some reason, it can either be redesigned to be "better" in the designer's eye, or the customer's mind. The designer can become disconnected from what the end-user wants, but the end-user can't always know what they want since they don't understand the technology in detail.

Paper Reading #18

Biofeedback game design: using direct and indirect physiological control to enhance game interaction

Authors:	Lennart Erik Nacke	University of Saskatchewan, Saskatoon, Saskatchewan, Canada
	Michael Kalyn	University of Saskatchewan, Saskatoon, Saskatchewan, Canada
	Calvin Lough	University of Saskatchewan, Saskatoon, Saskatchewan, Canada
	Regan Lee Mandryk	University of Saskatchewan, Saskatoon, Saskatchewan, Canada

Proceeding

CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems

Summary

Several physiological sensors are tested for enhancing a video game. Primary interaction is done on a normal Xbox controller while other sensor data such as respiration and heart rate are taken passively.

Hypothesis

1. How do users respond when physiological sensors are used to augment rather than replace game controllers?
2. Which types of physiological sensors (indirect versus direct) work best for which in-game tasks?

Methods
Six sensors are used to gather data from the user.

• Gaze iteration. Cameras track a user's eye movements to determine where they are looking. This is a direct physiological input.
• Electromyography. Sensors are place longitudinally along a muscle group to determine the electrical activation level of the muscle.
• Electrodermal activity. This determines the level of conductance of a user's skin. This insinuates psychological arousal and as such is an indirect input.
• Electrocardiography. This reads the activity of the heart. Even though this is autonomously controlled, it can be mildly influenced consciously. Heart rate is an indirect input. 
• Respiration. When a person breathes in, their chest expands. A strain sensor is placed around the body to determine the level of respiration. This is directly controllable.
• Temperature. Homeostasis dictates internal body temperature is constant, but in this case it is directly controllable by blowing hot air into the sensor.

Together, these sensors were used to control different aspects of a video game in different studies. The size of enemies were varied to make them easier to hit. Since larger enemies are more intimidating, only a shadow of the sprite was made larger. Speed and jumping height was varied as to make the avatar faster or able to jump higher. The weather is changed to make snow appear across the screen. The fallback weapon of a flamethrower has a variable length. Lastly "Medusa's Gaze" is a power-up that allows the user to look at enemies and freeze them by using eye tracking. These variables are changed in two tests as shown below.

Results

The researchers found that users preferred direct control devices, even though they enjoyed using the sensors. They concluded that these sensors are best for use in changing background variables instead of primary interactions. From individual sensors, users preferred ones they could more actively control, such as breath and muscle contraction as opposed to heart rate.

Discussion

I think the SensorLib framework could be used for very rich interactions in games. In any case, most users would probably prefer not having to wear anything though. One thing I was thinking about was the difference between natural mappings and relevant biofeedback mappings.

Paper Reading #17

Privacy Risks Emerging from the Adoption of Innocuous Wearable Sensors in the Mobile Environment

Authors: 
Andrew Raij - University of South Florida, Tampa, Florida, USA
Animikh Ghosh - SETLabs, InfoSys, Bangalore, India
Santosh Kumar - University of Memphis, Memphis, Tennessee, USA
Mani Srivastava - University of California, Los Angeles, Los Angeles, California, USA

Proceeding
CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems

Summary

AutoSense is a data collecting system concerning physiological status of humans. It collects inertial data along with cardiac, temperature, skin, and respiration.

Hypothesis

When creating the AutoSense apparatus, it was not known that certain inferences could be made. The researchers proposed that participants can not fully understand the stake in releasing their data unless they had a personal stake in it.

Methods

The first goal of the research was to assess how people regard their data sets, since it includes information about their physiological, behavioral, and psychological states. This was measured by having two primary groups, one that did not have personal stake in the data released, and one that did not.

The second goal was to design the study so that the participants could limit access to their data, or even prevent logging of certain points of data, such as acceleration.

The last goal is to determine how easily a participant can be reidentified using the data sets.

Results

People were less likely to share some parts of their data if they understood that it could be used against them (personal stake). Also there was a considerably larger amount of stress for the S-Post group as well. The data set of exercise was not as much stressed about, but there was still concern about privacy of their exercise preferences.

Discussion

Today we live in a world where the amount of data being shared is enormous. Some people take this as an opportunity to hide everything about themselves, or to embrace it by actively sharing their lives to strangers on the internet. Obviously, when privacy is removed and if there is any incriminating activity, a potential for issues arises. So if privacy was not an option, there is one of two choises: be exposed from "wrong" actions, or to chose not to do those actions knowing that people are watching.

From this, I do not believe that privacy should be an option since it allows unwise actions to be performed.

Paper Reading #15

Madgets: actuating widgets on interactive tabletops

Authors:
Malte WeissRWTH Aachen University, Aachen, Germany
Florian SchwarzRWTH Aachen University, Aachen, Germany
Simon JakubowskiRWTH Aachen University, Aachen, Germany
Jan BorchersRWTH Aachen University, Aachen, Germany

Proceeding
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology

Summary
Madgets are interactive physical objects what can be manipulated by a user or by the interactive tabletop itself. An array of permanent magnets below a multitouch display allows for such interactions.

Hypothesis
Madgets combine the benefits of untethered manipulable objects along with the system's ability to manipulate them as well. Other systems have this ability, such as interactive sound controllers, but do not share the Madgets' degree of physical freedom.

Methods
The system is composed of an array of 228 electromagnets which are all controlled from a PWM signal from an Arduino. On top of the table, the Madgets are held in place from permanent magnets attached to them, and the electromagnets inside the system. Directly above the magnet array is a TFT panel that allows images to be displayed in the interaction area. On top of that is an acrylic sheet which is lit from the edges by infrared LED lights. When a user touches this sheet, infrared light is reflected downwards. Since the magnet array would normally block the transmission of light, several fiber optic cables transmit the light around the magnet array. There, the light is picked up by an infrared camera which can then track any touch in the entire surface. Magnets are actuated by determining which electromagnet to use for attracting or repelling the permanent magnets.
There are 6 sample Magnets displayed

1. Buttons: Using the fact that the 3rd dimension can be utilized, buttons can be help upwards using repulsive force. When a user presses the button down, the system sees the difference in infrared light and can perform an action.
2. Clutch: Since now both vertical and horizontal actuation is possible, the button Madget is modified to have a slider that can physically disable pressing a button by sliding beneath a button. This can be useful when a button should not be logically pressed.
3. Force feedback: Madgets can be manipulated by the system to create force feedback. This can be done in the form of resistance, vibration feedback, or "notches."
4. Induction: The magnet array is powerful enough to transfer inductive power to an LED.
5. Motor: By actuating a tangential magnetic force, a pair of magnets can be spun around an axis at a constant velocity. This can be used to create much more complex systems.
6. Bell: By building on the concept of the button, a magnet can be forcefully launched against a bell to ring it.

Results
The system itself had very little problems, aside from a small overheating problem. If the electromagnets are held at full power for too long they would get too hot. An algorithm was developed to spread the effort of the array across several magnets if one was estimated to be too hot.
Conflicting goals: Having widgets that are actuated in a human understandable way, there is less confusion as to what they do, but the limitations of these simple actions prevent complex interactions. The researchers admit they need to find a balance between these goals.

Conclusion
I imagine this system uses quite a bit of electricity to operate. Although this concept is interesting, the gains in simplicity on the user side is not close to how easy it is to create a virtual interaction object within a touch environment. Ironically, the fusion of software and physical input can be just as easily prototyped with the Arduino that is used to control the magnet array as the Madgets themselves. Since the real world is always manipulated (especially in industrial environments) by software, there will always be a need for interaction, but I doubt it will be in the form of Madgets.

Paper Reading #14

TeslaTouch: electrovibration for touch surfaces

Authors

Olivier Bau Disney Research Pittsburgh, Pittsburgh, PA, USA and Université Paris-Sud, Orsay, France
Ivan Poupyrev Disney Research Pittsburgh, Pittsburgh, PA, USA
Ali Israr Disney Research Pittsburgh, Pittsburgh, PA, USA
Chris Harrison Disney Research Pittsburgh and Carnegie Mellon University, Pittsburgh, PA, USA

Proceeding
UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology

Summary
TeslaTouch is a touch interface that uses oscillating electric forces to create tactile feedback. It works by periodically attracting the surface of finger to the surface, changing the amount of friction based on voltage.

Hypothesis
The researchers believed this system could be used to simulate different types of surfaces dynamically.

Methods
The TeslaTouch System was created from a transparent electrode sandwiched between a glass plate and an insulator. Images were displayed using a projector. High voltages were sent to the electrode and attract the skin of the finger using electrostatic forces. Much better results were achieved if the user is grounded. The safety of the system is ensured by limiting the current possible from the power supply. Different types of signals and strengths were tested on participants and their opinions were noted in categories such as how slick the surface felt and how it felt like vibration or friction. A scale of pleasantness was also considered. First the minimum detection for a single user is determined using a step-wise learning cutoff. The main tests were between 400 and 80 cycles per second, and 115 and 85 volts.

Results
Higher frequencies rendered more "smooth" and "waxy" results, and higher voltages increased these observations. Also, compared to other vibration interfaces, this system does not generate any noise.

Conclusion
I think this system could be used for generating much more tactile sensations than mere sine-waves. I'm imagining having the system determine which direction a finger is moving and then creating a shaped wave based on the direction of the motion. This way there could be much richer types of sensations, and possibly types of sensations that may not be possible in the world.