What it does¶
ALTactileGesture is intended to manage tactile gestures on the head sensors.
With this module, you can:
- Detect tactile gestures performed on the head sensors,
- Respond to tactile gestures via qi.Signals and ALMemory events,
- Create new tactile gestures on-the-fly.
How it works¶
ALTactileGesture utilizes many features of the new qimessaging framework (see qimessaging-python).
When you interact with these sensors, you are making a tactile gesture. Tactile gestures can range from simple to complex. An example of a simple tactile gesture would be a single touch of the front head sensor (known as a “SingleFront”), while a more complex gesture example would be one known as “ZoomIn” – whereby you first touch the front and rear head sensors at the same time, then slide both fingers toward the middle sensor and lift up.
A tactile gesture is composed of two elements:
The name of a tactile gesture is either built into the system, or auto-generated based on the sequence. For further details see: Default Tactile Gestures.
The sequence of a tactile gesture is a binary representation of activity on the head sensors from moment to moment. This representation takes the form of a list of binary triplets. Each digit of the triplet corresponds to the activity on the Front, Middle and Rear sensors – in that order. For example, the tactile gestures known as “SingleFront” and “ZoomIn” (described previously) are [000, 100, 000] and [000, 101, 010, 000], respectively. Here, 0 means no activity on the sensor, while 1 means activity.
All sequences must start with “000”, as “000” means that there is no activity on any of the head sensors – which is always the case when you begin a gesture.
All sequences do not have to end with “000”, however. If a sequence ends in a triplet other than “000”, it indicates that it is a “hold” gesture (i.e. you are holding down a particular triplet). For example, as stated before, the “SingleFront” gesture is represented as [000, 100, 000]. Contrast this with the “SingleFrontHold” gesture, which is represented as [000, 100].
Once a particular triplet has been held the gesture is complete (and matched if possible). Any input following that is recognized as part of a new gesture. As a consequence something like “SingleFront SingleFrontHold SingleFront” isn’t a valid gesture.
The evaluation of an inputted tactile gesture depends on 3 key timing intervals. These are:
Settling time is the length of time between the initial sensor change (i.e. the moment you first make contact with any of the 3 head sensors) and when the full sensor read (of all 3 head sensors) occurs.
Hold time is the length of time between the full sensor read and when the currently active pattern is labeled as a hold. Furthermore, this is the length of time between each re-firing of the signal/event for a still-held gesture.
Sequence time is the length of time between inputted patterns. After every sensor read, if a tactile event was not registered on any of the 3 head sensors it will be evaluated as a gesture. If a tactile event was registered on any of the 3 head sensors, then it will be viewed as a part of an on-going tactile gesture.
Default Tactile Gestures¶
By default, there are a number of tactile gestures built into the system. They are:
- ‘SingleFront’: [000, 100, 000]
- ‘SingleMiddle’: [000, 010, 000]
- ‘SingleRear’: [000, 001, 000]
- ‘DoubleFront’: [000, 100, 000, 100, 000]
- ‘DoubleMiddle’: [000, 010, 000, 010, 000]
- ‘DoubleRear’: [000, 001, 000, 001, 000]
- ‘SingleTap’: [000, 111, 000]
- ‘DoubleTap’: [000, 111, 000, 111, 000]
- ‘CaressFtoR’: [000, 100, 010, 001, 000]
- ‘CaressRtoF’: [000, 001, 010, 100, 000]
- ‘ZoomIn’: [000, 101, 010, 000]
- ‘ZoomOut’: [000, 010, 101, 000]
- ‘TheClaw’: [000, 101, 000]
- ‘SingleFrontHold’: [000, 100]
- ‘SingleMiddleHold’: [000, 010]
- ‘SingleRearHold’: [000, 001]
- ‘SingleTapHold’: [000, 111]
- ‘TheClawHold’: [000, 101]
The recommended way to respond to tactile gestures is using the
onGesture signal. Whenever a tactile gesture is recognized,
onGesture signal is fired with the name of the tactile
gesture that was matched as its value. With this pattern, it is
convenient to attach a single “gestureHandler” function to the signal
and handle the processing of the matched gestures there.
For compatibility, a corresponding ALMemory event is also raised,
When carrying out a hold gesture, if the gesture is continuously held,
onGesture signal will continuously fire (after
every hold period has elapsed) until release. Often it is desirable to
only react to the initial
onGesture signal associated to a hold
gesture and ignore subsequent firings. This can be accomplished easily
onRelease signal. When a tactile gesture has completed
(i.e. the user has released from carrying out the tactile
onRelease signal will fire.
Again, for compatibility, a corresponding ALMemory event is also
The following Python script shows example usages of ALTactileGesture.
python altactilegesture_example.py --qi-url=tcp://[Robot IP]:[PORT]
import qi import argparse import sys """Example: ALTactileGesture Example Application""" class ReactToTactileGesture(): def __init__(self, app): """ ALTactileGesture example application. """ app.start() session = app.session # Connect services self.tts = session.service('ALTextToSpeech') self.tg = session.service('ALTactileGesture') # Connect tactile gesture handler to onGesture signal self.s1 = self.tg.onGesture.connect(self.tactile_gesture_handler) # Connect tactile gesture release handler to onRelease signal self.s2 = self.tg.onRelease.connect(self.tactile_gesture_release_handler) # Using getGesture(), get the name of the gesture we're labeling "DoubleTap" doubleTap = ['000', '111', '000', '111', '000'] self.DoubleTap = self.tg.getGesture(doubleTap) # Create new gestures self.add_gestures() # Boolean 'lock' useful for responding to 'hold' gestures in a more controlled manner self.gesture_hold_lock = False print 'ALTactileGesture Example Application :' print 'Please touch the robot head sensors, for example the front one.' def add_gestures(self): """ Add a couple new gestures into ALTactileGesture. """ # Create a new gesture "TripleTap" try: self.TripleTap = None tripleTap = ['000', '111', '000', '111', '000', '111', '000'] self.TripleTap = self.tg.createGesture(tripleTap) except RuntimeError as e: print e # Using getSequence(), create "QuadrupleTap" by building off of "TripleTap" try: tt_seq = self.tg.getSequence(self.TripleTap) if tt_seq: tt_seq.extend(['111', '000']) self.QuadrupleTap = self.tg.createGesture(tt_seq) except RuntimeError as e: print e def tactile_gesture_handler(self, value): """ Given a tactile gesture, say the gesture if we're listening for it. """ # A default gesture... if value == 'SingleFront': self.tts.say(value) # Another default gesture, via getGesture() ... if value == self.DoubleTap: self.tts.say('DoubleTap') # New gesture if value == self.TripleTap: self.tts.say('TripleTap') # New gesture, using getSequence() to build from ... if value == self.QuadrupleTap: self.tts.say('QuadrupleTap') # Hold gesture, repeats every 'hold period' if value == 'SingleFrontHold': self.tts.say(value) # Hold gesture, only responds to first firing if not self.gesture_hold_lock and value == 'SingleRearHold': self.gesture_hold_lock = True self.tts.say(value) def tactile_gesture_release_handler(self): """ Enables 'locking out' of multiple 'hold gesture' signal responses """ self.gesture_hold_lock = False def clean_up(self): """ Disconnect tactile gesture handler from signal """ self.tg.onGesture.disconnect(self.s1) self.tg.onRelease.disconnect(self.s2) if __name__ == "__main__": parser = argparse.ArgumentParser() parser.add_argument("--ip", type=str, default="127.0.0.1", help="Robot IP address. On robot or Local Naoqi: use '127.0.0.1'.") parser.add_argument("--port", type=int, default=9559, help="Naoqi port number") args = parser.parse_args() try: # Initialize qi framework. connection_url = "tcp://" + args.ip + ":" + str(args.port) app = qi.Application(["ReactToTouch", "--qi-url=" + connection_url]) except RuntimeError: print ("Can't connect to Naoqi at ip \"" + args.ip + "\" on port " + str(args.port) +".\n" "Please check your script arguments. Run with -h option for help.") sys.exit(1) react_to_tactile_gesture = ReactToTactileGesture(app) app.run() # will exit when the connection is over. react_to_tactile_gesture.clean_up()