As many of you, last week i bought a brand new Apple Magic Mouse. As expected, it's a very powerful device rich of potential. However, it's misteriously software limited. Yet.
The Magic Mouse is powered by a glass surface all touchable as a standard touchpad. This means that every single mm of the upper surface of your Magic Mouse is touchable and highly sensitive. Really amazing!
Apple official driver is really limited (see my other post about momentum scrolling support on Leopard 10.5.x). This include the lack of support for advanced gestures like pinch and rotate.
As many other big troubles, the answer is around us: Trackpad shipped with new MacBook already contains the support for this kind of gestures. Instead, Magic mouse uses a new driver (required to detect the basic multi-touch gestures like swipe and momentum scrolling). Without this driver your new toy is like a water bottle without water.
So, let's start from this new package, named WirelessMouseSoftware.pkg. As you can see from the shoot below, there're two interesting bundles that the package install onto your system disk:
- AppleMultitouchDriver.kext
- PrivateFrameworks/MultitouchSupport.framework
As you can imagine, this binaries contains all that you need in order to disclose the full potentials of our MagicMouse. Using otx tool we can dump and enhance the disassembled code from Mach-O executable files. Using as input files the above ones, we obtain an interesting result:
...
+44 000034d1 e858d20000 calll 0x0001072e _CFArrayCreateMutable
This call will grab our MultiTouch compliance device list, returning it as an NSArray.
So, a call like this:
Will give us a list of all device that support multi-touch (including trackpad and magicmouse, of course).
Very well!, now we need some kind of callback, in order to bind our device with a standard ObjC function. Searching for callback keyword, we obtain:
Nice shot! Now we need to pass all acquired devices to the callback assigner. Something like this:
One last thing that we need to do is to start the events sending queue:
At this point, we need to code a touchCallback function, in order to grab the desired multitouch events and send an appropriate response to the CGEvents queue:
...
}
The following proof of concept grab (very rawly) the pinch event, using the euclide's distance between the two fingers, and send a combined keystrokes as response to the front most application (kCGHIDEEventTap). Launching the binary and bring a Preview.app window to the front, you'll able to pinch in/out, using your magic mouse...amazing! :-)
gcc -o main main.m -F/System/Library/PrivateFrameworks -framework MultitouchSupport -lIOKit -framework CoreFoundation -framework ApplicationServices -lobjc
I've included a commented function, to dump the whole output from magicmouse. You can use it as a skeleton to write an userspace level wrapper driver, in order to catch appropriately all events.
I think that the next OSX update, will include a built-in support for multi-touches gestures of MagicMouse. However, you've all the details to write your own support.
#import <Foundation/Foundation.h>
#include <CoreFoundation/CoreFoundation.h>
#include <ApplicationServices/ApplicationServices.h>
/*
Costantino Pistagna <valvoline@gmail.com>
This code will read raw input from all multi-touch devices attached to chain deviceList.
Either if you're dealing with a trackpad or magic mouse, you'll get the raw multi-touch taps.
touchCallback function uses the nFinger parameter to detect a multitouch event. We don't want to
detect the one finger gestures.
If you don't want to handle a gesture, just return null from the callback function.
Compile with:
gcc -o main main.m -F/System/Library/PrivateFrameworks -framework MultitouchSupport -lIOKit \
-framework CoreFoundation -framework ApplicationServices -lobjc
If you make something usefull with this proof of concept, notify me, and i'll wrote some lines
on my page at: http://aladino.dmi.unict.it
*/
/*
These structs are required, in order to handle some parameters returned from the
MultiTouchSupport.framework
*/
typedef struct {
float x;
float y;
}mtPoint;
typedef struct {
mtPoint position;
mtPoint velocity;
}mtReadout;
/*
Some reversed engineered informations from MultiTouchSupport.framework
*/
typedef struct
{
int frame; //the current frame
double timestamp; //event timestamp
int identifier; //identifier guaranteed unique for life of touch per device
int state; //the current state (not sure what the values mean)
int unknown1; //no idea what this does
int unknown2; //no idea what this does either
mtReadout normalized; //the normalized position and vector of the touch (0,0 to 1,1)
float size; //the size of the touch (the area of your finger being tracked)
int unknown3; //no idea what this does
float angle; //the angle of the touch -|
float majorAxis; //the major axis of the touch -|-- an ellipsoid. you can track the angle of each finger!
float minorAxis; //the minor axis of the touch -|
mtReadout unknown4; //not sure what this is for
int unknown5[2]; //no clue
float unknown6; //no clue
}Touch;
//a reference pointer for the multitouch device
typedef void *MTDeviceRef;
//the prototype for the callback function
typedef int (*MTContactCallbackFunction)(int,Touch*,int,double,int);
//returns a pointer to the default device (the trackpad?)
MTDeviceRef MTDeviceCreateDefault();
//returns a CFMutableArrayRef array of all multitouch devices
CFMutableArrayRef MTDeviceCreateList(void);
//registers a device's frame callback to your callback function
void MTRegisterContactFrameCallback(MTDeviceRef, MTContactCallbackFunction);
//start sending events
void MTDeviceStart(MTDeviceRef, int);
//just output debug info. use it to see all the raw infos dumped to screen
void printDebugInfos(int nFingers, Touch *data) {
int i;
for (i=0; i<nFingers; i++) {
Touch *f = &data[i];
printf("Finger: %d, frame: %d, timestamp: %f, ID: %d, state: %d, PosX: %f, PosY: %f, VelX: %f, VelY: %f, Angle: %f, MajorAxis: %f, MinorAxis: %f\n", i,
f->frame,
f->timestamp,
f->identifier,
f->state,
f->normalized.position.x,
f->normalized.position.y,
f->normalized.velocity.x,
f->normalized.velocity.y,
f->angle,
f->majorAxis,
f->minorAxis);
}
}
//this's a simple touchCallBack routine. handle your events here
int touchCallback(int device, Touch *data, int nFingers, double timestamp, int frame) {
int i;
if(nFingers>=2) { //only report if two or more fingers are touching
//printf("Device: %d ",device);
//printDebugInfos(nFingers, data);
//use two different Touch pointers for the two fingers gestures
Touch *f1 = &data[0]; //first finger
Touch *f2 = &data[1]; //second finger
//compute the euclide distance between the two touches
float distAB = sqrt(((f1->normalized.position.x - f2->normalized.position.x) * \
(f1->normalized.position.x - f2->normalized.position.x) + \
(f1->normalized.position.y - f2->normalized.position.y) * \
(f1->normalized.position.y - f2->normalized.position.y)));
//if we pinch-in (zoom-in)
if(distAB > 0.40 && distAB < 0.41) {
printf("pinch-in detected\n");
/*
CGEventPost(kCGHIDEventTap, CGEventCreateKeyboardEvent (NULL, (CGKeyCode)55, true));
CGEventPost(kCGHIDEventTap, CGEventCreateKeyboardEvent (NULL, (CGKeyCode)69, true));
CGEventPost(kCGHIDEventTap, CGEventCreateKeyboardEvent (NULL, (CGKeyCode)69, false));
CGEventPost(kCGHIDEventTap, CGEventCreateKeyboardEvent (NULL, (CGKeyCode)55, false));
*/
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)55, true); // command (hit)
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)69, true); // + (hit)
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)69, false); // + (out)
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)55, false); // command (out)
} else if(distAB < 0.80 && distAB > 0.79) { //if we pinch-out (zoom-out)
printf("pinch-out detected\n");
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)55, true); // command (hit)
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)78, true); // command (hit)
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)78, false); // command (hit)
CGPostKeyboardEvent((CGCharCode)0, (CGKeyCode)55, false); // command (hit)
}
}
return 0;
}
int main(void) {
int i;
NSMutableArray* deviceList = (NSMutableArray*)MTDeviceCreateList(); //grab our device list
for(i = 0; i<[deviceList count]; i++) { //iterate available devices
MTRegisterContactFrameCallback([deviceList objectAtIndex:i], touchCallback); //assign callback for device
MTDeviceStart([deviceList objectAtIndex:i], 0); //start sending events
}
printf("Ctrl-C to abort\n");
sleep(-1);
return 0;
}
Happy hack!
References
- NSTouch Class Reference
- NSEvent Class Reference