I have worked yesterday on a simple implementation of a mouse gesture recognition algorithm.
Here's a demonstration.
I'll probabaly publish a MouseGesture.as class soon, so stay tuned.
Mouse Gesture [ADD : 2007-05-17]
 |
This project is OPEN SOURCE under RPL License. 2007-05-17 v1.0 mouse_gesture_v1_0.zip |
The zip file contains the MouseGesture class + the full code of sample demo above.
Algorithm
- Each letter is defined by a n ' 8-directions gesture sequence
- The mouse moves are saved with the same 8-directions sensibility
- A Levenshtein distance is calculated from each letter to the user moves
- The algorithm return the best candidate (lowest levenshtein cost)
Usage
Example for the B :
gesture=new MouseGesture(stage);
gesture.addGesture("B","260123401234");
gesture.addEventListener(GestureEvent.MATCH,matchHandler);
gesture.addGesture("B","260123401234");
gesture.addEventListener(GestureEvent.MATCH,matchHandler);
function matchHandler(e:GestureEvent):void{
trace (e.datas+" matched !")
}
Comments (65)
Tested and it worked greatly. I looking forward to see us your MouseGesture class
La classe ! C’est l’algo de Grafitti ou c’est un algo maison ?
PS : La typo de la fenêtre de droite déconne chez moi, j’ai une petite Times New Roman à la place, assez illisible
NICELY DONE !
awaesome
It works great !
As max saids (in french) the font size in the right box is too tiny to be easyly read.
In wait for the sources and the complete letter sign list (the fox hide the arrobase but it works).
Thanks for your comments, I’ve uploaded a new version :
- No more tiny font on the right textfield
- The @ is working now (will you find it ?)
Max > A part of the algorithm looks like grafitti (the mouse gesture capture), but the most important part (match process) is based on a Levenshtein distance algo.
Nice and smooth
Absolutely incredible! I have a touch screen application that I would love to try this on!
Oh, and nice bonus. I haven’t seen that pink, fluffy thing for many years.
Nice one. But not the first one
Here is an older Flash 8 version with the same character set:
http://www.gskinner.com/blog/archives/2005/03/gesture_recogni.html
hu hu I can’t get the secret picture with the flower. I need it, I want it, I will get it
Could you place the dot that marks the start of the drawing ?
Thank you for the flower drawing path and for this lovely bonus, even if I prefer her less hairy !
joa > I knew Grant Skinner one, he was not the first and I’m surely not the last. However, my algorithm is strongly different
Tek > you’re welcome
Yep, It can be cool for wii !
i got the secret picture finnaly !!!!
Well done! Yes very smooth and the easter egg is sweet, literally.
nice work, makes me remember the great “Black & White”. The “0″ and the “6″ seem to make problems in differentiating, though.
Right for the 6 … same problem here.
Sorry for you Foxy, probably the fourth special sign in your sign map
Really cool!
Ha I love it, I’ve been wanting for some bright spark to do a demo in flash.
You mind if I do a link to it, so user can try out gesture interaction.
You planing on posting the code so I can have a pick about at it?
Excellent
James
Hi there,
the sources are online now ! have fun
Bon job! J’était justement en train de faire une reconnaissance d’écriture mais pas la même méthode…interessant!
Eratum : the digit 1 is not correct (same for the i)
Replace : mg.addGesture(“1″,”2″)
by mg.addGesture(“1″,”72″);
in GestureDemo.as
’7′ for the first -45° segment of the “1″, then ’2′ for the vertical one.
Hi
Thanks for the source, much appreciated I’ve put the demo and a credit on mousegesture.org
I was playing with the idea of writing a demo in action script, but it is a little beyond me.
I’ve written a gesture engine myself in Vb, nowhere near as accurate though, http://www.cityzen451.co.uk there is a video of it on youTube
http://www.youtube.com/watch?v=RNggSmVJ7Ow
You dont have any plans to port your engine to vb by any chance?
Dr Rowan:)
Hi James, you’re welcome.
I think it’s not too difficult to port it to vb since the class is very simple, you should do this
Great work! I tried many ways to recognize mouse gestures, but all of them were slow and could recognize only the simplest curves… Your idea is so simple and so smart!
Very nice. Excellent work as usual.
Clever, the algorithm is very cool. I really think its fantastic that you took time to do a little diagram and illustration so that we could understand how you used the Levenshtein distance to detect mouse gestures.
so good idea!
Great idea on the algorithm – it’s very cool to see people solving the same problems in different ways.
My original version was actually Flash 6, and I believe it actually was the first time it was done with any success in AS.
Of interest, one other person released a mouse recognition system using a directional tracking algorithm similar to yours at the same time I released mine (spring 2003?), but it had no fault tolerance and as such was very inaccurate. You’ve solved that problem with the application of the Levenstein distance algorithm, which is a great idea.
Cheers.
Really accurate, too! Nice work all round!
Thanks guys
真是好东西！
in Chinese 高实在是高
always on the cutting edge Didier !
that one’s rather impressive and might come really handy when used in an appropritae project.
keep it up !
AWESOME. I’m waiting the class for implement it. Awesome work. Welldone
nicoptere > Yeh, thanks, see you soon around a beer
Yuchi > don’t wait, the class is still available
great job…
I have a problem with the demo file. When I am trying to open ‘GestureDemo.fla’ I get the following message:
Unexpected file format.
I am using Flash 8 Professional.
Thank you in advance.
Chris, it’s AS3/Flash CS3, you need CS3 to open it.
Really nice work on this. I’d never seen Grant’s example or anything like this done in Flash and before reading the comments was floored. I still am floored, but you know what I mean.
I’ve been playing alot of DS lately and the first thing that came to mind was to make a DS in Flash.
Excellent work! Is there any chance of releasing this source code under a commercial license? Please e-mail me to let me know. Thanks!
Good work!
But it can’t distinguish “A” and “1″. Though I try many time.
Chris > Thanks to Matt
Matt > Thanks !
Bernardo > Yes, I’ve sent you an email
ntvy > Yes, look at my comment #22, I have to update the demonstration.
The demo is fantastic and I highly appreciate your efforts.I tried to compile the GestureDemo.as file using flex SDK2,but faced the following errors:
Error: Call to a possibly undefined method stop through a reference with static type
if(fadeTween)fadeTween.stop();
Error:Access of undefined property Regular
fadeTween=new Tween(draw_mc,”alpha”,Regular.easeIn,draw_mc.alpha,0,20);
Error:Incorrect number of arguments.Expected no more than 5.
fadeTween=new Tween(draw_mc,”alpha”,Regular.easeIn,draw_mc.alpha,0,20);
The same errors were reported for fiabTween()function as well.
I am new to actionscript.Kindly tell me how to execute the GestureDemo.as file correctly.Thanks
Hi Dave,
Maybee the Tween classes are different on Flex 2, you can use MouseGesture.as class at least ?
Yes maybe the Tween classes are different in Flex 2.When I compiled the MouseGesture class using mxml compiler,I got the following error
Error:A file found in the source path must have the same package structure as the definition’s
package ‘com.foxaweb.ui.gesture’
Kindly tell me if there are other ways of executing AS3 and generate a .swf file without using a flash player (so that the generated .swf file can be viewed in a browser).Should I use another compiler?
Thanks
great ideal
I believe your Levenshtein algorithm is incorrect. After researching other implementations such as http://sourceforge.net/projects/simmetrics/
http://en.wikipedia.org/wiki/Levenshtein_distance
It appears costleven in MouseGesture:
for (x=1;x
Hi Marcus,
well I’ve not ‘strictly’ implemented Levenshtein, I’ve modified it for my need.
nice work,and i really enjoy the algorithm of the mouse gesture
Great work!
Very Nice.
I tried use this in Flex using Flex sdk 3.2, but failed.
I changed lines 141,142 in MouseGesture.as to disable weak reference and it works again.
Excellent class Didier.
Just one question, it seems if you have only few mouse gestures like simple ones:
mg.addGesture(“BACKSPACE”,”4″)
a lot of free drawing will be recognized as ‘backspace’ in this case since it falls into fiability? Is there a way to prevent this? For example will it make a lot difference if I do:
mg.addGesture(“BACKSPACE”,”44444″), in another word to make the gesture more recognizable for certain minimal length of drawing at one direction? The ’44444′ doesn’t seem to help.
Hi Kyle,
To solve this issue, you have to modify the algorithm. Try to add an extra cost when a letter is removed (this cost is null actually)
Nice work. This works well.
I actually built a gesture recognition system in Flash 4. It certainly wasn’t anywhere near as smooth as this implementation, but it used a similar home-brewed system in the back.
In those days, we didn’t have arrays – so there was a complicated and really irritating system of checking through the possible signatures.
I may have a version of it kicking around somewhere… I’ll have to crack out the backup drives and have a look.
How abaut Flash lite 2.1? is compatible with this?
Hi Didier:
Can you pls elaborate abit more? What do you mean by ‘when a letter is removed?’.. And by ‘modify the algorithm’, do you mean by changing the DEFAULT_PRECISION or DEFAULT_FIABILITY, or actual algorithm?
My problem again is lots of free-drawings are recognized as one of my small set of registered Guestures. I think similar problem is in your example as well, if you do an bottom-up I, it will be always recognized as N, is there a good solution for such cases?
Thanks alot
hey, just wondering if you could help out …
TypeError: Error #1009: Cannot access a property or method of a null object reference.
at GestureDemo/matchHandler()
at flash.events::EventDispatcher/dispatchEventFunction()
at flash.events::EventDispatcher/dispatchEvent()
at com.foxaweb.ui.gesture::MouseGesture/matchGesture()
at com.foxaweb.ui.gesture::MouseGesture/stopCapture()
The links appear to be dead. Hopefully, they’ll be back online so I can try this out. Thanks!
Salut,
Le lien semble mort depuis plusieurs jours. =( …
The links are up now
круто
кстати есть и другие темы. Я даже могу фотку вам скинуть но почты не нашел. лучше думаю просто залить на фото обменник.
This is great! Awesome work, man! I wish you could convert it to AS2 though, so then it could be used for both.
mouse gestures engine of ENCSOFT works like yours, but a little different, no mouse button holding, just moving the mouse.
their site: http://www.mouse-gesture.com
nice work! I really like this,
however i do have one question:
is there anyway i can resize or constrain the rectangle where you can draw? Because now it seems like i can draw anywhere on the stage and it will always listen to the mouse.
i tried drawzone.width and mouseZone.width/height but no luck..
plz help!
Great stuff that’s fun to play with
Just to clarify, the RPL license means i wouldn’t be able to sell any components that implement this class on say activeden.net or flashcomponents.net, right?
Trackbacks/Pingbacks (42)
[...] AS3 Mouse Gesture Recognition May 20th, 2007 — drawk Didier Brun of ByteArray just keeps throwing down. Here is another great showcase of the power of AS3 and the tools that can be built with it, strongly based on alogorithms proven in other hardcore languages. This project is a Mouse Gesture Recognition lib for AS3 [...]
[...] Didier Brun вновь поражает своим экспериментами на ActionScript3. В двух словах, он разработал алгоритм распознования движении мыши на AS3. Т.е. мышкой, вы рисуете букву, цифру (символ) – он автоматически появляется в текстовом поле. [...]
[資源]-超猛的ActionScript3.0資源網站 ~ ByteArray.org…
上禮拜回老家
再加上週日都在玩電吉他
所以又好幾天沒寫文章了~>0<
今天就來介紹一個滿實用的網站 – ByteArray.org
而這個網站是以ActionScript3.0為主
進行開發一些實驗性的Project
目前的Proj…
[...] Flex/Flash Mouse Gesture Recognition [...]
[...] i was able to understand the code of the gesture recognition script from Bytearray, and was able to implement it in a new package for use with the TUIO system. Concluding that i am [...]
[...] 说几个有趣的东西给大家瞧瞧（页面中都有源码下载）： 第一个，Flash识别鼠标手势。 点这儿直接观看演示 [...]
Mouse Gestures in Flash…
Ik was afgelopen week op zoek naar een algoritme voor een project toen ik hier tegen aan liep: “muis gebaren herkenning”. Een hoop vroegere Palm gebruikers herkennen het vast wel. Een snelle manier om met een muis of pen tekens in te voeren. Erg indr…
[...] wrote an interesting post today on Mouse Gesture Recognition [ by Didier Brun aka Foxy ]Here’s a quick [...]
[...] Mouse Gesture Recognition [ by Didier Brun aka Foxy ] ByteArray.org (tags: actionscript flash gesture mouse as3 interface interactive interesting download) [...]
[...] libraries. There’s a JPEG encoder, PDF generator, Zip file generator and even a class for gesture recognition, and many more besides. It’s worth a look. May 9, 2008 | In ActionScript | No [...]
[...] alternative input scheme. As a first step, I’ve taken the Flash Gesture recognition code from Didier Brun and modified it to work with a Wii remote instead of the mouse. It works pretty well – some of the [...]
[...] http://www.bytearray.org/?p=91 Pretty cool! Continue reading » · Written on: 10-10-08 · No Comments » [...]
[...] built with it, strongly based on alogorithms proven in other hardcore languages. This project is a Mouse Gesture Recognition lib for AS3 [...]
[...] Gesture recognition http://www.bytearray.org/?p=91 Explore posts in the same categories: [...]
[...] 一个基于ActionScript的鼠标动作实现 [...]
[...] with the stylus on the DS. To simulate this in Flash, I used mouse gesture recognition based on the excellent class provided by Didier Brun. This class was originally intended to identify written characters, but I [...]
[...] Here is the mouse gesture recognition flash library I showed you the other day. I’m pretty sure it can be rewritten to our needs. [...]
[...] Mouse gesture recognition class for Actionscript 3. If you’ve used a PDA with a stylus to jot notes down, then you have seen the way a touchscreen translates squiggly lines into recognized characters. This class by Didier Brun does that in Flash. I haven’t test-driven it, but Didier’s example code looks super easy to use. If you ever want to deploy an app in an environment with a stylus (like a DS), this could come in handy. [...]
[...] в иностранной части сети, нашлось замечательное решение, позволяющее распознавать жесты мыши. Такого [...]
[...] me rappelai très bien du “MouseGesture” de Didier et j’avais déjà songé à l’adapter. Sauf que j’étais trop teubé et [...]
[...] I wanted to use simple trig rules, in order to have as less computation as possible, but soon realised that I was in need with a grid and vectors… when I discovered the Levenshtein algorythme, and saw that It had already been ported to as3 by Didier Brun alias Foxy. [...]
[...] キーボードを切り替えてGestureを読み込み。初回起動は辞書データの読み込みに1分強かかります。気長に待ちましょう。 グレー部分が入力領域です。真ん中から左で文字入力。右で数字入力です。 入力方式はMouse Gesture Recognition [ by Didier Brun aka Foxy ] < ByteArray.orgを参考に。 [...]
[...] last import statement is a class that helps us recognize mouse gestures. The Mouse Gestures Class was written by Didier [...]
[...] Mouse Gesture Recognition [...]
[...] This script is a a port of the mouse gesture recognition action script by Didier [...]
[...] This script is a port of the mouse gesture recognition action script by Didier [...]
[...] plugin is a port of mouse gesture recognition with ActionScript & uses Walter Zorn’s VectorGraphics [...]
[...] Android port of Fancy Gestures (and original Mouse Gesture Recognition) [...]
[...] plugin is a port of mouse gesture recognition with ActionScript & uses Walter Zorn’s VectorGraphics [...]
[...] – mouse/keyboard Mouse Gestures Senocular’s Key Class – Awesome lightweight class that recreates the as2 Key object. [...]
[...] et les interaction se font avec un stylet et/ou à la main. Avec le stylet… on écrit. Grâce à foxy le mot est reconnu et s’affiche comme il faut à l’écran, il devient une [...]
[...] la base je voulais stocker les tracés sur un minimum d’espace alors j’ai pensé au mouse gesture de bytearray où les caractères sont représentés par une suite de directions. Finalement j’ai pensé [...]
[...] Mouse Gesture Recognition [...]
[...] [...]
[...] 你还记得 Mouse Gesture 吗? [...]
[...] you remember Mouse Gesture [...]
[...] 作者：http://www.bytearray.org/?p=91 ?View Code ACTIONSCRIPT3protected var mg:MouseGesture = null; mg = new MouseGesture(this.stage); mg.addGesture("UP", "6"); mg.addGesture("DOWN", "2"); mg.addGesture("RIGHT", "0"); mg.addGesture("LEFT", "4"); mg.addGesture("CLOCK", "67012345"); mg.addGesture("ANTICLOCK", "65432107"); mg.addEventListener(GestureEvent.GESTURE_MATCH,matchHandler); // 不同鼠标手势执行不同的处理 protected function matchHandler(event:GestureEvent):void { TweenMax.killAllTweens(); switch (event.datas) { case "UP": TweenMax.to(mesh, 1, {rotationX: mesh.rotationX – 360}); break; case "DOWN": TweenMax.to(mesh, 1, {rotationX: mesh.rotationX + 360}); break; case "LEFT": TweenMax.to(mesh, 1, {rotationY: mesh.rotationY + 360}); break; case "RIGHT": TweenMax.to(mesh, 1, {rotationY: mesh.rotationY – 360}); break; case "CLOCK": TweenMax.to(mesh, 1, {rotationZ: mesh.rotationZ + 360}); break; case "ANTICLOCK": TweenMax.to(mesh, 1, {rotationZ: mesh.rotationZ – 360}); break; } } [...]
[...] mal hier Mouse Gesture Recognition <font size='1'>[ by Didier Brun aka Foxy ]</font> – ByteArray…. hoffe das [...]
[...] into it, there is already a library in AS3 that’s really simple to use it and it works great. The Mouse Gesture Recognition lib for AS3, created by Didier Brun, implements all of the functionality that we need to detect mouse gestures; [...]
[...] into it, there is already a library in AS3 that’s really simple to use it and it works great. The Mouse Gesture Recognition lib for AS3, created by Didier Brun, implements all of the functionality that we need to detect mouse gestures; [...]
[...] At ByteArray.org [...]
[...] second package I looked into was posted in a short article at ByteArray.org by Didier Brun. I caught this post when it first came out (almost three years ago) and was very impressed with the [...]