I recently saw this tread:
http://forum.xda-developers.com/showthread.php?t=370632
i liked the idea, and when i thought about it, gesture recognition didn't seem to hard. And guess what - it really wasn't hard
I made a simple application recognizing gestures defined in an external configuration file. It was supposed to be a gesture launcher, but i didn't find out how to launch an app from a winCE program yet. Also, it turned out to be a bit to slow for that because of the way i draw gesture trails - i'd have to rewrite it almost from scratch to make it really useful and i don't have time for that now.
So i decided to share the idea and the source code, just to demonstrate how easy it is to include gesture recognition in your software.
My demo app is written in C, using XFlib for graphics and compiled using CeGCC so you'll need both of them to compile it (download and install instructions are on the Xflib homepage: www.xflib.net)
The demo program is just an exe file - extract it anywhere on your device, no installation needed. You'll also need to extract the gestureConfig.ini to the root directory of your device, or the program won't run.
Try some of the gestures defined in the ini (like 'M' letter - 8392, a rectangle - 6248, a triangle - 934), or define some of your own to see how the recognition works. Make sure that you have a string consisting of numbers, then space of a tabulator (or more of them) and some text - anything will do, just make sure that there's more than just the numbers in each line. Below you can set the side sensitivity to tweak recognition (see the rest of the post for description on how it works). Better leave the other parameter as it is - seems to work best with this value.
Now, what the demo app does:
It recognizes direction of drawn strokes, and prints them on the bottom of the screen as a string of numbers representing them (described bellow). If a drawn gesture matches one of the patterns in the config file, the entire drawn gesture gets highlited. It works the best with stylus, but is usable with finger as well.
Clicking the large rectangle closes the app.
And how it does it:
The algorithm i used is capable of recognizing strokes drawn in eight directions - horizontally, vertically and diagonally. Directions are described with numbers from 1 to 9, arranged like on a PC numerical keyboard:
Code:
7 8 9
4 6
1 2 3
So a gesture defined in config as 6248 is right-down-left-up - a ractangle.
All that is needed to do the gesture recognition is last few positions of the stylus. In my program i recorded the entire path for drawing if, but used only 5 last positions. The entire trick is to determine which way the stylus is moving, and if it moves one way long enough, store this direction as a stroke.
The easiest way would be to subtract previous stylus position from current one, like
Code:
vectorX=stylusX[i]-stylusX[i-1];
vectorY=stylusY[i]-stylusY[i-1];
[code]
But this method would be highly inaccurate due to niose generated by some digitizers, especially with screen protectors, or when using a finger (try drawing a straight line with your finger in some drawing program)
That's why i decided to calculate an average vector instead:
[code]
averageVectorX=((stylusHistoryX[n]-stylusHistoryX[n-5])+
(stylusHistoryX[n-1]-stylusHistoryX[n-5])
(stylusHistoryX[n-2]-stylusHistoryX[n-5])
(stylusHistoryX[n-3]-stylusHistoryX[n-5])
(stylusHistoryX[n-4]-stylusHistoryX[n-5]))/5;
//Y coordinate is calculated the same way
where stylusHistoryX[n] is the current X position of stylus, and stylusHistoryX[n-1] is the previous position, etc.
Such averaging filters out the noise, without sacrificing too much responsiveness, and uses only a small number of samples. It also has another useful effect - when the stylus changes movement direction, the vector gets shorter.
Now, that we have the direction of motion, we'll have to check how fast the stylus is moving (how long the vector is):
Code:
if(sqrt(averageVectorX*averageVectorX+averageVectorY*averageVectorY)>25)
(...)
If the vector is long enough, we'll have to determine which direction it's facing. Since usually horizontal and vertical lines are easier to draw than diagonal, it's nice to be able to adjust the angle at which the line is considered diagonal or vertical. I used the sideSensitivity parameter for that (can be set in the ini file - range its is from 0 to 100). See the attached image to see how it works.
The green area on the images is the angle where the vector is considered horizontal or vertical. Blue means angles where the vector is considered diagonal. sideSensitivity for those pictures are: left one - 10, middle - 42 (default value, works fine for me ), right - 90. Using o or 100 would mean that horizontal or vertical stroke would be almost impossible to draw.
to make this parameter useful, there are some calculations needed:
Code:
sideSensitivity=tan((sideSensitivity*45/100)*M_PI/180);
First, the range of the parameter is changed from (0-100) to (0-22), meaning angle in degrees of the line dividing right section (green) and top-right (blue). hat angle is then converted to radians, and tangent of this angle (in radians) is being calculated, giving slope of this line.
Having the slope, it's easy to check if the vector is turned sideways or diagonal. here's a part of source code that does the check, it is executed only if the vector is long enough (condition written above):
Code:
if( abs(averageVectorY)<sideSensitivity*abs(averageVectorX) ||
abs(averageVectorX)<sideSensitivity*abs(averageVectorY)) //Vector is turned sideways (horizontal or vertical)
{
/*Now that we know that it's facing sideways, we'll check which side it's actually facing*/
if( abs(averageVectorY)<sideSensitivity*averageVectorX) //Right Gesture
gestureStroke='6'; //storing the direction of vector for later processing
if( abs(averageVectorY)<sideSensitivity*(-averageVectorX)) //Left Gesture
gestureStroke='4';
if( abs(averageVectorX)<sideSensitivity*(averageVectorY)) //Down gesture
gestureStroke='2';
if( abs(averageVectorX)<sideSensitivity*(-averageVectorY)) //Up gesture
gestureStroke='8';
}
else
{ //Vector is diagonal
/*If the vector is not facing isdeways, then it's diagonal. Checking which way it's actually facing
and storing it for later use*/
if(averageVectorX>0 && averageVectorY>0) //Down-Right gesture
gestureStroke='3';
if(averageVectorX>0 && averageVectorY<0) //Up-Right gesture
gestureStroke='9';
if(averageVectorX<0 && averageVectorY>0) //Down-Left gesture
gestureStroke='1';
if(averageVectorX<0 && averageVectorY<0) //Up-Left gesture
gestureStroke='7';
}
Now, we have a character (i used char type, so i can use character strings for string gestures - they can be easily loaded from file and compared with strcmp() ) telling which way the stylus is moving. To avoid errors, we'll have to make sure that the stylus moves in the same direction for a few cycles before storing it as a gesture stroke by increasing a counter as long as it keeps moving in one direction, and resetting it if it changes the direction. If the counter value is bigger than some threshold (pathSensitivity variable is used as this threshold in my program), we can store the gestureStroke value into a string, but only if it's different from previous one - who needs a gesture like "44444" when dragging the stylus left?
After the stylus is released, you'll have to compare generated gesture string to some patterns (eg. loaded from a configuration file), and if it matches, do an appropriate acton.
See the source if you want to see how it can be done, this post already is quite long
If you have any questions, post them and i'll do my best to answer.
Feel free to use this method, parts of, or the entire source in your apps. I'm really looking forward to seeing some gesture-enabled programs
Very nice work. Reading your post was very insightful, and I hope this can provide the basis for some new and exciting apps!
great app... and well done for not just thinking that seems easy... but actually doing it...
ive been a victim of that myself
very nice work man.. one question in which tool did you write code.. i mean it looks like C but how you test and all..
Great app, i see that it is just proof of concept at this stage, but i see that it can be used in future applications control
Continiue with your great work
nik_for_you said:
very nice work man.. one question in which tool did you write code.. i mean it looks like C but how you test and all..
Click to expand...
Click to collapse
It is C, (no "++", no "#", no ".NET", just god old C ) compiled with opensource compiler CeGCC (works under linux, or under windows using cygwin - a unix emulator), developed in opensource IDE Vham (but even a notepad, or better notepad++ would do), tested directly on my Wizard (without emulator). I used XFlib which simplifies graphics and input handling to a level where anyone who ever programed anything at all should be able to handle it - it providea an additional layer between the programmer and the OS. You talk to Xflib, and Xflib talks to the OS. I decided to use this library, because i wanted to try it out anyway.
If i decide to rewrite it and make an actual launcher or anything else out of it, i'll have to use something with a bit faster and more direct screen access (probably SDL, since i already done some programing for desktop PC with it) - XFlib concentrates on usage of sprites - like 2D console games. every single "blob" of the gesture trail is a separate sprite , which has to be drawn each time the screen is refreshed - that is what slows down the app so much. The gesture recognition itself is really fast.
Very good program i just test it and it works very well some combinaison are pretty hard to realize but i like this blue point turning red with command2 and 934. Goond luck , i'll continue to see your job maybe you'll code a very interesting soft.
Interesting work.... would like to see this implemented in an app, could be very useful.
If you want I have some code I did for NDS coding, and which I ported on PocketPC for XFlib.
It works perfectly well and I use it in Skinz Sudoku to recognize the drawn numbers.
The method is pretty simple : when the stylus is pressed, enter the stylus coordinates in a big array. And when it's released, it takes 16 points (could be changed depending on what you need) at the same distance from each other, checks the angle, and gives you the corresponding 'char'.
To add new shapes, it's just a 15 character string which you link to any char (like, link right movement to 'r', or 'a', or a number, or whatever ^^). It works for pretty much any simple shape, and I even used it to do a graffitti-like thing on NDS which worked really well
Hey!
How do you get the last Stylus Positions?
and how often do you read them out
I want to realice such a code under vb.net, but I don't know how i should read out the last stylus positions, to get them perfectly for such calculations
Code:
Private Sub frmGesture_MouseMove(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles MyBase.MouseMove
If StylusJump = 1 Then
StylusJump += 1
If (CurrentStylusPosition.X <> frmGesture.MousePosition.X) Or (CurrentStylusPosition.Y <> frmGesture.MousePosition.Y) Then
LastStylusPosition(9).X = LastStylusPosition(8).X
LastStylusPosition(9).Y = LastStylusPosition(8).Y
LastStylusPosition(8).X = LastStylusPosition(7).X
LastStylusPosition(8).Y = LastStylusPosition(7).Y
LastStylusPosition(7).X = LastStylusPosition(6).X
LastStylusPosition(7).Y = LastStylusPosition(6).Y
LastStylusPosition(6).X = LastStylusPosition(5).X
LastStylusPosition(6).Y = LastStylusPosition(5).Y
LastStylusPosition(5).X = LastStylusPosition(4).X
LastStylusPosition(5).Y = LastStylusPosition(4).Y
LastStylusPosition(4).X = LastStylusPosition(3).X
LastStylusPosition(4).Y = LastStylusPosition(3).Y
LastStylusPosition(3).X = LastStylusPosition(2).X
LastStylusPosition(3).Y = LastStylusPosition(2).Y
LastStylusPosition(2).X = LastStylusPosition(1).X
LastStylusPosition(2).Y = LastStylusPosition(1).Y
LastStylusPosition(1).X = CurrentStylusPosition.X
LastStylusPosition(1).Y = CurrentStylusPosition.Y
CurrentStylusPosition.X = frmGesture.MousePosition.X
CurrentStylusPosition.Y = frmGesture.MousePosition.Y
End If
Dim LabelString As String
Dim iCount As Integer
LabelString = "C(" & CurrentStylusPosition.X & "\" & CurrentStylusPosition.Y & ")"
For iCount = 1 To 9
LabelString = LabelString & " " & iCount & "(" & LastStylusPosition(iCount).X & "\" & LastStylusPosition(iCount).Y & ")"
Next
lblGesture.Text = LabelString
ElseIf StylusJump <= 3 Then
StylusJump += 1
Else
StylusJump = 1
End If
End Sub
Sorry, i didn't notice your post before. I guess that you have the problem solved now, that you released a beta of gesture launcher?
Anyway, you don't really need 10 last positions, in my code i used only 5 for calculations and it still worked fine.
Nice thread, thanks for sharing.
Human-,achine interface has always been an interesting subject to me, and the release of ultimatelaunch has sparked an idea. I am trying to acheive a certain look-and-feel interface - entirely using components that are today screen and ultimatelaunch compatible. Basically a clock strip with a few status buttons at the top, and an ultimatelaunch cube for the main lower portion of the screen - gesture left/right to spin the cube, and each face should have lists of info / icons scrolled by vertical gesture. I'm talking big chunky buttons here - tasks, calendar appts, (quick) contacts, music/video playlists - all vertical lists, one item per row, scrolling in various faces of the cube.
Done the top bit using rlToday for now. Set it to type 5 so scrollbars never show for the top section. All good. Cobbling together bits for the faces, but few of the apps are exactly what I want, some (like that new face contacts one) are pretty close, and being a bit of an armchair coder, I thought now would be a good opportunity to check out WM programming and see if I can't at least come up with a mockup of what I want if not a working app.
I was wondering if anyone could advise me on whether I should bother recognising gestures in such a way as this. Does WM not provide gesture detection for the basic up, down, left, right? Actually, all the stuff I have in mind would just require up/down scrolling of a pane - I was thinking that I may well not need to code gesture support at all, just draw a vertical stack of items, let it overflow to create a scrollbar and then just use the normal WM drag-to-scroll feature (if it exists) to handle the vertical scrolling by gesture in the face of the cube. I would rather keep the requirements to a minimum (eg touchflo), both for dependancy and compatibility reasons, so maybe doing the detection manually would be the "way to go", I dunno.
Did you release source with the app launcher? A library maybe? I intend to go open source with anything I do, so if you are doing the same then I would love to have access to your working code
Nice work man.
Impressive.
Help...!! I'm new with WM developing,
There are 4 questions I'm about to ask about HTC HD2 using WM6.5:
1. How to disable hardware function and handle their event 'on press' for every HW button (such as Home, Window, Talk, End and Back button)?
So far, I just able to disable it using :
[DllImport("coredll.dll")]
private static extern bool UnregisterFunc1(KeyModifiers modifiers, int keyID);
[DllImport("coredll.dll", SetLastError = true)]
public static extern bool RegisterHotKey(IntPtr hWnd, // handle to window
int id, // hot key identifier
KeyModifiers Modifiers, // key-modifier options
int key //virtual-key code
);
Buttons that I can successfully disable are Volume Up, Volume Down, Back, and Talk.
There is no way I can handle or disable Windows, Home and end button.
I have a trick by disable Window button:
IntPtr hTaskBar = FindWindow("HHTaskBar", null);
EnableWindow(hTaskBar, false);
But when I try to lock and unlock the screen using end button, Window button function will not disable anymore.
So far, some forums mention about SHCMBM_OVERRIDEKEY and handle the message by using Microsoft.WindowsCE.Forms.MessageWindow,
and again I still don't know how to attach MessageWindow to my foreground application
2. I'd like if my panel finger gesture behavior works as well as ListView or DataGrid. So, I found a library that can help me.
code.msdn.microsoft.com/gestureswm/Release/ProjectReleases.aspx?ReleaseId=3146
Unfortunatelly, it makes my controls in panel flicker badly. Do you have any other alternative?
3. To handle bluetooth, I'm using
[DllImport("BthUtil.dll")]
public static extern int BthGetMode(out BluetoothMode dwMode);
[DllImport("BthUtil.dll")]
public static extern int BthSetMode(BluetoothMode dwMode);
and InTheHand.Net Library to send data.
But since I'm starting with this device, I even cannot deactive the Bluetooth.
Can you help me with this Bluetooth control and data communication?
4. Since I'm using virtual keyboard as my input, How to define what kind of input panel it will show?
For Example, if my Textbox1 is focused, I have to show Numeric Keypad.
But if Textbox2 is focused, I have to show Qwerty keyboard.
Thank you very much.
PS : I'm using C# for developing the application. It'll be very good if you give me a C# example for those questions.
I know this is a few months old, and I don't wanna ressurect it per-say - and i'm not farmiliar with the rules as it relates to posting on these forums, but i've been viewing them forever, and as a member for a few months at least, and i've got to say that disabling the END key is a must for the HTC HD2, especially for Android NAND. At present with my phone, and thousands of others, everytime we press the END button, the screen stops being responsive. Funny thing is, if i NEVER press the end button, my screen works great. I end calls using the screen, and i've installed 'Button Saviour' from the market to use the 'END' button virtually on-screen, and I wake the phone using any of the other 4 buttons, save for 'END', and i get no problems. It's the Damned signal sent to the screen from the END button that causes the screen to disable and never re-enable the digitizer - I am just supposing these things, I'm in no way or shape a developer... Just something to think about?
This could be the end to thousands of HTC HD2 end user issues relating to lockscreen lock up in Android or otherwise...
Hello everyone. My name is tony. First off sorry for being a noob. I should have been learned this. I would like to ask if i could get some help. I want to create a simple calculator that just has a GUI with a grid like layout of clickable buttons at the lower-left-hand side of the activity that has the value 0-9,addition, subtraction,
multiplication,division,decimal, percentage,calculate,clear and clear all . I would like a Text-view above the buttons to display the values of the buttons when clicked. I also would like the result of the calculation to be in the upper right hand corner of the screen that fades in when the user clicks the calculate button .I have searched around and have looked at lots of source code for calculators but there not really "nooob friendlyy"
Some Logic of what im wanting:
user click numbutton -> button pushes its value to Text-View -> user click numbutton -> button pushes its value to Text-View
->user clicks an operand-> Text-View clears its Text on next numbutton press (but still holding that prior value the user entered)->user clicks another numbutton (if they clicked a numbutton it's value would be set to the text-view) or the calculate button -> calculate the two values using the clicked operand and set it to the TextView Result->TextView Result fades in at the top right hand corner of the activity.
Here is were i get
(1)I'm trying to use int/longs and float/doubles depending if the decimal is in the text view when the user clicks the operand.
if is there, expect and return floats and if not, expect and return int's.
(2)parse commands parseInt().gettext().tostring coverts the int to string and respectively for the other data types correct?Is there some benefit to covering to strings?
Rather is it possible to have this implemented using a switch statement?
tone5992 said:
Hello everyone. My name is tony. First off sorry for being a noob. I should have been learned this. I would like to ask if i could get some help. I want to create a simple calculator that just has a GUI with a grid like layout of clickable buttons at the lower-left-hand side of the activity that has the value 0-9,addition, subtraction,
multiplication,division,decimal, percentage,calculate,clear and clear all . I would like a Text-view above the buttons to display the values of the buttons when clicked. I also would like the result of the calculation to be in the upper right hand corner of the screen that fades in when the user clicks the calculate button .I have searched around and have looked at lots of source code for calculators but there not really "nooob friendlyy"
Some Logic of what im wanting:
user click numbutton -> button pushes its value to Text-View -> user click numbutton -> button pushes its value to Text-View
->user clicks an operand-> Text-View clears its Text on next numbutton press (but still holding that prior value the user entered)->user clicks another numbutton (if they clicked a numbutton it's value would be set to the text-view) or the calculate button -> calculate the two values using the clicked operand and set it to the TextView Result->TextView Result fades in at the top right hand corner of the activity.
Here is were i get
(1)I'm trying to use int/longs and float/doubles depending if the decimal is in the text view when the user clicks the operand.
if is there, expect and return floats and if not, expect and return int's.
(2)parse commands parseInt().gettext().tostring coverts the int to string and respectively for the other data types correct?Is there some benefit to covering to strings?
Rather is it possible to have this implemented using a switch statement?
Click to expand...
Click to collapse
You should use Double.parseDouble method