Related
I recently saw this tread:
http://forum.xda-developers.com/showthread.php?t=370632
i liked the idea, and when i thought about it, gesture recognition didn't seem to hard. And guess what - it really wasn't hard
I made a simple application recognizing gestures defined in an external configuration file. It was supposed to be a gesture launcher, but i didn't find out how to launch an app from a winCE program yet. Also, it turned out to be a bit to slow for that because of the way i draw gesture trails - i'd have to rewrite it almost from scratch to make it really useful and i don't have time for that now.
So i decided to share the idea and the source code, just to demonstrate how easy it is to include gesture recognition in your software.
My demo app is written in C, using XFlib for graphics and compiled using CeGCC so you'll need both of them to compile it (download and install instructions are on the Xflib homepage: www.xflib.net)
The demo program is just an exe file - extract it anywhere on your device, no installation needed. You'll also need to extract the gestureConfig.ini to the root directory of your device, or the program won't run.
Try some of the gestures defined in the ini (like 'M' letter - 8392, a rectangle - 6248, a triangle - 934), or define some of your own to see how the recognition works. Make sure that you have a string consisting of numbers, then space of a tabulator (or more of them) and some text - anything will do, just make sure that there's more than just the numbers in each line. Below you can set the side sensitivity to tweak recognition (see the rest of the post for description on how it works). Better leave the other parameter as it is - seems to work best with this value.
Now, what the demo app does:
It recognizes direction of drawn strokes, and prints them on the bottom of the screen as a string of numbers representing them (described bellow). If a drawn gesture matches one of the patterns in the config file, the entire drawn gesture gets highlited. It works the best with stylus, but is usable with finger as well.
Clicking the large rectangle closes the app.
And how it does it:
The algorithm i used is capable of recognizing strokes drawn in eight directions - horizontally, vertically and diagonally. Directions are described with numbers from 1 to 9, arranged like on a PC numerical keyboard:
Code:
7 8 9
4 6
1 2 3
So a gesture defined in config as 6248 is right-down-left-up - a ractangle.
All that is needed to do the gesture recognition is last few positions of the stylus. In my program i recorded the entire path for drawing if, but used only 5 last positions. The entire trick is to determine which way the stylus is moving, and if it moves one way long enough, store this direction as a stroke.
The easiest way would be to subtract previous stylus position from current one, like
Code:
vectorX=stylusX[i]-stylusX[i-1];
vectorY=stylusY[i]-stylusY[i-1];
[code]
But this method would be highly inaccurate due to niose generated by some digitizers, especially with screen protectors, or when using a finger (try drawing a straight line with your finger in some drawing program)
That's why i decided to calculate an average vector instead:
[code]
averageVectorX=((stylusHistoryX[n]-stylusHistoryX[n-5])+
(stylusHistoryX[n-1]-stylusHistoryX[n-5])
(stylusHistoryX[n-2]-stylusHistoryX[n-5])
(stylusHistoryX[n-3]-stylusHistoryX[n-5])
(stylusHistoryX[n-4]-stylusHistoryX[n-5]))/5;
//Y coordinate is calculated the same way
where stylusHistoryX[n] is the current X position of stylus, and stylusHistoryX[n-1] is the previous position, etc.
Such averaging filters out the noise, without sacrificing too much responsiveness, and uses only a small number of samples. It also has another useful effect - when the stylus changes movement direction, the vector gets shorter.
Now, that we have the direction of motion, we'll have to check how fast the stylus is moving (how long the vector is):
Code:
if(sqrt(averageVectorX*averageVectorX+averageVectorY*averageVectorY)>25)
(...)
If the vector is long enough, we'll have to determine which direction it's facing. Since usually horizontal and vertical lines are easier to draw than diagonal, it's nice to be able to adjust the angle at which the line is considered diagonal or vertical. I used the sideSensitivity parameter for that (can be set in the ini file - range its is from 0 to 100). See the attached image to see how it works.
The green area on the images is the angle where the vector is considered horizontal or vertical. Blue means angles where the vector is considered diagonal. sideSensitivity for those pictures are: left one - 10, middle - 42 (default value, works fine for me ), right - 90. Using o or 100 would mean that horizontal or vertical stroke would be almost impossible to draw.
to make this parameter useful, there are some calculations needed:
Code:
sideSensitivity=tan((sideSensitivity*45/100)*M_PI/180);
First, the range of the parameter is changed from (0-100) to (0-22), meaning angle in degrees of the line dividing right section (green) and top-right (blue). hat angle is then converted to radians, and tangent of this angle (in radians) is being calculated, giving slope of this line.
Having the slope, it's easy to check if the vector is turned sideways or diagonal. here's a part of source code that does the check, it is executed only if the vector is long enough (condition written above):
Code:
if( abs(averageVectorY)<sideSensitivity*abs(averageVectorX) ||
abs(averageVectorX)<sideSensitivity*abs(averageVectorY)) //Vector is turned sideways (horizontal or vertical)
{
/*Now that we know that it's facing sideways, we'll check which side it's actually facing*/
if( abs(averageVectorY)<sideSensitivity*averageVectorX) //Right Gesture
gestureStroke='6'; //storing the direction of vector for later processing
if( abs(averageVectorY)<sideSensitivity*(-averageVectorX)) //Left Gesture
gestureStroke='4';
if( abs(averageVectorX)<sideSensitivity*(averageVectorY)) //Down gesture
gestureStroke='2';
if( abs(averageVectorX)<sideSensitivity*(-averageVectorY)) //Up gesture
gestureStroke='8';
}
else
{ //Vector is diagonal
/*If the vector is not facing isdeways, then it's diagonal. Checking which way it's actually facing
and storing it for later use*/
if(averageVectorX>0 && averageVectorY>0) //Down-Right gesture
gestureStroke='3';
if(averageVectorX>0 && averageVectorY<0) //Up-Right gesture
gestureStroke='9';
if(averageVectorX<0 && averageVectorY>0) //Down-Left gesture
gestureStroke='1';
if(averageVectorX<0 && averageVectorY<0) //Up-Left gesture
gestureStroke='7';
}
Now, we have a character (i used char type, so i can use character strings for string gestures - they can be easily loaded from file and compared with strcmp() ) telling which way the stylus is moving. To avoid errors, we'll have to make sure that the stylus moves in the same direction for a few cycles before storing it as a gesture stroke by increasing a counter as long as it keeps moving in one direction, and resetting it if it changes the direction. If the counter value is bigger than some threshold (pathSensitivity variable is used as this threshold in my program), we can store the gestureStroke value into a string, but only if it's different from previous one - who needs a gesture like "44444" when dragging the stylus left?
After the stylus is released, you'll have to compare generated gesture string to some patterns (eg. loaded from a configuration file), and if it matches, do an appropriate acton.
See the source if you want to see how it can be done, this post already is quite long
If you have any questions, post them and i'll do my best to answer.
Feel free to use this method, parts of, or the entire source in your apps. I'm really looking forward to seeing some gesture-enabled programs
Very nice work. Reading your post was very insightful, and I hope this can provide the basis for some new and exciting apps!
great app... and well done for not just thinking that seems easy... but actually doing it...
ive been a victim of that myself
very nice work man.. one question in which tool did you write code.. i mean it looks like C but how you test and all..
Great app, i see that it is just proof of concept at this stage, but i see that it can be used in future applications control
Continiue with your great work
nik_for_you said:
very nice work man.. one question in which tool did you write code.. i mean it looks like C but how you test and all..
Click to expand...
Click to collapse
It is C, (no "++", no "#", no ".NET", just god old C ) compiled with opensource compiler CeGCC (works under linux, or under windows using cygwin - a unix emulator), developed in opensource IDE Vham (but even a notepad, or better notepad++ would do), tested directly on my Wizard (without emulator). I used XFlib which simplifies graphics and input handling to a level where anyone who ever programed anything at all should be able to handle it - it providea an additional layer between the programmer and the OS. You talk to Xflib, and Xflib talks to the OS. I decided to use this library, because i wanted to try it out anyway.
If i decide to rewrite it and make an actual launcher or anything else out of it, i'll have to use something with a bit faster and more direct screen access (probably SDL, since i already done some programing for desktop PC with it) - XFlib concentrates on usage of sprites - like 2D console games. every single "blob" of the gesture trail is a separate sprite , which has to be drawn each time the screen is refreshed - that is what slows down the app so much. The gesture recognition itself is really fast.
Very good program i just test it and it works very well some combinaison are pretty hard to realize but i like this blue point turning red with command2 and 934. Goond luck , i'll continue to see your job maybe you'll code a very interesting soft.
Interesting work.... would like to see this implemented in an app, could be very useful.
If you want I have some code I did for NDS coding, and which I ported on PocketPC for XFlib.
It works perfectly well and I use it in Skinz Sudoku to recognize the drawn numbers.
The method is pretty simple : when the stylus is pressed, enter the stylus coordinates in a big array. And when it's released, it takes 16 points (could be changed depending on what you need) at the same distance from each other, checks the angle, and gives you the corresponding 'char'.
To add new shapes, it's just a 15 character string which you link to any char (like, link right movement to 'r', or 'a', or a number, or whatever ^^). It works for pretty much any simple shape, and I even used it to do a graffitti-like thing on NDS which worked really well
Hey!
How do you get the last Stylus Positions?
and how often do you read them out
I want to realice such a code under vb.net, but I don't know how i should read out the last stylus positions, to get them perfectly for such calculations
Code:
Private Sub frmGesture_MouseMove(ByVal sender As Object, ByVal e As System.Windows.Forms.MouseEventArgs) Handles MyBase.MouseMove
If StylusJump = 1 Then
StylusJump += 1
If (CurrentStylusPosition.X <> frmGesture.MousePosition.X) Or (CurrentStylusPosition.Y <> frmGesture.MousePosition.Y) Then
LastStylusPosition(9).X = LastStylusPosition(8).X
LastStylusPosition(9).Y = LastStylusPosition(8).Y
LastStylusPosition(8).X = LastStylusPosition(7).X
LastStylusPosition(8).Y = LastStylusPosition(7).Y
LastStylusPosition(7).X = LastStylusPosition(6).X
LastStylusPosition(7).Y = LastStylusPosition(6).Y
LastStylusPosition(6).X = LastStylusPosition(5).X
LastStylusPosition(6).Y = LastStylusPosition(5).Y
LastStylusPosition(5).X = LastStylusPosition(4).X
LastStylusPosition(5).Y = LastStylusPosition(4).Y
LastStylusPosition(4).X = LastStylusPosition(3).X
LastStylusPosition(4).Y = LastStylusPosition(3).Y
LastStylusPosition(3).X = LastStylusPosition(2).X
LastStylusPosition(3).Y = LastStylusPosition(2).Y
LastStylusPosition(2).X = LastStylusPosition(1).X
LastStylusPosition(2).Y = LastStylusPosition(1).Y
LastStylusPosition(1).X = CurrentStylusPosition.X
LastStylusPosition(1).Y = CurrentStylusPosition.Y
CurrentStylusPosition.X = frmGesture.MousePosition.X
CurrentStylusPosition.Y = frmGesture.MousePosition.Y
End If
Dim LabelString As String
Dim iCount As Integer
LabelString = "C(" & CurrentStylusPosition.X & "\" & CurrentStylusPosition.Y & ")"
For iCount = 1 To 9
LabelString = LabelString & " " & iCount & "(" & LastStylusPosition(iCount).X & "\" & LastStylusPosition(iCount).Y & ")"
Next
lblGesture.Text = LabelString
ElseIf StylusJump <= 3 Then
StylusJump += 1
Else
StylusJump = 1
End If
End Sub
Sorry, i didn't notice your post before. I guess that you have the problem solved now, that you released a beta of gesture launcher?
Anyway, you don't really need 10 last positions, in my code i used only 5 for calculations and it still worked fine.
Nice thread, thanks for sharing.
Human-,achine interface has always been an interesting subject to me, and the release of ultimatelaunch has sparked an idea. I am trying to acheive a certain look-and-feel interface - entirely using components that are today screen and ultimatelaunch compatible. Basically a clock strip with a few status buttons at the top, and an ultimatelaunch cube for the main lower portion of the screen - gesture left/right to spin the cube, and each face should have lists of info / icons scrolled by vertical gesture. I'm talking big chunky buttons here - tasks, calendar appts, (quick) contacts, music/video playlists - all vertical lists, one item per row, scrolling in various faces of the cube.
Done the top bit using rlToday for now. Set it to type 5 so scrollbars never show for the top section. All good. Cobbling together bits for the faces, but few of the apps are exactly what I want, some (like that new face contacts one) are pretty close, and being a bit of an armchair coder, I thought now would be a good opportunity to check out WM programming and see if I can't at least come up with a mockup of what I want if not a working app.
I was wondering if anyone could advise me on whether I should bother recognising gestures in such a way as this. Does WM not provide gesture detection for the basic up, down, left, right? Actually, all the stuff I have in mind would just require up/down scrolling of a pane - I was thinking that I may well not need to code gesture support at all, just draw a vertical stack of items, let it overflow to create a scrollbar and then just use the normal WM drag-to-scroll feature (if it exists) to handle the vertical scrolling by gesture in the face of the cube. I would rather keep the requirements to a minimum (eg touchflo), both for dependancy and compatibility reasons, so maybe doing the detection manually would be the "way to go", I dunno.
Did you release source with the app launcher? A library maybe? I intend to go open source with anything I do, so if you are doing the same then I would love to have access to your working code
Nice work man.
Impressive.
Hi,
I'm writing an open gl game and it requires the user to be able to type input for things such as high scores, saved games and whatnot.
The problem I'm having is that I can't get the soft input keyboard to display with the GLSurfaceView focused for input events.
Is there a trick to it? is it even possible? or do I have to draw my own keyboard with opengl?
I definitely do not want to show a separate activity with android controls because that would look really cheap and subtract from the game experience.
Any help would be appreciated,
Thank you.
Figured it out.
In the GLSurfaceView constructor I needed to set:
Code:
this.setFocusable(true);
this.setFocusableInTouchMode(true);
And then to show the keyboard:
Code:
((InputMethodManager) context.getSystemService(Context.INPUT_METHOD_SERVICE)).showSoftInput(view, 0);
And to hide:
Code:
((InputMethodManager) context.getSystemService(Context.INPUT_METHOD_SERVICE)).hideSoftInputFromWindow(view.getWindowToken(), 0)
Not difficult, but it just isn't documented anywhere, at least not the setFocusable() stuff.
Hi guys,
Teaching myself Android JAVA but am struggling with an issue that's affecting only one major version of Android, which appears to be all of 5.
I have an app using SupportFragments with the v4 support library (also tried it with framework fragments using v13 support library). On two of the fragments I have a ScrollView containing a large set of custom buttons. Both of these screen have an issue where the setColorFilter is being ignored when the scroll position is at the very top of the ScrollView. Oddly, if I scroll even 1 pixel the setColorFilter is working. Baffling.
Note, this only affects Android 5. Android 4, and Android 6 work perfectly. Only the various flavors of 5 I've tested with so far have this problem (5.1.1 - Cyanogen S4, and 5.0.2 - Galaxy Tab S2). The Nexus 9 (6.0), Galaxy Tab 2 (4.2.2) and Galaxy Tab 3 (4.4.2) all work fine.
Below is the example of the the scroll being at the top of the scrollview, and then scrolling by just a tiny margin showing the color filter applied.
******* won't let me post the images yet ****************
The buttons are initially added in the populate method called from the Fragments onCreateView using:
Code:
PasswordButton<Password> btn = (PasswordButton<Password>)inflater.inflate(R.layout.password_button, viewGroup, false);
btn.setObject(password);
The setObject(password) actually sets a password object to the button, and applies the color filter that it inherits from it's password group to the background using the following code (again all from the onCreateView):
Code:
public void setColour() {
int colour = ((ButtonInterface)this.object).getColour();
Drawable background = ContextCompat.getDrawable(getContext(), R.drawable.password_button_selector);
background.setColorFilter(new PorterDuffColorFilter(colour, PorterDuff.Mode.MULTIPLY));
this.setBackground(background);
}
The button's style is a made up of a 9patch background (which is where I'm applying the color filter), a small arrow image, and the text.
I've made a workaround by overriding the onDraw completely for Android 5, and all other variants simply calling super(canvas), but it would be nice to not have to do this as if I start skinning the buttons my custom onDraw will need constant updating.
The searching I've done found allot of issues the other way (i.e. 5 working, but 4 not), and the ones I've found that had issues with 5 I'm already using the methods they say got it working.
Any help would be appreciated
THIS MOD IS ONLY FOR THE JOYING / 2GB / Sofia models - It is stable and working well .
Please use V4.1 if you have not upgraded to Marshmallow!
History:
V6.4: Update using code base 9/1/2017 Joying Update ( Android 6.0 ) (updated Oct 7)
Changes:
- Meida keys - Removed support for media keys customization. No longer sent to launcher.sh due to slow response time.
- Media keys - Modified code to only send media keys to the active media player.
- Changed reboot timer from 1/2 second to 2 seconds. Should help eliminate random reboots when coming from standby.
- Updated player_app.txt and navi_app.txt to latest from @surfer63
- Fixed issue with button 33 not working
- Files included:
- V6.4-Sofia-1-C9-Server_NOKILL.zip - Original file with task killer removed
- V6.4-Sofia-1-C9-Server_NOKILL_GVS.zip - Task killer removed, Google Voice Search on 'SRC' key
- V6.4-Custom_Keys-Mods.zip - Complete key modifications ( requires launcher.sh - description below )
V6.3: Update using code base 5/25/2017 Joying Update ( Android 6.0 )
- Files included:
- V6.3-Sofia-1-C9-Server_NOKILL.zip - Original file with task killer removed
- V6.3-Sofia-1-C9-Server_NOKILL_GVS.zip - Task killer removed, Google Voice Search on 'SRC' key
- V6.3-Custom_Keys-Mods.zip - Complete key modifications ( requires launcher.sh - description below )
V6.2: Updated using code base 5/8/2017 Joying Update ( Android 6.0 )
- Added fix for google maps voice output on radio from @xdamember2
V6.1: Updated code base using 5/8/2017 Joying Update ( Android 6.0 )
V6: Updated code base using 4/27/2017 Joying Update ( Android 6.0 )
Android 5.1.1
V4.1 - Fixed issue with "Google Voice Search" not lowering music volume (3/23/2017) - Android 5.1.1
V4: Updated code base using 3/14/2017 Joying ROM
Changes/Added support for:
- MEDIA KEY - hard key on unit
- BAND KEY - hard key on unit
- ACC_ON - Triggers on key on
- Minor changes to example shell scripts to add cases for MEDIA, BAND, ACC_ON ( launcher.sh )
- FIX/UPDATE - Updated list of NAVI APPS in assets/property/navi_app.txt ( allows more applications to speak during radio )
- Many thanks to @surfer63 for navi app radio fix and windows install script
V3 - Rebuilt using latest Joying ROM dated 2/22/2017 - Includes V1 & V2 steering wheel / key modifications.
Changes/Added support for:
- ACC_OFF - Added support for running actions at key off (case 98 )
- Added case 98 to example scripts to perform "PAUSE" - to stop current music player when key off
V2 Removed killAppWhenSleep(); - Eliminates tasks being killed on sleep
Added support for additional keys / Minor changes to example shell script ( launcher.sh )
- HANG - steering wheel softkey
- DVD - hard key on unit (if available)
- EJECT- hard key on unit (if available)
- CALL/BT softkey - Fixed issue where key would not respond after call disconnect.
V1 Initial version
There is little risk with this mod, but please understand what you are doing, and make backup of the original apk file ( in case you want to revert back ). The installer script does make a backup, but will overwrite the backup if installed more than once.
I also added a simpler "launcher_simple.sh" into the zip, which only changes the functionality of the Source (rotates radio, pandora, spotify ) and Call buttons ( google voice command ). These scripts are working examples of what can be done. If you want the simpler version, just rename "launcher_simple.sh" to "launcher.sh" before installing.
---------------------- Original Post -----------------
I have spent quite a bit of time working on being able to customize the steering wheel keys of my Joying 2GB headunit (JY-UL135N2 / 2GB / Sofia unit). I have made some good progress and thought I would share what I have done so far.
The system package /system/app/Sofia-1-C9-Server-V1.0/Sofia-1-C9-Server-V1.0.apk has been modified to intercept the keys sent by the MCU, and thus able to change the behavior of the key presses. The keys listed below are supported, and will launch a "shell" process, so any action that can be done by the command line can be performed. If your not familiar with the Android command line, you might be surprised how powerful it is. As a side note, this works very fast and efficient. From my testing the overhead of the system call and shell script is almost non-existent, and key presses are registered with no delay. In other words, it is working very nicely.
In addition to the steering wheel button mods, also provided is the ability to launch a command or series of commands when a RESUME is received.
Included is my shell script (example) that can be used as a model for your own customization. The example script included performs the following actions:
CALL/BT PHONE(27) - One press is HOME, 2 presses is BACK, 3 presses brings up Google Voice Commands
MODE/SRC(37) - Rotates through music apps: Radio, Pandora, Spotify, Stock Music App - If it has been longer than 1 minute since the last press it will return you to the last selected music app with first press, then continue with the rotation. ( I'm working on a more intelligent way of doing this, that looks at the process list and will post soon)
NAVI(9) - One press is Torque, 2 presses is google maps
PAUSE/PLAY(85*),NEXT(87*), PREVIOUS(88*), FASTBACK(89*), FASTFWD(90*),HOME(3), BACK(4) - Currently set to perform stock functions. They ARE passed to script so they can be assigned other functions if wanted.
ACC_ON(97) - Executes at ignition switch on. Sample script has a 'play' here to restart music player that was paused at key off.
ACC_OFF(98) - Executes at ignition switch off. I suggest you assign "pause" on this case, to stop current music player. Resolves issue of music player advancing for several minutes after key is turned off.
RESUME(99) - Executes on system resume.
Please note the media control keys marked with a (*) are hardcoded to specific functions within the stock Joying applications. They are only passed through to the shell script when you are NOT using a Joying media application. They will ONLY work as the original keys when you are within the Joying media applications.
Remember, the apk is just handing codes to you, what you do with them after that is up to you! I have tried to comment the example script enough to be understandable to someone with basic programming knowledge.
Attached is a zip file with the updated Sofia-1-C9-Server-V1.0.apk, launcher.sh ( my example script), and install script. The launcher.sh script must be placed in the /data directory ( /data/launcher.sh ), as it is hardcoded in the apk file.
The install script is a shell/linux script, but you can easily copy/paste the adb commands if your using windows.
[NOTE: Please update to the latest version of busybox! I found some bugs in the version included by Joying that caused several issues.]
Awesome work. I will mess with this sometime this week. I still need to root my unit. Just installed it last week.
Sent from my SAMSUNG-SM-G935A using Tapatalk
Excellent work, sir! Before I start modifying the script for personal taste, could you help how to find key number for DVD and Eject buttons that currently have no function as the DVD drive is non-existent? Maybe logcat or something?
sonof said:
Excellent work, sir! Before I start modifying the script for personal taste, could you help how to find key number for DVD and Eject buttons that currently have no function as the DVD drive is non-existent? Maybe logcat or something?
Click to expand...
Click to collapse
I see a few references in the code for the DVD/eject button in the common code. My 'best guess' would be to add a system call to the cases listed below, but I seriously doubt the MCU would even send the codes if there is no DVD present.
bsp/FinalBsp.java
public static final int KEY_CODE_DVD = 242;
public static final int KEY_CODE_EJECT = 231;
bsp/HandlerBspKey.java
public static void keyEvent(int paramInt1, int paramInt2)
case 242:
bspKeyDvd(paramInt2);
return;
case 231:
bspKeyEject(paramInt2);
return;
gustden said:
I see a few references in the code for the DVD/eject button in the common code. My 'best guess' would be to add a system call to the cases listed below, but I seriously doubt the MCU would even send the codes if there is no DVD present.
bsp/FinalBsp.java
public static final int KEY_CODE_DVD = 242;
public static final int KEY_CODE_EJECT = 231;
bsp/HandlerBspKey.java
public static void keyEvent(int paramInt1, int paramInt2)
case 242:
bspKeyDvd(paramInt2);
return;
case 231:
bspKeyEject(paramInt2);
return;
Click to expand...
Click to collapse
Seems that MCU does send the codes as seen here.
I will see what I can do but I doubt I am able to do anything as I just installed apktool the first time today and am new with apk's. Thanks for your efforts! Very appreciated.
sonof said:
Excellent work, sir! Before I start modifying the script for personal taste, could you help how to find key number for DVD and Eject buttons that currently have no function as the DVD drive is non-existent? Maybe logcat or something?
Click to expand...
Click to collapse
sonof said:
Seems that MCU does send the codes as seen here.
I will see what I can do but I doubt I am able to do anything as I just installed apktool the first time today and am new with apk's. Thanks for your efforts! Very appreciated.
Click to expand...
Click to collapse
If I create and send you a apk file with additional debug statements, would you be able to install it, and capture the 'logcat' of the buttons being pressed? It makes it much more difficult since I don't have the DVD buttons. Hacking smali code is a LOT of trial and errors. Their code is quite complex, and doesn't decompile into Java very nicely in some critical areas.
gustden said:
If I create and send you a apk file with additional debug statements, would you be able to install it, and capture the 'logcat' of the buttons being pressed? It makes it much more difficult since I don't have the DVD buttons. Hacking smali code is a LOT of trial and errors. Their code is quite complex, and doesn't decompile into Java very nicely in some critical areas.
Click to expand...
Click to collapse
Yes I am capable of doing that!
sonof said:
Excellent work, sir! Before I start modifying the script for personal taste, could you help how to find key number for DVD and Eject buttons that currently have no function as the DVD drive is non-existent? Maybe logcat or something?
Click to expand...
Click to collapse
sonof said:
Yes I am capable of doing that!
Click to expand...
Click to collapse
I put in debugging code that parses the data stream from the MCU and dumps the first 4 bytes of EVERY message that is received from the MCU (skipping checksum info). Simply replace the original file (make a copy first ) and reboot. When looking for the keys, I suggest you do not use the radio, as it sends a constant flood of messages, such as RDS updates.
Even when the radio is off, there are still messages constantly being sent. If you filter on the debug messages MCUKEY in adb, and press the button in question a few times quickly, you should see a consistent pattern. I was able to identify the 'HANG' key in a couple minutes using this method. You should see something like this in the logcat:
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=0 B2=5 B2=224
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=7 B2=16 B2=0
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=0 B2=0 B2=49
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=7 B2=16 B2=0
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=3 B2=34 B2=0
In this case the HANG key is in bold ( 1, 7, 16, 0 ) messages. IF the DVD and eject keys are sent, they should definitely show up here. Not every key is sent by the MCU, if it is not relevant to the current state of the system. For example the media buttons are not sent when the radio is on, since the MCU is in control of the radio, however it does send the results such as a frequency change resulting from the button press.
It is possible that the MCU knows there is no DVD present and suppresses the messages. From looking at the code, it may be the action of a disc being ejected that is sent from the MCU, with the eject button press handled internally by the MCU. Your more likely to find the press of the DVD button than the eject button, but may find both.
When your done searching, you want to put back the original file and reboot. If you find the codes, I can certainly add them to trigger the shell script and you can take whatever action you want with them.
Thank you! I will try this the first thing next week when I get back home from a weekend trip.
Question: if I remove a customization from the script file for a key, the key should then perform its default functions, correct?
vassandrei said:
Question: if I remove a customization from the script file for a key, the key should then perform its default functions, correct?
Click to expand...
Click to collapse
The keys being passed to the shell script (launcher.sh) is hard-coded in the APK file and is just sent to the script. If you remove it from the script, the key would no longer do anything.
If you want to keep the stock functionality of a specific button, it should be fairly easy to duplicate it in the shell script. For example:
BT/CALL button: "am start com.syu.bt/com.syu.bt.BtAct"
NAVI: By default, I think NAVI just toggles between a Navi app, and the previous activity on the stack.
The code below should behave the same as the original for both the NAVI an BTPHONE buttons ( assuming your using google maps )
Code:
case $key in
9) # NAVI
x=`dumpsys activity activities | grep "Run #" | cut -f 13 -d" "| head -1 | grep maps`
if [ "$x" = "" ]
then
$maps
else
am start `dumpsys activity activities | grep "Run #" | cut -f 13 -d" " | grep -v launcher | grep -v systemui | grep -v maps |head -1`
fi
;;
27) # BTPHONE/PTT
am start com.syu.bt/com.syu.bt.BtAct
;;
Play/Pause: you can just remove the 85 case, it would just hit the default case and pass "input keyevent 85" which is identical in functionality.
gustden said:
The keys being passed to the shell script (launcher.sh) is hard-coded in the APK file and is just sent to the script. If you remove it from the script, the key would no longer do anything.
If you want to keep the stock functionality of a specific button, it should be fairly easy to duplicate it in the shell script. For example:
BT/CALL button: "am start com.syu.bt/com.syu.bt.BtAct"
NAVI: By default, I think NAVI just toggles between a Navi app, and the previous activity on the stack.
The code below should behave the same as the original for both the NAVI an BTPHONE buttons ( assuming your using google maps )
Code:
9) # NAVI
x=`dumpsys activity activities | grep "Run #" | cut -f 13 -d" "| grep -v launcher | grep -v systemui | head -1 | grep maps`
if [ "$x" = "" ]
then
$maps
else
am start `dumpsys activity activities | grep "Run #" | cut -f 13 -d" " | grep -v launcher | grep -v systemui | head -2 | tail -1`
fi
;;
27) # BTPHONE/PTT
am start com.syu.bt/com.syu.bt.BtAct
;;
Play/Pause: you can just remove the 85 case, it would just hit the default case and pass "input keyevent 85" which is identical in functionality.
Click to expand...
Click to collapse
brilliant! thank you so much!
Last thing: I want to have viper4android launch on resume (in the background ideally). where can I find the "app identifier" to put in the shell script?
vassandrei said:
Question: if I remove a customization from the script file for a key, the key should then perform its default functions, correct?
Click to expand...
Click to collapse
vassandrei said:
brilliant! thank you so much!
Last thing: I want to have viper4android launch on resume (in the background ideally). where can I find the "app identifier" to put in the shell script?
Click to expand...
Click to collapse
I made a slight change to the 'CODE' section of my previous post, so if you copy it make sure you get the edited version. I tested it and made a couple of changes.
There are several ways to get the package name and activity.
What I typically do is start the application on the unit and do the following in adb:
> adb shell
> dumpsys activity activities | grep "Run #"
This shows you a list of the activities ( package/activity ) in the task list with most recent at the top.
You should be able to pick it out pretty easily. If it is the app that I found with google, this should work to start it.
> am start com.vipercn.viper4android_v2/.activity.ViPER4Android
In most cases, you can just give the package name when using 'am start' with no need for the activity. It will default to the 'LAUNCHER' activity in the package. I usually always give the activity, just to be safe. I would sleep for a couple seconds to give it time to initialize and then $home to take you back to the Home screen. So whole command would look like this. It is easy to test using adb.
am start com.vipercn.viper4android_v2/.activity.ViPER4Android ; sleep 2 ; $home
Thanks for this, it's working great on my Joying (JY-UL135N2 ).
My only issue is that the CALL/BT PHONE (27) button is assigned to my "Pick Up/Call Out" button on my steering wheel. So I can't use it to pick up calls. I actually have another dedicated button on my steering to activate voice commands, so I'd rather assign the CALL/BT PHONE function to that. Do you happen to know how to find out what numbers corresponds to each button press?
akuma148 said:
Thanks for this, it's working great on my Joying (JY-UL135N2 ).
My only issue is that the CALL/BT PHONE (27) button is assigned to my "Pick Up/Call Out" button on my steering wheel. So I can't use it to pick up calls. I actually have another dedicated button on my steering to activate voice commands, so I'd rather assign the CALL/BT PHONE function to that. Do you happen to know how to find out what numbers corresponds to each button press?
Click to expand...
Click to collapse
I make and receive so few voice calls, It wasn't worth having a steering wheel button dedicated for the dialer/call pickup . I just tap the screen to answer and use google voice to make calls.
If you want to have a button answer a call, there *should* be a way to do it given the right command. I will play around with it more. Worse case I can just back out the changes for the (27) , and send you an updated apk.
I used the joying steering wheel control app and switched the dialer/call pickup to my voice button and that works, but my dialer/call pickup doesn't do anything. Please let me know if you want me to test anything. Thank you.
gustden said:
I put in debugging code that parses the data stream from the MCU and dumps the first 4 bytes of EVERY message that is received from the MCU (skipping checksum info). Simply replace the original file (make a copy first ) and reboot. When looking for the keys, I suggest you do not use the radio, as it sends a constant flood of messages, such as RDS updates.
Even when the radio is off, there are still messages constantly being sent. If you filter on the debug messages MCUKEY in adb, and press the button in question a few times quickly, you should see a consistent pattern. I was able to identify the 'HANG' key in a couple minutes using this method. You should see something like this in the logcat:
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=0 B2=5 B2=224
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=7 B2=16 B2=0
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=0 B2=0 B2=49
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=7 B2=16 B2=0
D/MCUKEY ( 693): MCUKEYINPUT: B1=1 B2=3 B2=34 B2=0
In this case the HANG key is in bold ( 1, 7, 16, 0 ) messages. IF the DVD and eject keys are sent, they should definitely show up here. Not every key is sent by the MCU, if it is not relevant to the current state of the system. For example the media buttons are not sent when the radio is on, since the MCU is in control of the radio, however it does send the results such as a frequency change resulting from the button press.
It is possible that the MCU knows there is no DVD present and suppresses the messages. From looking at the code, it may be the action of a disc being ejected that is sent from the MCU, with the eject button press handled internally by the MCU. Your more likely to find the press of the DVD button than the eject button, but may find both.
When your done searching, you want to put back the original file and reboot. If you find the codes, I can certainly add them to trigger the shell script and you can take whatever action you want with them.
Click to expand...
Click to collapse
Luckily the eject key was also found! Here are the codes:
DVD:
D/MCUKEY ( 691): MCUKEYINPUT: B1=1 B2=0 B2=16 B2=80
EJECT:
D/MCUKEY ( 691): MCUKEYINPUT: B1=1 B2=161 B2=2 B2=91
Looking forward to the modified apk. If you happen to take donations I'd be willing to give you a beer. Thank you.
Model number for reference: JY-AL003N2
sonof said:
Luckily the eject key was also found! Here are the codes:
DVD:
D/MCUKEY ( 691): MCUKEYINPUT: B1=1 B2=0 B2=16 B2=80
EJECT:
D/MCUKEY ( 691): MCUKEYINPUT: B1=1 B2=161 B2=2 B2=91
Looking forward to the modified apk. If you happen to take donations I'd be willing to give you a beer. Thank you.
Model number for reference: JY-AL003N2
Click to expand...
Click to collapse
Here is an updated apk file that should catch the code you listed. In the zip there is a file "keys.txt" and also updated APK file. The keys.txt is a complete list button codes for both the soft-keys ( steering wheel ) and hard-keys (on unit ) that the new apk sends to the script. This is a beta of V2.0 which I added:
3 - HOME ( On unit hard key )
4 - BACK ( On unit hard key )
28 - HANG/CALL END ( Steering wheel softkey)
31 - DVD ( On unit hard key )
32 - EJECT ( On unit hard key )
The HOME/BACK keys are sending the keycodes for the home/back actions. You don't need to map them in a script, as long as you have a default that passes keycodes at the end of the script.
*) input keyevent $key
;;
I have used it the last couple of days and it is working fine for me, but can't test the dvd or eject.
Let me know if it registers the keys.
sorry for the dumb question but do you need to root the unit to use this?
Is there a way to run a shell script when I switch ignition off and illumination on or off?
hi all again,
still in "beta", but seems works fine enough to public and allow to use this format even more effective than in vendors watch faces.
what does mean "TBUI"? it's just a tag in watch face (further "wf") file, what allow you identify it. you may find it at the end of all of this type of watch faces. just open it in any editor/viewer, go at the end of the file and you'll se it:
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
it's a very flexible format, perhaps a next version of "PUSH" (yep, a similar tag again). at least it looks similar but have more powerful abilities.
this format allow you to use in any combinations a lot of elements such as:
- clock hands (as discrete as smooth)
- digital data: time markers (49 in variants), sensors data (48 in variants)
- sequential frame-based elements (time, month, progress-bars, rolled-up elements... - 30 items)
- localizeable tags (chinesse/non chinesse)
- animations (sequential frames)
- buttons to call device menu items
- internal wf configuration (ability to enable/disable some groups of elements "on the fly")
and you not limited in one item per type (if you want).
but before to continue, to be honest I'd must warn:
WARNING - theoretically you may brick your device and may not recover it without special knowledge. while I'd investigated this format it happed few times in case of wrong frames compression. now it's solved and must not happens, but chances still not a zero. for sample, few days ago I've known what a zero mirror point for clock hands may cause stuck your device too. I did set a foolproof protection for exactly this issue, but can't check all of possibilities because of over 150 available elements.
ok. if you'd not escaped yet, let's continue.
Spoiler: few screenshots of interface
editor was maked closer to the file format as maximum as possible, so you must understand few basic definitions:
- frame - just an image
- char set - required for elements what uses strings to display. technically. it's a group of assignments frames to characters/symbols. if you use some char set for element, then each char in string what must be displayed, would be replaced by assigned frame. if the char would not assigned, than it'll be skipped.
- ui element - a part of a wf what do something. display some sort of data or mark some region for some action.
- region - rarely uses. if you not in plane to make a hard wf, than you may just ignore it. treat it as "layer". it assign new start point of a coordinate for child elements, but have no affect for sequential of global drawing queue and not cut elements if it's draws out of region area. each region have it's own "sets" - a group of elements what must be shown at once. what det must be shown, may be configured by special element class.
so in this case, editor have a 4 "subeditors": "UI" ("UI items" and "Regions") and "Resources" ("Char Sets" and "Frames")
main interface controls:
horizontal control panel:
"X" - remove element
"A" - add new element as next
"U" - up at one position
"D" - down at one position
"+"/"-" - expand/collapse element
vertical control panel (for ui items editor only):
"+"/"-" - expand/collapse element
"H" - hide element on branch
"L" - ignore element on branch (would be displayed, but would be ignored by mouse)
available classes:
Spoiler: Overlay
just a frame as is. used parameters:
Code:
Target ID - id of a frame
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
Spoiler: ClockHands
used parameters:
Code:
Target ID - id of a first frame in chain or alone frame id
X, Y - top left corner coordinates
X2, Y2 - coordinates of a mirror/rotate point
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
elements:
CHHoursHand16, CHMinutesHand16, CHSecondsHand16 - uses 16 sequential frames from "N" to "E". works as a discrete clock hands. all other frames would be used as a mirrored via lines goes through mirror point (x2, y2)
CHHoursHand1, CHMinutesHand1, CHSecondsHand1 - uses only one frame ("N"). would be rotated around x2, y2. works as a discrete clock hands. a lot of cpu usage - reduce size of a frame as possible.
CHHoursHandSmooth, CHMinutesHandSmooth, CHSecondsHandSmooth - uses only one frame ("N"). would be rotated around x2, y2. works as a smooth clock hands. a lot of cpu usage - reduce size of a frame as possible.
Spoiler: Char2FrameTime
various elements what display time parts (hours, minutes, seconds, date, month, year, day of the week in different variants) as a numeric values
used parameters:
Code:
Target ID - id of a char set
X, Y - top left corner coordinates
X2, Y2 - used only for calendar table parts as a size of a cell
Align - align of an element to x, y point
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
Params 1 - second field only - kerning (interval between chars)
Spoiler: Char2FrameStatistic
various elements what display sensors data (steps, hear rate, o2, battery level etc. in different variants) as a numeric values
used parameters:
Code:
Target ID - id of a char set
X, Y - top left corner coordinates
Align - align of an element to x, y point
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
Params 1 - second field only - kerning (interval between chars)
Spoiler: Localizable
elements what display various tags for chinesse and non-chenesse localizations ("am/pm","bpp","o2","km"/"ml" etc. )
each item must have two sequential frames: for "chinesse" locale, then for "non-chinesse" locale.
useless if you have no plans to make a chinesse variant.
used parameters:
Code:
Target ID - id of a first frame (chinesse)
X, Y - top left corner coordinates
Align - align of an element to x, y point
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
Spoiler: SequentialFrames
various elements displays some data as sequential frames. like a progress-bars of some data, month etc.
used parameters:
Code:
Target ID - id of a first frame in chain
X, Y - top left corner coordinates
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
Spoiler: Specialized
some sort of elements what not assigned to any other classes
used parameters:
Code:
Target ID - id of a first frame in chain
X, Y - top left corner coordinates
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
Spoiler: OverlayGroup
not analyzed yet
Spoiler: Button
uses to call internal watch menus if exists
used parameters:
Code:
X, Y - top left corner coordinates
X2, Y2 - width and height of a button
Parent Region ID - as named
Set ID - as named for region assigned in "Parent Region ID"
Spoiler: RegionSettings
selecting id of a set what must be shown in appropriate region. works like a button, but have a some specific.
used parameters:
Code:
X, Y - top left corner coordinates
X2, Y2 - width and height of a button
Region Type - how it must be used - via internal config or by tap
Params 1 - for "RSConfig" only. first field as a count of a selectable items
Params 2 - for "RSConfig" only. list of frames id what would be used as a thumbnails for selection menu
sorry for so short manual, it's a bit hard to write that's all on not my first language. you may use an original watch faces to learn how it works in details or ask me here or in pm.
~200 original watch faces for dm50 (466x466) available on my gdrive you may use it as a sample or just a template/resources.
download link TBUIWFTool.zip
Spoiler: list of tested compatible devices
- DM50, Lemfo (466х466)
- HD11, Huadai (240x280)
- HK28 (368х448)
- HK46 (360x360)
- i20, Colmi (360x360)
- i30, Colmi (390?x390?)
- i31, Colmi (466x466)
- C60, Colmi (240x280)
- C80, Colmi (368x448)
- LA24, Linwear, Tiroki (360х360)
- LF26 Max, Lemfo (360х360)
- Vibe 7 Pro, Zeblaze (466x466)
- x7, Gejian (360x360)
Spoiler: not tested devices with TBUI wf
- Dizo Watch R, Realme (360х360)
- HK3Pro (360х360)
- L20, microwear (240x280)
- W3Pro+, XO (360х360)
all devices except dm50 tested by other users, not by me.
watch face for sample:
it was maked by request (converted from push 240x240 to tbui 360x360) and because my watch have a 466x466 resolution it looks shifted. but on 360x360 would be fine.
if you hold tap on display to get watch face selector, you'll see a little gear icon at bottom of preview:
tap it and you'll enter in internal config menu:
you can setup:
- ability to show/hide weather applet
- using static or roll-up month/day elements
- using battery level by 10% or by 1%
- select discrete seconds clock hand or smooth
also it have a few buttons in regular mode:
- weather (by tap weather applet)
- data (by tap steps)
- heart rate (by tap heart or it's numbers)
- timer (by center of a clock)
Very good
I tried running the tool in a clean Windows 11 VM, and it wont start. Does it require a runtime or framework (eg, .NET, Python, etc.) to be installed?
EDIT: Nevermind, Windows' had blocked the file. Anyone else has this problem, right-click->propoerties->unblock. Windows does not give a helpful error message when failing to run. FYI I also got a hit in Windows Defender, which immediately deleted this when I first downloaded it. I scanned in in Kaspersky online and it appears to be clean, so it appears to be a false-positive, but that might have something to do with why Windows is making it so hard to run it.
danjayh said:
...Does it require a runtime or framework (eg, .NET, Python, etc.) to be installed?
...
Click to expand...
Click to collapse
.net 4.7.2 only.
danjayh said:
...FYI I also got a hit in Windows Defender, which immediately deleted this when I first downloaded it...
Click to expand...
Click to collapse
I use it on win 10 and never seen nothing similar. also never heard about this issue (shared on other forum from march).
also it's not finished and would be updated. m.b. even not an once.
Great work... But how I can upload to Colmi i30?
Thanks!
EDIT: Learned to upload and all ok, thanks again!
It's possible to easy convert 466px to 390px? Or need export and import resized all frames?
Jean-DrEaD said:
It's possible to easy convert 466px to 390px? Or need export and import resized all frames?
Click to expand...
Click to collapse
no. resizing in ".net" is a... not a good quality point. you must resize frames via external specialized tools. I use a xnview (for sample). it have a fine enough algorithms and batch processing.
with next build you'll be allowed to remove all frames by one click (already implemented in debugging build). also I'll integrate local browser (at first) for viewing graphical resources of local watch face files and export it or import to current watch face.
at this time, in debugging build it supports push format for viewing too (editing not in plans, just only as sourse of images). seems public build would be able at this week.
in plans some other formats and add web browser too.