Okay, I'm writing this because I want to help any other newbies trying to learn how to reverse engineer. The technical details involved in this are extremely daunting, so the purpose of this tutorial is to first explain in layman terms exactly what you're trying to accomplish and what to expect. Then we'll go over the details. That way you're not completely blind going into this. I'm fairly new to the scene, so I'm not as knowledgeable as everyone else. If you see any errors in my post, let me know so I can change. I'm going to assume you know a little bit of Java, can find your way around a computer, and know nothing about Android. The techniques used should work with other Android phones. For this tutorial I'm using Windows 7, Cygwin, and my stock (not rooted) EVO 4G mobile phone.
The FM tuner for the Evo is run by a Broadcom chip: BCM4329. This chip is pretty amazing in that it does wireless, bluetooth, and it has an FM receiver/transmitter. We're interested in the FM receiver / transmitter.
Now, all android phones are based on a Linux kernel. Basically they're Linux running computers. The Android operating system is then installed onto the linux system. Every app is then run off of Android.
Android is based on Java but it is not a Java system. It uses a virtual machine called Dalvik. Google did this to get around licensing issues with Sun Microsystems. So they pretty much invented their own machine language (called byte code) for the Java language. This makes things complicated for the reverse engineer because from what I've read, once Java is converted into this machine language or byte code, it can't be converted back.
So let's rehash.
If you were programming strictly in Java, you would see these extensions:
Java source code = .java
Compiled Java source code = Java byte code = .class
Compressed file to package your program = .jar (Java Archive)
But since you're programming in Android and Dalvik, you will see these:
Java source code = .java
Compiled Java source code = Dalvik byte code = .dex
Compressed file to package your program = .apk
(I haven't mentioned this, but HTC further Optimizes their .dex code)
Optimized Dalvik byte code = .odex
I'm writing all of these down because it's very easy to get confused with all of the extensions. (for me at least!). remember how I said once you go dex, you can't go back to java? That's where JesusFreke comes in. He's a senior member of XDA, and he created "baksmali" and "smali", two programs that can convert the Dalvik code back into a human readable format. These files have extensions of .smali
Decompiled Dalvik byte code = .smali
But what can you do with .smali files? That's where this other senior member, brut.all comes in: He developed apktool. apktool takes JesusFreke's work to the next level. This program in conjunction with NetBeans, actually lets you trace through any program using the .smali code taken from JesusFreke's programs!
apktool does this by converting those .smali files into "fake" .java files that can be used by the NetBeans (program that compiles and makes java programs) IDE. I say "fake" because apktool embeds the .smali code into java files as comments. However, once you attach a debugger to NetBeans, you'll see that the debugger will follow line by line every execution statement found in the smali code!
So...... you can take the program you want, plug it into Net Beans using a debugger (using the default ddms command provided by Android SDK), and you can trace everything you do in the program. I have it connected to my phone, so whenever I push a button while running my HTC FMRadio app or unplug my headphones,I see the corresponding response to the HTCFMRadio code I have loaded in NetBeans. I can now see in real-time how the program operates from my own interactions... JAM.
Technical Aspects: How to get from ground zero to tracing HTCFMRadio?
1.) Download Android SDK - Go to google development site and follow instructions: Make sure to download the latest Java JDK. Once that is installed, download NetBeans 6.8. Unfortunately, smali debugging does not work with the lastest versions of NetBeans.
Download the "Java SE" version for minimal space
http://netbeans.org/downloads/6.8/index.html
You can follow the rest of Google walkthrough and download Eclipse and ADT plugin, but it's not pertinent to this. You're going to be using adb and ddms from the android SDK extensively, so make sure the path for </android SDK/tools> is included in the PATH variable in your ENVIRONMENT SETTINGS. To get here, right click My computer, click properties, Advanced Settings, ENVIRONMENT SETTINGS.
2.) Search for 7z and download it. It is an awesome and free compression tool that will be extremely useful. It can be used to "unzip" .jar, .apk, and other compressed formats.
3.) Get the Radio app. You can do this by going to "shipped-roms" website, downloading the latest Supersonic image, and following the directions in the unlockr tutorial for HTC kitchens at the unlockr website... (once you have extracted the files from the image, you can look in the system/app and system/framework directories to get the files listed below) or:
you can pull the following files from your phone:
Using the command prompt type (and with phone plugged in, and with USB debugging enabled on phone):
adb pull /system/app/HtcFMRadio.odex
adb pull /system/app/HtcFMRadio.apk
adb pull /system/framework ./framework
This will put HtcFMRadio.odex and HtcFMRadio.apk in the current directory and create a framework directory with more files. A couple of the files in the framework are needed for the HtcFMRadio app, but for simplicity, we're just going to pull the whole directory.
Now that we have the files, we have to make a few changes to make the app installable and to be viewable by the debugger. To do this we have to decompile the .odex format into a human readable format we can edit. That brings us to:
3.) Download baksmali and smali from Project Hosting on Google Code (google search smali).
Usually an Android application is made up of one file, an apk file. Inside the apk file is an AndroidManifest.xml file, a classes.dex file (compiled Java code for the program), and other folders. The other folders contain either graphics or other .xml files that tell the program how it should look to the user. We don't have to worry about those for now. This is important because APKTOOL only opens programs set up this way. But wait up? We didn't download one .apk file, we downloaded an .apk file and an .odex file! What gives? Well, if you right click the apk file and open it (using 7z), you'll see that it's missing the classes.dex file. The dex file for the app is actually the HtcFMRadio.odex file we downloaded. So, to make this system app more like a nominal app, we have to find a way to convert the HtcFMRadio.odex to a classes.dex file. That's easy with baksmali and smali!
Once you download goto command prompt and type:
java -jar baksmali-<version>.jar -d framework -x HtcFMRadio.odex
(Remember to match baksmali-<version>.jar with the filename of baksmali you downloaded)
If done correctly, you should see a newly created \out directory
This creates an out\com\htc\fm directory with many .smali files.
Now let's reverse the process and put it back as a dex file. Type at command prompt:
java -jar smali-<version>.jar out -o classes.dex
If done correctly you'll see a newly created classes.dex.
now, right click on HtcFMRadio.apk (select 7z and open). Drag classes.dex into the file. Say yes to the prompt. Now you have a normal apk file APKTOOL can read!
4.) Download APKTOOL from Project Hosting on Google Code and the helper apps for your OS. (If you're extracting files for windows OS you should have apktool.bat and aapt.exe). Extract (again using 7z, don't you love this program?) apktool.jar (keep it as a jar file, don't extract the stuff inside of it), apktool.bat, and aapt.exe to the directory you're working on. To make things neat, you can also delete HtcFMRadio.odex (you don't need it anymore) and classes.dex (make sure you put it in the HtcFMRadio.apk file first!)
If this is the first time you're using apktool, then you have to install the htc framework so apktool can baksmali the Radio app. You only have to do this once:
apktool if ./framework/com.htc.resources.apk
Alright, at the command prompt:
apktool d -d HtcFMRadio.apk
This extracts the contents of HtcFMRadio.apk and places them in the HtcFMRadio directory. However, there are two major differences between this content and the content created in step 3. If you go into the smali directory you'll see that instead of .smali files, you'll see .java files. And if you go back and edit the AndroidManifest.xml file, you will also see that it's in text! Android applications convert their xml files to binary format. Now that APKTOOL has converted everything to an IDE friendly format, we can use NetBeans to edit everything. The first thing we're going to do is edit AndroidManifest.xml (using notepad) and add the following:
android:debuggable="true" to the Application tag.
IT should now look like this:
<application android:theme="@android:style/Theme.Black.NoTitleBar" android:label="@string/fm_app_name" android:icon="@drawable/fm_radio" android:taskAffinity="android.task.fmradio" android:description="@string/htc_corp" android:allowTaskReparenting="true" android:debuggable="true">
This permission lets the debugger watch the program while it's running on the phone.
We are going to run into two problems if we try to install this program. One is that Android doesn't let you install more than one copy of a system app. The second issue is that if we change the signature of our system app, then we'll have to change the signatures of our other system apps as well! Ahh.... So, to get around that, we're going to trick Android into thinking we have a completely new program. We're going to do that by renaming the com.htc.fm class to com.htc.modradio class. Next step:
5.) Cygwin (or Linux virtual machine)
The easiest way that I can think of to replace strings in multiple files is by using linux. You can most definitely do it in WIndows, but I dont know how. If you let me know how, I can put it in this tutorial.
(update: you can use Notepad++ to easily find/replace strings in multiple files for Windows. You still, however, want to download Cygwin if you're going to develop with Android-NDK.)
For now, just search for Cygwin (Cygwin is a program that lets you run Linux commands from a command prompt using your Windows directories), and install it. Make sure to have the Perl option selected. You'll need Perl to make the following commands work.
Once you get Cygwin up and running
cd <to your HtcFMRadio directory>
in my case it's
cd /cygdrive/c/Users/Jerry/Desktop/HtcFMRadio
now type the following commands in this order:
this command changes all occurances of htc/fm to htc/modradio in your xml and .java files.
find ./ -type f | xargs perl -pi -e 's/htc\/fm/htc\/modradio/g'
this command changes all occurances of htc.fm to htc.modradio
find ./ -type f | xargs perl -pi -e 's/htc.fm/htc.modradio/g'
If you don't follow this order, your source code will get messed up.
If using cygwin, a bunch of .bak files will be created. Using windows search, find all .bak files in your HtcFMRadio directory, then select them all and delete them (Make sure they are only files with .bak!)
Now just rename the fm directory to modradio. It is located in HtcFMRadio/smali/com/htc
Now go to your windows command prompt and type:
apktool b -d .\HtcFMRadio modradio.apk
Now sign and install modradio.apk on your phone.
adb install modradio.apk
If you have never signed before, then you need to use keytool and jarsigner. These two files are in your JDK directory, so make sure you include your JDK directory in the PATH variable of your ENVIRONMENT SETTINGS. (To get here, right click on My Computer, click Properties, Advanced Settings, Environment Variables. Once you make change, open up a new COMMAND prompt to see changes).
cd to the directory which has modradio.apk
now type:
keytool -genkeypair
Answer all questions, then use the same password for all password prompts.
Next type:
jarsigner -verbose modradio.apk mykey
Type in the password you created in the above step. Your apk should now be signed.
Next install:
adb install modradio.apk
Success!
6.) Testing the app on phone
Go to your phone and you'll now see a new FMRadio icon next to your first. Click on it and watch it open. It should now be able to play music. Keep it open.
7.) Using Netbeans
Go into HtcFMRadio and delete the build directory created by APKTOOL.
Now open up Net Beans and click on File, New Project, Select Java Project with Existing Sources, click on Next
Select HtcFMRadio directory for Project Folder, rename Project Name to whatever you want. Let's type in ModRadio. click on Next
Next to "Source Package Folders" click on "Add Folder" and select the smali directory.
Click Finish. For a quick tutorial by Brut.all, search APKTOOL in youtube and click on: Apktool Demo 2 - Smali improvements
Right click on Libraries. Click on "Add Jar / Folder". You want to add Android.Jar. Since I have Android 2.1 loaded I went to /platforms/android-7 located in my android SDK directory.
Your project is now ready for editting!
8.) Running the Debugger to trace through program.
Next go back to Windows command prompt and type ddms. This runs the Dalvik Debug Monitor. A window should open up. In the left hand side you should see com.htc.modradio. That's our app! To the right you're going to see 2 numbers, you're interested in the one to the right, 4 cells away from com.htc.modradio. This number is a port number, and you're going to use it to communicate with NetBeans. (In my case it is 8603)
Go back to NetBeans and click on Debug, Attach Debugger.
In the host field type: localhost
In the Port field: type in the second number you saw. (8603)
If everything is working you'll see a bug appear next to com.htc.modradio in the Dalvik Debug Monitor. Look at the bottom bar of NetBeans for feedback. If you get errors make sure the numbers match, or try port 8700 and make sure you select com.htc.modradio in the Dalvik Debug Monitor. Port 8700 is the default port used for whatever program you select in Dalvik Debug Monitor.
9.) Setting a breakpoint
I'm making this a seperate step because it is completely arbitrary. When creating a break point be sure to follow this rule:
You must select line with some instruction, you can't set breakpoint on lines starting with ".", ":" or "#".
Rather than looking for a spot to breakpoint, though, I'll tell you where to put one so you can quickly see how the debugger traces through the code. You aren't "REQUIRED" to do the next step, but if you want to trace you have to put a breakpoint somewhere.
In Net Beans click on the Project tab, click on Source Packages, com.htc.modradio, and then doubleclick on BroadcomFMTuner.java
We're going to insert a breakpoint. Scroll down to line 3226 and on your keyboard press: CTRL-SHIFT-F8, select line in dropdown box and hit ok. (To keep it simple, I usually look for "invoke" instructions to set breakpoints at)
Now go to your phone and click on the physical "back" button on your phone. This will clear the radio,(you should still be able to listen to music). Drag your status bar down. You should see a radio icon. Click on it again. The radio backgroudn will appear, but you wont' see any text or anything. Now go back to your netbeans application. You should now see debug options highlighted! Click on Step Over (F8) to step through!
nice tutorial
Nice tutor! I'm glad that finally someone is interested in smali debugging feature ;-)
About package name changing: I must warn you, that this isn't always that easy. It's advanced task, you must understand, what are you doing - otherwise you will get FCs for some apps and you won't know, why. And you don't have to change Java classes package name, so actually changing com.example.android should be enough - there is no need for changing com/example/android and dir names.
And I have few words about the background/theory: dexes, smali, (de)compiling, etc., but I will write them tomorrow
So I don't have to change the directory names for all of the invokes? Which means I don't have to rename the directory either? That sounds much easier. So you're saying to only replace the name in the application tag in the AndroidManifest.xml file?
Wow I'm a noob
Your badass tutorial just made me realize how much further I have to go. Great work!
BTW, if someone could figure how to make use of the FM transmitter and create an FM modulation app I think that could be popular. Not everyone has cars with input jacks and I know I really could of made use of it for my vacation I just took.
Once you understand how the software interacts with the Broadcom chip it shouldn't take too much should it? I would envision something working just like wifi tether does. No need to turn on the radio, the app just takes control.
Cool! Thanks! Will this work with eclipse also?
oikjdfkrjhr said:
Cool! Thanks! Will this work with eclipse also?
Click to expand...
Click to collapse
According to Brut.all, no.
"Google smali debugging - you'll find an article written by Brut.all"
Eclipse can open the files, but certain workarounds have to be done to get the same tracing to work.
A FM Transmitter that is able to transmit audio to a receiver without any addition cords hardware etc would IMO make the EVO 4G Un-Stoppable!!! I'll donate to support the dev/s in this effort
Will this get FM Radio working on any android phone with that broadcom chip?
Wow, great tutorial. I was looking for the hitch-hackers guide to my new galaxy and I think I found it. Cheers
Thanks this is great. I almost got it 100% working. But there's one problem. I'm debugging some other apk. And I can't set any breakpoints because, since the apktool made fake java files, all the code is comments, so there's no place to set any breakpoints since there's no real executable code!
Help how to get around this?
At the beginning of a class I tried to add something like... int a=0; while keeping the same amount of lines... just before comment start...
Code:
package com.example.app; class MainActivity {
int a = 0; /*
.class public Lcom/example/app/MainActivity;
.super Landroid/app/Activity;
.source "MainActivity.java"
and put a breakpoint there, but it still says its not an executable location.
Any ideas? This is cool and I'm so close with your great tutorial. Thanks!
oikjdfkrjhr said:
Thanks this is great. I almost got it 100% working. But there's one problem. I'm debugging some other apk. And I can't set any breakpoints because, since the apktool made fake java files, all the code is comments, so there's no place to set any breakpoints since there's no real executable code!
Click to expand...
Click to collapse
http://code.google.com/p/android-apktool/wiki/SmaliDebugging
I tried to use Eclipse, but seems it forbids to add breakpoint on unknown (commented out) line :-/ I workarounded this and added breakpoint, then I was able to debug normally, but you should treat apktool debuging as unworkable on Eclipse - at least until someone will find some solution.
Click to expand...
Click to collapse
And actually midnjerry said this to you just few posts above.
neomagik said:
Will this get FM Radio working on any android phone with that broadcom chip?
Click to expand...
Click to collapse
Unfortunately not.
Refer to IntersectRaven's posts regarding enabling the FM Tuner for the Nexus One. He needed to create drivers for a custom Linux kernel to be used with a Desire ROM.
Smali debugging is important, however, because it allows developers to learn the code that implements the "undocumented" hardware for their phone (in this case, HTC phones with a working FM Radio app).
oikjdfkrjhr said:
Thanks this is great. I almost got it 100% working. But there's one problem. I'm debugging some other apk. And I can't set any breakpoints because, since the apktool made fake java files, all the code is comments, so there's no place to set any breakpoints since there's no real executable code!
Help how to get around this?
At the beginning of a class I tried to add something like... int a=0; while keeping the same amount of lines... just before comment start...
Code:
package com.example.app; class MainActivity {
int a = 0; /*
.class public Lcom/example/app/MainActivity;
.super Landroid/app/Activity;
.source "MainActivity.java"
and put a breakpoint there, but it still says its not an executable location.
Any ideas? This is cool and I'm so close with your great tutorial. Thanks!
Click to expand...
Click to collapse
if you're using NetBeans, just keep trying to insert breakpoints at every "invoke" type statement.
A really interesting thread. Thanks for sharing your knowledge with us. A topic like this should have its own Forum like ‘Android Engineering Tutorial’. We need more tutorials like this one!
Bluetooth Driver Files released by HTC!
Just went to
http://developer.htc.com/
Bluetooth driver files released! We should be able to better control the radio now.
Any progress? I'll donate to the first person who gets an app to broadcast my music to my radio.
Sent from my HTC EVO using XDA App
@midnjerry,
sorry if this is way off topic, but it does deal with the fundamental aspects that you have established here.
Can I use this method to reverse engineer the Google Voice app, so that I can have the same program installed twice for my two different Google voice accounts?
I just want to name the application something else, so that I can install it twice with two different sign in names.
reekotubbs said:
@midnjerry,
sorry if this is way off topic, but it does deal with the fundamental aspects that you have established here.
Can I use this method to reverse engineer the Google Voice app, so that I can have the same program installed twice for my two different Google voice accounts?
I just want to name the application something else, so that I can install it twice with two different sign in names.
Click to expand...
Click to collapse
Yeah, I think you can do that. Just change package name of an app to something else.
Brut.all said:
Yeah, I think you can do that. Just change package name of an app to something else.
Click to expand...
Click to collapse
@Brut.all
Can I do that in Eclipse or do I need to use need to use Smali Debugging? Just asking because I am not looking to debug the program only install it twice.
I am new to programming Java. Can anyone help me with source code for a simple android app that copies one file from sdcard to any directory in Phone. I have eclipse and emulator setup for testing.
cheers
Im not into java... but you could write a small bash script!
Bashs can be run from a terminal (connectbot, irrsi, ssh-server)
Or you simply use the app (GScript), where you can add your own scripts and run them just with 1 click!
Code:
#!/system/bin/sh
file=/sdcard/FOLDER_OF_FILE/FILES_I_WANT_TO_COPY
target=/TARGET_FOR_FILE
cp -frv $file $target
Hope I could help you...
I would try searching stackoverflow.com ....
.... be prepared though, file operations in Android are not a quick 2 lines of code.
sebsch1991 said:
Im not into java... but you could write a small bash script!
Bashs can be run from a terminal (connectbot, irrsi, ssh-server)
Or you simply use the app (GScript), where you can add your own scripts and run them just with 1 click!
Code:
#!/system/bin/sh
file=/sdcard/FOLDER_OF_FILE/FILES_I_WANT_TO_COPY
target=/TARGET_FOR_FILE
cp -frv $file $target
Hope I could help you...
Click to expand...
Click to collapse
Thanks for reply. But i am looking for a pure android app and not script or adb commands via terminal !
@eatmold
eatmold said:
I would try searching stackoverflow.com ....
.... be prepared though, file operations in Android are not a quick 2 lines of code.
Click to expand...
Click to collapse
Thanks a lot man, i look forward to see some code soon .....
user_000 said:
I am new to programming Java. Can anyone help me with source code for a simple android app that copies one file from sdcard to any directory in Phone. I have eclipse and emulator setup for testing.
cheers
Click to expand...
Click to collapse
First of all, you can't copy a file to any directory on the phone. The app can only write to directories to which it has permission to write. Second, some parts of the phone can't be written to at all because the filesystem is read-only. Third, file I/O is non-trivial in the dalvik VM. It would take a substantial amount of code to do what you want.
Do you have some goal in mind for this starter project? There is the "Hello World" project at http://developer.android.com if you just want some example code.
Gene Poole said:
First of all, you can't copy a file to any directory on the phone. The app can only write to directories to which it has permission to write. Second, some parts of the phone can't be written to at all because the filesystem is read-only. Third, file I/O is non-trivial in the dalvik VM. It would take a substantial amount of code to do what you want.
Do you have some goal in mind for this starter project? There is the "Hello World" project at http://developer.android.com if you just want some example code.
Click to expand...
Click to collapse
Well the aim is to get an idea of how a very simple app can work on phone. So it can even be copying a file from one directory of SDcard to another directory on SDcard on Phone. And also lets assume that we have the permission to write to any directory on phone (i.e superuser or its writable etc).
Yea 'hello world' is a nice example and on the same lines a simple non-gui app to copy file is what i am looking for. It will give a good overview of how to do some basic I/O operation on phone via android app.
cheers
If you want to do this as a Java learning experience, then you'll probably want to handle the file I/O using strictly Java which can be a real pain. Nonetheless, all you need is here:
http://developer.android.com/reference/java/io/package-summary.html
although, typically things like file I/O are best left to the operating system which is usually more optimized for that sort of thing--especially when the OS is *nix. To leverage the OS to do your dirty work natively, you can use the Runtime class:
http://developer.android.com/reference/java/lang/Runtime.html
Thanks for the references. But interest is to see a working code on an Android device and not Java as a whole. So a simple app source code can be a really good starting point. Hello World is good first example but does not include I/O, so someone with experience can lead the newbies in a good way by showing a simple example.
Come on guys share some experience !