Related
Update 2010-02-26 Added examples for linking either dynamically or statically (see page 2)
I decided to try and set up a kitchen using the closest thing to a native kernel dev environment that I could set up.
At this point I have:
- cross-compiled the HTC Desire (kernel) sources using the Android 1.6 NDK
- figured out the repo/git stuff to pull the Android sources corresponding to a given Android release point
- cross-compiled the entire Cupcake release to an ARM target using the NDK tools
- built a dynamically linked "hello, world" executable using the NDK arm-eabi-* tools and verified it works on the phone.
Yah, I know - not much yet, but it's a start for a newb.
I have some notes documenting the Ubuntu setup process in case anyone wants it - but it is full of links (URLs), so I can't post anything with links as a new user.
If a mod can "verify" my account, I'll post it up. I promise, I'll be good.
bftb0
OK, I guess I need to make a couple of posts before I can embed links.
I'll post it without the URL protocol prefix (it's gonna look fugly), and then come back and edit it once the restriction is removed. I hope someone finds it useful.
Android kernel dev is not officially supported on Windows; closest match to Google Dev environment seems to be -> Ubuntu 8.04 LTS + JDK 5 (32 bit, not x64).
I realize that this is nothing close to the "kitchens" that the WinMo folks have been using, but figured that (as a newb) I would just end up confusing myself trying to figure out which of those tools could be useful for Android, in addition to needing the native binary cross-compilation tools anyway.
(Note: I briefly tried to set this up in a Ubuntu VMware Player VM, but ran into limitations of available Ubuntu 8.04 LTS player machine file size limits (as well as problems hosting a VM bigger than about 10 Gb on a USB drive - insufficient space on my primary drive)
If anyone wants to follow in my footsteps, here is a thumbnail sketch of useful links. I'll admit, this is not for the faint of heart - you need Linux/Unix skills to work through the kinks.
1) (ubuntu.com/getubuntu/downloadmirrors#mirrors) Ubuntu Download Mirrors list - Find a server near you
2) Download ubuntu-8.04.4-alternate-i386.iso and burn it to 700 Mb CD-R
3) Install Ubuntu 8.04 LTS Desktop Version (gulp!)
4) Use the Ubuntu graphical package manager (System -> Administration -> Synaptic Package Manager) and install the packages (git, flex, bison, etc) mentioned in the "Ubuntu Linux (32-bit x86)" section in this (source.android.com/download)Android Source setup page
5) Note that JDK6 does not work correctly; rather than going to Sun/Oracle for JDK5, install the sun-java5-{bin,jdk,jre,doc,demo} packages (1.5.0-22-0ubuntu) using the Ubuntu package installer mentioned above. Note if you install the docs package, you will also need to go to Sun/Oracle and get jdk-1_5_0-doc.zip towards the end of the package install, and drop it into /tmp for the package manager to finish.
6) Go to the (developer.android.com/sdk/index.html) Android SDK Download page and get android-sdk_r04-linux_86.tgz
7) Unpack ( gunzip -c and...tgz | tar xf - ) to desired location, add tools dir to PATH
Optional Steps (For Java App Devs)
o-8) (eclipse.org/downloads/ Eclipse IDE Download page - chose 'Eclipse Classic 3.5.1 (162 MB)' for Linux, 32 bit (eclipse-SDK-3.5.1-linux-gtk.tar.gz)
o-9) Install by unpacking ( gunzip -c ecli....tar.gz | tar xf - ) to desired location, add eclipse folder to PATH
o-10) Run eclipse, and follow the (developer.android.com/sdk/eclipse-adt.html) Installing and Updating ADT instructions for adding Android functionality to Eclipse 3.5 (Galileo)
o-11) From within Eclipse, launch Window -> Android SDK and AVD Manager, and install all available & compatible Android SDK Tools (all versions). (You might need to use http instead of https... YMMV) Note that this is the same functionality as simply running the command "android" from the command prompt if the Android SDK (steps 6,7) tools directory is in your PATH
12) Familiarize yourself with the command line tool "adb" tool in the SDK; in particular, since Ubuntu needs root privs to access the USB hardware, the best way to start the adb server on your PC is to
Code:
$ sudo /bin/bash
- set up root's environment so the SDK "tools" directory is in root's PATH
- plug your phone into the USB cable
- On the Eris, set the options:
Settings -> Applications -> Development -> USB Debugging (on)
Settings -> Applications -> Development -> Stay awake (on)
Code:
# adb devices
This last command only lists the available USB-connected (real) devices and any emulators you have running, but as a side effect it starts up the adb server on the Ubuntu machine running as root. Thereafter, you can use adb (or ddms) as an unprivileged user.
13) Have a peek at the SDK "ddms" monitor. A variety of things can be done from here, the least of which is screen shots (Select device, then Device-> Screen capture)... and some more twisty things too.
14) Download the (developer.android.com/sdk/ndk/1.6_r1/index.html) Android 1.6 NDK r1 (Native Development Kit) - android-ndk-1.6_r1-linux-x86.zip, and unzip into your desired location. Add the ARM cross-compiler tools to your path (see example below).
15) At this point you might be using something like this to set up your path wherever you please (.profile or elsewhere, according to how you want to manage your environment)
Code:
_INSLOC='/opt/android/'
_JAVAPATH='/usr/lib/jvm/java-1.5.0-sun/bin'
_BASEPATH='/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
_ANDTOOLS="${_INSLOC}"'/android-sdk-linux_86/tools'
_ECLPSPATH="${_INSLOC}"'/eclipse-3.5.1'
_ARMXCOMPILE="${_INSLOC}"'/android-ndk-1.5_r1/build/prebuilt/linux-x86/arm-eabi-4.2.1/bin'
PATH="${_JAVAPATH}"':'"${_BASEPATH}"':'"${_ANDTOOLS}"':'"${_ECLPSPATH}"':'"${_ARMXCOMPILE}"
export PATH
16) Download the (member.america.htc.com/download/RomCode/Source_and_Binaries/desirec-be918f7b.tgz) HTC Desire Kernel Source Code and unpack to a location of your choice.
17) Test your enviroment by building the HTC Kernel code:
$ cd v2_6_27/kernel_msm7k; make
I can't remember if I needed to stuff anything into the environment here (e.g. 'CC' definitions, etc.) - but I know that I didn't need to modify any Makefiles. I think the make system will find all the "arm-eabi-*" executables if they are in your path. You will know soon enough.
18) Try to compile a "Hello, world!" program for the phone. Before you get too far along in this (e.g. "where is crt0.o?"), (honeypod.blogspot.com/2007/12/dynamically-linked-hello-world-for.html) read this blog post - steps 3 and 4 only.
You might find a Makefile similar to this useful:
Code:
AR = arm-eabi-ar
AS = arm-eabi-as
CC = arm-eabi-gcc
CXX = arm-eabi-c++
LD = arm-eabi-ld
NDK_KIT = /opt/android/android-ndk-1.5_r1
PLATF_KIT = build/platforms/android-1.5
ARM_INC = $(NDK_KIT)/$(PLATF_KIT)/arch-arm/usr/include
ARM_LIB = $(NDK_KIT)/$(PLATF_KIT)/arch-arm/usr/lib
PLATF_INC = $(NDK_KIT)/$(PLATF_KIT)/common/include
OBJS = hello.o start.o
EXES = hello
hello: hello.o start.o
$(LD) \
--entry=_start \
--dynamic-linker /system/bin/linker -nostdlib \
-rpath /system/lib -rpath $(ARM_LIB) \
-L $(ARM_LIB) -lc -o hello hello.o start.o
hello.o: hello.c
$(CC) -I $(ARM_INC) -I $(PLATF_INC) -c hello.c
start.o: start.c
$(CC) -I $(ARM_INC) -I $(PLATF_INC) -c start.c
clean:
rm -f $(OBJS) $(EXES)
and then
Code:
$ make hello 2>&1 | tee make_hello-log.txt
if all goes well, push to the phone
Code:
$ adb push hello /sqlite_stmt_journals/hello
and then use a shell on the phone to watch the lights gerblinken:
Code:
PC-path$ adb shell
$ cd /sqlite_stmt_journals
$ ./hello
Hello, world!
$ exit
PC-path$
19) Now, download the Android stock kernel sources:
- Make sure you have "git" and "repo" installed; see the (source.android.com/download) Android "Get source" page, starting from "Installing Repo".
- Make a clean directory to initialize "repo" in, and download the code:
Code:
$ mkdir ~/mydroid
$ cd ~/mydroid
$ repo init -u git://android.git.kernel.org/platform/manifest.git -b cupcake
$ repo sync
The sync step above will download the entire source tree corresponding to the Cupcake release (90 minutes on my link).
20) Build the sources
Code:
$ cd ~/mydroid
$ make 2>&1 | tee make_Android-log.txt
On my old PC, this step took about two hours.
Well, that's a little start; everything that comes after this is heavy lifting, I suppose.
bftb0
Thank you for that.
I forgot to add a disk space usage report to this; here it is:
O/S ( /, /boot, /var, /tmp) ~ 4 Gb **
Android SDK, all versions: 1.3 Gb
Android "Cupcake" full sources + full build: 5.5 Gb (2.5 Gb sources, 3.0 Gb build tree)
Android NDK ~0.05 Gb
Eclipse 3.5.1: 0.21 Gb
HTC Desire Kernel Sources + build ~ 0.77 Gb
All up, thats about 12 Gb minimum needed for a full environment. Obviously, you would want some headroom above that.
Note that if you only want to build native binaries, and not the full Android source tree (or develop Java applications), this could be done handily in a virtual machine which is less than 10 Gb in size - for Windows users, there are VMware "Player" versions of Ubuntu 8.0.4 available for that. That eliminates the need for a separate machine, or a dual-boot PC - and the worries of messing up trying to create a dual-boot machine.
All that is really needed for that is the Desktop OS and the NDK. Adding a single version of the SDK so you have 'adb' and 'ddms' available to you would be helpful, but not absolutely necessary if you are using a Windows host machine with those tools already present.
bftb0
** beware that Ubuntu uses a lot of space on /var if you put it in a separate partition - 600+ Mb in my configuration.
bftb0 said:
I forgot to add a disk space usage report to this; here it is:
O/S ( /, /boot, /var, /tmp) ~ 4 Gb **
Android SDK, all versions: 1.3 Gb
Android "Cupcake" full sources + full build: 5.5 Gb (2.5 Gb sources, 3.0 Gb build tree)
Android NDK ~0.05 Gb
Eclipse 3.5.1: 0.21 Gb
HTC Desire Kernel Sources + build ~ 0.77 Gb
All up, thats about 12 Gb minimum needed for a full environment. Obviously, you would want some headroom above that.
Note that if you only want to build native binaries, and not the full Android source tree (or develop Java applications), this could be done handily in a virtual machine which is less than 10 Gb in size - for Windows users, there are VMware "Player" versions of Ubuntu 8.0.4 available for that. That eliminates the need for a separate machine, or a dual-boot PC - and the worries of messing up trying to create a dual-boot machine.
All that is really needed for that is the Desktop OS and the NDK. Adding a single version of the SDK so you have 'adb' and 'ddms' available to you would be helpful, but not absolutely necessary if you are using a Windows host machine with those tools already present.
bftb0
** beware that Ubuntu uses a lot of space on /var if you put it in a separate partition - 600+ Mb in my configuration.
Click to expand...
Click to collapse
great very descriptive thread
Hey bftb0
thanks for posting this guide. I am trying to just play around with someone of the exploit binaries on my eris (from the tattoo/hero rooting threads) , so these guidelines are coming in handy. However for some reason whenever I try and do anything over adb i get this error:
Code:
error: insufficient permissions for device
I cant use adb shell at all, even trying to push to the sdcard gives permissions denied. In the adb devices output the phone is coming up as "?????? no permissions", Did I miss a step ?
My setup so far: Ubuntu 9.10 i386,
Hey bftb0
thanks for posting this guide. I am trying to just play around with someone of the exploit binaries on my eris (from the tattoo/hero rooting threads) , so these guidelines are coming in handy. However for some reason whenever I try and do anything over adb i get this error:
Code:
error: insufficient permissions for device
I cant use adb shell at all, even trying to push to the sdcard gives permissions denied. In the adb devices output the phone is coming up as "?????? no permissions", Did I miss a step ?
My setup so far: Ubuntu 9.10 i386, android-sdk_r04-linux_86, droid eris with most recent OTA
someguy, you have to go to developer.android.com/guide/developing/device.html and install the linux driver (as root) and restart your system.
@someguy
Type "adb kill-server" followed by "sudo adb". You need to run adb as root in unix for it to work.
zifnab06, that was it, I must have not run the first command as root which caused the server to be run without su.
What exactly does this do?
bigcj55 said:
What exactly does this do?
Click to expand...
Click to collapse
Because of the lineage between Linux and Android, the primary platform for compilation of the kernel and other ARM instruction-set binaries ("native" code) has been on Linux. It appears (at the moment...reading between the lines) that Google engineers are using Ubuntu 8.04 LTS as their choice of Linux platform for maintaining Android kernel & utilities code... so, in principle, it is the "best supported" platform for doing "native" development work, if for no other reason than it's what Google has been using. (Not because it is somehow a "better" O/S, but rather because you are likely to run into fewer quirks, and have more resources on the internet who might have experienced the same difficulties that you run into.)
For instance, the folks that have been building kernel exploit programs have been using the Google "NDK". They are not writing/compiling those in Java - they are writing or compiling programs written in "C", and compiling them into native (ARM) object code.
I provided notes from my own installation for anyone considering "dipping their toe" into building Android kernels or native binaries using Linux as a host O/S - most forum readers are not going to be interested in doing that.
Note that Google now releases an NDK for Windows, Linux, and Mac OS/X. The strange thing about doing development on Windows using Cygwin is that if you have sufficient skills to do so - and are capable of resolving problems that inevitably crop up - then you probably already have sufficient skills to be using Unix/Linux in the first place. Sort of depends on your past experience, though. I'd be willing to bet that Devs with a history of doing WinMo development probably would prefer to climb the Android development learning curve by trying their hand with the Windows/Cygwin NDK because of their familiarity with other Windows-based development tools (IDEs, hex editors, assemblers, GUI generators, etc).
That reminds me though - I need to update this thread with something else.
Cheers
bftb0
Static OR Dynamic Linking of Native Binaries
I was a little unsatisfied with the "hack" used in the above examples to circumvent the need to launch programs without linking to C runtime initialization using that strange code shown above, i.e.
Code:
start() { exit( main() ); }
Moreover, for purposes of debugging applications running on the phone (using "gdbserver"), it simply seems easier to link statically - it allows you to step into the syscall library routines and actually see the assembler implementations wrapped around "STI" interrupts.
So, I spent some time experimenting, and came up with a Makefile that allows me to easily toggle between building an ARM executable linked statically or dynamically. Moreover, it lets "argc" and "argv" work they way they are supposed to, and also allows main() to do a return().
Here's an example Makefile, for a program "hello.c". Make sure you read the notes which follow it.
Code:
#
# FIX THESE (to match your installation)
# Top of the Android Source/build tree (retrieved w/ repo)
ANDSRCROOT := /home/user/android/repo-root
# link dynamically against the libraries shipped on the phone!
# e.g.: $cd /opt/android/Eris_libs/ruu1_16_605_1 ; adb pull /system/lib/
PHONELIBS := /opt/android/Eris_libs/ruu1_16_605_1/lib
# Choose one or the other; static is easiest for debugging use
LINKMODE := dynamic
#LINKMODE := static
# tools
CROSS_COMPILE := arm-eabi-
CC := $(CROSS_COMPILE)gcc
LD := $(CROSS_COMPILE)ld
AR := $(CROSS_COMPILE)ar
CXX := $(CROSS_COMPILE)c++
OBJDUMP := $(CROSS_COMPILE)objdump
RANLIB := $(CROSS_COMPILE)ranlib
READELF := $(CROSS_COMPILE)readelf
# Note STATLIBROOT is relative to the Android Source root (ANDSRCROOT)
STATLIBROOT := out/target/product/generic/obj/STATIC_LIBRARIES
INCLUDE := -I $(ANDSRCROOT)/bionic/libc/include \
-I $(ANDSRCROOT)/bionic/libc/arch-arm/include \
-I $(ANDSRCROOT)/bionic/libc/kernel/common \
-I $(ANDSRCROOT)/bionic/libc/kernel/arch-arm
ifeq ($(LINKMODE),dynamic)
LIBDIRS := -L $(PHONELIBS)
else
LIBDIRS := -L $(ANDSRCROOT)/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/lib/gcc/arm-eabi/4.2.1 \
-L $(ANDSRCROOT)/$(STATLIBROOT)/libc_intermediates \
-L $(ANDSRCROOT)/$(STATLIBROOT)/libc_common_intermediates
endif
ifeq ($(LINKMODE),dynamic)
CRTBEG := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtbegin_dynamic.o
LDFLAGS := -rpath /system/lib -rpath-link /system/lib --dynamic-linker /system/bin/linker
EXTLIBS := -lc -ldl
else
CRTBEG := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtbegin_static.o
LDFLAGS := -static
EXTLIBS := -lc -lgcc -lc_common
endif
CRTEND := $(ANDSRCROOT)/out/target/product/generic/obj/lib/crtend_android.o
CFLAGS := -g -Wall
%.o: %.c
$(CC) -c $(INCLUDE) \
$(CFLAGS) $< -o [email protected]
%.s: %.c
$(CC) -S -c $(INCLUDE) \
$(CFLAGS) $< -o [email protected]
all: hello
hello: hello.o
$(LD) $(LDFLAGS) \
-o [email protected] --entry=_start -nostdlib \
$(CRTBEG) \
$^ \
$(CRTEND) \
$(LIBDIRS) $(EXTLIBS)
This makefile has certain assumptions built into it:
(1) Static Linkage
If you are going to link statically, you will need to have already built the entire Android source tree - the Android NDK does not provide any static libraries (In the OP I provided instructions on how to do this download and build the Android "Cupcake" release.)
(2) Dynamic Linking
If you want to compile code which is "as close to the phone as possible" then there is really only one way to do that: compile against the headers that HTC used, and perform linking against the dynamic libraries that you literally pull off the phone ("adb pull /system/lib").
There's only one problem with this strategy: HTC did not release their copy of the bionic (or other library) source code, so you are forced to punt... the second-best choice is to use headers from the Android (Cupcake) source tree. So... you will notice that for both the dynamic-link or static-link cases in the above makefile, the $(INCLUDE) paths used during compilation come from the Android source tree.
bftb0
PS I guess there is a mechanism to configure the top-level "make" of the Android source tree for not only a target type (defaults to "arm"), but also a "machine" dependency. I didn't research that, as I have only built user-space code. I'm doubt there should be any differences at the syscall interface API - excepting perhaps things like ptrace() which have a machine dependency (register structures, etc).
bftb0 said:
Because of the lineage between Linux and Android, the primary platform for compilation of the kernel and other ARM instruction-set binaries ("native" code) has been on Linux. It appears (at the moment...reading between the lines) that Google engineers are using Ubuntu 8.04 LTS as their choice of Linux platform for maintaining Android kernel & utilities code... so, in principle, it is the "best supported" platform for doing "native" development work, if for no other reason than it's what Google has been using. (Not because it is somehow a "better" O/S, but rather because you are likely to run into fewer quirks, and have more resources on the internet who might have experienced the same difficulties that you run into.)
For instance, the folks that have been building kernel exploit programs have been using the Google "NDK". They are not writing/compiling those in Java - they are writing or compiling programs written in "C", and compiling them into native (ARM) object code.
I provided notes from my own installation for anyone considering "dipping their toe" into building Android kernels or native binaries using Linux as a host O/S - most forum readers are not going to be interested in doing that.
Note that Google now releases an NDK for Windows, Linux, and Mac OS/X. The strange thing about doing development on Windows using Cygwin is that if you have sufficient skills to do so - and are capable of resolving problems that inevitably crop up - then you probably already have sufficient skills to be using Unix/Linux in the first place. Sort of depends on your past experience, though. I'd be willing to bet that Devs with a history of doing WinMo development probably would prefer to climb the Android development learning curve by trying their hand with the Windows/Cygwin NDK because of their familiarity with other Windows-based development tools (IDEs, hex editors, assemblers, GUI generators, etc).
That reminds me though - I need to update this thread with something else.
Cheers
bftb0
Click to expand...
Click to collapse
Wow, i was a little off in this case. I was under the assumption y'all were somehow going to compile ubuntu to run on the eris. :O
I did an extensive search to try and find if anyone built binutils to run native on android and came up empty. I did, however, find someone who claims to have gotten Fortran to compile and run on android (specificimpulses.blogspot.com/2011/01/my-android-speaks-fortran-yours-can-too.html) He seems to have problems at the linker phase. I was wondering what people thought about the possibility. I would like to mess around with arm assembler on my android device whilst out and about...
So this is awesome, I set the build and the host environment to arm-android-eabi and I magickally got out i686-elf binaries that run on my linux workstation...amazing! I didn't know that cirrus made a processor with the smarts to automagically execute arm-android-eabi target code when presented with it...
../binutils-2.19.1/configure --prefix=/opt/compilers/gcc/native-android --build=arm-android-eabi --host=arm-android-eabi --target=arm-android-eabi --with-sysroot=../android-ndk-r5/platforms/android-8/arch-arm
So in addition to that I tried to wipe my path clean and point only to the binutils and gcc under the ndk-toolchain directory but it gave me the infamous 'cannot create working executable'....wow? ya think, config?
Anyone know how to turn off the test for building working executables when cross compiling, or perhaps to point in the configure script options where to find crti.o which is what was missing at the link phase?
Strike that, I mean Via. my workstation uses a via processor, not that that is important here...
Success! Well, in as much as I was able to compile binaries of some of the binutils and run them on my g-tab. I haven't tried to assemble a piece of code and link it yet. What I did was use the info in android-ndk-r5/docs/STANDALONE-TOOLCHAIN.html
which suggested that I run:
$NDK/build/tools/make-standalone-toolchain.sh --platform=android-5 --install-dir=/tmp/my-android-toolchain
which created a standalone toolchain self enclosed. I then set path carefully so I was pointing to the toolchain directory's bins. Then I ran configure using
../binutils-2.19.1/configure --prefix=/root/android --host=arm-android-eabi --target=arm-android-eabi
and made a few comments to code--real hacks like casting &stat.time to a time_t * and changing #ifdef tests to force inclusion of the correct code. I also had to link:
ln -s /root/my-android-toolchain/bin/arm-linux-androideabi-ar /root/my-android-toolchain/arm-android-eabi-ar
just because I was tired and didn't want to f**k with the compile script to get it to point to the correct set of toolchain binaries. In the end, I copied over nm-new, ld-new, as-new, and strip-new to my gtab and ran them and got errors that I expected like:
as-new aa.s
Error: can't open aa.s for reading: No such file or directory. Indicating that at least gas was running native on my gtab...
Of course, that is a first tiny step....
Btb, I had to unzip into the /data directory on android for the executables to run. Ran them in a terminal emulator...
Hi all, I'm just starting getting into App development and I want to attach the Android source code into Eclipse so that I can reference it locally. Is there anyone here who has experience doing this on Windows who is willing to walk me through it?
Alternatively, I'll go it on my own if someone could help me get past this one roadblock. Thus far I've installed Python and msysGit (and updated my PATH variable accordingly) so that I can use git and repo. Something isn't working however and repo runs into an error that I can't seem to figure out. I'm contemplating uninstalling all that and starting from scratch with Cygwin.
Here's a tutorial on setting up Eclipse on windows (albeit over a year old so no guarantees) link
personally I would dual boot linux... it was a breeze setting it up and there's plenty of tutorials for that.
You could install ubuntu (i think) side by side with win7 using wubi, as if it were a program or something. That way you're not messing with partitions.
I already have Eclipse set up. This is not the problem. I'm having trouble downloading the Android source code - NOT the SDK - on my Windows PC. Specifically, I always run into errors when I try to use "repo init -u blah blah" as instructed on the android source page. If someone has successfully done this on Windows, I'd like someone to give me some guidance, because either I'm doing something wrong or my setup is not correct.
Right now I'm trying to use Cygwin (basically creates a linux environment inside Windows). The last time I tried using repo, I got this:
EDIT: Code removed. I found a solution, outlined below, for anyone else who wants to get the Android source into Eclipse.
Okay, I got the source code, it was actually pretty easy to do, and I didn't even need repo to do it . For those who may find this useful, here's how.
0. It is assumed you have already installed both Eclipse and the Android SDK. If you don't have those yet, download them and set those up. If you aren't on Windows but still want to know how to get the source and import it into Eclipse, you can skip step 1.
1. Install Cygwin on your computer. During the installation, you will be presented with a package explorer and you can choose specific packages to install. Choose whatever you like, but make sure to include automake, bison, curl, flex, gcc, git, gnupg, python, zip, and an editor of your choice, like nano or vim (you can use the search field at the top to find these quickly).
This will basically give you a Linux shell on Windows whose root directory is something you've designated, like C:\cygwin.
2. A few more things to set up. Fire up Cygwin and make a bin folder in your home directory. Add this directory to your PATH variable, then cd into it.
Code:
mkdir ~/bin
PATH=~/bin:$PATH
cd bin
This blog post has code for a python script that will be used later. Make a new file using an editor of your choice. Copy the code and save the file as anything you want (I named it "pack.py"). When you're done, cd back to your home directory.
3. Downloading source. From your home directory (or a subdirectory if you wish), use git to clone the platform frameworks base into a new directory (here I've called it "android").
Code:
git clone http://android.git.kernel.org/platform/frameworks/base.git android
cd into that directory and examine the git tags.
Code:
cd android
git tag
You'll get a list of tags. Choose the one that matches the API level you want to build your project(s) in. I'm using API level 10, so I chose android-2.3.3_r1. That's what you're going to check out.
Code:
git checkout android-2.3.3_r1
4. Reorganizing the source code. You need to locate all the java source files and restructure everything into a directory structure that matches the package naming. Fortunately, that script from earlier does exactly this and then zips it up into a nice little package for you called "sources.zip".
Code:
python ~/bin/pack.py
5. Move this zip file to the proper SDK platform folder. Since I downloaded sources for API level 10, I want to move this zip file to
Code:
C:\android-sdk-windows\platforms\android-[B]10[/B]
You can do this with your regular old file explorer on Windows. Extract the contents into a subfolder named "sources".
6. Get Eclipse to recognize them. Open up a project (or start a new one) whose target is the API level you just did all that stuff for. In the package explorer, right-click the project root and click "Refresh". Now when you browse the class files under project-root/android 2.3.3/android.jar/whatever, you'll see actual source code instead of that nasty "Source not found" page.
Enjoy ^_^
New: SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone
This can (in combination with tesseract-ocr) be very useful to recognize text from scanned images.
For pictures see also http://forum.xda-developers.com/showpost.php?p=23046120&postcount=187
Update 01/30/2012: Now with a small android app (ScriptStarter) to start services and other shell scripts easily from the GUI. See bottom of the post.
WHAT'S POSSIBLE WITH THIS SCRIPT:
With this bash script your are able to build relatively easy from source a lot of native software (over 90 at the moment) from the GNU/Linux community for your Android phone and you will have
the good experience with free software.
All these tools and programs are not "apps for the GUI" for Android but native running command line tools "under the surface" of your Android GUI like the kernel and other services.
Don't ask me if it makes any sense to install program xy (like MySQL) on a phone. This is your decision. It's possible, so try it and see what you can do with it.
Note: I tested most of the compiled software on a Samsung Galaxy S2 (my phone). I cannot test it on a variety of phones because I have no mobile phone shop...
Currently buildable with this scripts (abstract, see the full list in package_selection.conf):
PHP 5.3.10 (with almost all extensions and XCache)
MySQL 5.5.19 (if you want a full database server on your phone...)
lighttpd 1.4.30
OpenSSH 5.9p1
OpenSSL 1.0.0f
Samba 3.6.3
cURL 7.23.1
bash 4.2
BusyBox 1.19.3
ImageMagick 6.7.4-10
OpenVPN 2.2.2
Python 2.7.2
Node.js 0.6.7
git 1.7.9
Wget 1.13.4
Rsync 3.0.9
TCPDUMP 4.2.0
Midnight Commander 4.8.1
Pure-FTPd 1.0.35
screen 4.0.3
iptables 1.4.10 (without bionic bug "getnetbyaddr()" and with iptables-save/restore!)
BIND 9.8.1-P1
Lua 5.2.0
Ruby 1.9.3-p0
usbutils 004
MTDUtils latest git
GCC 4.6.2 (c, c++)
GNU Coreutils 8.14, Binutils 2.22, Findutils 4.4.2, Inetutils 1.8
QEMU 0.15.1
SANE 1.0.22
tesseract-ocr 3.01
...
a lot of libraries, development tools and other useful programs
...
FEATURES:
automatic download and untar the cross-compilers (if not already done) for cross compiling
automatic download and untar the source packages (if not already done), that you have selected
solve dependences and reserves additional needed packages
apply patches to the sources if necessary
build them, "install" them in a folder for later "ready to transfer" to your phone and stripes the binaries and libraries to reduce the filesize if selected
autogenerate a tar file and adb push the file to device if selected
allows you to use well-known software like PHP, MySQL, OpenSSL, OpenSSH, OpenVPN, bash and lighttpd on your phone (see the full list in file package_selection.conf, see attachement)
scripting and development on your phone
no limits with BusyBox (without want to be derogative, BusyBox is a wonderful tool), full featured tools like Coreutils, Binutils etc.
build scripts are expandable with other or newer packages you eventual want to have
QUESTIONS AND ANSWERS:
Q: Do I need native GNU/Linux programs and what is that?
A: If you do not know what native Linux applications are, then stop here - you don't need them.
Q: What can I do with these tools?
A: Many things. As a developer you can use the libraries to include them in Android GUI apps, as user you can start for example a webserver on your phone and write some HTML or PHP pages
just for fun or replace BusyBox with full featured tools (or only some commands).
Some tools are also very useful to find network errors, check the network traffic and many more.
Q: What do I need?
A: A Linux installation (VM or real), some Linux knowledge and a little bit time to read all instructions carefully. A rooted Android device with enouth free memory (ext2/3/4 formatted)
See README for details.
Q: Costs?
A: Nothing. The scripts are Open Source (GPL v3), the packages are GPL licensed or under a similar license. The cross-compilers are Open Source or free to use for non-profit purposes.
But I would be pleased if I get a donation and/or a press on the "Thanks button", if the scripts are useful for you. It was a long and hard work...
Note: The attachement scriptset-x.yy.zip is double packed (inside .bz2), because I want preserve the right settings of the scripts.
STEPS:
Selected the desired packages in the config file
Adjust the compiler settings for your phone
Run the build script to download and compile the software
Copy the generated TAR files to your phone or let "adb" this do for you and unpack them in the prepared sysroot folder
For more information please read the instructions in "README"
Please let me also know if you have any problems or if you have questions or further suggestions.
Thanks go to the Open Source community and a lot of people/webpages with hints about cross-compiling software.
Thanks
smitna
CHANGELOG:
01/14/2012 First public release v0.9
01/14/2012 Update: Removed unneeded big patch for ncurses
01/15/2012 Release of v1.0
Update: Fixed build of gcc. Note: the header files are not copied automatically with the script.
If you want to compile on your phone, set TAR_HEADERS=1 in configuraton.conf
Typos: some translations corrected in build.sh
01/16/2012 Release of v1.1
Update: Removed building of fsck from util-linux-ng, it's in e2fsprogs
Update: ISC Cron / vixie cron: set correct paths in pathnames.h before compiling
Enhancement: More detailed explanation of device specific compiler settings with list of possible values
Update: Removed dependence e2fsprogs libs from e2fsprogs
01/17/2012 Release of v1.2
Update: Build of zip, zipcloak, zipnote, zipsplit, unzip, unzipsfx and funzip now possible without errors
01/17/2012 Release of v1.3
Update: Added Node.js
Typos: some translations and language errors corrected in build.sh and README (sorry, I'm not a native English speaker...)
01/18/2012 Release of v1.4
Fixed TMP_PATH in PHP extension XCache (mmap.c) to prevent startup warning
Fixed BusyBox problems with static build against glibc ("__getpagesize" segfault with ping, traceroute and other networking parts)
and added some BusyBox patches.
BusyBox is now dynamic linked, so don't replace the system BusyBox with this build, because of missing mount/paths to libraries at phone startup
01/20/2012 Release of v1.5
Update: Cleanup in scripts and separation of configuration and code. Now it's only necessary to edit configuration.conf and package_selection.conf
Update: Added gzip
Update: Added experimental adb push to device if the device is connected and permissions are ok
Update: Check for installed programs. Script now exists on check failure to minimize errors while compiling.
01/21/2012 Release of v1.6
Bugfix: MySQL build could fail if gen_lex_hash could not generate the file sql/lex_hash.h in pass 2
Update: adb push should now work, check for connected device (and possible multiple devices) and check for right permissions to use the device
Enhancement: Even more explanation of device specific compiler flags and some hints to find the right settings.
01/21/2012 Release of v1.7
Enhancement: To eliminate sure a failing MySQL build, with every new run the source folder will be removed before completely
Enhancement: file version is now strictly checked (the host version must be the same as the cross version)
Some minor changes and filechecks (e.g. autoconf)
01/23/2012 Release of v1.8
Bugfix: Python has SQLite dependencies and SQLite must be ready before Python configure/make
Bugfix: In certain circumstances the optimizer settings get lost
Bugfix: Removed --enable-multibyte from bash, no chance to use it with NDK
01/24/2012 Release of v1.9
Update: Added ngIRCd (Please read the comment for this server, because of running as root)
Update: Sorted the links and package names in the config a little bit...
01/26/2012 Release of v2.0
Bugfix: OpenSSH wants to touch /var/empty while compiling, should be fixed --with-privsep-path now set (thanks to member dateno1 for his reports of such warnings/errors)
Bugfix: Removed a wrong CFLAG -I path from NDK config (only useful for me, really useless for others...)
Enhancement: Example for a user db for Pure-FTPd virtual user database in README (please use virtual users, it's the best and easiest solution)
Update: OpenSSH build need ssh-keygen for the generation of the default keys, added this info to README and the package selection
Bugfix: Not really a bugfix, but some installers don't set the right permissions of libraries in the sysroot folder, so set them global in the final task
Bugfix: Removed --with-lastlog=$TARGET_SYSROOT/var/log/lastlog from OpenSSH and added --with-lastlog=no, I don't know why this is not working and throws a warning, but we can live without it...
Enhancement: Added option to choose the sysroot variant from the CodeSourcery libs: v5TE (default), v4T (old), thumb2 (v7, fastest if supported by the phone)
Enhancement: Added option to tar the headers for development on the device with CodeSourcery (hint: always set rpath in LDFLAGS to make use of the CodeSourcery libs)
01/26/2012 Release of v2.1
Update: Added getcap/setcap and libraries from wishlist, needs libtoolize. Note: This is in a testing stage, kernel and filesystem support needed!
01/30/2012 Added a ScriptStarter app
09/02/2012 Release of v2.2
Update: Local compilation of static "tic" from ncurses for buildhost, if ncurses installation is missing
Enhancement: "downloading only" mode without compiling the packages (and force download)
Enhancement: some parameters for starting the script with additional functions (show with ./build.sh help)
Enhancement: more logging and outputs
Enhancement: language specs, all outputs can be changed to other languages in build.sh
Update: Added dosfstools
Update: Added Ruby
Update: Added example configs and starter scripts for Samba and OpenSSH
Update: Use of http-URLs if possible and update of some broken external links since last release
Bugfix: util-linux-ng needs gettext, added dependence
Bugfix: Added parameter to Samba to solve user/group settings and socket_wrapper support to set path vars at runtime
Update: Changed PHP version to 5.3.10, Samba to 3.6.3 and git to 1.7.9
02/11/2012 internal Release v2.3
Bugfix: openVPN patch for /dev/tun and links to ip/netstat/ifconfig/route, see hints for symlinks
02/16/2012 internal Release v2.4
02/25/2012 Release of v2.5
Update: Added LAME
Update: Added experimental CUPS
Update: Changed PNG version to 1.5.9
Update: Added Net-SNMP
Update: Added Netpbm
Update: Added PNG 1.2.47 for Netpbm and phpSANE
Update: Added LibTIFF
Update: Added SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone
Update: Split patches for bash (4.1 and 4.2) for better and easier (automatic) inclusion of original patches
02/28/2012 Release of v2.6
Update: Added Cuneiform OCR
Update: Added tesseract-ocr (see hints for more details howto use it)
ScriptStarter
02/03/2012 Added a small ScriptStarter for the GUI. With this app, written with JQuery, PhoneGap and Java (wrapper to execute the shell scripts) it's possible to start for example the lighttpd web server, Pure-FTPd or MySQL from within the Android GUI without a terminal and read the output. If a script needs root access, there is a checkbox to allow this. For this to work, copy the script sush in the script folder. All scripts ending with .sh can be started (please choose after the first start your script folder on the device to add the scripts to the selection, normally your [SYSROOT]/sbin folder).
Three buttons (start, stop and status) are predefined for start-stop-status scripts, there is also a parameter input field for more start values.
Free GPL v3 source code also included (Eclipse project). Please don't expect too much eye-candy. The GUI could be much better, but my CSS skills are very limited...
I added a few example scripts in the zip file, for example a battery stats reader. Before starting these scripts, have a look in the source code to be sure that they are compatible with your device.
Update 0.97 02/03/2012: Now the stderr is also printed in the output window
Update 0.99 02/10/2012: Replaced scrolling api with a much better solution (jScrollPane) for the output window. Not the real Android scroll feeling, but it's usable.
Thanks! I would hit the Thanks button, but I am on my mobile atm.
Sent from MIUI powered phone
Someone tried it? Problems?
Please, I need your feedback...
Sounds interesting, I'll try once I get back home and share feedback here
now i try to all packages for test
i will test on cortex a8 (motolora xt720)
TARGET_MARCH="armv7-a" (not work on armv7 )
TARGET_MTUNE="cortex-a8"
TARGET_MFPU="neon"
TARGET_MFLOAT="softfp"
it make dynamic binarys but i don't want dymanic type (because it difficult to divide one thing and maybe not work on android's libs)
how can i make static type?
ps : oh...it has so many comile error (even can't read or count all)
dateno1 said:
i will test on cortex a8 (motolora xt720)
TARGET_MARCH="armv7-a" (not work on armv7 )
TARGET_MTUNE="cortex-a8"
TARGET_MFPU="neon"
TARGET_MFLOAT="softfp"
it make dynamic binarys but i don't want dymanic type (because it difficult to divide one thing and maybe not work on android's libs)
Click to expand...
Click to collapse
I hope I understand your question. Why not dynamic?
Yes, you have to set the parameters for your mobile phone. For TARGET_MARCH try "armv7", for TARGET_MFLOAT "softfp" and for TARGET_MFPU "vfp" but check this last setting carefully for this phone.
It's very hard to build a full static binary (and sometimes nearly impossible) and it's also hard to build against bionic (with NDK). I had no good experience with it . A lot of compiler runs wasn't sucessful. Edit: I also had for example massive problems with openssl. I couldn't find a way to build it with NDK or static and then build other programs like cURL with openssl support.
Static binaries are also too big and not necessary. For this we have a sysroot environment to be independent from bionic and without the restrictions. The compiled libs and binaries are linked to the CodeSourcery glibc and have set the custom "rpath" for the dynamic loader. That's no problem I think.
that's simple
if someone want to make some binary for all version of android (even same cpu) it will not work well (defend on basic lib(ex : libc) is ok but complex things will not work)
i know make static type binary is very difficult but it valable
dateno1 said:
that's simple
if someone want to make some binary for all version of android (even same cpu) it will not work well (defend on basic lib(ex : libc) is ok but complex things will not work)
i know make static type binary is very difficult but it valable
Click to expand...
Click to collapse
You are right, if you want to build a "portable" binary, you have to set very safe settings, like TARGET_MARCH="armv5", the full list is in the "README". But it's not my main target to offer a solution that is easy portable to other phones, instead you can build the software with optimizations for your phone to "get the best" for it
smitna said:
You are right, if you want to build a "portable" binary, you have to set very safe settings, like TARGET_MARCH="armv5", the full list is in the "README". But it's not my main target to offer a solution that is easy portable to other phones, instead you can build the software with optimizations for your phone to "get the best" for it
Click to expand...
Click to collapse
just i want to add some more function on the phone
thanks for patch files
I want to try this, to install some apps(most interested fro mc), but I don' t wanna change my sd card.
Can I use phone' s /data partition as mysysroot directory? I have 1.5GB space available on data partition, it is 2GB all.
And If yes, I must create the dir: /data/mysyroot? And where to create this dir on pc? On root directory of my linux system?
dancer_69 said:
I want to try this, to install some apps(most interested fro mc), but I don' t wanna change my sd card.
Can I use phone' s /data partition as mysysroot directory? I have 1.5GB space available on data partition, it is 2GB all.
And If yes, I must create the dir: /data/mysyroot? And where to create this dir on pc? On root directory of my linux system?
Click to expand...
Click to collapse
You can use the /data partition. Just create a new folder, for example /data/sysroot. On your PC you have to create the same temporary with full user rights. This is not your working dir for building. Create a new dir in /home/user and unpack the files there. In settings insert the /data/sysroot as TARGET_SYSROOT and change the other settings for your phone and choose mc=yes. Then start the ./build.sh
Thanks for your quick answer.
Just to make clear because my english understanding is not in a high level
I must create the /data/sysroot dir under "/"(root directory) on linux pc with superuser rights?
dancer_69 said:
Thanks for your quick answer.
Just to make clear because my english understanding is not in a high level
I must create the /data/sysroot dir under "/"(root directory) on linux pc with superuser rights?
Click to expand...
Click to collapse
Yes, that is right. But this is the only step you need to do as root or better with root rights. After this work as usual as user.
oh it can't make mysql
package selection : mysql only
target directory (prefix) : /sddata/mysql (on sdext on my system)
armv7-a
cortex-a8
neon
softfp
then run ./build.sh 2> error.txt
it is my error messages
http://pastebin.com/g7Yi4bGz
sorry you can see this link only today (my mistake)
this setting can make some binary and lib run on my phone but fail to make gcc or mysql (when i try gcc it just fail to making gcc only (other things were made) )
dateno1 said:
package selection : mysql only
target directory (prefix) : /sddata/mysql (on sdext on my system)
armv7-a
cortex-a8
neon
softfp
then run ./build.sh 2> error.txt
it is my error messages
http://pastebin.com/g7Yi4bGz
sorry you can see this link only today (my mistake)
this setting can make some binary and lib run on my phone but fail to make gcc or mysql (when i try gcc it just fail to making gcc only (other things were made) )
Click to expand...
Click to collapse
Please notice first, that you should not run this script as root!!
Then there is something wrong with the build of the first needed packages:
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
- The script cannot find the readline sources. Do you have a subfolder readline-6.2 in folder sources in your builddir? Same for the other source folders. The script first downloads the packages to folder packages, extract the tars in folder sources, repeated for all selections.
(look at line 67 in your log: ./build.sh: 1402: cd: Python-2.7.2: No such file or directory)
- You have no "xz", install it first and make sure, that the other necessary packages are installed. MySQL and gcc are "two pass" compilations and this builds generating temporary binaries, runable on your linux pc. So you need all for a successful compilation of such binaries like gcc, make, python etc. on your linux machine. If you are using Ubuntu, then install the "meta-package" build-essential, look in the README.
Do you have the other unpacking tools like gunzip etc. Please install also "bison"
You should do following steps:
1. First select only the necessary packages (see above), all others to no.
2. If zlib, python, readline an the others are cross-compiled (without errors) and the libraries are in ./sysroot/lib, then deselect them (=no) and select mysql=yes and start a second run with ./build.sh
smitna said:
Please notice first, that you should not run this script as root!!
Then there is something wrong with the build of the first needed packages:
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
- The script cannot find the readline sources. Do you have a subfolder readline-6.2 in folder sources in your builddir? Same for the other source folders. The script first downloads the packages to folder packages, extract the tars in folder sources, repeated for all selections.
(look at line 67 in your log: ./build.sh: 1402: cd: Python-2.7.2: No such file or directory)
- You have no "xz", install it first and make sure, that the other necessary packages are installed. MySQL and gcc are "two pass" compilations and this builds generating temporary binaries, runable on your linux pc. So you need all for a successful compilation of such binaries like gcc, make, python etc. on your linux machine. If you are using Ubuntu, then install the "meta-package" build-essential, look in the README.
Do you have the other unpacking tools like gunzip etc. Please install also "bison"
You should do following steps:
1. First select only the necessary packages (see above), all others to no.
2. If zlib, python, readline an the others are cross-compiled (without errors) and the libraries are in ./sysroot/lib, then deselect them (=no) and select mysql=yes and start a second run with ./build.sh
Click to expand...
Click to collapse
i'm not edit necessary part
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
i just edit mysql=no => mysql=yes
ok try to install packages and run as not root
dateno1 said:
i'm not edit necessary part
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
i just edit mysql=no => mysql=yes
ok try to install packages and run as not root
Click to expand...
Click to collapse
Good luck!
If you can wait a little bit:
In a short time I release a new version with better error and program checks and some small bug fixes. With this version it's not longer necessary to edit the build.sh, it's now all in only two separated configuration files (configuration and package selection).
hmm...fail to compile mysql again
with this error
note: the mangling of 'va_list' has changed in GCC 4.4
full error log on http://pastebin.com/jd1Pw9Xw
dateno1 said:
with this error
note: the mangling of 'va_list' has changed in GCC 4.4
full error log on http://pastebin.com/jd1Pw9Xw
Click to expand...
Click to collapse
Hello dateno1,
first: you have *NOT* changed to a user account as promised
The va_list is only a note, not an error, no problem (but ask the MySQL devs why they haven't adapted the code to gcc 4.6.1).
Your Python cross binaries and libraries are not ok (from you broken first build?). Please start from a "fresh state" (delete the folder sources/Python-2.7.2 and sysroot) or start from a total clean state.
Please use the latest 1.5 version of the script because of the xargs warnings at the end
Delete the complete sysroot folder, there are some folders not in place (data and docs). There is a script remove_all_files.sh
Start this script in your buildroot with ./remove_all_files.sh to remove all files and symlinks in subfolder sysroot.
Edit: delete the sysroot folder complete, there are some other missing folders from your first build!
Start again with ./build.sh and activated packages (the "ALWAYS NEEDED" one and MySQL)
smitna said:
Hello dateno1,
first: you have *NOT* changed to a user account as promised
The va_list is only a note, not an error, no problem (but ask the MySQL devs why they haven't adapted the code to gcc 4.6.1).
Your Python cross binaries and libraries are not ok (from you broken first build?). Please start from a "fresh state" (delete the folder sources/Python-2.7.2 and sysroot) or start from a total clean state.
Please use the latest 1.5 version of the script because of the xargs warnings at the end
Delete the complete sysroot folder, there are some folders not in place (data and docs). There is a script remove_all_files.sh
Start this script in your buildroot with ./remove_all_files.sh to remove all files and symlinks in subfolder sysroot.
Edit: delete the sysroot folder complete, there are some other missing folders from your first build!
Start again with ./build.sh and activated packages (the "ALWAYS NEEDED" one and MySQL)
Click to expand...
Click to collapse
make new account (compile.compile) and use su -c "./build.sh" compile
i think it work (compiled file's owner is changed to compile.compile)
now i delete almost files and folders (exclude android-ndk, readline-6.2, toolchains) and retrying...oh failed (attach new log (full) )
'를 위해 할 일이 없습니다' mean 'no job for~'
'오류' mean 'error'
DexPatcher
A toolchain for modifying Android APK files at source-level using Java, graphical resource editors, and the full power of Android Studio in all major platforms.
Fully integrated with Android Studio and the Gradle build system.
Includes support for coding assistance and on-demand class decompilation.
Patch Java code in Java using declarative syntax provided by the DexPatcher tool.
Manifest merging enables piece-wise changes to the original app manifest.
Modify existing resources or create new ones using Android Studio's standard resource editors.
Use Android Studio's code template wizards for creating activities, etc.
Pull in Android libraries (a.k.a. '.aar' Android archives) and have their manifests, code and resources automatically merged into the patched app.
And enjoy debugging support.
All in your favorite platform: Linux, Windows or macOS.
DexPatcher tool (Dalvik bytecode patcher)
Release notes: https://github.com/DexPatcher/dexpatcher-tool/releases
Sources: https://github.com/DexPatcher/dexpatcher-tool
DexPatcher Gradle plugins (Android build system and Android Studio integration)
Release notes: https://github.com/DexPatcher/dexpatcher-gradle/releases
Artifacts: https://plugins.gradle.org/search?term=dexpatcher
Sources: https://github.com/DexPatcher/dexpatcher-gradle
Samples: https://github.com/DexPatcher/dexpatcher-gradle-samples
Deprecated tools
Release notes for DexPatcher Gradle v1 plugins: https://github.com/DexPatcher/dexpatcher-gradle/releases?after=v2.0.0-alpha1
Support tools for DexPatcher Gradle v1 plugins: https://github.com/DexPatcher/dexpatcher-gradle-tools
Workflow automation scripts (Linux-only): https://github.com/DexPatcher/dexpatcher-tool-scripts
License
GPL v3 (or later)
THANK YOU ! - The DexPatcher tool uses JesusFreke's dexlib2 (part of smali) to read and write dex files. Many thanks to him for repeatedly helping me in #smali on freenode. When creating apk libraries, the DexPatcher Gradle plugin uses iBotPeaches' Apktool to decode compiled resources and pxb1988's dex2jar to translate Dalvik bytecode. DexPatcher could not exist without the invaluable work of these guys.
Documentation
PATCHING JAVA CODE IN JAVA
The DexPatcher tool uses declarative semantics based on Java annotations to patch the bytecode of the source application. There is no formal definition but hopefully you will find everything you need in this sample:
The sample code being patched.
The patch itself with explanatory comments.
The output of DexPatcher and test runs of the original and patched code.
START HERE !
With the new DexPatcher Gradle v2 plugins
Get the sample code working, here is how:
Clone the samples.
Start with the 'patched-app' sample: open the project with Android Studio, disable instant run, and run or debug your patched app!
Please review the release notes of recent versions of the Gradle plugins for more information on tool compatibility and environment setup.
In the 'patched-app' sample, browse the two 'build.gradle' files (main and app) to get an idea of what is happening. The plugins used are briefly described here. Next get inside the 'app' subproject and take a look at its manifest and resource files. These files are merged with the ones coming from the source app, which is located in the 'apk' directory. Finally look into the 'MainActivity.java' file to see how the compiled code of the app is patched using Java. This is handled by the DexPatcher tool, a key piece of the DexPatcher toolchain. See the section 'Patching Java code in Java' above for more details on this tool, and please review its recent release notes.
The DexPatcher Gradle plugins use Apktool to decode applications and optionally create APK libraries. They optionally use dex2jar to display decompiled application code and to import application code symbols into patch projects. They use the DexPatcher tool to patch the Dalvik bytecode of applications. And finally they use the Andoird build system to merge manifests, code, resources, assets, and extra files, and to repackage applications.
With the old DexPatcher Gradle v1 plugins
To get the old sample code working:
Install the support tools by cloning the repo anywhere you like.
Clone the 'v1' branch of the samples.
Set the 'dexpatcher.dir=<support-tool-dir>' property in identical files named 'local.properties' in the root directory of each sample so that it points to your local clone of the support tools. Create the files if necessary, or have Android Studio create them for you by opening the samples.
Start with the 'patched-app' sample: open the project with Android Studio, disable instant run, and run or debug your patched app!
Please review the old release notes of the Gradle v1 plugins for more information on tool compatibility and environment setup.
OLD NEWS: DexPatcher Featured On XDA Portal !
GermainZ wrote an excellent introductory article for XDA that walks you though the complete process of modding an app using the old deprecated Linux-only workflow automation scripts. This only covers the DexPatcher tool itself, ie: only patching of code, not resources. The workflow is deprecated, but the Java patch code and accompanying explanations are very valuable and continue to be current. I am grateful to him for having taken the time to do this. Please make sure you give it a read:
DexPatcher: Patch Android APKs Using Java.
Also available: [WARNING: Deprecated, Linux-only]
A how-to that will help you get started with the workflow automation scripts.
LICENSING UPDATE:
tl;dr: DexPatcher patches are no longer forcefully covered by the GPL.
Recent versions of DexPatcher no longer require that users bundle the DexPatcher annotations with every patch (although continuing to do so has no ill effects). The DexPatcher licensing terms no longer impose licensing restrictions on patches, as long as users refrain from bundling the DexPatcher annotations with them. In particular, DexPatcher patches are no longer considered to be derivative works of DexPatcher and thus are no longer automatically covered by the GPL. Legals aside, the DexPatcher project urges you not to use copyright laws to introduce artificial scarcity in the world. Please give back to the community: share your work.
XDA:DevDB Information
DexPatcher, Tool/Utility for all devices (see above for details)
Contributors
Lanchon
Source Code: https://github.com/DexPatcher
Version Information
Status: Stable
Created 2015-03-21
Last Updated 2019-11-09
DEPRECATED: Workflow Automation Scripts HOW-TO
DEPRECATED: Workflow Automation Scripts HOW-TO
These scripts bundle several tools (including DexPatcher) and automate boilerplate actions that are part of a typical workflow. They are available only for Linux. Take the leap today, get yourself free, get Linux Mint Cinnamon 64-bit.
Installation
Code:
# cd to a suitable install dir such as '~/opt' or '~/android/tools'.
git clone [url]https://github.com/Lanchon/DexPatcher-scripts.git[/url] dexpatcher
cd dexpatcher
chmod +x dxp-*
# edit 'dxp.config':
# -set the path to android sdk (defaults to '~/android/sdk').
# -disable bundled tools to use the ones in $PATH if desired.
# add 'dxp-*' scripts to $PATH:
# -symlink the 'dxp-*' scripts in a suitable dir in the $PATH such as '~/bin'.
# -or add the 'dexpatcher' dir to $PATH.
Sample Workflow
Code:
# cd to a suitable workspace dir.
dxp-setup-for-apk path/to/TheApk.apk
cd TheApk
dxp-create-keystore
# in 'src-cfr' you will find decompilation of the app to Java (with errors).
# you can use these files to plan your changes.
# in 'src-cfr-nocode' you will find decompilation of empty method stubs.
# you can use selected files as a basis for your patch.
# create your patch as Java files in 'src'.
# the symbols of the original app will be accessible to your patch code.
dxp-make
adb install -r patched.apk
The Big TO-DO List
The Big TO-DO List
...of things I would like done in this project, but that I will probably not have the time to implement myself.
DexPatcher-tool
Moved here: https://github.com/Lanchon/DexPatcher-tool/issues
DexPatcher-gadle
Moved here: https://github.com/Lanchon/DexPatcher-gradle/issues
DexPatcher and javac
Modify javac so that all source symbols -even those defined inside edited classes- are available during compilation. This could produce a patched javac executable, or the standard javac could be patched in-memory using java agents, or its in-memory data structures hacked from an annotation processor (a la Project Lombok).
Decide if dex2jar and jar2dex/dx are transparent and mature enough to move DexPatcher 2.0 to the java bytecode realm, and stop operating at the dalvik bytecode level. [UPDATE: It is not.] Or a VM abstraction could be implemented as a backend to DexPatcher so that the same font end could be used for both dex and class files.
If the project is moved to the java bytecode realm, decide if the new DexPatcher should be a standalone tool like it is today, or a java agent/annotation processor that modifies javac to do the patching during compile time. This would make the tool very easy to integrate into most build systems.
just added the documentation section
I checked docs and wow, massive Maybe you can make some easier examples how to use it (some small patches).
there is a lot of docs missing, especially regarding the workflow, how to extract and use symbols from the source apk, etc. for now im working on 0.2
UPDATE: version 0.2.0 released!
UPDATE: version 0.3.0 released!
Looks good
I will look at it more deeply when I found time.
I'm surprised there isn't so much feedback… May be could you add some examples / details in the OP?
PerfectSlayer said:
Looks good
I will look at it more deeply when I found time.
I'm surprised there isn't so much feedback… May be could you add some examples / details in the OP?
Click to expand...
Click to collapse
i still need to evolve the tool a little with respect to handling of fields, and then it's mostly done. but i need to document a full workflow, including obtaining symbols against which to build the patch (you want your patch code to be able to use all of the source app's code).
Very interested in seeing how this tool progresses. :good:
I still have no idea how to use this tool
UPDATE: version 0.4.0 released!
laura almeida said:
I still have no idea how to use this tool
Click to expand...
Click to collapse
take a look a the test run and samples. docs are very incomplete, but i guess you should understand them if you have experience patching apks in smali.
Lanchon said:
UPDATE: version 0.4.0 released!
Click to expand...
Click to collapse
Sorry for the small change; this beaks patches and so I wanted to push it out ASAP.
Click to expand...
Click to collapse
Don't worry for breaking patches.. Focus on making the better API / usage you can while you are in development. We will thanks you later
UPDATE: version 1.0.0-beta1 released!
An explanation of the workflow and some helper scripts should follow soon.
UPDATE: added Workflow Automation Scripts and HOW-TO
Lanchon said:
UPDATE: added Workflow Automation Scripts and HOW-TO
Click to expand...
Click to collapse
Tips: you could use the 2nd argument of git clone instead of running mv command
Lanchon said:
UPDATE: added Workflow Automation Scripts and HOW-TO
Click to expand...
Click to collapse
The speed of development is atonishing...
Keep up the good work!
UPDATE: DexPatcher featured on XDA portal !
includes full sample, please see the OP for details.