[TOOL] DexPatcher: Modify Android applications at source-level in Android Studio - Android Software Development

DexPatcher
A toolchain for modifying Android APK files at source-level using Java, graphical resource editors, and the full power of Android Studio in all major platforms.
Fully integrated with Android Studio and the Gradle build system.
Includes support for coding assistance and on-demand class decompilation.
Patch Java code in Java using declarative syntax provided by the DexPatcher tool.
Manifest merging enables piece-wise changes to the original app manifest.
Modify existing resources or create new ones using Android Studio's standard resource editors.
Use Android Studio's code template wizards for creating activities, etc.
Pull in Android libraries (a.k.a. '.aar' Android archives) and have their manifests, code and resources automatically merged into the patched app.
And enjoy debugging support.
All in your favorite platform: Linux, Windows or macOS.
DexPatcher tool (Dalvik bytecode patcher)
Release notes: https://github.com/DexPatcher/dexpatcher-tool/releases
Sources: https://github.com/DexPatcher/dexpatcher-tool
DexPatcher Gradle plugins (Android build system and Android Studio integration)
Release notes: https://github.com/DexPatcher/dexpatcher-gradle/releases
Artifacts: https://plugins.gradle.org/search?term=dexpatcher
Sources: https://github.com/DexPatcher/dexpatcher-gradle
Samples: https://github.com/DexPatcher/dexpatcher-gradle-samples
Deprecated tools
Release notes for DexPatcher Gradle v1 plugins: https://github.com/DexPatcher/dexpatcher-gradle/releases?after=v2.0.0-alpha1
Support tools for DexPatcher Gradle v1 plugins: https://github.com/DexPatcher/dexpatcher-gradle-tools
Workflow automation scripts (Linux-only): https://github.com/DexPatcher/dexpatcher-tool-scripts
License
GPL v3 (or later)
THANK YOU ! - The DexPatcher tool uses JesusFreke's dexlib2 (part of smali) to read and write dex files. Many thanks to him for repeatedly helping me in #smali on freenode. When creating apk libraries, the DexPatcher Gradle plugin uses iBotPeaches' Apktool to decode compiled resources and pxb1988's dex2jar to translate Dalvik bytecode. DexPatcher could not exist without the invaluable work of these guys.
Documentation
PATCHING JAVA CODE IN JAVA
The DexPatcher tool uses declarative semantics based on Java annotations to patch the bytecode of the source application. There is no formal definition but hopefully you will find everything you need in this sample:
The sample code being patched.
The patch itself with explanatory comments.
The output of DexPatcher and test runs of the original and patched code.
START HERE !
With the new DexPatcher Gradle v2 plugins
Get the sample code working, here is how:
Clone the samples.
Start with the 'patched-app' sample: open the project with Android Studio, disable instant run, and run or debug your patched app!
Please review the release notes of recent versions of the Gradle plugins for more information on tool compatibility and environment setup.
In the 'patched-app' sample, browse the two 'build.gradle' files (main and app) to get an idea of what is happening. The plugins used are briefly described here. Next get inside the 'app' subproject and take a look at its manifest and resource files. These files are merged with the ones coming from the source app, which is located in the 'apk' directory. Finally look into the 'MainActivity.java' file to see how the compiled code of the app is patched using Java. This is handled by the DexPatcher tool, a key piece of the DexPatcher toolchain. See the section 'Patching Java code in Java' above for more details on this tool, and please review its recent release notes.
The DexPatcher Gradle plugins use Apktool to decode applications and optionally create APK libraries. They optionally use dex2jar to display decompiled application code and to import application code symbols into patch projects. They use the DexPatcher tool to patch the Dalvik bytecode of applications. And finally they use the Andoird build system to merge manifests, code, resources, assets, and extra files, and to repackage applications.
With the old DexPatcher Gradle v1 plugins
To get the old sample code working:
Install the support tools by cloning the repo anywhere you like.
Clone the 'v1' branch of the samples.
Set the 'dexpatcher.dir=<support-tool-dir>' property in identical files named 'local.properties' in the root directory of each sample so that it points to your local clone of the support tools. Create the files if necessary, or have Android Studio create them for you by opening the samples.
Start with the 'patched-app' sample: open the project with Android Studio, disable instant run, and run or debug your patched app!
Please review the old release notes of the Gradle v1 plugins for more information on tool compatibility and environment setup.
OLD NEWS: DexPatcher Featured On XDA Portal !
GermainZ wrote an excellent introductory article for XDA that walks you though the complete process of modding an app using the old deprecated Linux-only workflow automation scripts. This only covers the DexPatcher tool itself, ie: only patching of code, not resources. The workflow is deprecated, but the Java patch code and accompanying explanations are very valuable and continue to be current. I am grateful to him for having taken the time to do this. Please make sure you give it a read:
DexPatcher: Patch Android APKs Using Java.
Also available: [WARNING: Deprecated, Linux-only]
A how-to that will help you get started with the workflow automation scripts.
LICENSING UPDATE:
tl;dr: DexPatcher patches are no longer forcefully covered by the GPL.
Recent versions of DexPatcher no longer require that users bundle the DexPatcher annotations with every patch (although continuing to do so has no ill effects). The DexPatcher licensing terms no longer impose licensing restrictions on patches, as long as users refrain from bundling the DexPatcher annotations with them. In particular, DexPatcher patches are no longer considered to be derivative works of DexPatcher and thus are no longer automatically covered by the GPL. Legals aside, the DexPatcher project urges you not to use copyright laws to introduce artificial scarcity in the world. Please give back to the community: share your work.
XDA:DevDB Information
DexPatcher, Tool/Utility for all devices (see above for details)
Contributors
Lanchon
Source Code: https://github.com/DexPatcher
Version Information
Status: Stable
Created 2015-03-21
Last Updated 2019-11-09

DEPRECATED: Workflow Automation Scripts HOW-TO
DEPRECATED: Workflow Automation Scripts HOW-TO
These scripts bundle several tools (including DexPatcher) and automate boilerplate actions that are part of a typical workflow. They are available only for Linux. Take the leap today, get yourself free, get Linux Mint Cinnamon 64-bit.
Installation
Code:
# cd to a suitable install dir such as '~/opt' or '~/android/tools'.
git clone [url]https://github.com/Lanchon/DexPatcher-scripts.git[/url] dexpatcher
cd dexpatcher
chmod +x dxp-*
# edit 'dxp.config':
# -set the path to android sdk (defaults to '~/android/sdk').
# -disable bundled tools to use the ones in $PATH if desired.
# add 'dxp-*' scripts to $PATH:
# -symlink the 'dxp-*' scripts in a suitable dir in the $PATH such as '~/bin'.
# -or add the 'dexpatcher' dir to $PATH.
Sample Workflow
Code:
# cd to a suitable workspace dir.
dxp-setup-for-apk path/to/TheApk.apk
cd TheApk
dxp-create-keystore
# in 'src-cfr' you will find decompilation of the app to Java (with errors).
# you can use these files to plan your changes.
# in 'src-cfr-nocode' you will find decompilation of empty method stubs.
# you can use selected files as a basis for your patch.
# create your patch as Java files in 'src'.
# the symbols of the original app will be accessible to your patch code.
dxp-make
adb install -r patched.apk

The Big TO-DO List
The Big TO-DO List
...of things I would like done in this project, but that I will probably not have the time to implement myself.
DexPatcher-tool
Moved here: https://github.com/Lanchon/DexPatcher-tool/issues
DexPatcher-gadle
Moved here: https://github.com/Lanchon/DexPatcher-gradle/issues
DexPatcher and javac
Modify javac so that all source symbols -even those defined inside edited classes- are available during compilation. This could produce a patched javac executable, or the standard javac could be patched in-memory using java agents, or its in-memory data structures hacked from an annotation processor (a la Project Lombok).
Decide if dex2jar and jar2dex/dx are transparent and mature enough to move DexPatcher 2.0 to the java bytecode realm, and stop operating at the dalvik bytecode level. [UPDATE: It is not.] Or a VM abstraction could be implemented as a backend to DexPatcher so that the same font end could be used for both dex and class files.
If the project is moved to the java bytecode realm, decide if the new DexPatcher should be a standalone tool like it is today, or a java agent/annotation processor that modifies javac to do the patching during compile time. This would make the tool very easy to integrate into most build systems.

just added the documentation section

I checked docs and wow, massive Maybe you can make some easier examples how to use it (some small patches).

there is a lot of docs missing, especially regarding the workflow, how to extract and use symbols from the source apk, etc. for now im working on 0.2

UPDATE: version 0.2.0 released!

UPDATE: version 0.3.0 released!

Looks good
I will look at it more deeply when I found time.
I'm surprised there isn't so much feedback… May be could you add some examples / details in the OP?

PerfectSlayer said:
Looks good
I will look at it more deeply when I found time.
I'm surprised there isn't so much feedback… May be could you add some examples / details in the OP?
Click to expand...
Click to collapse
i still need to evolve the tool a little with respect to handling of fields, and then it's mostly done. but i need to document a full workflow, including obtaining symbols against which to build the patch (you want your patch code to be able to use all of the source app's code).

Very interested in seeing how this tool progresses. :good:

I still have no idea how to use this tool

UPDATE: version 0.4.0 released!

laura almeida said:
I still have no idea how to use this tool
Click to expand...
Click to collapse
take a look a the test run and samples. docs are very incomplete, but i guess you should understand them if you have experience patching apks in smali.

Lanchon said:
UPDATE: version 0.4.0 released!
Click to expand...
Click to collapse
Sorry for the small change; this beaks patches and so I wanted to push it out ASAP.
Click to expand...
Click to collapse
Don't worry for breaking patches.. Focus on making the better API / usage you can while you are in development. We will thanks you later

UPDATE: version 1.0.0-beta1 released!
An explanation of the workflow and some helper scripts should follow soon.

UPDATE: added Workflow Automation Scripts and HOW-TO

Lanchon said:
UPDATE: added Workflow Automation Scripts and HOW-TO
Click to expand...
Click to collapse
Tips: you could use the 2nd argument of git clone instead of running mv command

Lanchon said:
UPDATE: added Workflow Automation Scripts and HOW-TO
Click to expand...
Click to collapse
The speed of development is atonishing...
Keep up the good work!

UPDATE: DexPatcher featured on XDA portal !
includes full sample, please see the OP for details.

Related

[APP] php 5.4.2 + lighttpd 1.4.29 in FCGI mode, crosscompiled for ARM

Update 7 May 2012: New version released, containing PHP 5.4.2 with curl / openssl modules built in (still for armv5). Besides the version number, installation steps remain the same.
Introduction
The web page of the project, containing more information, licenses, configuration fils, list of modules for php and lighttpd, etc is here:
http://hex.ro/wp/blog/php-and-lighttpd-for-android/
It also contains and example, how to install SMF forum onto the phone
Download
This comes with NO WARRANTY. Use it at your own risk!
php-5.4.2_lighttpd_1.4.29_armv5.tar.gz
php-5.3.8_lighttpd_1.4.29_armv5.tar.gz
Installing
After you copy the archive above into the root of your SD Card, installation is easy:
Code:
cd /
tar xf /sdcard/php-5.3.8_lighttpd_1.4.29_armv5.tar.gz
reload-fcgi.sh
reload-lighttpd.sh
Make sure you are on the root file system (/) before executing tar command.
Also, mandatory: rooted phone + busybox + /system partition mounted read only.
Below information is older - which was before the bundle was ready
Coming soon:
(excerpt from phpinfo() now running on my phone)
PHP:
System Linux localhost 2.6.32.9-perf #1 PREEMPT Fri Jul 8 13:45:08 2011 armv7l
Build Date Aug 30 2011 22:07:32
...
PHP Version 5.3.8
...
Server API CGI/FastCGI
Virtual Directory Support disabled
Configuration File (php.ini) Path /system/etc/php
Loaded Configuration File /system/etc/php/php.ini
...
_SERVER["_"] /system/bin/php
_SERVER["FCGI_ROLE"] RESPONDER
_SERVER["SERVER_SOFTWARE"] lighttpd/1.4.29
_SERVER["SERVER_NAME"] 192.168.1.3
_SERVER["GATEWAY_INTERFACE"] CGI/1.1
I have managed to compile lighttp 1.4.29 with fcgi support as well as php 5.3.8 (both latest version).
All graphics libraries are in as well as SQLite.
I will still have to play with OpenSSL to get it running. Maybe MySQL in the future
It will take a while to set everything up (a web page for source code, wiki on how to install), but we're close to have a fuller Linux experience on Android powered phones.
This would be amazing, I've been hoping for something like this.
wow, I'm really looking forward this
It's cool to have this functionality, cuz sharing files via BT is a pain the ass
I'm waiting too. And wish you luck. Thanks for the efforts.
Thank you for the kind words
Yes, everything works now, and I've tried out a PHP gallery as well as PHP file editor (so you can edit .php files within Opera connected to local host).
It works. Unfortunately, on my Arc, it takes about 3 seconds to generate a thumbnail of a camera image. Now I understand why the whole thing with Media Server indexing files first time after reboot, when phone is slow and etc .. and why the need that apps have to notify Media Server to index additional files when apps create them. So by the time user gets to open the gallery, their thumbnails are already there (it masks the delay).
I want to try out more scripts and then create an installer for the whole thing. If members here report it works, then I will upload it to the market.
I think I will try a bunch of forum engines.
How many additional modules will there be? ImageMagik is really useful… or at least GD
EDIT: ohh, didn't see OP says all graphic libraries included (^_^; )
viulian said:
Thank you for the kind words
Click to expand...
Click to collapse
We should thank you for your efforts. And we are.
viulian said:
I want to try out more scripts and then create an installer for the whole thing. If members here report it works, then I will upload it to the market.
Click to expand...
Click to collapse
You can count on me. I'm not a full time PHP developer, but I wrote few little "systems" in "my way" so it should test non-standard approach at least. but I believe you've already tested basic PHP behaviour
By the way - you mentioned something about mysql... I would personally prefer PostgreSQL, but I do not know anything about a way it's written so theoretically it can be harder to move to arm platform. Postgres is more flexible in many ways, and sometimes (I've heard) a way faster because of simplicity of mysql.
But if anything will be problematic with sql, I can always fall back to sqlite
Update:
libcurl won't be bundled (PHP will be compiled without --with-curl argument).
It seems that libcurl has a dependency on libc - and it tries to load libc.so causing php executable to crash.
Output from strace:
Code:
open("/system/lib/libc.so", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=273912, ...}) = 0
close(3) = 0
--- {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0xfde0004} (Segmentation fault) ---
+++ killed by SIGSEGV +++
Segmentation fault
that's really sad. cURL is sooo cool. (´Д`)
but so far you always found a way to make things work and I hope the same will apply to this as well.
If you have an idea that would work, then will this released later? or you let us play with the cURL-less boundle?
I won't mind lacking of cURL. We have wget onboard for simple downloads. For more sophisticated actions android isn't the best platform
Here's the info.php generated using lighttpd / php statically built for Android:
info.php.zip
I've been on a trip with limited access to internet, and so far I managed to find a fix for lighttpd which, when statically built, would not load the modules in the correct order. It was a mess to use I've documented it here: http://redmine.lighttpd.net/issues/2343
ImageMagick is not ported yet (just standard gd as you will see). I'm tempted to port ImageMagick also, since it seems that batch operations are faster with it than with the built in gd (and on my Arc, generating the thumbnail of a camera taken photo takes about 3 seconds, which is slow).
So far I've tested fotogallery and monoedit (to edit php files on the phone using any browser) and they work fine.
Please expect updates soon.
I've just managed to cross-compile ImageMagick for Android, and it takes mogrify about 4.2 seconds (on my Arc) to create a 120x90 px thumbnail for a 1.8Mb photo from the camera.
Code:
[[email protected]]/sdcard/DCIM/100ANDRO# time mogrify -path thumbs -thumbnail 120x90 DSC_0172.jpg
real 0m4.176s
user 0m3.950s
sys 0m0.130s
Also, each executable has about 5.3Mb when statically linked (convert, mogrify, etc...) which makes the ImageMagick bundle to around 57Mb in total (which is huge).
Thus, no ImageMagick for now waiting for better hardware.
When will you release any alpha to allow us killing it?
I'm working on a web page on my blog where I'll present everything - and will post an update here. If not later tonight, then tomorrow I'll release it.
For now it is compiled for armv5 CPUs (meaning it will work on many phones, even older and slower ones). I've tested it on a Sony X10 Mini (MSM7227, 600Mhz, ArmV5) and Sony Arc (MSM8255, 1Ghz, ArmV7) and works a treat!
Also installed SMF (Simple Machines Forum with sqlite) and it works.
Great . I'll try it as soon as you publish it
krzych said:
Great . I'll try it as soon as you publish it
Click to expand...
Click to collapse
Please check first post and let me know! I've just released it.
i think max request size is too small
it will make 413 error when try to attach file
default value is 2Gb but i think viulian set too small value (even i can't attache 250kb jpg )
if someone want to increase it
add this lines on your lighttpd.conf
server.max-request-size=10240
server.network-backend=”writev”
server.upload-dirs=( “/sddata/www/tmp” )
size=kb
dirs=must same as upload_tmp_dir on your php.ini
there are my config files
Thank you for the updates
Both php and lighttpd come with the default values - I did not change anything ... (in an attept to offer as much as possible the Linux experience with them, when you have to reconfigure things).
The only change I did was to set the temp folder for php to /data/local/tmp (although your suggestion to use the /sdcard might work out better in case of a lot of uploads since the /data space is limited compared to /sdcard).
i also think sdcard is better but /sdcard can't save permission and ownner
it's big secure hole
i think sd-ext is best
i compile the execwrap
using by gcc for android (you can download from here (you can extract from main files from https://market.android.com/details?id=com.n0n3m4.gcc4droid&feature=more_from_developer)
you can find documents about execwrap from lighty's site

[SCRIPTSET v2.6][02/28/2012] Build native GNU/Linux applications: the easy way

New: SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone
This can (in combination with tesseract-ocr) be very useful to recognize text from scanned images.
For pictures see also http://forum.xda-developers.com/showpost.php?p=23046120&postcount=187
Update 01/30/2012: Now with a small android app (ScriptStarter) to start services and other shell scripts easily from the GUI. See bottom of the post.
WHAT'S POSSIBLE WITH THIS SCRIPT:
With this bash script your are able to build relatively easy from source a lot of native software (over 90 at the moment) from the GNU/Linux community for your Android phone and you will have
the good experience with free software.
All these tools and programs are not "apps for the GUI" for Android but native running command line tools "under the surface" of your Android GUI like the kernel and other services.
Don't ask me if it makes any sense to install program xy (like MySQL) on a phone. This is your decision. It's possible, so try it and see what you can do with it.
Note: I tested most of the compiled software on a Samsung Galaxy S2 (my phone). I cannot test it on a variety of phones because I have no mobile phone shop...
Currently buildable with this scripts (abstract, see the full list in package_selection.conf):
PHP 5.3.10 (with almost all extensions and XCache)
MySQL 5.5.19 (if you want a full database server on your phone...)
lighttpd 1.4.30
OpenSSH 5.9p1
OpenSSL 1.0.0f
Samba 3.6.3
cURL 7.23.1
bash 4.2
BusyBox 1.19.3
ImageMagick 6.7.4-10
OpenVPN 2.2.2
Python 2.7.2
Node.js 0.6.7
git 1.7.9
Wget 1.13.4
Rsync 3.0.9
TCPDUMP 4.2.0
Midnight Commander 4.8.1
Pure-FTPd 1.0.35
screen 4.0.3
iptables 1.4.10 (without bionic bug "getnetbyaddr()" and with iptables-save/restore!)
BIND 9.8.1-P1
Lua 5.2.0
Ruby 1.9.3-p0
usbutils 004
MTDUtils latest git
GCC 4.6.2 (c, c++)
GNU Coreutils 8.14, Binutils 2.22, Findutils 4.4.2, Inetutils 1.8
QEMU 0.15.1
SANE 1.0.22
tesseract-ocr 3.01
...
a lot of libraries, development tools and other useful programs
...
FEATURES:
automatic download and untar the cross-compilers (if not already done) for cross compiling
automatic download and untar the source packages (if not already done), that you have selected
solve dependences and reserves additional needed packages
apply patches to the sources if necessary
build them, "install" them in a folder for later "ready to transfer" to your phone and stripes the binaries and libraries to reduce the filesize if selected
autogenerate a tar file and adb push the file to device if selected
allows you to use well-known software like PHP, MySQL, OpenSSL, OpenSSH, OpenVPN, bash and lighttpd on your phone (see the full list in file package_selection.conf, see attachement)
scripting and development on your phone
no limits with BusyBox (without want to be derogative, BusyBox is a wonderful tool), full featured tools like Coreutils, Binutils etc.
build scripts are expandable with other or newer packages you eventual want to have
QUESTIONS AND ANSWERS:
Q: Do I need native GNU/Linux programs and what is that?
A: If you do not know what native Linux applications are, then stop here - you don't need them.
Q: What can I do with these tools?
A: Many things. As a developer you can use the libraries to include them in Android GUI apps, as user you can start for example a webserver on your phone and write some HTML or PHP pages
just for fun or replace BusyBox with full featured tools (or only some commands).
Some tools are also very useful to find network errors, check the network traffic and many more.
Q: What do I need?
A: A Linux installation (VM or real), some Linux knowledge and a little bit time to read all instructions carefully. A rooted Android device with enouth free memory (ext2/3/4 formatted)
See README for details.
Q: Costs?
A: Nothing. The scripts are Open Source (GPL v3), the packages are GPL licensed or under a similar license. The cross-compilers are Open Source or free to use for non-profit purposes.
But I would be pleased if I get a donation and/or a press on the "Thanks button", if the scripts are useful for you. It was a long and hard work...
Note: The attachement scriptset-x.yy.zip is double packed (inside .bz2), because I want preserve the right settings of the scripts.
STEPS:
Selected the desired packages in the config file
Adjust the compiler settings for your phone
Run the build script to download and compile the software
Copy the generated TAR files to your phone or let "adb" this do for you and unpack them in the prepared sysroot folder
For more information please read the instructions in "README"
Please let me also know if you have any problems or if you have questions or further suggestions.
Thanks go to the Open Source community and a lot of people/webpages with hints about cross-compiling software.
Thanks
smitna
CHANGELOG:
01/14/2012 First public release v0.9
01/14/2012 Update: Removed unneeded big patch for ncurses
01/15/2012 Release of v1.0
Update: Fixed build of gcc. Note: the header files are not copied automatically with the script.
If you want to compile on your phone, set TAR_HEADERS=1 in configuraton.conf
Typos: some translations corrected in build.sh
01/16/2012 Release of v1.1
Update: Removed building of fsck from util-linux-ng, it's in e2fsprogs
Update: ISC Cron / vixie cron: set correct paths in pathnames.h before compiling
Enhancement: More detailed explanation of device specific compiler settings with list of possible values
Update: Removed dependence e2fsprogs libs from e2fsprogs
01/17/2012 Release of v1.2
Update: Build of zip, zipcloak, zipnote, zipsplit, unzip, unzipsfx and funzip now possible without errors
01/17/2012 Release of v1.3
Update: Added Node.js
Typos: some translations and language errors corrected in build.sh and README (sorry, I'm not a native English speaker...)
01/18/2012 Release of v1.4
Fixed TMP_PATH in PHP extension XCache (mmap.c) to prevent startup warning
Fixed BusyBox problems with static build against glibc ("__getpagesize" segfault with ping, traceroute and other networking parts)
and added some BusyBox patches.
BusyBox is now dynamic linked, so don't replace the system BusyBox with this build, because of missing mount/paths to libraries at phone startup
01/20/2012 Release of v1.5
Update: Cleanup in scripts and separation of configuration and code. Now it's only necessary to edit configuration.conf and package_selection.conf
Update: Added gzip
Update: Added experimental adb push to device if the device is connected and permissions are ok
Update: Check for installed programs. Script now exists on check failure to minimize errors while compiling.
01/21/2012 Release of v1.6
Bugfix: MySQL build could fail if gen_lex_hash could not generate the file sql/lex_hash.h in pass 2
Update: adb push should now work, check for connected device (and possible multiple devices) and check for right permissions to use the device
Enhancement: Even more explanation of device specific compiler flags and some hints to find the right settings.
01/21/2012 Release of v1.7
Enhancement: To eliminate sure a failing MySQL build, with every new run the source folder will be removed before completely
Enhancement: file version is now strictly checked (the host version must be the same as the cross version)
Some minor changes and filechecks (e.g. autoconf)
01/23/2012 Release of v1.8
Bugfix: Python has SQLite dependencies and SQLite must be ready before Python configure/make
Bugfix: In certain circumstances the optimizer settings get lost
Bugfix: Removed --enable-multibyte from bash, no chance to use it with NDK
01/24/2012 Release of v1.9
Update: Added ngIRCd (Please read the comment for this server, because of running as root)
Update: Sorted the links and package names in the config a little bit...
01/26/2012 Release of v2.0
Bugfix: OpenSSH wants to touch /var/empty while compiling, should be fixed --with-privsep-path now set (thanks to member dateno1 for his reports of such warnings/errors)
Bugfix: Removed a wrong CFLAG -I path from NDK config (only useful for me, really useless for others...)
Enhancement: Example for a user db for Pure-FTPd virtual user database in README (please use virtual users, it's the best and easiest solution)
Update: OpenSSH build need ssh-keygen for the generation of the default keys, added this info to README and the package selection
Bugfix: Not really a bugfix, but some installers don't set the right permissions of libraries in the sysroot folder, so set them global in the final task
Bugfix: Removed --with-lastlog=$TARGET_SYSROOT/var/log/lastlog from OpenSSH and added --with-lastlog=no, I don't know why this is not working and throws a warning, but we can live without it...
Enhancement: Added option to choose the sysroot variant from the CodeSourcery libs: v5TE (default), v4T (old), thumb2 (v7, fastest if supported by the phone)
Enhancement: Added option to tar the headers for development on the device with CodeSourcery (hint: always set rpath in LDFLAGS to make use of the CodeSourcery libs)
01/26/2012 Release of v2.1
Update: Added getcap/setcap and libraries from wishlist, needs libtoolize. Note: This is in a testing stage, kernel and filesystem support needed!
01/30/2012 Added a ScriptStarter app
09/02/2012 Release of v2.2
Update: Local compilation of static "tic" from ncurses for buildhost, if ncurses installation is missing
Enhancement: "downloading only" mode without compiling the packages (and force download)
Enhancement: some parameters for starting the script with additional functions (show with ./build.sh help)
Enhancement: more logging and outputs
Enhancement: language specs, all outputs can be changed to other languages in build.sh
Update: Added dosfstools
Update: Added Ruby
Update: Added example configs and starter scripts for Samba and OpenSSH
Update: Use of http-URLs if possible and update of some broken external links since last release
Bugfix: util-linux-ng needs gettext, added dependence
Bugfix: Added parameter to Samba to solve user/group settings and socket_wrapper support to set path vars at runtime
Update: Changed PHP version to 5.3.10, Samba to 3.6.3 and git to 1.7.9
02/11/2012 internal Release v2.3
Bugfix: openVPN patch for /dev/tun and links to ip/netstat/ifconfig/route, see hints for symlinks
02/16/2012 internal Release v2.4
02/25/2012 Release of v2.5
Update: Added LAME
Update: Added experimental CUPS
Update: Changed PNG version to 1.5.9
Update: Added Net-SNMP
Update: Added Netpbm
Update: Added PNG 1.2.47 for Netpbm and phpSANE
Update: Added LibTIFF
Update: Added SANE (USB Scanner backend and drivers) - OTG equipped device needed to work for this - YES, Scanner Access Now possible and Easy with your phone
Update: Split patches for bash (4.1 and 4.2) for better and easier (automatic) inclusion of original patches
02/28/2012 Release of v2.6
Update: Added Cuneiform OCR
Update: Added tesseract-ocr (see hints for more details howto use it)
ScriptStarter
02/03/2012 Added a small ScriptStarter for the GUI. With this app, written with JQuery, PhoneGap and Java (wrapper to execute the shell scripts) it's possible to start for example the lighttpd web server, Pure-FTPd or MySQL from within the Android GUI without a terminal and read the output. If a script needs root access, there is a checkbox to allow this. For this to work, copy the script sush in the script folder. All scripts ending with .sh can be started (please choose after the first start your script folder on the device to add the scripts to the selection, normally your [SYSROOT]/sbin folder).
Three buttons (start, stop and status) are predefined for start-stop-status scripts, there is also a parameter input field for more start values.
Free GPL v3 source code also included (Eclipse project). Please don't expect too much eye-candy. The GUI could be much better, but my CSS skills are very limited...
I added a few example scripts in the zip file, for example a battery stats reader. Before starting these scripts, have a look in the source code to be sure that they are compatible with your device.
Update 0.97 02/03/2012: Now the stderr is also printed in the output window
Update 0.99 02/10/2012: Replaced scrolling api with a much better solution (jScrollPane) for the output window. Not the real Android scroll feeling, but it's usable.
Thanks! I would hit the Thanks button, but I am on my mobile atm.
Sent from MIUI powered phone
Someone tried it? Problems?
Please, I need your feedback...
Sounds interesting, I'll try once I get back home and share feedback here
now i try to all packages for test
i will test on cortex a8 (motolora xt720)
TARGET_MARCH="armv7-a" (not work on armv7 )
TARGET_MTUNE="cortex-a8"
TARGET_MFPU="neon"
TARGET_MFLOAT="softfp"
it make dynamic binarys but i don't want dymanic type (because it difficult to divide one thing and maybe not work on android's libs)
how can i make static type?
ps : oh...it has so many comile error (even can't read or count all)
dateno1 said:
i will test on cortex a8 (motolora xt720)
TARGET_MARCH="armv7-a" (not work on armv7 )
TARGET_MTUNE="cortex-a8"
TARGET_MFPU="neon"
TARGET_MFLOAT="softfp"
it make dynamic binarys but i don't want dymanic type (because it difficult to divide one thing and maybe not work on android's libs)
Click to expand...
Click to collapse
I hope I understand your question. Why not dynamic?
Yes, you have to set the parameters for your mobile phone. For TARGET_MARCH try "armv7", for TARGET_MFLOAT "softfp" and for TARGET_MFPU "vfp" but check this last setting carefully for this phone.
It's very hard to build a full static binary (and sometimes nearly impossible) and it's also hard to build against bionic (with NDK). I had no good experience with it . A lot of compiler runs wasn't sucessful. Edit: I also had for example massive problems with openssl. I couldn't find a way to build it with NDK or static and then build other programs like cURL with openssl support.
Static binaries are also too big and not necessary. For this we have a sysroot environment to be independent from bionic and without the restrictions. The compiled libs and binaries are linked to the CodeSourcery glibc and have set the custom "rpath" for the dynamic loader. That's no problem I think.
that's simple
if someone want to make some binary for all version of android (even same cpu) it will not work well (defend on basic lib(ex : libc) is ok but complex things will not work)
i know make static type binary is very difficult but it valable
dateno1 said:
that's simple
if someone want to make some binary for all version of android (even same cpu) it will not work well (defend on basic lib(ex : libc) is ok but complex things will not work)
i know make static type binary is very difficult but it valable
Click to expand...
Click to collapse
You are right, if you want to build a "portable" binary, you have to set very safe settings, like TARGET_MARCH="armv5", the full list is in the "README". But it's not my main target to offer a solution that is easy portable to other phones, instead you can build the software with optimizations for your phone to "get the best" for it
smitna said:
You are right, if you want to build a "portable" binary, you have to set very safe settings, like TARGET_MARCH="armv5", the full list is in the "README". But it's not my main target to offer a solution that is easy portable to other phones, instead you can build the software with optimizations for your phone to "get the best" for it
Click to expand...
Click to collapse
just i want to add some more function on the phone
thanks for patch files
I want to try this, to install some apps(most interested fro mc), but I don' t wanna change my sd card.
Can I use phone' s /data partition as mysysroot directory? I have 1.5GB space available on data partition, it is 2GB all.
And If yes, I must create the dir: /data/mysyroot? And where to create this dir on pc? On root directory of my linux system?
dancer_69 said:
I want to try this, to install some apps(most interested fro mc), but I don' t wanna change my sd card.
Can I use phone' s /data partition as mysysroot directory? I have 1.5GB space available on data partition, it is 2GB all.
And If yes, I must create the dir: /data/mysyroot? And where to create this dir on pc? On root directory of my linux system?
Click to expand...
Click to collapse
You can use the /data partition. Just create a new folder, for example /data/sysroot. On your PC you have to create the same temporary with full user rights. This is not your working dir for building. Create a new dir in /home/user and unpack the files there. In settings insert the /data/sysroot as TARGET_SYSROOT and change the other settings for your phone and choose mc=yes. Then start the ./build.sh
Thanks for your quick answer.
Just to make clear because my english understanding is not in a high level
I must create the /data/sysroot dir under "/"(root directory) on linux pc with superuser rights?
dancer_69 said:
Thanks for your quick answer.
Just to make clear because my english understanding is not in a high level
I must create the /data/sysroot dir under "/"(root directory) on linux pc with superuser rights?
Click to expand...
Click to collapse
Yes, that is right. But this is the only step you need to do as root or better with root rights. After this work as usual as user.
oh it can't make mysql
package selection : mysql only
target directory (prefix) : /sddata/mysql (on sdext on my system)
armv7-a
cortex-a8
neon
softfp
then run ./build.sh 2> error.txt
it is my error messages
http://pastebin.com/g7Yi4bGz
sorry you can see this link only today (my mistake)
this setting can make some binary and lib run on my phone but fail to make gcc or mysql (when i try gcc it just fail to making gcc only (other things were made) )
dateno1 said:
package selection : mysql only
target directory (prefix) : /sddata/mysql (on sdext on my system)
armv7-a
cortex-a8
neon
softfp
then run ./build.sh 2> error.txt
it is my error messages
http://pastebin.com/g7Yi4bGz
sorry you can see this link only today (my mistake)
this setting can make some binary and lib run on my phone but fail to make gcc or mysql (when i try gcc it just fail to making gcc only (other things were made) )
Click to expand...
Click to collapse
Please notice first, that you should not run this script as root!!
Then there is something wrong with the build of the first needed packages:
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
- The script cannot find the readline sources. Do you have a subfolder readline-6.2 in folder sources in your builddir? Same for the other source folders. The script first downloads the packages to folder packages, extract the tars in folder sources, repeated for all selections.
(look at line 67 in your log: ./build.sh: 1402: cd: Python-2.7.2: No such file or directory)
- You have no "xz", install it first and make sure, that the other necessary packages are installed. MySQL and gcc are "two pass" compilations and this builds generating temporary binaries, runable on your linux pc. So you need all for a successful compilation of such binaries like gcc, make, python etc. on your linux machine. If you are using Ubuntu, then install the "meta-package" build-essential, look in the README.
Do you have the other unpacking tools like gunzip etc. Please install also "bison"
You should do following steps:
1. First select only the necessary packages (see above), all others to no.
2. If zlib, python, readline an the others are cross-compiled (without errors) and the libraries are in ./sysroot/lib, then deselect them (=no) and select mysql=yes and start a second run with ./build.sh
smitna said:
Please notice first, that you should not run this script as root!!
Then there is something wrong with the build of the first needed packages:
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
- The script cannot find the readline sources. Do you have a subfolder readline-6.2 in folder sources in your builddir? Same for the other source folders. The script first downloads the packages to folder packages, extract the tars in folder sources, repeated for all selections.
(look at line 67 in your log: ./build.sh: 1402: cd: Python-2.7.2: No such file or directory)
- You have no "xz", install it first and make sure, that the other necessary packages are installed. MySQL and gcc are "two pass" compilations and this builds generating temporary binaries, runable on your linux pc. So you need all for a successful compilation of such binaries like gcc, make, python etc. on your linux machine. If you are using Ubuntu, then install the "meta-package" build-essential, look in the README.
Do you have the other unpacking tools like gunzip etc. Please install also "bison"
You should do following steps:
1. First select only the necessary packages (see above), all others to no.
2. If zlib, python, readline an the others are cross-compiled (without errors) and the libraries are in ./sysroot/lib, then deselect them (=no) and select mysql=yes and start a second run with ./build.sh
Click to expand...
Click to collapse
i'm not edit necessary part
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
i just edit mysql=no => mysql=yes
ok try to install packages and run as not root
dateno1 said:
i'm not edit necessary part
ZLIB=yes
OPENSSL=yes
CURL=yes
GETTEXT=yes
LIBXML2=yes
READLINE=yes
NCURSES=yes
i just edit mysql=no => mysql=yes
ok try to install packages and run as not root
Click to expand...
Click to collapse
Good luck!
If you can wait a little bit:
In a short time I release a new version with better error and program checks and some small bug fixes. With this version it's not longer necessary to edit the build.sh, it's now all in only two separated configuration files (configuration and package selection).
hmm...fail to compile mysql again
with this error
note: the mangling of 'va_list' has changed in GCC 4.4
full error log on http://pastebin.com/jd1Pw9Xw
dateno1 said:
with this error
note: the mangling of 'va_list' has changed in GCC 4.4
full error log on http://pastebin.com/jd1Pw9Xw
Click to expand...
Click to collapse
Hello dateno1,
first: you have *NOT* changed to a user account as promised
The va_list is only a note, not an error, no problem (but ask the MySQL devs why they haven't adapted the code to gcc 4.6.1).
Your Python cross binaries and libraries are not ok (from you broken first build?). Please start from a "fresh state" (delete the folder sources/Python-2.7.2 and sysroot) or start from a total clean state.
Please use the latest 1.5 version of the script because of the xargs warnings at the end
Delete the complete sysroot folder, there are some folders not in place (data and docs). There is a script remove_all_files.sh
Start this script in your buildroot with ./remove_all_files.sh to remove all files and symlinks in subfolder sysroot.
Edit: delete the sysroot folder complete, there are some other missing folders from your first build!
Start again with ./build.sh and activated packages (the "ALWAYS NEEDED" one and MySQL)
smitna said:
Hello dateno1,
first: you have *NOT* changed to a user account as promised
The va_list is only a note, not an error, no problem (but ask the MySQL devs why they haven't adapted the code to gcc 4.6.1).
Your Python cross binaries and libraries are not ok (from you broken first build?). Please start from a "fresh state" (delete the folder sources/Python-2.7.2 and sysroot) or start from a total clean state.
Please use the latest 1.5 version of the script because of the xargs warnings at the end
Delete the complete sysroot folder, there are some folders not in place (data and docs). There is a script remove_all_files.sh
Start this script in your buildroot with ./remove_all_files.sh to remove all files and symlinks in subfolder sysroot.
Edit: delete the sysroot folder complete, there are some other missing folders from your first build!
Start again with ./build.sh and activated packages (the "ALWAYS NEEDED" one and MySQL)
Click to expand...
Click to collapse
make new account (compile.compile) and use su -c "./build.sh" compile
i think it work (compiled file's owner is changed to compile.compile)
now i delete almost files and folders (exclude android-ndk, readline-6.2, toolchains) and retrying...oh failed (attach new log (full) )
'를 위해 할 일이 없습니다' mean 'no job for~'
'오류' mean 'error'

[TOOL] [PROJECT] New bugreport analysis tool - open source project by Sony Ericsson

Hi all,
ChkBugReport is a tool that helps you analyze your crash files. Because this tool is such a great help for our internal developers at Sony Ericsson, and as a part of our ambition to step up our open source activities, we want to give all of you the chance to use it by making it open source. Pál Szász (XDA username: pal.szasz), a software developer at Sony Ericsson, is the creator of this tool and will monitor this thread for questions, discussions and suggestions. Feel free to have a look at the open source project and contribute to make it even better!
https://github.com/sonyericssondev/ChkBugReport
For the full story follow the full story in Developer World
/Karl-Johan Dahlström
Sony Ericsson Developer Program
Nice. Any intention of integrating this into some IDE ?
I haven't thought about IDE integration. What do you have in mind exactly?
What I'm working on is automatically opening the result in a browser when processing done (already in the source code, although I tested only in osx at the moment).
Another functionality in progress (not uploaded yet) is to use ADB to automatically save a bugreport. So you would just execute
Code:
chkbugreport --browser adb://
and it will save the bugreport, analyze it and open the result in a browser (as a bonus a screenshot is saved as well).
When this would be implemented I think it would be easy to add it as an external tool in eclipse and launch it from there.
pal.szasz said:
I haven't thought about IDE integration. What do you have in mind exactly?
What I'm working on is automatically opening the result in a browser when processing done (already in the source code, although I tested only in osx at the moment).
Another functionality in progress (not uploaded yet) is to use ADB to automatically save a bugreport. So you would just execute
Code:
chkbugreport --browser adb://
and it will save the bugreport, analyze it and open the result in a browser (as a bonus a screenshot is saved as well).
When this would be implemented I think it would be easy to add it as an external tool in eclipse and launch it from there.
Click to expand...
Click to collapse
One possible IDE integration would be for the report to have linkable parts to the source code when applicable.
Also selecting the process from a windows, having UI buttons starting and stopping the capture of profiling data would be a bonus.
Anyway, the application works very well, and thumbs up for open sourcing this tool .
BTW in my case I've came across some missing resources when generating output:
Code:
Writing chapter: Trace charts/Thread-17 (Thread-18)...
Writing chapter: Main thread activity...
Writing footer...
Cannot find resource: /style.css
Cannot find resource: /icons.png
Cannot find resource: /ftrace-legend-dred.png
Cannot find resource: /ftrace-legend-black.png
Cannot find resource: /ftrace-legend-yellow.png
Cannot find resource: /ftrace-legend-red.png
Cannot find resource: /ftrace-legend-cyan.png
Cannot find resource: /ftrace-legend-dcyan.png
Cannot find resource: /pcy_p0.png
Cannot find resource: /pcy_p1.png
Cannot find resource: /pcy_p2.png
Copying extra resources...
Cannot find resource: /pcy_p3.png
Cannot find resource: /pcy_p4.png
Cannot find resource: /pcy_un.png
Cannot find resource: /pcy_rt.png
Cannot find resource: /pcy_fg.png
Cannot find resource: /pcy_bg.png
Cannot find resource: /main.js
Cannot find resource: /jquery.js
Cannot find resource: /jquery.cookie.js
Cannot find resource: /jquery.jstree.js
Cannot find resource: /jquery.hotkeys.js
Cannot find resource: /jquery.tablesorter.js
Cannot find resource: /jquery.tablednd.js
Cannot find resource: /themes/classic/d.png
Cannot find resource: /themes/classic/dot_for_ie.gif
Cannot find resource: /themes/classic/throbber.gif
Cannot find resource: /themes/classic/style.css
Cannot find resource: /themes/blue/desc.gif
Cannot find resource: /themes/blue/bg.gif
Cannot find resource: /themes/blue/style.css
Cannot find resource: /themes/blue/asc.gif
DONE!
Regarding the resource errors: did you used the precompiled binary or compiled the source code yourself? If it's the later one, how did you compile it? While developing I use eclipse, and I have the "res" folder added a source folder as well. This way the contents of the res folder will be copied to the jar. If you compiled it in some other way, then probably the resources from "res" where not packed into the jar. Building with the ant file should work as well ("ant -f chkbugreport.xml" if I remember correctly)
Regarding the IDE integration: yes, linking/binding the source code would be nice. For tracing data there is already something in progress: running "chkbugreport -t adb://" will show a window to select a process and start/stop it's profiling, but since that code uses code from the android sdk, I need to check how I can link the two codes together (since one of the codes is GPL, the other is Apache) before I can publish it.
I uploaded the changes for adb support. Now an extra jar file is needed: ddmlib.jar. You can find this either in the source code repo, or in the android sdk (a recent one is needed).
I updated the wrapper script as well, to use ddmlib.jar. However if the ddmlib.jar is missing, then the tool will still work like before, simply the new functionality will be missing, which is:
Code:
chkbugreport --browser adb://
The above command will run the bugreport tool on the phone (if it's connected to the PC, of course), processes it and open the result in the browser. As extra bonus, a screenshot is saved as well ;-)
Code:
chkbugreport --browser -t adb://
This will show a window, where you can select which process to trace. Click START then STOP, and the tracefile will be created and processed (and opened in the browser). BUT: there are some limitations:
* adb must be run as root (the tracefile is saved to the /data partition), which means either run en ENG build or USERDEBUG (but then you must first execute "adb root")
* it seems that the fileformat for traceview files have changed, the tool currently supports only version 1, so this will not work if you have a recent android version. I'll fix this later.
pal.szasz said:
Regarding the resource errors: did you used the precompiled binary or compiled the source code yourself? If it's the later one, how did you compile it? While developing I use eclipse, and I have the "res" folder added a source folder as well. This way the contents of the res folder will be copied to the jar. If you compiled it in some other way, then probably the resources from "res" where not packed into the jar. Building with the ant file should work as well ("ant -f chkbugreport.xml" if I remember correctly)
Regarding the IDE integration: yes, linking/binding the source code would be nice. For tracing data there is already something in progress: running "chkbugreport -t adb://" will show a window to select a process and start/stop it's profiling, but since that code uses code from the android sdk, I need to check how I can link the two codes together (since one of the codes is GPL, the other is Apache) before I can publish it.
Click to expand...
Click to collapse
I compiled my own using IDEA, but since I didn't get any errors during compilation (I think) I didn't assume I was missing something in the compilation process.
About the other changes, why not output the traces to /sdcard ? Since that one is always R/W to all users should work right? I say this without actually having looked at the code behind the magic, so perhaps is not a valid suggestion.
I am planning to add support for that as well (i.e. the user can select where to save the tracelogs). AFAIK the sdcard is only readable for everyone by default, apps need to have a certain permission (something with WRITE_EXTERNAL_STORAGE) in order to be able to write to sdcard. But if you are debugging your own app, that's not a problem.
I haven't updated this thread in a looong time. I'm pretty close to put up some more detailed documentation and examples, hopefully I can do it tomorrow.
Meanwhile as a teaser here is an example report generated from a bugreport: http://sonyericssondev.github.com/ChkBugReport/examples/bugreport_aidldeadlock_out/index.html
The shell user does have permissions to write to sdcard, so if you're doing that via adb it should be good.
Sent from my GT-P1000
Well, finally I had time to add some documentation: https://github.com/sonyxperiadev/ChkBugReport/wiki (it's not complete, but at least a good start)
Also source code is pushed and added a new precompiled jar (I started to add the version number in the filename so I can keep the old versions as well): https://github.com/sonyxperiadev/ChkBugReport/downloads
And as usual, any kind of feedback is welcome
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
New link: https://github.com/sonyxperiadev/ChkBugReport
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Since I cannot edit the first post (since it was not created by me), and the link is broken, I started a new thread: http://forum.xda-developers.com/showthread.php?p=34960691
clear post
pal.szasz said:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
New link: https://github.com/sonyxperiadev/ChkBugReport
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Click to expand...
Click to collapse
There are many people, new to development.
Always try to post step by step instruction which will help others....
Is there anyone here that can look at a bug report that popped up on my friend's phone and be able to tell who hacked her phone? She is in bitter custody battle and we are trying to prove it is the ex doing this with intent for court purposes. Please help! If willing please respond and I will give a private email address to private message due to complex circumstances and we all fear for child and our lives. Thanks in advance.
Hi, what are these colorful icons mean in the VM traces info?

[EXPERIMENTAL/TOOLS/GUIDE/ADV.] Toolchain gcc 4.6.3 Cygwin/Linux/OSX; kernel builds

The following contains binaries, and instructions for use of an experimental newer toolchain than those included in the android pre-built package.
This is provided to allow existing linux kernel developers experiment with the updated version of GCC (besides the complicated building instructions the code just has a small patch for a known issue with android and 4.6.* otherwise is directly from GNU (see build information below)
The packages I'm also providing to help developers current on windows (With cygwin but no other virtual environment) to start exploring android Linux kernel development for their devices. (In addition to a OSX toolchain for more advanced mac users)
due to the slight differences between kernels on different devices this How To is intended to be used for advanced users who can adjust for the differences required for the devices specific kernel.
I may add additional guides to this thread if I find a need for them.
[size=+1]Binary Toolchains[/size]
I have two versions of a GCC-4.6.3 toolchain
1) For x86 linux: toolchain-4.6.3.tar.bz2 (Sha1: c8c57aba6ad92e9acddf29ba8620ba880be09a81)
2) For Cygwin (windows): toolchain-4.6.3.cygwin.tar.bz2 (Sha1: 6947e1c1ba95195019f542eb8ba0708667b63eca)
3) For OSX (mac/darwin): toolchain-4.6.3.darwin.tar.bz2 (Sha1: 9a977f0672863fdd9501383a6ad1e30723281f68)
[size=-1]> The linux version was built via this script: http://pastebin.com/b2dZ8YtG (or in the included toolchain_notes.txt)
> The cygwin/darwin version uses the same source however has a slightly modified script (see the included toolchain_notes.txt for the updated script)[/size]
[size=+1]System Requirements[/size]
Note I have a 64bit debian linux box, and a 64bit version of WIndows7 Pro running cygwin, while I believe these binaries ought to be compatible with all x86 linux and cygwin installs these have not been tested by a wide number of people yet.
> Linux users: If you have a working android build environment likely no additional requirements are needed. In addition you must build any kernel/android utilities from a case sensitive filesystem or disk image.
> OSx users: the cygwin packages will give you an idea of what is required, please also see the README in the tar.bz2 about manually installing elf.h
> Cygwin users: as its unlikely you have compiled a kernel with cygwin up to this point you may wish to ensure the following packages are installed (as well as any cygwin recommends to be used with them):
vim
vim-common
make
cmake
lzop
gcc
gcc-core
gcc-g++
wget
git
git-compleation
libncurses-devel
python​
Cygwin users must also set their NTFS file partition to allow case sensitive files:
To do this open regedit and change:
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive
to 0
after this reboot and the NTFS kernel will allow cygwin to use case sensitive files (ie 'AbC' and 'aBc' as two different files)​
[size=+1]Advanced Kernel Building Guide[/size]
Since may devices are slightly different this can only be a high level kernel building guide
Please ask for any specific directions from the current kernel maintainer.
To most people who have built kernels before this will look familiar, the most important part is preparing your local environment to use the new toolchain,
1) If you have not already download the tar.bz2 above, (pick the one for the computer you are preparing to build kernels on)
2) extract the file (for the sake of this guide I will assume it is extracted into a ~/android directory)
ie on cygwin you might:
mkdir -p ~/android
cd ~/android
tar -xjvf toolchain-4.6.3.cygwin.tar.bz2
Note: steps one and two are just properly installing the toolchain, you will not need
to repeat them to build a different kernel.
2.b) OSX Only: check elf.h
open ~/android/toolchain-4.6.3/README in a text editor and check if elf.h is installed, otherwise manually install it as per the readme
3) fetch your kernel source into the ~/android directory and change into the
source codes root directory
(this can be via a git repo or tarball)
example if you wanted my ACER a100/a500 source you might run:
git clone git://github.com/ezterry/AcerTabKernel.git
cd AcerTabKernel
change this to match the source you are fetching
4) In the root directory of the kernel source (~/android/AcerTabKernel in the about a100/a500 example) update the environment for the cross compiler
add the toolchain to your path:
export PATH=$HOME/android/toolchain-4.6.3/bin:$PATH​
set the cross compile:
export ARCH=arm
export CROSS_COMPILE=arm-linux-androideabi-
export KERNEL_DIR=`pwd`​
(this will need to be done once for each terminal you use to run any of the following make commands)
5) Grab your configuration
From the device:
adb pull /proc/config.gz
gzip -dc config.gz > .config
or from a prepared configuration if it's included in your code base by running
make _defconfig
example for the a100/a500 code base is 'make vangogh_defconfig' for the a100 and 'make picasso_defconfig' for the a500
6) Customize configuration
just run menuconfig and use the UI to update settings
make menuconfig​
7) building the kernel (zImage)
This is normal at this point with 'make -j'
so for a quad core CPU you can run
make -j5​
(note: the wrong number in the -j argument will not harm the build but may slow down the building process by either using too few threads or needing to keep track of too many)
8) If it worked you will now have the kernel in arch/arm/boot/zImage
this can be injected into the boot.img (as the kernel, using an existing ramdisk) as is
If it failed for your kernel your kernel source may need one of the common patches
9) Building modules
the following will prepare the modules in subdirectory mod:
make modules
rm -rf mod
mkdir mod
cp `find ./ | grep .ko$` modules.order mod/​
After these commands [if no errors] you can simply copy the contents of mod/ to /system/lib/modules on your device, note it is recommended usually to remove the existing contents of /system/lib/modules unless there is a closed source module required from the previous build despite the new kernel.
[size=+1]Common Patches Required[/size]
If your kernel has not been built for the new toolchain or on cygwin before you may require some of these common patchs: (I've thus far only tested this on my Acer A100/A500 source)
wireless/bcm4329 for GCC-4.6.3:
https://github.com/ezterry/AcerTabKernel/commit/123f32e27e2c74f1c1789ae5d6d5a1c04e1e264c
linux kernel module patch for cygwin (broken elf.h):
https://github.com/ezterry/AcerTabKernel/commit/220db49593cf6b9f3b556e2f4b75b2f6d3ff556c
Error compiling security/smc/bridge_pub2sec.S (I required the cygwin elf patch and this patch to build the Franco galaxy nexus kernel)
0001-Fix-build-error-with-4.6.3-toolchain-smc-0.patch
(use 'git am 0001-Fix-build-error-with-4.6.3-toolchain-smc-0.patch' to apply)
OSX stat patches (fixes errors between GNU and BSD stat):
https://github.com/ezterry/AcerTabKernel/commit/0c49df3cc1a05a0ccd98201511cdc0534aaeb35a
Errors loading newly built modules (modules appear to build cleanly but wont install):
simply add -fno-pic to CFLAGS_MODULE
https://github.com/ezterry/AcerTabKernel/commit/c5ed0fcb014d36936a86ad253f15af43de2f644a
(I can add others here if highlighted to me, and as I find them)
[size=+1]Other Toolchain uses[/size]
The linux toolchain can be used to build android components, however this requires various manipulation to the android /build git repository as currently various other toolchains have hard coded paths into the prebuilt repo. If you intend to incorporate this into your build it may be best to inject the toolchain into your prebuilt repo rather than expect your users to download the links above, ensure to keep the compile notes as it explains how to get the source code to the toolchains)
On cygwin/osx it may also be possible to build android applications however I've not yet tested this as the scripts to make a proper build are complex when attempted outside the android build tree.
Excellent!!! thanks Man!!! Has been looking for one of these
Added in the OSX version of the toolchain (and some related patches)
Also if you are having problems with your modules built with this toolchain a CFLAG fixes it. (see note in common patches)
FINALLY!!! Great Instructions followed them to the T and BAM got my kernel compiled!!!
wow your toolchain is so small, many thanks, just compiled a kernel for my LG P500 gonna test it, thanks again
Bookmarked for testing when I wake up...
I can almost put: "Building under Windows is not currently supported." (quote from google), unofficially at least, out of my mind...
I have been spoiled by Windows for far too long I fear, my attempt for this last week at getting Linux setup and building CM9 has been nothing but a complete FAILURE, too many single line entrys for different distributions/package combo's, commands I dont know and cant find because of wildcard portions of them and lack of understanding, and not enough scripts like the wonderful Compile CM9 script someone put up (cant run it because all the pre-req's arent setup).
Good god Ill be jumping for joy if this works tomorrow on WinBlows!
EDIT: never mind, problem solved, what I need to know now won't be solved here
I wish i knew how to go about doing this..
Thanks
Thanks For ........... i Really need It .........
Hammerfest said:
Bookmarked for testing when I wake up...
I can almost put: "Building under Windows is not currently supported." (quote from google), unofficially at least, out of my mind...
I have been spoiled by Windows for far too long I fear, my attempt for this last week at getting Linux setup and building CM9 has been nothing but a complete FAILURE, too many single line entrys for different distributions/package combo's, commands I dont know and cant find because of wildcard portions of them and lack of understanding, and not enough scripts like the wonderful Compile CM9 script someone put up (cant run it because all the pre-req's arent setup).
Good god Ill be jumping for joy if this works tomorrow on WinBlows!
Click to expand...
Click to collapse
If you succeed tell me how to do it in a more familiar way
I get errors regading the processor not being supported?
brfield said:
I get errors regading the processor not being supported?
Click to expand...
Click to collapse
Sorry I can't support devs that have not learned cut copy and paste.
You probably forgot to export some of the environment variables or are just trying to run a arm binary on your PC and or a x86 binary on your arm device.
But why am I guessing? Why don't we know what you tried to run and what the actual text of the error was.. what type of kernel you tried to build, what you are running?
These packages are for developers if you feel like using them its time to learn to how to trouble shoot a problem and write a ticket to explain said problem clearly to others with at least some information to give us a chance to understand.
Sent from my Galaxy Nexus using Tapatalk 2
Great guide. Have the bcm4329 wireless module, so followed the tips and everything compiled and works!
Thanks!
one question,how to compiled android kernel with -O3 optimization?
jxxhwy said:
one question,how to compiled android kernel with -O3 optimization?
Click to expand...
Click to collapse
Um, in general its recommended you don't:
-O3 is not recommended for the kernel as many low level parts require the assembly code generated to remain as is and not be optimized as will be by some of the options -O3 enables thus likely reducing the stability of your kernel.
---
That said the menuconfig has an option to optimize for size (-Os) if enabled else -O2
For other optimization (including tuning for your cpu) you can add them to the CFLAGS_KERNEL/CFLAGS_MODULE in the make file.)
For -O3 like functionality you could add the increment to the line:
-finline-functions, -funswitch-loops, -fpredictive-commoning, -fgcse-after-reload, -ftreel-vectorize, -ftree-partial-pre and -fipa-cp-clone
-Ofast is -O3 with -ffast-math as well
If you do build with all the -O3 or-Ofast options ensure you test the kernel for stability and run some benchmarks. (as it may not actually be faster depending on the cache misses)
The first option I'd start pruning was the one mentioned in my reference below -fgcse-after-reload.
If you really insist on -O3 rather than just adding the optimization you want that are not in O2,it can be added also in the Makefile where the configure flag is checked and its added to KBUILD_CFLAGS
Obviously all and any of this is at your own risk, anything that breaks you have to find a way to fix it.
Reference:
Gcc optimize options:
http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
Talk of O3 and the kernel (not android specific)
http://unix.stackexchange.com/questions/1597/compiling-gnu-linux-with-o3-optimization
ezterry,thank you very much!
I want to build cm9 kernel using toolchain 4.6.3.
How to replace the default GCC 4.4.3 to 4.6.3 version?
BTW,I use the way -- make bootimage
jxxhwy said:
ezterry,thank you very much!
I want to build cm9 kernel using toolchain 4.6.3.
How to replace the default GCC 4.4.3 to 4.6.3 version?
BTW,I use the way -- make bootimage
Click to expand...
Click to collapse
If you are asking in a cm9 build tree.. its a pain, best is to update the cm kernel makefile to point the build chain temporarily.
If you mean just to compile by hand as described in the op
Once the kernel is compiled find an anykernel update.zip or manually use mkbootimg or fastboot to merge it with the ram disk. Cm9 targets mkbootimg and unpackbootimg (if I'm not mistaken) will build the applications to unpack your current (or the default cm9) boot image and re-create them with your custom kernel built from hand.
Now,I runing script file:
#Let's make sure the environment is clean and ready to compile the kernel
echo "Cleaning house!!"
make mrproper
echo "House cleaned, lets build a kernel!!!"
#
# Lets set the kernel defconfig
echo "defconfig = cyanogenmod_iprj_defconfig"
make ARCH=arm cyanogenmod_iprj_defconfig
#
# Let's build a kernel
echo "Now compiling kernel, go get a soda! "
ARCH=arm CROSS_COMPILE=~/Android/sourcecm9/cm9/toolchain-4.6.3/bin/arm-linux-androideabi- make zImage -j4
#
if [ -f arch/arm/boot/zImage ]; then
echo "Plague has been compiled!!! You can find it in arch/arm/boot/zImage"
else
echo "Kernel did not compile, please check for errors!!"
fi
but,I got some error output:
/home/xxx/cm9/toolchain-4.6.3/bin/arm-linux-androideabi-ld: cannot find usr/initramfs_data.o: No such file or directory
thanks again!!!
I think that means there is an issue with your config..
Most android kernels require initramfs, but don't specify a file, as its provided by the bootloader.. However in your case it is trying to embed it into the kernel.
Sent from my A500 using Tapatalk 2
ezterry said:
I think that means there is an issue with your config..
Most android kernels require initramfs, but don't specify a file, as its provided by the bootloader.. However in your case it is trying to embed it into the kernel.
Sent from my A500 using Tapatalk 2
Click to expand...
Click to collapse
issue has been resolved by myslef.Thanks for your time.

android-platform-headers - platform headers for all Android vers. at your fingertips

I recently looked into stagefright support in ffmpeg and vlc and found that one of the chore they have is having platform headers to compile against around. Actually, they didn't work for me at all - likely because they were written for one Android version, and I run them on the other. Again, if supporting compilation against even one (random) Android version is a chore, who can expect good support for multiple versions?
To fight against such poor state of things, I'd like to introduce android-platform-headers, which provides headers for all important Android versions side by side.
More info is in the README:
Code:
These are sets of header files extracted from main components of
AOSP (Android Open Source Project), for each main release of AOSP.
Android NDK provided by Google contains subset of AOSP platform
headers which are deemed "external interface" by Google. However,
writing advanced software for Android necessitates access to complete
platform. Many aspects of it change, sometimes drastically, between
releases, so compiling user source code against platform headers for
different releases is required. This project intends to streamline
this process, allowing to access headers for all important Android
releases at once.
In this respect, it is similar to "linux-kernel-headers" found in
many Linux distributions - it allows to compile software without
the need to have hundreds of megabytes (or gigabytes) of complete
kernel (or platform) source code around.
To prepare these headers, all *.h and *.hpp files were extracted
from following AOSP projects:
* system/core
* frameworks/base
* frameworks/native
FAQ
---
Q: Google doesn't recommend usage of non-NDK headers.
A: Who cares.
Q: As these headers don't represent external interface, they can be
changes by vendor of some device.
A: True, so we can figure out what particular vendors work on -
on bringing up innovative products, cutting time to market, increasing
power efficiency, or fragmenting platform and breaking compatibility.
Q: How do I know that headers provided are correct?
A: You are encouraged to verify. Each header project directory contains
.git-org/refs/heads/master with revision of tag from which it was extracted.
Actually, extraction was automatic and script is provided. To easily verify
a particular tag:
rm -rf android-1.6_r1
scripts/fetch-headers android-1.6_r1
git diff
TODO
----
1. Add platform "import" libraries to link with (devoid of code, only with
exported symbols).
2. Similar "implib" jars to link Java code against.
Status update: repo now contains platform headers for the most popular Android versions from 1.6 to 4.2, and started to work on scripts to produce "import libraries" to link native apps against.

Categories

Resources