When trying to compile Android from source for a particular device, what files are usually pulled from the device to inject into the build? I'm very familiar with compiling applications from source, but compiling a complete system is new territory for me.
myersn024 said:
When trying to compile Android from source for a particular device, what files are usually pulled from the device to inject into the build? I'm very familiar with compiling applications from source, but compiling a complete system is new territory for me.
Click to expand...
Click to collapse
Check the cyanogen source tree at github. If you i.e. check under:
/android/system/device/htc/bravo
you will find a file called: extract-files.sh
In that file is all the proprietary files needed for the build.
The following contains binaries, and instructions for use of an experimental newer toolchain than those included in the android pre-built package.
This is provided to allow existing linux kernel developers experiment with the updated version of GCC (besides the complicated building instructions the code just has a small patch for a known issue with android and 4.6.* otherwise is directly from GNU (see build information below)
The packages I'm also providing to help developers current on windows (With cygwin but no other virtual environment) to start exploring android Linux kernel development for their devices. (In addition to a OSX toolchain for more advanced mac users)
due to the slight differences between kernels on different devices this How To is intended to be used for advanced users who can adjust for the differences required for the devices specific kernel.
I may add additional guides to this thread if I find a need for them.
[size=+1]Binary Toolchains[/size]
I have two versions of a GCC-4.6.3 toolchain
1) For x86 linux: toolchain-4.6.3.tar.bz2 (Sha1: c8c57aba6ad92e9acddf29ba8620ba880be09a81)
2) For Cygwin (windows): toolchain-4.6.3.cygwin.tar.bz2 (Sha1: 6947e1c1ba95195019f542eb8ba0708667b63eca)
3) For OSX (mac/darwin): toolchain-4.6.3.darwin.tar.bz2 (Sha1: 9a977f0672863fdd9501383a6ad1e30723281f68)
[size=-1]> The linux version was built via this script: http://pastebin.com/b2dZ8YtG (or in the included toolchain_notes.txt)
> The cygwin/darwin version uses the same source however has a slightly modified script (see the included toolchain_notes.txt for the updated script)[/size]
[size=+1]System Requirements[/size]
Note I have a 64bit debian linux box, and a 64bit version of WIndows7 Pro running cygwin, while I believe these binaries ought to be compatible with all x86 linux and cygwin installs these have not been tested by a wide number of people yet.
> Linux users: If you have a working android build environment likely no additional requirements are needed. In addition you must build any kernel/android utilities from a case sensitive filesystem or disk image.
> OSx users: the cygwin packages will give you an idea of what is required, please also see the README in the tar.bz2 about manually installing elf.h
> Cygwin users: as its unlikely you have compiled a kernel with cygwin up to this point you may wish to ensure the following packages are installed (as well as any cygwin recommends to be used with them):
vim
vim-common
make
cmake
lzop
gcc
gcc-core
gcc-g++
wget
git
git-compleation
libncurses-devel
python
Cygwin users must also set their NTFS file partition to allow case sensitive files:
To do this open regedit and change:
HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive
to 0
after this reboot and the NTFS kernel will allow cygwin to use case sensitive files (ie 'AbC' and 'aBc' as two different files)
[size=+1]Advanced Kernel Building Guide[/size]
Since may devices are slightly different this can only be a high level kernel building guide
Please ask for any specific directions from the current kernel maintainer.
To most people who have built kernels before this will look familiar, the most important part is preparing your local environment to use the new toolchain,
1) If you have not already download the tar.bz2 above, (pick the one for the computer you are preparing to build kernels on)
2) extract the file (for the sake of this guide I will assume it is extracted into a ~/android directory)
ie on cygwin you might:
mkdir -p ~/android
cd ~/android
tar -xjvf toolchain-4.6.3.cygwin.tar.bz2
Note: steps one and two are just properly installing the toolchain, you will not need
to repeat them to build a different kernel.
2.b) OSX Only: check elf.h
open ~/android/toolchain-4.6.3/README in a text editor and check if elf.h is installed, otherwise manually install it as per the readme
3) fetch your kernel source into the ~/android directory and change into the
source codes root directory
(this can be via a git repo or tarball)
example if you wanted my ACER a100/a500 source you might run:
git clone git://github.com/ezterry/AcerTabKernel.git
cd AcerTabKernel
change this to match the source you are fetching
4) In the root directory of the kernel source (~/android/AcerTabKernel in the about a100/a500 example) update the environment for the cross compiler
add the toolchain to your path:
export PATH=$HOME/android/toolchain-4.6.3/bin:$PATH
set the cross compile:
export ARCH=arm
export CROSS_COMPILE=arm-linux-androideabi-
export KERNEL_DIR=`pwd`
(this will need to be done once for each terminal you use to run any of the following make commands)
5) Grab your configuration
From the device:
adb pull /proc/config.gz
gzip -dc config.gz > .config
or from a prepared configuration if it's included in your code base by running
make _defconfig
example for the a100/a500 code base is 'make vangogh_defconfig' for the a100 and 'make picasso_defconfig' for the a500
6) Customize configuration
just run menuconfig and use the UI to update settings
make menuconfig
7) building the kernel (zImage)
This is normal at this point with 'make -j'
so for a quad core CPU you can run
make -j5
(note: the wrong number in the -j argument will not harm the build but may slow down the building process by either using too few threads or needing to keep track of too many)
8) If it worked you will now have the kernel in arch/arm/boot/zImage
this can be injected into the boot.img (as the kernel, using an existing ramdisk) as is
If it failed for your kernel your kernel source may need one of the common patches
9) Building modules
the following will prepare the modules in subdirectory mod:
make modules
rm -rf mod
mkdir mod
cp `find ./ | grep .ko$` modules.order mod/
After these commands [if no errors] you can simply copy the contents of mod/ to /system/lib/modules on your device, note it is recommended usually to remove the existing contents of /system/lib/modules unless there is a closed source module required from the previous build despite the new kernel.
[size=+1]Common Patches Required[/size]
If your kernel has not been built for the new toolchain or on cygwin before you may require some of these common patchs: (I've thus far only tested this on my Acer A100/A500 source)
wireless/bcm4329 for GCC-4.6.3:
https://github.com/ezterry/AcerTabKernel/commit/123f32e27e2c74f1c1789ae5d6d5a1c04e1e264c
linux kernel module patch for cygwin (broken elf.h):
https://github.com/ezterry/AcerTabKernel/commit/220db49593cf6b9f3b556e2f4b75b2f6d3ff556c
Error compiling security/smc/bridge_pub2sec.S (I required the cygwin elf patch and this patch to build the Franco galaxy nexus kernel)
0001-Fix-build-error-with-4.6.3-toolchain-smc-0.patch
(use 'git am 0001-Fix-build-error-with-4.6.3-toolchain-smc-0.patch' to apply)
OSX stat patches (fixes errors between GNU and BSD stat):
https://github.com/ezterry/AcerTabKernel/commit/0c49df3cc1a05a0ccd98201511cdc0534aaeb35a
Errors loading newly built modules (modules appear to build cleanly but wont install):
simply add -fno-pic to CFLAGS_MODULE
https://github.com/ezterry/AcerTabKernel/commit/c5ed0fcb014d36936a86ad253f15af43de2f644a
(I can add others here if highlighted to me, and as I find them)
[size=+1]Other Toolchain uses[/size]
The linux toolchain can be used to build android components, however this requires various manipulation to the android /build git repository as currently various other toolchains have hard coded paths into the prebuilt repo. If you intend to incorporate this into your build it may be best to inject the toolchain into your prebuilt repo rather than expect your users to download the links above, ensure to keep the compile notes as it explains how to get the source code to the toolchains)
On cygwin/osx it may also be possible to build android applications however I've not yet tested this as the scripts to make a proper build are complex when attempted outside the android build tree.
Excellent!!! thanks Man!!! Has been looking for one of these
Added in the OSX version of the toolchain (and some related patches)
Also if you are having problems with your modules built with this toolchain a CFLAG fixes it. (see note in common patches)
FINALLY!!! Great Instructions followed them to the T and BAM got my kernel compiled!!!
wow your toolchain is so small, many thanks, just compiled a kernel for my LG P500 gonna test it, thanks again
Bookmarked for testing when I wake up...
I can almost put: "Building under Windows is not currently supported." (quote from google), unofficially at least, out of my mind...
I have been spoiled by Windows for far too long I fear, my attempt for this last week at getting Linux setup and building CM9 has been nothing but a complete FAILURE, too many single line entrys for different distributions/package combo's, commands I dont know and cant find because of wildcard portions of them and lack of understanding, and not enough scripts like the wonderful Compile CM9 script someone put up (cant run it because all the pre-req's arent setup).
Good god Ill be jumping for joy if this works tomorrow on WinBlows!
EDIT: never mind, problem solved, what I need to know now won't be solved here
I wish i knew how to go about doing this..
Thanks
Thanks For ........... i Really need It .........
Hammerfest said:
Bookmarked for testing when I wake up...
I can almost put: "Building under Windows is not currently supported." (quote from google), unofficially at least, out of my mind...
I have been spoiled by Windows for far too long I fear, my attempt for this last week at getting Linux setup and building CM9 has been nothing but a complete FAILURE, too many single line entrys for different distributions/package combo's, commands I dont know and cant find because of wildcard portions of them and lack of understanding, and not enough scripts like the wonderful Compile CM9 script someone put up (cant run it because all the pre-req's arent setup).
Good god Ill be jumping for joy if this works tomorrow on WinBlows!
Click to expand...
Click to collapse
If you succeed tell me how to do it in a more familiar way
I get errors regading the processor not being supported?
brfield said:
I get errors regading the processor not being supported?
Click to expand...
Click to collapse
Sorry I can't support devs that have not learned cut copy and paste.
You probably forgot to export some of the environment variables or are just trying to run a arm binary on your PC and or a x86 binary on your arm device.
But why am I guessing? Why don't we know what you tried to run and what the actual text of the error was.. what type of kernel you tried to build, what you are running?
These packages are for developers if you feel like using them its time to learn to how to trouble shoot a problem and write a ticket to explain said problem clearly to others with at least some information to give us a chance to understand.
Sent from my Galaxy Nexus using Tapatalk 2
Great guide. Have the bcm4329 wireless module, so followed the tips and everything compiled and works!
Thanks!
one question,how to compiled android kernel with -O3 optimization?
jxxhwy said:
one question,how to compiled android kernel with -O3 optimization?
Click to expand...
Click to collapse
Um, in general its recommended you don't:
-O3 is not recommended for the kernel as many low level parts require the assembly code generated to remain as is and not be optimized as will be by some of the options -O3 enables thus likely reducing the stability of your kernel.
---
That said the menuconfig has an option to optimize for size (-Os) if enabled else -O2
For other optimization (including tuning for your cpu) you can add them to the CFLAGS_KERNEL/CFLAGS_MODULE in the make file.)
For -O3 like functionality you could add the increment to the line:
-finline-functions, -funswitch-loops, -fpredictive-commoning, -fgcse-after-reload, -ftreel-vectorize, -ftree-partial-pre and -fipa-cp-clone
-Ofast is -O3 with -ffast-math as well
If you do build with all the -O3 or-Ofast options ensure you test the kernel for stability and run some benchmarks. (as it may not actually be faster depending on the cache misses)
The first option I'd start pruning was the one mentioned in my reference below -fgcse-after-reload.
If you really insist on -O3 rather than just adding the optimization you want that are not in O2,it can be added also in the Makefile where the configure flag is checked and its added to KBUILD_CFLAGS
Obviously all and any of this is at your own risk, anything that breaks you have to find a way to fix it.
Reference:
Gcc optimize options:
http://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
Talk of O3 and the kernel (not android specific)
http://unix.stackexchange.com/questions/1597/compiling-gnu-linux-with-o3-optimization
ezterry,thank you very much!
I want to build cm9 kernel using toolchain 4.6.3.
How to replace the default GCC 4.4.3 to 4.6.3 version?
BTW,I use the way -- make bootimage
jxxhwy said:
ezterry,thank you very much!
I want to build cm9 kernel using toolchain 4.6.3.
How to replace the default GCC 4.4.3 to 4.6.3 version?
BTW,I use the way -- make bootimage
Click to expand...
Click to collapse
If you are asking in a cm9 build tree.. its a pain, best is to update the cm kernel makefile to point the build chain temporarily.
If you mean just to compile by hand as described in the op
Once the kernel is compiled find an anykernel update.zip or manually use mkbootimg or fastboot to merge it with the ram disk. Cm9 targets mkbootimg and unpackbootimg (if I'm not mistaken) will build the applications to unpack your current (or the default cm9) boot image and re-create them with your custom kernel built from hand.
Now,I runing script file:
#Let's make sure the environment is clean and ready to compile the kernel
echo "Cleaning house!!"
make mrproper
echo "House cleaned, lets build a kernel!!!"
#
# Lets set the kernel defconfig
echo "defconfig = cyanogenmod_iprj_defconfig"
make ARCH=arm cyanogenmod_iprj_defconfig
#
# Let's build a kernel
echo "Now compiling kernel, go get a soda! "
ARCH=arm CROSS_COMPILE=~/Android/sourcecm9/cm9/toolchain-4.6.3/bin/arm-linux-androideabi- make zImage -j4
#
if [ -f arch/arm/boot/zImage ]; then
echo "Plague has been compiled!!! You can find it in arch/arm/boot/zImage"
else
echo "Kernel did not compile, please check for errors!!"
fi
but,I got some error output:
/home/xxx/cm9/toolchain-4.6.3/bin/arm-linux-androideabi-ld: cannot find usr/initramfs_data.o: No such file or directory
thanks again!!!
I think that means there is an issue with your config..
Most android kernels require initramfs, but don't specify a file, as its provided by the bootloader.. However in your case it is trying to embed it into the kernel.
Sent from my A500 using Tapatalk 2
ezterry said:
I think that means there is an issue with your config..
Most android kernels require initramfs, but don't specify a file, as its provided by the bootloader.. However in your case it is trying to embed it into the kernel.
Sent from my A500 using Tapatalk 2
Click to expand...
Click to collapse
issue has been resolved by myslef.Thanks for your time.
I recently looked into stagefright support in ffmpeg and vlc and found that one of the chore they have is having platform headers to compile against around. Actually, they didn't work for me at all - likely because they were written for one Android version, and I run them on the other. Again, if supporting compilation against even one (random) Android version is a chore, who can expect good support for multiple versions?
To fight against such poor state of things, I'd like to introduce android-platform-headers, which provides headers for all important Android versions side by side.
More info is in the README:
Code:
These are sets of header files extracted from main components of
AOSP (Android Open Source Project), for each main release of AOSP.
Android NDK provided by Google contains subset of AOSP platform
headers which are deemed "external interface" by Google. However,
writing advanced software for Android necessitates access to complete
platform. Many aspects of it change, sometimes drastically, between
releases, so compiling user source code against platform headers for
different releases is required. This project intends to streamline
this process, allowing to access headers for all important Android
releases at once.
In this respect, it is similar to "linux-kernel-headers" found in
many Linux distributions - it allows to compile software without
the need to have hundreds of megabytes (or gigabytes) of complete
kernel (or platform) source code around.
To prepare these headers, all *.h and *.hpp files were extracted
from following AOSP projects:
* system/core
* frameworks/base
* frameworks/native
FAQ
---
Q: Google doesn't recommend usage of non-NDK headers.
A: Who cares.
Q: As these headers don't represent external interface, they can be
changes by vendor of some device.
A: True, so we can figure out what particular vendors work on -
on bringing up innovative products, cutting time to market, increasing
power efficiency, or fragmenting platform and breaking compatibility.
Q: How do I know that headers provided are correct?
A: You are encouraged to verify. Each header project directory contains
.git-org/refs/heads/master with revision of tag from which it was extracted.
Actually, extraction was automatic and script is provided. To easily verify
a particular tag:
rm -rf android-1.6_r1
scripts/fetch-headers android-1.6_r1
git diff
TODO
----
1. Add platform "import" libraries to link with (devoid of code, only with
exported symbols).
2. Similar "implib" jars to link Java code against.
Status update: repo now contains platform headers for the most popular Android versions from 1.6 to 4.2, and started to work on scripts to produce "import libraries" to link native apps against.
Hey guys, I downloaded the LG D800 lollipop source files from the opensource LG site, and I was trying to build it.
From my understanding, the instructions.
Code:
1. Android build
- Download original android source code ( Lollipop 5.0.1_r1 ) from http://source.android.com
- Untar opensource packages of LGD800_L_Android.tar.gz into downloaded android source directory
a) tar -xvzf LGD800_L_Android.tar.gz
- And, merge the source into the android source code
- Run following scripts to build android
a) source build/envsetup.sh
b) lunch 1
c) make -j4
- When you compile the android source code, you have to add google original prebuilt source(toolchain) into the android directory.
- After build, you can find output at out/target/product/generic
2. Kernel Build
- Uncompress using following command at the android directory
tar xvzf LGD800_L_Kernel.tar.gz
- When you compile the kernel source code, you have to add google original prebuilt source(toolchain) into the android directory.
- Run following scripts to build kernel
a) cd kernel
b) export PATH=$PATH:tools/lz4demo
c) make ARCH=arm CROSS_COMPILE=../prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/bin/arm-eabi- g2-att-perf_defconfig zImage -j4
* "-j4" : The number, 4, is the number of multiple jobs to be invoked simultaneously.
* lz4demo : More information can be found at "https://code.google.com/p/lz4/"
- After build, you can find the build image(zImage) at arch/arm/boot/
3. how to build chromium34_lge (vendor\lge\external\chromium34_lge\src),
please refer to README.txt at the folder mentioned above.
say to build the AOSP_ARM generic image.
I am building it now, and it did include the make files for the vendor directories. But I am wondering if this is actually going to work, and if it's missing anything?
If someone has ever done this before, and can accurately describe the process I would appreciate it.
I understand it as download android>extract the files to that directory and "Merge"(Overwrite?) the files, then run the envsetup, lunch for AOSP_ARM and make.
I don't understand what this line means "When you compile the android source code, you have to add google original prebuilt source(toolchain) into the android directory."
I also don't understand what the Chromium34_LGE thing is, and what it's for.
If anyone can shine some information on this, I would really appreciate it. Thanks!
Download site instructions, file size, and Linux knowledge
Silentwidow said:
Hey guys, I downloaded the LG D800 lollipop source files from the opensource LG site, and I was trying to build it.
- Download original android source code ( Lollipop 5.0.1_r1 ) from http://source.android.com
Click to expand...
Click to collapse
Go to http://opensource.lge.com, and search for LGD800. The downloaded file named LGD800_Lollipop_Android_D800_v30f.zip is a 633,894KB file containing LGD800_L_Android.tar.gz 505,244 KB and LGD800_L_Kernel.tar.gz 133,042 KB. When both are extracted the result is over 2GB of files and folders.
To build you'll need a Linux OS and be very knowledgeable with Linux commands and file structures.
As for a prebuilt, there is one buried in the folders at LGD800_L_Android.tar.gz\LGD800_L_Android.tar\android\vendor\lge\prebuilt\LGBackup\tar\LGBackup_tar.zip\
I don't know if there is another prebuilt in the LGD800_L_Kernel.tar.gz because I didn't look. Linux is far out of my reach, so I'll stay away from build scripts and makefiles...etc..
edited (solution): I forgot I had another thread on this same basic topic. You have to merge AOSP + Device Sources + Hardware Drives and set it up to build the sources.
https://forum.xda-developers.com/android/software/toolbox-c-struct-define-toolname-tool-s-t3568409
If I try to compile my own customize 'toolbox' binary for example, I can't find the headers anywhere in the toolchains or included headers. I can search them down from Google sources but I gave up after tools.h. I have a feeling this is NOT the way I am supposed to be doing it.
Is there a shared library on my device that I pull off to get these headers? (lib?????.so).. Where do I get headers for Android/log.h and cutils properties.h and selinux.h ??? Or are we not supposed to compile these binaries on a Ubuntu machine? (do I compile them on the device itself?)
ediit: (here's compiling it another way, obviously I can make a hack easily but working with Android is so much harder.)
arm-linux-gnueabi-gcc -static -march=armv8-a toolbox.c -o toolbox
toolbox.c:6:25: fatal error: android/log.h: No such file or directory
compilation terminated.
edit, solved i think
I think it's stand alone toolchain not defined. If that's not the answer, it's a fun answer so I'm playing with that. If I don't have the answer I will pass the question up, download sources and start trying to compile Android proper. (which is my new goal now, I want open source phone and I want to compile my own OS 100%, besides the closed sources, it's so exciting!!) ~so much for sleep