Database Users

[GUIDE] Compile the stock android kernel

I have already discussed how to unpack/repack the boot.img file.
Now let's try to recompile the android kernel.
You will need some packages not included in a default Ubuntu 11.04 installation:
Code:
sudo apt-get install git unrar libncurses5-dev qt3-dev-tools
Unrar and git are the only necessary packages.
Install libncurses if you want to modify the kernel configuration with the command line menu.
Otherwise, if you want a graphical interface, go with qt3.
Please refer to the documentation of your linux distro for help on how to install additional packages.
For the purpose of this guide we are going to rebuild the official asus kernel with no modifications.
Download the latest kernel source from the ASUS site (atm 2.6.36.3)
Code:
wget http://dlcdnet.asus.com/pub/ASUS/EeePAD/TF101/Eee_PAD_TF101_Kernel_Code_8_4_4_11.rar
Now unpack it:
Code:
unrar e Eee_PAD_TF101_Kernel_Code_8_4_4_11.rar
tar xvfz kernel-3eb19ec-fix-lic.tar.gz
We need a config file for the compile process.
The config file essentially sets features ON or OFF from the final kernel (yes, I know that is much more than this...).
If your tablet is upgraded to the latest build (8.4.4.11) you can extract the config file from your running device:
Code:
adb pull /proc/config.gz ./
Or you can find it attached to this post (the forum won't allow to upload a .gz file, so i made a .zip file; use unzip instead of gunzip).
Now uncompress and move it into the kernel dir (renaming it in .config).
Code:
gunzip config.gz
cp config kernel-3eb19ec/.config
Now the not-so-fun part.
We will need a compiler for the arm platform.
The android source comes with the arm compiler. You can download it following this instructions http://source.android.com/source/downloading.html .
The only problem is that the repository is about 2.6GB...
So we are going to download only the prebuilt toolchain (1,8GB):
Code:
git clone git://android.git.kernel.org/platform/prebuilt.git
After git finishes his download, you will have a new directory called "toolchain".
We have to set some environment variables (i assume you have downloaded the toolchain in your home directory):
Code:
export ARCH=arm
export CROSS_COMPILE=arm-eabi-
export PATH=$PATH:~/toolchain/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/
Ok, the compiler is installed and configured.
It's time to come back to the kernel source.
If you want to modify some settings in your config file (!!AT YOUR OWN RISK!!) you can edit the .config file or use a graphical interface.
Code:
cd kernel-3eb19ec
make xconfig (make menuconfig if you prefer a command-line menu)
Now we can compile the android kernel:
Code:
make -j4
Our new kernel is stored in : arch/arm/boot/zImage
We can use it for make a boot image and flash in our TF101 (use this guide http://forum.xda-developers.com/showthread.php?t=1193737)
Let me say this again: I'm not an expert in android kernel/compiler.
If you find something wrong with this guide, please tell me.
I've learned this with google and by trial and error.
If you brick something don't blame on me.
If you want to hit "thanks", you're welcome.
Have Fun!

Pulling out config.gz you will get the current kernel configuration...it could not fit the stock kernel..

gnufabio said:
Pulling out config.gz you will get the current kernel configuration...it could not fit the stock kernel..
Click to expand...
Click to collapse
Of course the kernel source has to be exactly the same version as the running kernel for this to work.
In this example, the source and running kernel were both 2.6.36.3 (build 8.4.4.11).
For use an older .config (or a newer source tree), we should run:
Code:
make oldconfig
to adapt them (...kinda).
Thanks for point this out.

I was eventually planning on making this guide, good work!
gnufabio said:
Pulling out config.gz you will get the current kernel configuration...it could not fit the stock kernel..
Click to expand...
Click to collapse
This is a possibility if anything in the config changes from the kernel you're running in the actual source Asus has posted
I've always done the following:
Code:
make tegra_defconfig
make
I haven't actually checked to see if this is up to date with a running kernel, just noticed Asus pops up a few times in there.

Actually this is the default TF101 config: tf101_defconfig

Thanks for the guides. These are really helpful -
Hashi

I'm getting this set of errors. I've built and modified many linux kernels, so I'm guessing this is in the toolchain area, and am wondering if it needs to know something else for where to look at dynamic load libraries, etc?
I created a shell script to setup / export the shell environment vars. It looks like this:
Code:
export ARCH=arm
export CROSS_COMPILE=arm-eabi-
export PATH=$PATH:~/sandbox/android-sdk-linux_x86/toolchain/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/
I'm using a PC I use for work ARM cross compiles that is 64 bit Ubuntu 9.10.
Thanks --
--- make -j4 errors ---
Code:
[email protected]:~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$ make -j4
arm-eabi-gcc: /lib32/libc.so.6: version `GLIBC_2.11' not found (required by arm-eabi-gcc)
CHK include/linux/version.h
CHK include/generated/utsrelease.h
CC scripts/mod/empty.o
arm-eabi-gcc: /lib32/libc.so.6: version `GLIBC_2.11' not found (required by arm-eabi-gcc)
make[2]: *** [scripts/mod/empty.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: `include/generated/mach-types.h' is up to date.
make[1]: *** [scripts/mod] Error 2
CC kernel/bounds.s
arm-eabi-gcc: /lib32/libc.so.6: version `GLIBC_2.11' not found (required by arm-eabi-gcc)
make: *** [scripts] Error 2
make: *** Waiting for unfinished jobs....
make[1]: *** [kernel/bounds.s] Error 1
make: *** [prepare0] Error 2
[email protected]:~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$
In searching for the basic problem, that arm*gcc can't see libc.so.6 in the version it wants, I think it's not seeing this at all:
~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$ find ../../toolchain/ -iname libc.so.6 -exec ls -al {} \;
lrwxrwxrwx 1 hashi hashi 11 2011-08-01 15:58 ../../toolchain/prebuilt/linux-x86/toolchain/i686-linux-glibc2.7-4.4.3/sysroot/usr/lib/libc.so.6 -> libc-2.7.so
[email protected]:~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$

thanks. I'm not new to kernel compiling but I'm new to android

hachamacha said:
I'm getting this set of errors. I've built and modified many linux kernels, so I'm guessing this is in the toolchain area, and am wondering if it needs to know something else for where to look at dynamic load libraries, etc?
I created a shell script to setup / export the shell environment vars. It looks like this:
Code:
export ARCH=arm
export CROSS_COMPILE=arm-eabi-
export PATH=$PATH:~/sandbox/android-sdk-linux_x86/toolchain/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/
I'm using a PC I use for work ARM cross compiles that is 64 bit Ubuntu 9.10.
Thanks --
--- make -j4 errors ---
Code:
[email protected]:~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$ make -j4
arm-eabi-gcc: /lib32/libc.so.6: version `GLIBC_2.11' not found (required by arm-eabi-gcc)
CHK include/linux/version.h
CHK include/generated/utsrelease.h
CC scripts/mod/empty.o
arm-eabi-gcc: /lib32/libc.so.6: version `GLIBC_2.11' not found (required by arm-eabi-gcc)
make[2]: *** [scripts/mod/empty.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: `include/generated/mach-types.h' is up to date.
make[1]: *** [scripts/mod] Error 2
CC kernel/bounds.s
arm-eabi-gcc: /lib32/libc.so.6: version `GLIBC_2.11' not found (required by arm-eabi-gcc)
make: *** [scripts] Error 2
make: *** Waiting for unfinished jobs....
make[1]: *** [kernel/bounds.s] Error 1
make: *** [prepare0] Error 2
[email protected]:~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$
In searching for the basic problem, that arm*gcc can't see libc.so.6 in the version it wants, I think it's not seeing this at all:
~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$ find ../../toolchain/ -iname libc.so.6 -exec ls -al {} \;
lrwxrwxrwx 1 hashi hashi 11 2011-08-01 15:58 ../../toolchain/prebuilt/linux-x86/toolchain/i686-linux-glibc2.7-4.4.3/sysroot/usr/lib/libc.so.6 -> libc-2.7.so
[email protected]:~/sandbox/android-sdk-linux_x86/kernel/kernel-3eb19ec$
Click to expand...
Click to collapse
The toolchain has everything it needs for compile the kernel.
The problem seems to be your 64 bit...
The arm compiler is built against 32bit version of glibc library; I think you already have a multilib environment (you have a /lib32 dir). Maybe the 32bit verion of the glibc is not the correct version.
Try this command:
Code:
ldd ~/sandbox/android-sdk-linux_x86/toolchain/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-gcc
And look what it says for the libc.so.6. In my system it says it is located in /lib/i386-linux-gnu/libc.so.6. So if I run it:
Code:
$ /lib/i386-linux-gnu/libc.so.6
GNU C Library (Ubuntu EGLIBC 2.13-0ubuntu13) stable release version 2.13, by Roland McGrath et al.
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.5.2.
Compiled on a Linux 2.6.38 system on 2011-04-11.
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.
The first line tell the version number. If it's older than 2.11 you need to update your multilib libc.

gnufabio said:
Actually this is the default TF101 config: tf101_defconfig
Click to expand...
Click to collapse
This is for 2.6.36.3 but seems way older. It says "Fri Mar 25 19:50:17 2011".
Maybe it's from an old build?
The last ASUS config (the one attached to the first post) is "Mon Jul 11 14:47:43 2011".

Brilliant!
I had problems cloning using the git protocol because of a corporate firewall, but the HTTP method worked fine:
Code:
http://android.git.kernel.org/platform/prebuilt.git

I also found the the toolchain with the Android NDK failed to build the kernel but the one linked to in the OP worked perfectly.

SammyC said:
I also found the the toolchain with the Android NDK failed to build the kernel but the one linked to in the OP worked perfectly.
Click to expand...
Click to collapse
Yes, I have tried this also, with the same result.
Someone said that the codesourcery compiler can compile the android kernel.
I have tried, it compiles all, but when i flash on the tablet it stucks on the boot screen.
Initially I wrote the guide using this compiler (it's smaller and has a graphical installer), when i realized that it doesn't work , i switched back to the android official compiler.
If someone want to try with the codesourcery compiler here's the (unsuccessful) process I have used:
!!ATTENTION!!: ATM THIS DOESN'T WORK
An easier (and somewhat faster) way is downloading the Code Sorcery's "Code Bench" compiler (http://www.codesourcery.com/) and install it.
Code:
wget http://www.codesourcery.com/sgpp/lite/arm/portal/package8741/public/arm-none-linux-gnueabi/arm-2011.03-41-arm-none-linux-gnueabi.bin
This compiler need some extra packages to work in Ubuntu:
Code:
sudo apt-get install git-core gnupg flex bison gperf libsdl-dev libesd0-dev libwxgtk2.6-dev build-essential zip curl libncurses5-dev zlib1g-dev valgrind
For some reason, in Ubuntu 11.04 the /bin/sh link refer to the dash shell and not bash.
The compiler's installer will comply for this, so we do as follows:
Code:
sudo dpkg-reconfigure -plow dash
and when it asks to use dash as the default shell, click on "NO"
Code:
sh arm-2011.03-41-arm-none-linux-gnueabi.bin
Leave all the default options. The installer will copy the files in your home directory.
We should set some environmet variables:
Code:
export ARCH=arm
export CROSS_COMPILE=arm-none-linux-gnueabi-
export PATH=$PATH:~/CodeSourcery/Sourcery_G++_Lite/bin
Click to expand...
Click to collapse
Then continue with the guide from the "make xconfig".
If someone can make it work with this compiler, could be so gentle to tell us how?

rebound821 said:
The toolchain has everything it needs for compile the kernel.
The problem seems to be your 64 bit...
The arm compiler is built against 32bit version of glibc library; I think you already have a multilib environment (you have a /lib32 dir). Maybe the 32bit verion of the glibc is not the correct version.
Try this command:
Code:
ldd ~/sandbox/android-sdk-linux_x86/toolchain/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-gcc
And look what it says for the libc.so.6. In my system it says it is located in /lib/i386-linux-gnu/libc.so.6. So if I run it:
Code:
$ /lib/i386-linux-gnu/libc.so.6
GNU C Library (Ubuntu EGLIBC 2.13-0ubuntu13) stable release version 2.13, by Roland McGrath et al.
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.5.2.
Compiled on a Linux 2.6.38 system on 2011-04-11.
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
libc ABIs: UNIQUE IFUNC
For bug reporting instructions, please see:
<http://www.debian.org/Bugs/>.
The first line tell the version number. If it's older than 2.11 you need to update your multilib libc.
Click to expand...
Click to collapse
Thanks again -- I'll try that out as soon as I'm back in TF mode.

Code:
~/sandbox/android-sdk-linux_x86/toolchain/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-gcc: /lib32/libc.so.6: version `GLIBC_2.11' not found (required by ~/sandbox/android-sdk-linux_x86/toolchain/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-gcc)
linux-gate.so.1 => (0xf77e0000)
libc.so.6 => /lib32/libc.so.6 (0xf7677000)
/lib/ld-linux.so.2 (0xf77e1000)
^^ that was from the first command
Code:
GNU C Library (EGLIBC) stable release version 2.10.1, by Roland McGrath et al.
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.4.1.
Compiled on a Linux >>2.6.24-27-server<< system on 2011-01-11.
Available extensions:
crypt add-on version 2.1 by Michael Glad and others
GNU Libidn by Simon Josefsson
Native POSIX Threads Library by Ulrich Drepper et al
BIND-8.2.3-T5B
For bug reporting instructions, please see:
<http://www.eglibc.org/issues/>.
And ^^ from the second.
It looks like with 9.10 Ubuntu, (backporting something from 10 or 11.x notwithstanding), I can't easily get to GLIBC 2.11 from GLIBC 2.10.x. Thanks. I'll put together an 11.x VM for this task.

For 64bit machines, you might need to do the following?
apt-get install ia32-libs

raypou said:
For 64bit machines, you might need to do the following?
apt-get install ia32-libs
Click to expand...
Click to collapse
Thanks, I managed to download a ARM toolchain from their site that works fine with Ubuntu 9.10 64 bit.
Whether or not it'll run when I try to install a kernel is another question, but it builds without problem and produces ARM code.

rebound821 said:
Yes, I have tried this also, with the same result.
Someone said that the codesourcery compiler can compile the android kernel.
I have tried, it compiles all, but when i flash on the tablet it stucks on the boot screen.
Initially I wrote the guide using this compiler (it's smaller and has a graphical installer), when i realized that it doesn't work , i switched back to the android official compiler.
If someone want to try with the codesourcery compiler here's the (unsuccessful) process I have used:
!!ATTENTION!!: ATM THIS DOESN'T WORK
Then continue with the guide from the "make xconfig".
If someone can make it work with this compiler, could be so gentle to tell us how?
Click to expand...
Click to collapse
Wow can't believe I found this thread. I just want to confirm that I'm also having a horrible time with the code sourcery toolchain trying to compile the android kernel, the moto atrix kernel specifically. The zimage compiles, but flashing to the device it just hangs.
So im currently downloading the official android toolchain.
Also some learned knowledge for everybody, the correct code sourcery toolchain supposedly android equivalent is arm-none-linux-gnueabi-, NOT the arm-none-eabi- or any of the others. This info is very hard to find fyi.
However as this thread seems to prove, even using the correct one doesn't exactly work.

Hi,
I'm trying to build the stock Memo Pad smart(ME301t) stock kernel 10.6.1.29 downloaded directly from asus.
I used 4.3.3 to 4.7 toolchains with similar results(give or take some deprecated compiler arguments).
What I really want to know is why I'm getting a lot of warnings(NOT ERRORS), some of them are minor errors like "unused function" and some of them are major warnings breaking function functionality. Some of the builds don't even boot.
What toolchain should I use with this kernel?
Why a stock kernel(from asus) should produce warnings at the first place?
I'm on linux BTW.

[GUIDE][REVISED] Build CM12.1 from Source! [NOW-CAT&GRANDMA-FRIENDLY]

Im sick of seeing guides around that tell you all about how to build a rom but never ever go into detail about the hardest part in my opinion which is obtaining device specific code, its ridiculous!? So im here to help.
Just a prior note, im no dev, ive never built from source before (hopefully this will change in few days ) and the real devs are people like Luk & Luca who built a lovely repos for us with lots of good documentation. So please go and give Luca & Luk a thanks on any one of there posts!
Also i would like to personally thank Luca for helping me when i asked for info regarding building.
Lets get started shall we!
1 - Pre-requisites
4+ GB RAM (I have 4GB)
Preferably a Quad-Core CPU (I have an i5)
At least 90GB of HDD/SSD space for the repo & ccache (SSD recommended for faster compile times, i have a 750GB HDD)
GPU - (not needed but may aswell throw it in here) GTX-740M & Intel HD 4000 (Optimus technology - not working on Linux though)
And lastly a Github account, please go make one (its free!)
2 - Installing Linux
We need a Linux based 64 bit OS, i use Lubuntu with my own tweaks
Download and install it, either dual-boot or use it as your primary OS like the bad-ass you are! (If you want help on doing this there at least +100000 guides in google and youtube alone)
3 - Grabbing build dependencies
Code:
sudo apt-get install git && cd ~/ && git clone https://github.com/DiamondBond/Scripts/ .init-scripts && cd ~/.init-scripts && ./dev_init.sh
Copy & Pasta that code above into your terminal and accept/press ok/press yes to any prompts that appear
That should get your system fit with all the necessary things to build and work with androids source, its about 900mb of a download! So sit back and relax...
4 - Preparing to download the massive source-code
I recommend downloading a small movie before doing this so you can relax while it all syncs! (just be sure to keep your eye on that sync window to make sure its still scrolling...)
Lets make a folder to hold all our stuff:
Code:
mkdir -p ~/android/titan/roms/cm && cd ~/android/titan/roms/cm
Now your inside the cm folder, you have to be in the cm directory every-time you want to start building!
Now initiate the cm repo: (one-time thing)
Code:
repo init -u git://github.com/CyanogenMod/android.git -b cm-12.1
Now it might give a small error at the end stating for some email and user id stuff.
So now is the time to add your github infos (run this in terminal):
Code:
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
EG:
Code:
git config --global user.email "[email protected]"
git config --global user.name "Diamond Bond"
We havnt yet downloaded the CM source, but we will in the next part...
(source code is roughly 35GB for CM12.1 , pure AOSP is slightly smaller)
5 - Downloading & waiting
Note: Straight after syncing official CM12.1 repo this is its size: (31GB)
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
It is time!, run one of the two commands below (depending on your networking speed) to initiate the long wait of syncing!)
Code:
repo sync
Finished? sweet, lets have a rave party!
6 - Building!
Now once youve downloaded all the source code you should be able to run your first build after init!
Thanks to some nice build scripts by Luca and Luk, running builds is super duper easy!
To initialize the last thing, run:
Code:
. build/envsetup.sh
then to build run
Code:
brunch titan
to build, ive read around and on my system it should only take 1.5 to 3 hours tops, and my system is mediocre + its a laptop so take in account that it may thermally throttle, if you have a desktop it should only take around 1-2 hours with relatively decent specs!
7 - Party Time!
If you want help please leave a comment in this thread and i/others will try to help.
Also if you do get a successful build done please share it with us, we would love to be a part of your victory party!
Thanks to this guide: http://forum.xda-developers.com/chef-central/android/guide-android-rom-development-t2814763
&
Luca & Luk again!

@Pizza_Dox
Replace
Code:
mkdir ~/titan/ && cd ~/titan/ && mkdir cm12 && cd ~/titan/cm12/
With
Code:
mkdir -p ~/titan/cm12 && cd ~/titan/cm12/
Also users don't need github account to just build the rom.

I would try this, but I don't have a cat. Thanks for sharing tho

Exactly what I was looking .
Thank you for sharing.

LuK1337 said:
@Pizza_Dox
Replace
Code:
mkdir ~/titan/ && cd ~/titan/ && mkdir cm12 && cd ~/titan/cm12/
With
Code:
mkdir -p ~/titan/cm12 && cd ~/titan/cm12/
Also users don't need github account to just build the rom.
Click to expand...
Click to collapse
Thanks updated.
hippy dave said:
I would try this, but I don't have a cat. Thanks for sharing tho
Click to expand...
Click to collapse
Haha welcome
CaptainCaptcha said:
Exactly what I was looking .
Thank you for sharing.
Click to expand...
Click to collapse
Enjoy!

LuK1337 said:
@Pizza_Dox
Replace
Code:
mkdir ~/titan/ && cd ~/titan/ && mkdir cm12 && cd ~/titan/cm12/
With
Code:
mkdir -p ~/titan/cm12 && cd ~/titan/cm12/
Also users don't need github account to just build the rom.
Click to expand...
Click to collapse
Oh right i know you dont need a github account but its better to have one

Updating this soon
Sent from my Motorola Titan

I dont know why but every time i repo sync it gets stuck here for quiet a long time and after sometimes ends up in error like below.. Is it that its downloading below package file again

koradiavatsal said:
I dont know why but every time i repo sync it gets stuck here for quiet a long time and after sometimes ends up in error like below.. Is it that its downloading below package file again
Click to expand...
Click to collapse
Sent from MicroTitan

There are some tricks to before compiling.
This is just the introduction to compile Cyanogenmod
Good job @Pizza_Dox :good:

klander said:
There are some tricks to before compiling.
This is just the introduction to compile Cyanogenmod
Good job @Pizza_Dox :good:
Click to expand...
Click to collapse
And they are? Please let us know
Sent from MicroTitan

Code:
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=4.4.4
CM_VERSION=
TARGET_PRODUCT=full
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
TARGET_CPU_VARIANT=generic
HOST_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-3.16.0-4-amd64-x86_64-with-debian-8.0
HOST_BUILD_TYPE=release
BUILD_ID=KTU84Q
OUT_DIR=/data/deepak/titan/cm11/out
============================================
Checking build tools versions...
************************************************************
You are attempting to build with an unsupported JDK.
You use OpenJDK but only Sun/Oracle JDK is supported.
Please follow the machine setup instructions at
https://source.android.com/source/download.html
Continue at your own peril!
************************************************************
************************************************************
You are attempting to build with an unsupported version
of java.
Your version is: java version "1.7.0_75".
The correct version is: Java SE 1.6 or 1.7.
Please follow the machine setup instructions at
https://source.android.com/source/download.html
************************************************************
find: `bootable/recovery/res-720': No such file or directory
No private recovery resources for TARGET_DEVICE generic
Entire build directory removed.
Set cache size limit to 50.0 Gbytes
/tmp/hudson8243709547868758627.sh: line 13: ./resync.sh: No such file or directory
including device/generic/armv7-a-neon/vendorsetup.sh
including device/generic/goldfish/vendorsetup.sh
including device/generic/mips/vendorsetup.sh
including device/generic/x86/vendorsetup.sh
including vendor/cm/vendorsetup.sh
including sdk/bash_completion/adb.bash
including vendor/cm/bash_completion/git.bash
including vendor/cm/bash_completion/repo.bash
including vendor/cm/vendorsetup.sh
ls: cannot access device/*/titan/cm.mk: No such file or directory
build/core/product_config.mk:234: *** Can not locate config makefile for product "cm_titan". Stop.
Device titan not found. Attempting to retrieve device repository from CyanogenMod Github (http://github.com/CyanogenMod).
Found repository: android_device_motorola_titan
Default revision: cm-11.0
Checking branch info
Default revision cm-11.0 not found in android_device_motorola_titan. Bailing.
Branches found:
cm-12.0
Use the ROOMSERVICE_BRANCHES environment variable to specify a list of fallback branches.
ls: cannot access device/*/titan/cm.mk: No such file or directory
build/core/product_config.mk:234: *** Can not locate config makefile for product "cm_titan". Stop.
** Don't have a product spec for: 'cm_titan'
** Do you have the right repo manifest?
No such item in brunch menu. Try 'breakfast'
real 0m17.121s
user 0m0.256s
sys 0m0.024s
Build step 'Execute shell' marked build as failure
Sending e-mails to: [email protected]
[BFA] Scanning build for known causes...
[BFA] Done. 0s
Finished: FAILURE
Can someone help please? or just list me with cm11 commands of repo initialisation and device trees?

klander said:
There are some tricks to before compiling.
This is just the introduction to compile Cyanogenmod
Good job @Pizza_Dox :good:
Click to expand...
Click to collapse
Oh okay, well thank you, ill try and add whatever more into this so more people can compile,

Quick steps for compiling cm11
I hope you dont mind adding to OP
repo init -u git://github.com/CyanogenMod/android -b cm-11.0
curl --create-dirs -L -o .repo/local_manifests/titan.xml -O -L https://raw.githubusercontent.com/motog2014devteam/roomservice/cm-11.0/manifest.xml
repo sync
Thanks @LuK1337
Sent from MicroTitan

koradiavatsal said:
Quick steps for compiling cm11
I hope you dont mind adding to OP
repo init -u git://github.com/CyanogenMod/android -b cm-11.0
curl --create-dirs -L -o .repo/local_manifests/titan.xml -O -L https://raw.githubusercontent.com/motog2014devteam/roomservice/cm-11.0/manifest.xml
repo sync
Thanks @LuK1337
Sent from MicroTitan
Click to expand...
Click to collapse
done

Great, I'll try this guide, thank you very much I was always curious about ROM development. In case, where can I ask questions about it ?
Regarding this guide : is necessary to create a bin folder in home directory ? Usually I put large binaries in /opt/. Is there any permission problem ?

davcri91 said:
Great, I'll try this guide, thank you very much I was always curious about ROM development. In case, where can I ask questions about it ?
Regarding this guide : is necessary to create a bin folder in home directory ? Usually I put large binaries in /opt/. Is there any permission problem ?
Click to expand...
Click to collapse
Thanks
Um I think its best to keep the binary as is since other things might link to it and etc... Questions can be asked here I or others will try to help! Glad this helps.

Is there a way to use these steps to similarly build a kernel? As Moto G2 lollipop kernel sources are out (http://www.androidpolice.com/2015/02/23/motorola-finally-posts-kernel-source-lollipop-2014-moto-x), I want to try building a stock kernel.

@LuK1337 Is this guide still work since titan switched to new kernel?
I get this errors while building
<code>
find: `dummy': No such file or directory
build/core/tasks/kernel.mk:130: **********************************************************
build/core/tasks/kernel.mk:131: * Kernel source found, but no configuration was defined *
build/core/tasks/kernel.mk:132: * Please add the TARGET_KERNEL_CONFIG variable to your *
build/core/tasks/kernel.mk:133: * BoardConfig.mk file *
build/core/tasks/kernel.mk:134: **********************************************************
make: *** No rule to make target 'bacon'. Stop.
</code>

koradiavatsal said:
@LuK1337 Is this guide still work since titan switched to new kernel?
I get this errors while building
<code>
find: `dummy': No such file or directory
build/core/tasks/kernel.mk:130: **********************************************************
build/core/tasks/kernel.mk:131: * Kernel source found, but no configuration was defined *
build/core/tasks/kernel.mk:132: * Please add the TARGET_KERNEL_CONFIG variable to your *
build/core/tasks/kernel.mk:133: * BoardConfig.mk file *
build/core/tasks/kernel.mk:134: **********************************************************
make: *** No rule to make target 'bacon'. Stop.
</code>
Click to expand...
Click to collapse
So
rm -rf out
rm -rf .repo/local_manifests
rm -rf build* resync.sh patches
And follow the guide
Or join irc hannel isntead of using thread xd

Compiling dropbear 2015.67 for Zenfone 2 (x86-64)

Hi,
This is a guide on compiling the latest (as of 2015-06-24) dropbear SSH daemon (2015.67) for the Zenfone 2 but should also work for other x86-64 architecture Android devices. The guide is mainly based off the work from the blog http://blog.xulforum.org/index.php?post/2013/12/19/Compiling-Dropbear-for-a-Nexus-7-tablet which is an excellent guide to getting an older version of dropbear running on Android devices.
I've modified the patch made by the original author to work on the latest dropbear version. There are not much changes from ver 2013.58 to 2015.67. The main difference seems to be the support for Elliptical Curve encryption.
** START DISCLAIMER **
I did not write the original codes/patch myself and have not scrutinized it for any security issues. USE IT AT YOUR OWN RISK.
** END DISCLAIMER **
Here's the list of requirements :-
1) Dropbear 2015.67 source code - dropbear-2015.67.tar.bz2 (https://matt.ucc.asn.au/dropbear/releases/dropbear-2015.67.tar.bz2)
2) Patch to compile for Android - dropbear-v67-android-patch-20150629 (https://goo.gl/LBokgG)
3) A recent Linux distro (I use Ubuntu 14.04.2 LTS) running x86-64.
4) Development tools i.e. C compiler, linker, etc... all the necessary tools to run configure, make, etc..
Steps :-
(1) Extract the source code to a directory of its own :-
Code:
tar xjf dropbear-2015.67.tar.bz2
cd dropbear-2015.67
(2) Patch the source :-
Code:
patch -p1 < dropbear-v67-android-patch-20150629
(3) Run configure :-
Code:
./configure --disable-zlib --disable-largefile --disable-loginfunc \
--disable-shadow --disable-utmp --disable-utmpx --disable-wtmp \
--disable-wtmpx --disable-pututline --disable-pututxline --disable-lastlog
(4) Run make :-
Code:
STATIC=1 MULTI=1 SCPPROGRESS=0 PROGRAMS="dropbear dropbearkey scp dbclient" make strip
(5) You should end up with a single static binary "dropbearmulti" which you should link dropbear, dbclient/ssh, dropbearkey and scp to.
Code:
./dropbear -h
Dropbear server v2015.67 [URL]https://matt.ucc.asn.au/dropbear/dropbear.html[/URL]
Usage: ./dropbear [options]
-A Android Mode, specify a user explicitly
-N Android Mode, user name
-C Android Mode, password
-R Android Mode, public key file (authorized_keys)
-U Android Mode, UID
-G Android Mode, GID
-b bannerfile Display the contents of bannerfile before user login
(default: none)
-r keyfile Specify hostkeys (repeatable)
defaults:
dss /etc/dropbear/dropbear_dss_host_key
rsa /etc/dropbear/dropbear_rsa_host_key
ecdsa /etc/dropbear/dropbear_ecdsa_host_key
-F Don't fork into background
-E Log to stderr rather than syslog
-m Don't display the motd on login
-w Disallow root logins
-s Disable password logins
-g Disable password logins for root
-B Allow blank password logins
-j Disable local port forwarding
-k Disable remote port forwarding
-a Allow connections to forwarded ports from any host
-p [address:]port
Listen on specified tcp port (and optionally address),
up to 10 can be specified
(default port is 22 if none specified)
-P PidFile Create pid file PidFile
(default /var/run/dropbear.pid)
-i Start for inetd
-W <receive_window_buffer> (default 24576, larger may be faster, max 1MB)
-K <keepalive> (0 is never, default 0, in seconds)
-I <idle_timeout> (0 is never, default 0, in seconds)
-V Version
I will assume you know what to do with the binary file generated so will not elaborate on the process.
Hope it works for you guys. I'll be happy to help out anyone who needs more details.
Cheers.
UPDATE: 2015-06-29
- I've updated the patch to fix dbclient/ssh client using password authentication.

Hi all,
For those of you using the dropbear ssh client and encountering a seg fault when using password authentication, I've fixed the patch now and it should be working.
Cheers.

wolfdude said:
I will assume you know what to do with the binary file generated so will not elaborate on the process.
Hope it works for you guys. I'll be happy to help out anyone who needs more details.
Click to expand...
Click to collapse
Ehm...
Replacing the not working dropbear binaries of ARM-oriented SSHD servers with a link to this should solve the problem discussed here ...
is it right ?!
Thanks again

Val3r10 said:
Ehm...
Replacing the not working dropbear binaries of ARM-oriented SSHD servers with a link to this should solve the problem discussed here ...
is it right ?!
Thanks again
Click to expand...
Click to collapse
In a nutshell, you will need to copy dropbearmulti to /system/xbin, create some symbolic links to it (dropbear, dropbearkey, dbclient, scp), run dropbearkey to generate your host keys, copy your client pubkey to /data/dropbear/authorized_keys and then startup dropbear with parameters like :-
Code:
dropbear -A -N root -R /data/dropbear/authorized_keys
Hope this helps.
Cheers.

wolfdude said:
[...] and then startup dropbear with parameters like :-
Hope this helps.
Click to expand...
Click to collapse
Thanks. Got that.
Actually I mean I would replace - with the symlink - the x86 binary file inside one of the above mentionned packages, i.e. QuickSSHd or SSHDroid, to make them compatible with ZE55xML...
I'm trying and reporting here

[GUIDE] Build CM13 from source for the Z5/Z5C/Z5P

The Sony-Kitakami dev team is glad to announce a guide on how to build CM13 or any CM13 based ROM for the entire Z5- series
Note: If you plan to release a ROM using our sources then credits aren't required, however we would deffinately appriciate it very much!
Before we get started
What you need
Any of our supported devices
A PC that is capable of running linux
An Internet connection
And lots of lots of patience
Getting started
Setting up Linux
To be able to build a ROM fully from source a Linux based operating system is required. This is due to the fact that windows is unable to compile the android source code due to lacking tools/libaries. After all you're compiling Linux on Linux (Because android is Linux). We strongly recommend to have AT LEAST 50 GB of space for Ubuntu!!!
Please be aware that this guide is target audiences with Ubuntu specifically and so may not work on other distros. Because of this we prefer ubuntu for everyone that is coming from windows.
Because this guide isn't about how to install ubuntu, I'll let this guide explain all for you : http://www.linuxtechi.com/ubuntu-15-10-desktop-installation-guide/
Installing the required packages
We will start off by downloading openjdk 7 (Note: Open JDK 8 isn't supported by CM yet).
Please open the terminal and enter or copy paste the following :
Code:
sudo apt-get update && sudo apt-get install openjdk-7-jdk
After entering the password (Note: you won't be able to see any output when typing the password) hit enter and then type "y" and hit enter again.
Once this is completed we will download the required packages. Please copy paste the following into your terminal :
Code:
sudo apt-get install git-core gnupg flex bison gperf build-essential
zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386
lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev ccache
libgl1-mesa-dev libxml2-utils xsltproc unzip maven schedtool
You may have to enter your password again after hitting enter. Once that is done enter "y" again and hit enter again.
Additionally you can set up ccache by entering the following into the terminal :
Code:
echo 'export USE_CCACHE=1' >> ~/.bashrc
after this has been done, restart your PC.
Then we will set up git. Replace "Your name" with your name and "[email protected]" with your email :
Code:
git config --global user.name "Your Name"
git config --global user.email "[email protected]"
Now we will need to set up the repo tool. Copy the following into the terminal:
Code:
mkdir ~/bin && PATH=~/bin:$PATH && echo 'PATH=~/bin:$PATH' >> ~/.bashrc && curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo && chmod a+x ~/bin/repo
Guides
CyanogenMod 13
Getting the CM13 source
Start out by making a directory called cm13 in your home dir by running the following command :
Code:
mkdir ~/cm13/
Now we will initialize the CM13 repository by entering the following in your terminal :
Code:
cd ~/cm13/ && repo init -u https://github.com/CyanogenMod/android.git -b cm-13.0
We are now coming to the worst part and that is downloading the entire 20GB big CM13 source code. To sync the source code, run the following command :
Code:
repo sync
And now wait.... and wait..... and wait....... until it's done
Getting the Z5/Z5C/Z5P source
To clone the device specific source code we'll have to introduce you to some of the codenames as those are used for simplifying reasons when building from source
Code:
NAME CODENAME
Sony Xperia Z5 sumire
Sony Xperia Z5 Compact suzuran
Sony Xperia Z5 Premium satsuki
We will start off by cloning the appropriate manifests for your device. Please enter the following and make sure to replace yourdevicecodename with your device codename :
Code:
cd ~/cm13/ && git clone https://github.com/Sony-Kitakami/android_.repo_local_manifests.git -b cm-13.0-yourdevicecodename .repo/local_manifests
Now we will sync our device specific source like this :
Code:
repo sync --force-sync
This could again take some time depending on your connection as the kernel source is quite heavy.
Building the ROM
We will now get to the big part, which is compiling. Note this could take from 1 hour to more than 10 hours depending on how good your PC is. Replace yourdevicecodename with your targeted device codename.
Please copy the following to your terminal line by line
Code:
cd ~/cm13/
cd .repo/local_manifests/ && git pull && cd ../.. && repo sync --force-sync
. build/envsetup.sh
lunch
cm_yourdevicecodename-userdebug
mka clobber
mka bacon
After a long time of waiting you should be done and your CyanogenMod package should be located in home/cm13/out/target/product/yourdevicecodename/
Congratulations! You have now build your very own CM13 ROM from source!
CyanogenMod 13 based ROMs
If you are planning to build a CM13 based or similar ROM (Bliss Pop, RR, Slim etc.), we're expecting you to be a more advanced user. As much as we would like to give a strict guide, we simply can't cover all as there are always differences between each and every ROM.
You will have to start off by downloading the ROM source. Again this should be similar to CM but the git links are different.
Ex:
Code:
RR = repo init -u [url]https://github.com/ResurrectionRemix/platform_manifest.git[/url] -b marshmallow
Bliss = repo init -u [url]https://github.com/BlissRoms/platform_manifest.git[/url] -b mm6.0
and much more. Most of them can be found on the ROMs official git team under the platform_manifest repo.
Here is where things get hard. You will have to edit our manifests and adjust them to work with your target ROM. This is nearly impossible to explain as each ROM contains different repos that may overlap with the CM ones or it may lack repos that you're trying to remove. In other words, you will need to learn how to use the local manifests. A nice guide exists here https://wiki.cyanogenmod.org/w/Doc:_Using_manifests
Once this is done you will have to adjust your device tree to work with your specific ROM. Again this is always different. The best way to learn is to compare a CM tree with a tree from a different ROM
Ex:
Samsung Galaxy S5 mini tree for CyanogenMod : https://github.com/cm-3470/android_device_samsung_kminilte
Samsugn Galaxy S5 mini tree for BlissPop : https://github.com/cm-3470/android_device_samsung_kminilte/tree/bosp-6
The main differences are the cm.mk or device.mk files etc.
Here is a commit that show the transfer from CM to Bliss Pop which usually isn't very hard
https://github.com/cm-3470/android_...mmit/5e902e7dd228c6804b2735af9a96ea02969aa2cf
Once this is completed you can run your build and release it!
XDA:DevDB Information
How to build CM13 from source for the Z5, ROM for the Sony Xperia Z5
Contributors
CTXz, zacharias.maladroit, Moonbutt74
Source Code: https://github.com/Sony-Kitakami
ROM OS Version: 6.0.x Marshmallow
ROM Firmware Required: Stock Marshmalllow
Based On: CyanogenMod
Version Information
Status: Alpha
Created 2016-04-12
Last Updated 2016-04-15

Reserved

Reserved

Thanks for posting the guide
I followed the steps, but got stuck at the command: "cd ~/cm13/ && git clone [email protected]:Sony-Kitakami/android_.repo_local_manifests.git -b cm-13.0-sumire"
It returned this to me:
Cloning into 'android_.repo_local_manifests'...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
EDIT: Never mind, I investigated, and found out that I had to generate an SSH key and link that to my GitHub account. If anyone needs help with this, I used this guide: https://help.github.com/articles/generating-an-ssh-key/

Uclydde said:
Thanks for posting the guide
I followed the steps, but got stuck at the command: "cd ~/cm13/ && git clone [email protected]:Sony-Kitakami/android_.repo_local_manifests.git -b cm-13.0-sumire"
It returned this to me:
Cloning into 'android_.repo_local_manifests'...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
EDIT: Never mind, I investigated, and found out that I had to generate an SSH key and link that to my GitHub account. If anyone needs help with this, I used this guide: https://help.github.com/articles/generating-an-ssh-key/
Click to expand...
Click to collapse
Hey, sorry for that. I had git set to ssh like and forgot that they require a keygen. I've changed it to https://github.com/Sony-Kitakami/android_.repo_local_manifests.git -b cm-13.0-yourdevicecodename so it no longer needs a key

Actually, is you are building CM13 and using Ubuntu 14.04 or up, you just need to type:
Code:
sudo apt-get install bison g++-multilib git gperf libxml2-utils make zlib1g-dev:i386 zip

Okay, I hit another bump. When I type: "cd .repo/local_manifests/ && git pull && cd ../.. && repo sync --force-sync"
It returns:
bash: cd: .repo/local_manifests/: No such file or directory

zhangyang_haha said:
Actually, is you are building CM13 and using Ubuntu 14.04 or up, you just need to type:
Code:
sudo apt-get install bison g++-multilib git gperf libxml2-utils make zlib1g-dev:i386 zip
Click to expand...
Click to collapse
No, you'll be missing quite some packages
Sent from my E5823 using XDA-Developers mobile app

Uclydde said:
Okay, I hit another bump. When I type: "cd .repo/local_manifests/ && git pull && cd ../.. && repo sync --force-sync"
It returns:
bash: cd: .repo/local_manifests/: No such file or directory
Click to expand...
Click to collapse
Ok updated again. Please run this.
cd ~/cm13/ && git clone https://github.com/Sony-Kitakami/android_.repo_local_manifests.git -b cm-13.0-yourdevicecodename .repo/local_manifests

CTXz said:
Ok updated again. Please run this.
cd ~/cm13/ && git clone https://github.com/Sony-Kitakami/android_.repo_local_manifests.git -b cm-13.0-yourdevicecodename .repo/local_manifests
Click to expand...
Click to collapse
Yay, that worked! However, I got to the "cm_yourdevicecodename-userdebug" line, so I entered "cm_sumire-userdebug", but that returned: "cm_sumire-userdebug: command not found"

Uclydde said:
Yay, that worked! However, I got to the "cm_yourdevicecodename-userdebug" line, so I entered "cm_sumire-userdebug", but that returned: "cm_sumire-userdebug: command not found"
Click to expand...
Click to collapse
This needs a
Code:
repo --sync
in between

Uclydde said:
Yay, that worked! However, I got to the "cm_yourdevicecodename-userdebug" line, so I entered "cm_sumire-userdebug", but that returned: "cm_sumire-userdebug: command not found"
Click to expand...
Click to collapse
When you type lunch it will show a big list of devices and prompt for an input. Enter cm_sumire-userdebug. You can also however run
Code:
lunch cm_sumire-userdebug
Sent from my E5823 using XDA-Developers mobile app

Nice guide, buddy

Binaries have been updated to v5a,
v4 is included in this, can you update please? I did it manually and got it to boot
I'm not able to push them to github.
http://developer.sonymobile.com/downloads/tool/software-binaries-for-aosp-m-android-6-0/

celderic said:
Binaries have been updated to v5a,
v4 is included in this, can you update please? I did it manually and got it to boot
I'm not able to push them to github.
http://developer.sonymobile.com/downloads/tool/software-binaries-for-aosp-m-android-6-0/
Click to expand...
Click to collapse
Is the camera working for you ?
Did the focus improve ? or are still changes needed ?

zacharias.maladroit said:
Is the camera working for you ?
Did the focus improve ? or are still changes needed ?
Click to expand...
Click to collapse
It doesn't work for me with v4, or v5a
Using the guide posted here, nothing changed only updated to v5a manually.
I haven't bothered to look through the log yet, bur it says camera hardware inaccessible or something like that I forget the exact wording. With v4 and v5a, I even downloaded the source again to make sure it wasn't a problem on my end.
I'll try and have a proper look tomorrow

celderic said:
It doesn't work for me with v4, or v5a
Using the guide posted here, nothing changed only updated to v5a manually.
I haven't bothered to look through the log yet, bur it says camera hardware inaccessible or something like that I forget the exact wording. With v4 and v5a, I even downloaded the source again to make sure it wasn't a problem on my end.
I'll try and have a proper look tomorrow
Click to expand...
Click to collapse
Yeah, we kinda apologize for that. My linux environment has broken hard after upgrading to ubloatu, i mean Ubuntu 16.04... I'm busy updating to debian now, and will most likely stick with debian jessie as sid and stretch ended in an huge catastrophy. Thanks to @moonbutt74 for not warning me. (<- Only a joke, he actually predictied that sid will be a catastrophy but I didn't listen cuz that's me). I'll also most likely need to update the guide too as we no longer prefer/recommend ubuntu 16.04
Sent from my E5823 using XDA-Developers mobile app

celderic said:
It doesn't work for me with v4, or v5a
Using the guide posted here, nothing changed only updated to v5a manually.
I haven't bothered to look through the log yet, bur it says camera hardware inaccessible or something like that I forget the exact wording. With v4 and v5a, I even downloaded the source again to make sure it wasn't a problem on my end.
I'll try and have a proper look tomorrow
Click to expand...
Click to collapse
@celderic
did you do any modifications to the repos ?
If not did the kernel built with the ROM work for you ?
Thanks
edit:
v04 and v05a binary blobs are the same for us (Kitakami)
Changes in v05a was for kanuti, rhine, shinano and yukon

zacharias.maladroit said:
@celderic
did you do any modifications to the repos ?
If not did the kernel built with the ROM work for you ?
Thanks
edit:
v04 and v05a binary blobs are the same for us (Kitakami)
Changes in v05a was for kanuti, rhine, shinano and yukon
Click to expand...
Click to collapse
Sorry been on holiday not had time to reply on here, oh I didn't realise they where the same for our device.
I didn't modify the repos, and yes the stock cm kernel booted fine.

openjdk-7 isnt available under ubuntu 16.04. Is 8 or 9 supported yet or do I have to manually install v7?

[GUIDE]Creating your own Firefox ‘Data Saver’ proxy Browser

Introduction
Hello XDA. My name’s Chris and in this guide I’ll try to show you how to set up your own independent “Data Saver” proxy similar to the one we know from Google Chrome browser. I have to warn you that the steps needed require moderate knowledge and might not be easy to follow for the beginners. It also requires setting up some software on your server (Ubuntu 16.04 in my case) prior to following this guide. You can find tutorials on how to do so in Prerequisites section below.
Prerequisites
- Server running Linux distro (Ubuntu 16.04 x64 in my case as I said above)
- Wireguard or other VPN (OpenVPN) setup on the server that is useable on your device (I’ll go with wireguard, so here’s a great tutorial to set that up: https://www.digitalocean.com/commun...t-to-point-vpn-with-wireguard-on-ubuntu-16-04 . It's pretty straightforward.)
- Machine to build (preferably gcloud or some other buildserver)
- Time
Environment
We’ll start of with installing libraries & additional tools needed to build Android itself just to make sure everything’s ready to move on later:
Code:
sudo apt-get install git ccache automake lzop bison gperf build-essential zip curl zlib1g-dev zlib1g-dev:i386 g++-multilib python-networkx libxml2-utils bzip2 libbz2-dev libbz2-1.0 libghc-bzlib-dev squashfs-tools pngcrush schedtool dpkg-dev liblz4-tool make opting libjpeg8 libjpeg-dev openssl ssl-cert maven mercurial libnss3-tools
Okay, so that’s about it for automated install of packages. We’ll also need newest version of golang and Ubuntu repos do not provide that by default so we will install It and set work directory manually:
Code:
mkdir -p ~/golang
cd ~/golang
wget https://dl.google.com/go/go1.10.1.linux-amd64.tar.gz
tar -xvf go1.10.1.linux-amd64.tar.gz
sudo mv go /usr/local
mkdir -p ~/go
Now that it is installed let’s add it to our PATH environment by editing .profile:
Code:
sudo nano ~/.profile
Add these lines at the end of the file:
Code:
export PATH=$PATH:/usr/local/go/bin
export GOROOT=/usr/local/go
export GOPATH=$HOME/go
Save the file and run
Code:
source ~/.profile
To refresh your env setup.
That’s it for now when it comes to your environment setup. We may be installing other packages needed in this guide later.
Installing the Proxy server
Let’s proceed to setting our proxy server that will do all the compression work we want it to. We’ll be using Compy proxy here since it supports ssl which is a must for almost all modern websites nowadays. Let’s download the source we need:
Code:
go get github.com/barnacs/compy
Now we are ready to compile our proxy server binary:
Code:
cd go/src/github.com/barnacs/compy/
go install
Our binary should be located in ~/go/bin so lets try to run it and see if it’s okay
Code:
cd ~/go/bin
./compy -help
This should print all of the arguments we can pass to our proxy. I prefer to have easy access to running it on boot without setting any paths so I’ll move my binary to /usr/bin of my server.
Code:
sudo mv compy /usr/bin
sudo chmod a+x /usr/bin/compy
Generating Self-signed Certificate & Running the Server
Since w plan on using our proxy with SSL traffic we have to provide our own SSL certificate. I just went ahead an generated self-signed to not bother myself with getting it the other way:
Code:
mkdir -p ~/keys
cd ~/keys
openssl req -x509 -newkey rsa:2048 -nodes -keyout cert.key -out cert.crt -days 36500
openssl is going to ask you a few details. Fill those in however you want as it doesn’t matter much since you will be the only one using that certificate anyway. After doing so you should have two files in your keys folder:
Code:
cert.key
cert.crt
Now you may test run compy with those keys to see if everything’s okay. I’ll run it on port 8888:
Code:
compy -host :8888 -ca ~/keys/cert.crt -cakey ~/keys/cert.key -gif false -brotli 8 -gzip 7 -jpeg 75 -png -minify
As you can see we have our keys specified twice. That’s due to workaround applied earlier, don’t worry about it. If everything’s okay you should now see something like this:
Code:
2018/04/05 09:46:49 compy listening on :8888
Since our proxy server seems to run correctly, we may also add some iptables to make it accessible only from our VPN peer address:
Code:
sudo apt-get install iptables-persistent
sudo iptables -A INPUT -p udp -s 10.0.0.0/24 --dport 8888 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 10.0.0.0/24 --dport 8888 -j ACCEPT
sudo iptables -A INPUT -p udp -s 0.0.0.0/0 --dport 8888 -j DROP
sudo iptables -A INPUT -p tcp -s 0.0.0.0/0 --dport 8888 -j DROP
sudo dpkg-reconfigure iptables-persistent
From now on your proxy should be accessible only through your Wireguard VPN connection. At this point it’s safe to run our proxy in the background at system bootup. I do that via crontab:
Code:
sudo crontab -e
Add this line in the end of a file (change the settings if you want to):
Code:
@reboot compy -host :8888 -ca ~/keys/cert.crt -cakey ~/keys/cert.key -gif false -brotli 8 -gzip 7 -jpeg 75 -png -minify
Save the file and you're done with server config.
Server is ready. What now?
Now we need a way to make use of our server and that’s the moment where the fun begins. We could of course add our SSL certificate to Android system certs and try to use our proxy globally BUT that will result in most of apps refusing to connect to their services due to SSL Pinning as our MiTM proxy signs the traffic with our self-signed certificate so non-browser traffic would be essentially dead. Since I haven’t found reliable way to set proxy for individual apps nor disable SSL Pinning on Android 8.1 I had to try my luck finding browser that support proxy setup and Firefox does just that via about:config.
Here comes another problem. Firefox doesn’t use system certificate store so trying to setup our proxy there will result in “Unsafe connection” errors that won’t allow us to visit any website due to the fact the browser doesn’t trust our certificate. I also haven’t found any reliable way to add the cert into newest versions of the browser so the easiest way around that was compiling it from source with the certificate bundled in. Well, let’s go then.
Sync Fennec source
Development variant of Firefox is named Fennec. I recommend doing these steps on some build server as it will be faster than on regular machine. We will use Mercurial to download newest source. It got installed when we were setting up our Env so you can just proceed to clone the repo:
Code:
mkdir -p ~/fennec
cd ~/fennec
hg clone https://hg.mozilla.org/mozilla-central
Wait for the sync to complete. It can take a while so in the meantime we can go forward with other things.
Prepare certificate for Fennec
So we have our cert files we generated earlier used by the proxy. Now we need to convert our cert to format used by mozilla in their browser sources. First we need to convert our certificate into .der format
Code:
cd ~/keys
openssl x509 -in cert.crt -out cert.der -outform DER
Then we will convert it into certdata.txt format for us to use.
Code:
nss-addbuiltin -n ProxyCert -t "C,C,C" -i cert.der >> certdata.txt
We should get quite lengthy file certdata.txt that contains our certificate in the format needed by Fennec sources. We will add it there in a few moments.
Adding the certificate into browser source
Okay so the sync finished and our certdata is ready. Now we need to append it to file provided in Fennec sources:
Code:
cat ~/keys/certdata.txt >> ~/fennec/mozilla-central/security/nss/lib/ckfw/builtins/certdata.txt
Now our browser is ready to be built.
Building Fennec
Let’s move on to our final part: Building the browser. First let the building environment setup by itself.
Code:
cd ~/fennec/mozilla-cental
./mach bootstrap
You will be prompted with a question which version of Firefox you want to build. Naturally, I chose 4. Firefox for Android. Some additional package installs and dependencies downloads will occur at this point. This might take some time. You may need to agree to Android SDK licence and make some other chocies like Mercurial install. Choose Do not install Mercurial as we have it already. Then in another Mercurial dialogue choose No.
After the sync is complete we will get the suggested content of mozconfig file that is needed for the build:
Code:
# Build Firefox for Android:
ac_add_options --enable-application=mobile/android
ac_add_options --target=arm-linux-androideabi
# With the following Android SDK and NDK:
ac_add_options --with-android-sdk="/home/chris/.mozbuild/android-sdk-linux"
ac_add_options --with-android-ndk="/home/chris/.mozbuild/android-ndk-r15c"
Create the file with nano and paste your config there:
Code:
nano mozconfig
Now before we start the build we will need to update gcc toolchain since Ubuntu 16.04 comes with gcc 5.4 and Fennec needs at least 6.1 to build:
Code:
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y
sudo apt-get update
sudo apt-get install gcc-6 g++-6
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-6 60 --slave /usr/bin/g++ g++ /usr/bin/g++-6
Verify our install with gcc -v
Code:
gcc version 6.3.0 20170519 (Ubuntu/Linaro 6.3.0-18ubuntu2~16.04)
We also need to download rust toolchain for arm target and then we may start the build:
Code:
source ~/.profile
rustup target add armv7-linux-oideabi
./mach build
This will take a lot of time. On Google Cloud build took me around 50 minutes so you may take your rest now and go out for a walk/grab some coffee etc. Anyway, after it’s done run:
Code:
./mach package
To generate your new browser fennec-61.0a1.en-US.android-arm.apk file. It should be somewhere in
Code:
~/fennec/mozilla-central/ objdir-droid/dist
Copy it over to your device and install. Time for some tests.
Setting up the browser & Testing
After installing your fresh Fennec build proceed to open about:config in the address bar and setup proxy as follows (replace ip and port with the ones you used):
Code:
network.proxy.http
10.0.0.1
network.proxy.http_port
8888
network.proxy.ssl
10.0.0.1
network.proxy.ssl_port
8888
network.proxy.type
1
network.proxy.no_proxies_on
10.0.0.1
Try to load any https website while running compy proxy on your server. It should work now:
Screenshot of browser and console output
Enjoy your self-made data compressing browser. Remember to support the developers of open source software used in this guide. Perhaps later I'll attach some more screenshot of the final product.
Links & References:
Compy proxy
Wireguard
Mozilla Cert Storage Info

Hi! Thanks for the guide. I managed to compile Compy using Termux with golang.