Related
This project is no longer being updated. Feel free to use the code.
Android Builder (AB v.5)
kiel123 has joined the AB team!
Thanks to dsixda, Armin Coralic, and cteneyck. I do have permission to reuse their code.
Requirements:
Ubuntu Linux or variant
dsixda's HTC Android Basic Kitchen - http://forum.xda-developers.com/showthread.php?t=633246
In my crazy quest to learn more about Android, I decided to create a companion to dsixda's kitchen that will take some of the drudgery out of downloading and compiling AOSP and the HTC kernel.
Here's what the current menus looks like:
Code:
Select the device that you would like to build for.
1) HTC Droid Eris
a) About Builder
0. Exit
Please enter option number:
and then
Code:
1. Sync AOSP - Cupcake (1.5) repo
2. Sync AOSP - Donut (1.6) repo
3. Sync AOSP - Eclair (2.1) repo
4. Sync AOSP - Froyo (2.2) repo
0. Exit
Please enter option number:
You can follow these two steps, and you will get a freshly compiled, but unsigned ROM in the original_update folder. After running it through the kitchen, it boots up just fine. Of course, most of the hardware doesn't work. That's the part that I am working on now (seriously, I am). I've been working on this for a while, because as I said - this is really about me learning. If I can make a tool that someone finds useful, then great!
Install instructions:
1) download and unzip dsixda's kitchen to whatever directory you want
2) download Android Builder into kitchen's scripts/plugins folder and unzip
3) back out and start the kitchen
4) select 'Run plugin scripts' from Advanced menu and select 'Android Builder'
5) you're there!
NOTE: This will try to install/update all of the packages that it needs, including Java5. Please look at the code so that you know what it's doing. That's just a good general practice with any code that you can actually look at and that you have to run as root...
Future plans:
1) add more kernels (devices)
2) add more vendor trees (goes along with devices)
3) add more repos (CM5, CM6, AOSP Master)
I hope someone finds this useful, and feel free to post useful suggestions/bug reports.
DOWNLOAD - (https://github.com/gnarlyc/android_builder)
CHANGELOG -
v5. - kiel123 re-wrote v.4 as a plugin to dsixda's kitchen with me testing afterward, minor fixes, removed Java6 as no longer needed by kitchen
v.4 - re-wrote to add structure to support more devices in the future
added option for Cupcake, Donut, and Froyo
added .config files that enable touchscreen in kernel by default
v.3 - unreleased build
v.2 - corrected misspellings in code and dialogs
added sudo to launch setup script (Still need to sudo bash first. Sorry.)
v.1 - initial release
Very nice. I have been doing the same. I have used darchstar's post here as the basis of my work. But, I am always glad to look how others are doing things. Thanks for the post.
arockj said:
Very nice. I have been doing the same. I have used darchstar's post here as the basis of my work. But, I am always glad to look how others are doing things. Thanks for the post.
Click to expand...
Click to collapse
Thanks for the thanks! I don't quite get the vendor tree thing, but I just need to look at it again a few times. And, that's where I'm at. It would be really neat to be able create a working AOSP ROM from scratch with minimal manual intervention. It might not be possible, but I haven't found that out yet. (But wait! If you can do it manually, it can be scripted. Right?)
there's typo in your install script
look for line: android_builder*
should be android-builder*
and your install script should use mv command instead of cp to avoid files duplication
EDIT: also you should use sudo command along with ./ab_setup script or include within script as well.
When i ran ./abmenu and i get this message
Code:
Please copy 'menu-kd' to the root of your kitchen folder with 'menu'
menu-kd?.... don't you mean abmenu? turned out it was looking for menu file which i had it renamed to kitchen_menu but I fixed that on my part and all is working well.
Dont let any of my criticism scare you away, I can see there's a lot of potential use for this scripts, great start by the way.
firestrife23 said:
there's typo in your install script
look for line: android_builder*
should be android-builder*
and your install script should use mv command instead of cp to avoid files duplication
EDIT: also you should use sudo command along with ./ab_setup script or include within script as well.
When i ran ./abmenu and i get this message
Code:
Please copy 'menu-kd' to the root of your kitchen folder with 'menu'
menu-kd?.... don't you mean abmenu? turned out it was looking for menu file which i had it renamed to kitchen_menu but I fixed that on my part and all is working well.
Dont let any of my criticism scare you away, I can see there's a lot of potential use for this scripts, great start by the way.
Click to expand...
Click to collapse
I renamed all of the scripts at the last minute and thought that I had them fixed. I'll make the corrections and repost. The goal is to get this thing working well. Your criticism is just more motivation. Thanks!
I'm new to all this but I would love this to work with Froyo source.
Geo411m said:
I'm new to all this but I would love this to work with Froyo source.
Click to expand...
Click to collapse
The current 'internal' build works with Cupcake (1.5), Eclair (2.1), and Froyo (2.2). It's not ready to release though. Give me another week or two. I'm working on proprietary bits now, so that you get a more usable ROM from the start.
Although from your sig, it appears that you don't have an Eris, so you probably don't care much about it's proprietary bits. If you want it to get Froyo instead of Eclair, you can just change 'eclair' to 'froyo' in the script that creates the repo.
I'm with you on Froyo. I have a feeling in a month or two very few people will want to run a pre-Froyo ROM. It's really nice.
rm error on install
i keep getting "rm: cannot remove './bkup': No such file or directory"
the file is there untill i run the ./install script then it disappears and shoots me an error any help would be appriciated
Sjflowerhorn said:
i keep getting "rm: cannot remove './bkup': No such file or directory"
the file is there untill i run the ./install script then it disappears and shoots me an error any help would be appriciated
Click to expand...
Click to collapse
That files was used by me while developing the code. It's not needed. I'll fix the install script for v.5, but if you really want to fix the script now... Just remove the line -
'rm ./bkup'
from the 'install' script
You should still be able to start Android Builder by typing './abmenu', as long as it's in the same folder with the kitchen's menu. The error shouldn't matter.
gnarlyc said:
That files was used by me while developing the code. It's not needed. I'll fix the install script for v.5, but if you really want to fix the script now... Just remove the line -
'rm ./bkup'
from the 'install' script
You should still be able to start Android Builder by typing './abmenu', as long as it's in the same folder with the kitchen's menu. The error shouldn't matter.
Click to expand...
Click to collapse
k thanks i think it worked the same even with the error but got it =)
Sjflowerhorn said:
k thanks i think it worked the same even with the error but got it =)
Click to expand...
Click to collapse
Good deal. Let me know how it goes.
gnarlyc --
Have you thought of making this into a .plugin file that can be placed in the scripts/plugins folder of the HTC Android Kitchen? Then that way you won't need to update your scripts each time I update my kitchen
In my thread I could provide a link to your tool, so that people can add that plugin if they want.
dsixda said:
gnarlyc --
Have you thought of making this into a .plugin file that can be placed in the scripts/plugins folder of the HTC Android Kitchen? Then that way you won't need to update your scripts each time I update my kitchen
In my thread I could provide a link to your tool, so that people can add that plugin if they want.
Click to expand...
Click to collapse
The thought has occurred to me.
Give me a bit. A link in your thread would be great. It might even motivate me to add more devices. Getting stuck on adding Eris proprietary bits has lead me down the road of making a ROM (http://forum.xda-developers.com/showthread.php?t=725447) and documenting the process. I'll get back to Android Builder soon though.
Ok, so...
TO-DO
-------
1) rewrite AB as a plugin to dsixda's HTC Android Kitchen
2) add proprietary bits for Eris
3) add more devices
4) add CM repos
5) add Gingerbread repo
6) add compatibility with more distros
i am getting line 28: update-java-alernatives: command not found?
lord194409 said:
i am getting line 28: update-java-alernatives: command not found?
Click to expand...
Click to collapse
Interesting... I've tested from a freshly installed Ubuntu installation several times. It should install everything you need.
You could try 'update-alternatives --config java' and let me know what it says. You could also look for a file called 'setup_ran' in the root of the Kitchen, and delete it. Start './abmenu' after that and try again.
thanks for answering. Could the problem be that i installed it on cygwin.
lord194409 said:
thanks for answering. Could the problem be that i installed it on cygwin.
Click to expand...
Click to collapse
Yes. Android Builder will not work under cygwin. I don't know that it ever will. That's on the TO-DO list, but it's so far down that it's not on the posted TO-DO list.
I've only tested under Xubuntu 10.04 and minimal Ubuntu 10.04.
If you happen to work it out on cygwin, let me know! I'm sure others would be happy to have the option. Heck, I might even be. I'm mainly using Ubuntu because of AB anyway. (Although, it is kind of growing on me.)
thanks i will look in to it.
Sent from my Eris using XDA App
just wanted to let you know that it didn't work for me. It runs, gives some errors and outputs a update.zip called AOSPbase but the file is only 1.1kb in size.
Geo411m said:
just wanted to let you know that it didn't work for me. It runs, gives some errors and outputs a update.zip called AOSPbase but the file is only 1.1kb in size.
Click to expand...
Click to collapse
make sure you have java5 and java6 jre/jdk installed. it should work then. i had a similar issue the first time through... took it literally 10hours plus, then output was the 1.1k AOSPBase.zip file.
manually installed all java components, and everything worked perfectly then (and took half the time)
I would like to mess with trying to install my own customized ROM's to my Nexus 7, but the first place to probably start is with being able to build AOSP as-is from source.
As I understand currently, building is only supported on Linux and OS X, but I can easily get Ubuntu 10.04 and re-partition my HDD to give it about 100GB (if that much is even needed).
Looking at:
http://source.android.com/source/initializing.html
I need to choose a branch and setup the Linux environment. I'm a bit confused as to what branch I should choose though. I want the latest source of Android available at the time, so I should pick the master branch? Or since I'm only building for the Nexus 7, should I choose it's device-specific branch instead? Although looking at:
http://source.android.com/source/build-numbers.html
the Nexus 7 is only at android-4.1.1_r1.1, but I could of sworn I heard there was r4 out already.
As for setting up the Linux environment, I hope I can just follow all the commands listed there without any problem.
Proceeding on with:
http://source.android.com/source/downloading.html
It looks like a pretty straightforward process that I'm also hoping can be done successfully if I follow the commands exactly as presented. I don't have a proxy nor the need for a local mirror either.
And then moving onto:
http://source.android.com/source/building-devices.html
Some stuff there I find a little bit confusing. It would seem I have to first get proprietary drivers, which all 4 seem to be placed conveniently at:
https://developers.google.com/android/nexus/drivers#grouper
From there, I imagine I can move the script that's bundled inside to the root of the source folder, run it, and follow the instructions. I don't exactly know what the root of the source folder is, but it would probably be obvious once I did start trying to build this. But once I did find it, I would run (using Nvidia's Graphics driver for the example) sh extract-nvidia-grouper.sh in Terminal, and it would place the right files where they need to be.
I don't understand the make clobber part too well at all; should I run this on the very first build, later builds, or all builds?
And once the source and drivers are all downloaded and available, I should then run lunch full_grouper-userdebug and then finally make -j# (# being some number in accordance with how many cores on my CPU I have). I have a triple-core CPU at 3.5Ghz, and I have the ability to unlock to quad-core at 3.3Ghz (but prefer to stay on triple). Should I just run -j32? Also will this build the Kernel as well, or will I have to get the source for that and compile it separately?
And once the build completes, my plan from there was to just go back to Windows and flash it. And if I managed to get it to flash and boot properly, I assume I would of succeeded with compiling AOSP from source
I noticed that userdebug part on full_grouper-userdebug gives "root access and debuggability". Does this mean it comes with some program like Superuser or SuperSU already installed? Or does this mean I can easily install those?
Perhaps after I get comfortable with the basics of flashing AOSP as-is, I can then try to mess with different types of optimizations, like Linaro and perhaps even messing with many types of optimizations from different kernels like faux123 has done .
I also have a 360kb/s DSL connection, so downloading the entire source the first time will probably take a good while. But once I have the source, I take it I don't have to redownload the entire thing for patches and stuff?
Any and all guidance is welcome
Bump before I go for tonight
Bump
You have a bunch of questions. I will answer some. And while I whole-heartedly support learning to build you don't need to build to flash roms.
The best advice I can give you is to just start building. You have found a bunch of instructions and links, obviously. Go ahead and begin, and tackle problems as they arise.
Environment
Okay...really the hardest part is setting upi the environment, if you don' t know linux. After downloading and installing Java and the SDK, make sure you add them to your path.
Most guides will have adding the path in the directions. But make sure to check that it works! It will be extremely frustrating, and you won't know what is wrong. Go to a random directory, Documents would be good, and enter java -version and then adb devices. If the computer says it cannot find the commands, then your path is the problem.
Make sure to setup udev. It is easy, Google it.
Building
Branch
You want to build from the tags.
Code:
repo init -u https://android.googlesource.com/platform/manifest -b android-4.1.1_r4
For the proprietary blobs, whatever directory you repo sync from (~/android/system or whatever) is the root directory. run the extraction from there.
when the proprietary blobs are extracted, and the source has been downloaded, these are your commands.
Code:
. build/envsetup.sh
lunch
Lunch will return a list of devices, Grouper is the Nexus 7, it is number 4. eng and user-debug do have root access, but SU and SuperSU are more than just root, they manage the root access for your apps as well. You can download them from Play or install them as a flashable-zip.
Choose 4 and then
Code:
make otapackage
don't worry about the -j# part. Your machine almost definitley cannot handle -j32. It is -j4 by default, that should be fine for your cpu.
If you want to enable faster builds, you can enter
ENABLE_CCACHE=1
before make otapackage, but it will take up a lot of space on your hd. Your subsequent builds will use some thing from your intial build instead of rebuilding them each time (kernel and other things). So even if you repo sync, some changes won't be reflected in your later builds. For instance, if you do not clean your prebuilts and build system, your build date in the build.prop will always stay the same as the first build.
The way you clear the build directory and make new everything is with make clean or make clobber. You can run it before any build, but the build will take much much longer than one that uses prebuilts. Non-clobbered and with ccache enabled are the fastest of all. But subsequent builds are pretty fast even without ccache.
When you want to update your source, you can just go to your root dir and repo sync. It will only update your source, it won't take nearly as long.
Okay, I answered more than I intended. There are a million guides that show you every step in the process.
Don't ask anymore generic worry questions...you're ready. You understand more than most people do before their first build before I even posted. Get started and if you run into problems, search. If you can't find the answer, then come back and ask us.
Good luck. it is easy, and very satisfying.
I finally got around to installing a Virtual Machine, and Ubuntu 10.04 After doing that, I fully updated Ubuntu, installed VMWare Tools, and then proceeded to start trying to acquire the AOSP source.
Getting sun-java-6 was a bit tricky, but not too hard (I ran the commands exactly as listed on the site, but the package didn't exist; had to get it from somewhere else). After that, I proceeded to do everything else, except CCache (I didn't know what .bashrc was, but I'll look further into this with future AOSP builds).
I then made the folder, did repo sync, and I'm now acquiring the source now from android-4.1.1_r4. As a quick question, does it matter whether I choose to build from android-4.1.1_r4, or master? Would master be more up-to-date?
espionage724 said:
I finally got around to installing a Virtual Machine, and Ubuntu 10.04 After doing that, I fully updated Ubuntu, installed VMWare Tools, and then proceeded to start trying to acquire the AOSP source.
Getting sun-java-6 was a bit tricky, but not too hard (I ran the commands exactly as listed on the site, but the package didn't exist; had to get it from somewhere else). After that, I proceeded to do everything else, except CCache (I didn't know what .bashrc was, but I'll look further into this with future AOSP builds).
I then made the folder, did repo sync, and I'm now acquiring the source now from android-4.1.1_r4. As a quick question, does it matter whether I choose to build from android-4.1.1_r4, or master? Would master be more up-to-date?
Click to expand...
Click to collapse
Sorry for late answer, no, use the r4 branch as it is more up to date. Also, make clobber every time isn't needed but you should as it remove then entire out folder (wich is where compiled stuff go) and this make sure you rebuild a clean thing.
Building CyanogenMod 10
Dunno if this is of any interest, but I have a thread started with a complete walkthrough for building CyanogenMod10 for Nexus 7.
Most of the info is the same, and there are some tips in the comments as well.
espionage724 said:
I would like to mess with trying to install my own customized ROM's to my Nexus 7, but the first place to probably start is with being able to build AOSP as-is from source.
Click to expand...
Click to collapse
So, how did you get on? I've been following the same path I think - repo sync the source and follow Google's own tutorial on compiling Android but with the added step of incorporating the binary drivers for the grouper.
I've built the .img files using make -j8, that all works, fastboot flash worked, but I get no video out when booting up using the new OS. I can ADB into the Nexus and it's certainly booted and working okay apart from, I'm guessing, the missing binary drivers.
I've used each of the 5 binary driver scripts to populate the "vendor" directory in the root of the downloaded source before compiling from scratch, but perhaps I've missed a step, so I'm curious as to whether you've got a fully working AOSP+binary driver compile working.
(By the way, my build environment was Ubuntu 12.04 64bit, SDK r20.0.3, Android 4.1.1 (JRO03R) source, Sun Java 1.6, and it all seems to work well using 8 threads on a Core i5 2500K + 4GB RAM).
Edit:
I re-ran the binary extraction, did a make clean; make clobber, and re-compiled - and now video works. Everything works now apart from the compass, camera and rotation sensor. I also tried compiling CyanogenMod from source, too, and had the exact same three problems. Everything works, and works well, apart from camera, compass and rotation sensor. All of which work in the stock Google ROM. Weird.
OK, So I've just compiled an OTA update package from AOSP source... my question is this:
I already have unlocked the bootloader on my wife's Nexus 7, installed Clockworkmod, rooted it, installed busybox, etc, manually on the stock 4.2 update I downloaded from Google on the device when it asked me to upgrade.
Is the otapackage I just compiled going to replace my custom recovery if I flash it as is? I've looked, and it has a "recovery" folder in the .zip, whereas any of the custom ROMs I have downloaded for my phone do not. Do I simply delete this recovery folder, and flash away? Do I need to edit the updater-script? I'm still trying to read and learn about this, but I haven't gotten a good answer from google or searching this site for my specific problem... maybe I'm wording my searches incorrectly.
I would just rather not have to go back and reinstall Clockworkmod... I know that if I want to have busybox, SuperSU, and other apps installed when I flash I'm going to have to add them to the zip and resign... I just don't want to mess my recovery. And being that this is my wife's tab (and not mine to play with, as she pointed out ) I don't want her to get the impression that I'm having to "fix" something I "broke" lol.
hallowed.mh said:
OK, So I've just compiled an OTA update package from AOSP source... my question is this:
I already have unlocked the bootloader on my wife's Nexus 7, installed Clockworkmod, rooted it, installed busybox, etc, manually on the stock 4.2 update I downloaded from Google on the device when it asked me to upgrade.
Is the otapackage I just compiled going to replace my custom recovery if I flash it as is? I've looked, and it has a "recovery" folder in the .zip, whereas any of the custom ROMs I have downloaded for my phone do not. Do I simply delete this recovery folder, and flash away? Do I need to edit the updater-script? I'm still trying to read and learn about this, but I haven't gotten a good answer from google or searching this site for my specific problem... maybe I'm wording my searches incorrectly.
I would just rather not have to go back and reinstall Clockworkmod... I know that if I want to have busybox, SuperSU, and other apps installed when I flash I'm going to have to add them to the zip and resign... I just don't want to mess my recovery. And being that this is my wife's tab (and not mine to play with, as she pointed out ) I don't want her to get the impression that I'm having to "fix" something I "broke" lol.
Click to expand...
Click to collapse
Sorry if a bit late, but here are some answers:
yes, the rom will replace your recovery. but if you delete the recovery folder and delete every line containing the word "recovery" in the updater-script, you should be good to go.
And if you accidentally remove the recovery, you can always flash it back very easily using: "fastboot flash recovery [filename.img]" (your n7 has to be in the bootloader)
And again, yes, you will have to put the extra apps into the zip and update the updater-script to install them too.
Also, you will need the gapps package if you want to use the play store and other google apps.
Hope this helped
Nexus 7 3G does not boot after flashing AOSP
Hi,
I followed the steps provided on source.android.com to build and flash the AOSP for Nexus 7 3G Tilapia. After successful flash, the device does not show anything after Google logo. Please help me out.
Thanks,
Veeren
Compile with ccache makes build time extremely fast.
How to do:
_Open a terminal
_Install ccache:
sudo apt-get install ccache
_Open .bashrc:
sudo gedit ~/.bashrc
_Add these lines:
#ccache
export USE_CCACHE=1
_Save and exit
_Sync source code
_After source synced, run in same terminal (in root directory of your source):
prebuilts/misc/linux-x86/ccache/ccache -M 20G (20G is the size in giga of space allocated for ccache, change it as you want)
_Start building
How to see if ccache works:
_Open another terminal in the root directory of your source and type:
watch -n1 -d prebuilts/misc/linux-x86/ccache/ccache -s
First build using ccache may be a little much longer but the others will be faster...
veerndra said:
Hi,
I followed the steps provided on source.android.com to build and flash the AOSP for Nexus 7 3G Tilapia. After successful flash, the device does not show anything after Google logo. Please help me out.
Thanks,
Veeren
Click to expand...
Click to collapse
Did you pull the proprietary files for your nexus and include them in the build? I believe things like your video drivers are included in there, so if those are missing....
I think the prop files are available for download from Google on source.android.com... If not, they tell you how to use an included script to pull them via adb. I can't remember... It's been a while since I built vanilla AOSP.
Sent from my Inspire 4G using xda app-developers app
Modifying stock AOSP
I have built AOSP following the Google tutorial.
I am compiling using the master branch and
Code:
aosp_grouper-userdebug
.
I have downloaded and extracted the appropriate proprietary binaries.
I am modifying two files in the source tree (see attachments; search for "// MODIFICATION ADDED HERE" to find my changes). Will these changes work? I am using Eclipse, set up in the exact way the tutorial explains, and I am not receiving any new errors.
When I compile the source using the following commands
Code:
$ . build/envsetup.sh
$ lunch aosp_grouper-userdebug
$ make fastboot adb
and flash it to my device with
Code:
$ fastboot -w flashall
BEFORE my modifications, it works just fine. The android-info.txt file and all the image files are produced properly.
However, AFTER adding the modifications, the build completes with no errors, but android-info.txt and all image files are no longer produced.
Why am I experiencing these problems? What can I do to make it work the way I want?
P.S. YES, I am aware that my modifications are not secure; these are for my own purposes, not for a public build.
i present to you the latest STABLE release from vvn's secret underground laboratory.......
THE HALF-ASSED ONE PLUS ONE TOOLKIT v1.2b!
UPDATED!!! LATEST RELEASE: August 24, 2014
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
i'm copying and pasting from the thread i have on the oneplus one forum because i am lazy and i already worked hard enough coding the damn thing read on for changes/updates/fixes, or check out the README file which is also the changelog!
LATEST UPDATE RELEASED (cumulative v1.1 and v1.2b update since oneplus decided to release 33R so damn fast)!
TOOLKIT SOURCE CODE/PYTHON SCRIPT:
http://pastebin.com/y511TjV1 -or- http://notworth.it/opo/opotoolkit.py
PYADB LIBRARY SOURCE CODE:
http://pastebin.com/7VSpinAz -or- http://notworth.it/opo/pyadb.py
README FILE:
http://notworth.it/opo/README
download ZIP file containing scripts and adb/fastboot binaries:
http://notworth.it/opo/1plus1-halfassedtoolkit_v1.2b.zip
SHA1: eb2ddd5eeddd51cbc863100422ffa84bfb568a8e
same ZIP above PLUS a few useful apps:
http://notworth.it/opo/1plus1-halfassedtoolkit_v1.2b-withapps.zip
SHA1: 6528af01ce50e544f5b1c143659f0ef357b0895b
##################################################################
HALF-ASSED ONEPLUS ONE TOOLKIT v1.2b
BY vvn (eudemonics on xda-developers)
release date: AUGUST 24, 2014
##################################################################
PASTED FROM README FILE:
this is a very half-assed project, as you can guess from the name, and i cannot guarantee fast or frequent updates.
REQUIREMENTS FOR SCRIPT TO WORK:
* opotoolkit.py is the main script. that's the one file you REALLY need.
- get it here: http://pastebin.com/y511TjV1 -or- here: http://notworth.it/opo/opotoolkit.py
* my PYADB library needs to be in the same directory as filename "pyadb.py".
- get it here: http://pastebin.com/7VSpinAz -or- here: http://notworth.it/opo/pyadb.py
* obviously, you'll need python 2.7. download python here: https://www.python.org/downloads/
* you need ADB and FASTBOOT from the android SDK. download the SDK here: https://developer.android.com/sdk/
* finally, you need an OS that supports Python and the android SDK, which I'm afraid narrows it down to:
- Linux (all flavors)
- Mac OSX (exotic jungle cats and beyond)
- Windows (pretty much all releases, or starting from whichever one could support Python.)
i apologize for limiting your options like that.
you can either put the scripts and other files in the same directory as your android SDK,
or set an environmental path variable for your android SDK directory.
##################################################################
USE AT YOUR OWN RISK. i am not responsible for any damage to your device.
i have tested every function except the unlock bootloader and sync functions.
everything works except the uninstall APK function, but i think i just don't quite understand the adb command.
to report bugs, ask questions, offer suggestions, explain the adb uninstall and sync functions(?!), ***** at me, propose marriage, or send anonymous death threats, email me: vvn (at) eudemonics (dot) org
feel free to share, modify, whatever.
some credit would be nice but it's not a big deal if you don't. donations are super nice.
but buying & sharing my EP would be the most awesome way to show your appreciation. really, it would mean the world to me.
you can stream and buy the EP at: http://dreamcorp.bandcamp.com or any major online music retailer (itunes, google play, amazon, spotify, cdbaby, etc.) - just search for "the dream corporation" and album title "last night on earth"
follow on facebook: http://www.facebook.com/dreamcorporation
and of course, more music on soundcloud: http://www.soundcloud.com/dreamcorp
##################################################################
CHANGES IN v1.2B (released same day as v1.1):
new SuperSU binary - we are now on v2.02! many thanks to chainfire for the development and maintenance of SuperSU. also updated the TWRP image, which is bacon 2.7.1 or 2.7.1.1 i believe.... it's the one that's about 12mb. the 15mb version was the one that was giving me the problems i complain about below.
also added some more flashing options and made the sideload/install from device/fastboot options a bit more flexible instead of always following the same path.
probably not even noticeable to most people, but for some reason TWRP hasn't been behaving all the time, especially with installing the SuperSU binaries for rooting. so instead of TWRP or stock recovery being the only options, i added ClockworkMod and Philz as options to whatever functions (flash and sideload) that use a custom recovery and were previously only able to use TWRP. i've tested rooting with the towelroot method on my samsung galaxy note 3 and it works perfectly, doesn't even trigger the knox 0x1 bit (as long as firmware is NE6 or earlier) and no reboot required. towelroot will not work for the oneplus one though - i tested the latest superSU (2.02) update on the OPO's latest update (33R) and it flashes successfully through Philz Recovery.
developing and running the script primarily on a Macbook Pro OSX 10.8 with Python 2.7.x, though sometimes i work on it in a Linux Debian environment. i have no clue how this script will run in Python 3, i could try it and find out, but i just don't care.
also new - check out the sweet ASCII art i added (even in oneplus' signature colors)! since it's the first thing you see when you run the script, you probably can't miss it. added because, obviously, it's an absolute REQUIREMENT that every terminal application (at least, the ones that matter) include some uber leet ASCII. like, the functionality doesn't even matter. handling exceptions doesn't matter (i don't do much of that here, by the way). all that matters is whether or not you have ASCII art, and how uberleet it is. otherwise, nobody will give a **** about you or your app. (at least, that's what i was told. mommy????)
in a future release there will be more functionality for other phones. i plan to add a script for deodexing, and maybe if i am not too tired i'll create a stock ROM with root already injected into it.
my github repository is still being a jerk and won't let me commit anything. sorry. keep checking my pastebin until then to get the latest updates:
http://pastebin.com/u/eudemonics
there might be some errors. i don't know. i thought i fixed the ones i came across. i really need sleep.
##################################################################
CHANGES IN v1.1:
- most files in script can be downloaded directly from script by demand to proper location, making it an easier install and a more seamless user experience
- added support and files for latest updates: XNPH33R released 8/22/2014, and XNPH30O updates #1 and #2
- can now flash entire factory stock ROMs - full XNPH30O and XNPH25R stock images - or flash your own custom ROM
- updated PYADB library to return STDOUT response instead of just a '0' success or error if not 0.
- reboot functions should behave a bit more sanely now that the piped STDOUT response can be used as qualifiers
- added more details and instructions in certain procedures, especially those having to do with booting into recovery.
- several root options available now: superSU is recommended for OnePlus One. TowelRoot is recommended for Android firmware releases earlier than June 2014. I also included a "superuser" zip file which is also supposed to be for rooting, but unless you are well acquainted with what it does, which devices it supports, and what to do with it, I would advise you not to try flashing it.
- i shuffled a lot of items around, it's very likely that there may be some syntax errors floating around. Please report any errors you come across to me at vvn (at) eudemonics (dot) org. thanks!!
~*~
NOTES FROM RELEASE v1.0:
MORE UNIVERSAL FEATURES TO SUPPORT ALL ANDROID DEVICES!
added a bunch of features to the pyadb python library, as well as including them in the toolkit. if there are any features you'd like to see in the toolkit, whether for the oneplus one or for android devices in general, please let me know and if it's possible, i'll add it! wiping and flashing options work now - you can choose the specific partitions you want to wipe, or flash everything back at once. i have also added the towelroot option, which should work on quite a few android devices as long as they are running firmware released before june 2014. also included a few APK's in the package that aren't available from the google play store, so you can install 'em if you want. tested most functions on my macbook pro (running OSX mountain lion) and they all seem to be working, except the uninstall APK function, but it's not a problem with the code, it's my failure to grasp what the <package> parameters should be. also, the latest OTA updates (30O) aren't installing from the fastboot update method, so i've added other options to install them such as ADB sideload and through custom recovery. i haven't tested the sync, wipe, restore, or unlock bootloader functions yet. they should all be working though, except maybe the sync (as i don't completely understand the ADB command myself. i just know the sync command alone without any args syncs your device's /system and /data directories).
one thing that may or may not work sometimes is running the reboot function from the toolkit while you're in recovery mode. i'm not sure if it's even possible to send a command from the computer to make it reboot while in recovery mode, though it shows up under "adb devices" as device type "recovery". i've added to the dialogue of the script directions to carry out certain functions from the device, such as rebooting from recovery back into android. also, the latest OTA updates (30O) *might* not install from the fastboot update method, so i've added both the ADB sideload method as well as installing updates from a custom recovery.
USE AT YOUR OWN RISK. i'm not responsible for anything that might happen to your device as a result of using the toolkit. if you use the correct files in the package and follow the toolkit directions onscreen, your device should be just fine. i assume if you have the technical know-how to install python, the android SDK, and run the script, then you've got a pretty good idea of what you're doing already. if you need help, send me a message. the best way to reach me, to get the quickest response, is probably on facebook, as i do not log onto xda-developers or my email accounts regularly:
http://www.facebook.com/dreamcorporation
requirements: python 2.7, android SDK, opotoolkit.py, pyadb.py
you can either supply the files referenced in the script, or download them from my site. links are in the opotoolkit.py source code pastebin link.
opotoolkit.py source code: http://pastebin.com/ciAj8NJy | download: http://notworth.it/opo/opotoolkit.py
pyadb.py source code: http://pastebin.com/g2Z08JN1 | download: http://notworth.it/opo/pyadb.py
for your convenience, i put together a ZIP file with most of the files you need - scripts, superSU, recovery images, apps, adb/fastboot binaries, etc. all that's missing are the stock images referenced in option #8.
download package here: http://notworth.it/opo/android-1plus1-halfassedtoolkit_v1.zip
there are also a couple large ZIP files available if you want to use the flash stock images functions in option #8 - links are in the source code for opotoolkit.py. or you can just download the XNPH25R stock firmware file from the official cyanogenmod page and unzip the contents into a subdirectory called "XNPH25R" of the script location, and the OTA updates for XNPH30R can be placed into another subdirectory of the script location called "XNPH30O". i will eventually add the functionality to download all the files directly from the toolkit itself, to make it easier for you guys so you don't have to scramble all over the place downloading and collecting files.
any APK's you'd like to install from the toolkit to the phone should go into the /apps subdirectory (already included in the ZIP package file along with a few apps). i have created the github repository for the project, but github is still refusing to acknowledge its existence on the web. once i get it sorted out i'll add a git link. keep checking this space as well as the pastebin links for any updates!
##################################################################
installation:
download and install python 2.7: https://www.python.org/downloads/
download and install android SDK: https://developer.android.com/sdk/
download toolkit package: http://notworth.it/opo/android-1plus1-halfassedtoolkit_v1.zip
-or-
download the python scripts/copy + paste source code from pastebin links into a text editor:
opotoolkit.py source - http://pastebin.com/ciAj8NJy | download: http://notworth.it/opo/opotoolkit.py
pyadb.py source - http://pastebin.com/g2Z08JN1 | download: http://notworth.it/opo/pyadb.py
extract files from the ZIP package or save the *.py scripts to a new directory, "opotoolkit" or whatever you want to name it.
it should work with the adb and fastboot binaries provided in the ZIP, but if not, you should install the android SDK. i recommend even MORE that you create an environmental path variable to the android SDK so you can run the commands from any directory.
setting up an environmental path variable (optional - recommended):
if you're on windows you can go to my pastebin, find the only powershell script on there, and steal/adapt the code to create your own environment path variable. but it's much easier to configure in system properties - i'm not on windows right now so these may not be exact instructions, but you should be able to right click on "my computer", select "properties", go to the "environment" tab in system settings, and add the environment path there. linux and OSX users just need to add the android SDK directory to their ~/.bash_profile or ~/.bashsrc or wherever environment paths are defined. if you still don't understand environmental path variables or symbolic links, i highly recommend google (or startpage.com, the private version).
if you don't want to go to the trouble of creating the environmental path variables, and you want to use the android SDK on your computer, then just extract all the files from the ZIP into your android SDK directory.
##################################################################
how to run the toolkit:
plug phone to computer via USB, turn on android debugging.
open command prompt or terminal window to scripts directory. start toolkit by entering:
"python opotoolkit.py"
if everything is installed and in the right places, you should see a menu like the attached screenshot. if for some reason you get a permission denied error, try launching the command prompt or terminal as administrator or superuser.
i'll continue working on it and adding more when i can so keep checking this space. everything's open source; use, share, steal whatever you want from the code. some credit would be nice, though.
you can use the pyadb.py library to incorporate adb/fastboot commands into your own python projects. i'll be adding more features to that, but for now most of the common features are covered.
here's the github link - it still won't acknowledge my repository exists, and if i try to create it again locally it gives me a fatal error. so it's not letting me add or commit any files, since it says the repository doesn't exist - though trying to create the repository gives me the warning that it already exists. [[email protected]#[email protected]#] let's just say github and i are not exactly BFF's.
project github home whenever it decides to start working: https://github.com/eudemonics/1plus1toolkit
i've been working to expand this so it can be used with all android devices, though it'll require people to supply their own device files to use with it! just because the toolkit can flash device firmware DOES NOT MEAN that the included files meant for the oneplus one will work on a NON oneplus one device! hopefully that's not something i needed to emphasize. if there is enough positive feedback and support i'll create a GUI since people like to click things more than type into a terminal window
my half-assed oneplus one toolkit was featured on cyberwarzone!
http://cyberwarzone.com/android-toolkit-python-2-7/
not gonna bug you for donations - all my code is open source - if you'd like to donate, please do so by purchasing my EP. hey, you might even like the music, too:
buy it here: http://dreamcorp.bandcamp.com
or search for it on itunes, google play, amazon, spotify, last.fm, cdbaby, and so on.
there are also links in the description for my video:
https://www.youtube.com/watch?v=2i-F4jiKtGg
##################################################################
special thanks to chainfire for superSU, cyanogenmod for CM11S, geohot for the towelroot exploit, oneplus for manufacturing such an excellent and affordable product, and everyone here on xda-developers for growing and developing the android community into the vital and creative force it is today!
XDA:DevDB Information
Half-Assed One Plus One Toolkit v1 in Python/ADB-python library - ALL OS' SUPPORTED!, Tool/Utility for the ONEPLUS ONE
Contributors
eudemonics
Version Information
Status: Stable
Current Stable Version: 1.2b STABLE
Stable Release Date: 2014-08-24
Current Beta Version: 1.2.3 BETA BLOCKERS
Beta Release Date: 2014-08-22
Created 2014-08-18
Last Updated 2014-08-25
Reserved
Reserved
Questions? Use Q&A!
Please read the FAQ before reporting any bugs or errors!
If you post in the main thread not having read the FAQ or error message itself, not included a debug log when reporting a malfuction or reporting a Force Closure without a logcat, your post will be ignored by the developers!
Not because we are evil, but because the same questions keep popping up over and over again and too often we get a "X doesn't work, plz fix" without any clue what is happening. We don't have telepathic connection to your device and all the time unnecessarily wasted on this can't be spend on development of Open GApps itself.
The Latest builds of Open GApps for Android can easily be downloaded from the:
Open GApps Homepage -> All architectures & download options
Open GApps App
FAQ @ GitHub
Package comparison @ GitHub
Advanced Features and Options @ GitHub
Development Repository @ GitHub
I work on this project for FREE and putting in a lot of hours into it. While not mandatory, donations encourage me to continue to further pursue this project and I'd deeply appreciate them, if you feel generous.
Donate to The Open GApps Project
Are you a ROM developer and want to hotlink to the latest Open GApps package? Then check this wiki entry for details.
Please don't publicly mirror the prebuilt packages without explicit consent of @MastahF, to ensure that users will always be directed to the very latest version and the source code of the project.
About The Open GApps Project
Open GApps is a Google Apps package completely developed by writing buildscripts which allow for the automated creation of new up-to-date packages automatically.
The development process is completely open-source (GPLv3) and the goal is to have multiple contributors involved, to secure and reinforce the sustainability of Open GApps development.
Builds are generated every (European) night automatically (if there are any changes) and uploaded to GitHub.
Official AROMA Open GApps package is developed in collaboration with long-time LP-AROMA-developer @raulx222 and has a dedicated XDA thread
For any questions about the AROMA installer development, please refer to that thread. Of course, general support questions can also be asked in our own Q&A thread.
Official Open GApps For Stock support is developed in collaboration with @Rapper_skull and has a dedicated XDA thread
For any questions about the GApps for Stock development, please refer to that thread. Of course, general support questions can also be asked in our own Q&A thread.
The x86 package branch of the package is focused on Zenfone support and is maintained by @deadman96385 of the famous Zenfone GApps packages and has its own topic for x86 related questions
For those that cook their own ROM, an AOSP-build mechanism for Open GApps has been developed by @blystad and can be found at GitHub, remember that you should not bundle any pre-packaged Google Apps with any ROMs you want to distribute further though.
To gather all the various APKs that are necessary for the packages our master of the APK Universe @MNBooZe has written a tool called APKCrawler that scrape these from the internet, e.g. from APKMirror, it can be found at GitHub too.
Characteristic of Open GApps:
Some highlights about the characteristics of the Open GApps packages:
All platforms and and all Android versions are supported
DPI-optimized support for all Google packages (unlike other GApps)
Frequently updated Google Apps: The pre-built OpenGApps.org packages are updated every (European) night (if there are any updated Google Apps available)
Strong compression, allowing for relatively small downloads of even the most complete packages
Automatic backup: It is not necessary to re-flash Google Apps when you flash a ROM update. Most ROMs support this (addon.d) function
The installer checks your device’s capabilities, like the system partition size. It will notify you, before making any changes, if it finds any problems
Several package variations, from a Google Super Package (includes all applications that ever shipped on a Google device), to a Stock package that equals the set of applications found on the most current and complete Nexus, to smaller, minimalist packages and an AROMA package that allows graphically selection of what to install
A special ‘for Stock ROM’ installation mode that allows to update the Google Apps on Stock ROMs that conform to the original Google Nexus filesystem structure
All package installations can be customized to your individual preferences using our Advanced Features and Options
The idea behind this project:
I believe a big source of the problem for many GApps packages to stay up-to-date (or not be forfeited) is the lack of time for developers to do labour-intensive repetive every time a new google-app apk is released.
That is why I have taken it upto myself to write some Linux shell scripts to automate the packaging and to share these efforts with the world with the goal to create a team to continue this package together under the name Open GApps.
This project should not be managed by a person, but by a team, so volunteers willing to help are more than welcome!
Team management and projectleader: @[COLOR="blue"]MastahF[/COLOR]
Writing of the scripts: @[COLOR="blue"]MastahF[/COLOR] & @[COLOR="blue"]Rapper_skull[/COLOR] or check the team on GitHub and many other contributors
Updating GApp sources: @[COLOR="blue"]MNBooZe[/COLOR] & @[COLOR="blue"]DJAlik[/COLOR] & @[COLOR="blue"]bgiesing[/COLOR] & @[COLOR="blue"]mc1100[/COLOR] & @[COLOR="blue"]deadman96385[/COLOR] or check the team on GitHub
AROMA installer: @[COLOR="blue"]raulx222[/COLOR] using LibAroma
Custom compiled commandline tools: @[COLOR="blue"]YashdSaraf[/COLOR]
Website: @[COLOR="blue"]raulx222[/COLOR] & @[COLOR="blue"]MastahF[/COLOR] & signalv hosted by GitHub and created with Material Design Lite and JQuery
Artwork: @[COLOR="blue"]Yeti12[/COLOR]: Yeti-Designs website
Documentation: @[COLOR="blue"]Trafalgar-Square[/COLOR] and @[COLOR="blue"]MastahF[/COLOR] and many other contributors.
Open GApps installer uses open source third-party tools, like busybox and xzdec, compiled by @YashdSaraf; See his busybox thread for more info.
Open GApps is originally based on the now discontinued PA GApps package of @TKruzze and @osm0sis
I suggest to @hellowasif and @sir*mez to take a look at this
Hi, @provolinoo suggested me to take a look at this topic.
I made my own GApps, derived from PA GApps by TKruzze and without knowing I solved some problems that the people that were trying to continue the PA GApps are having.
I completely removed the sizes.prop file and now I measure the sizes of the apps on the fly using unzip. I took a look at your scripts and they're really useful, and reminded me that the APKs need to be zipaligned, which I forgot.
Maybe we can join your scripts, my changes, and a fast internet connection to bring PA GApps back to life.
If you're interested here's the link to my topic, please take a look at it: http://forum.xda-developers.com/android/general/alpha-gapps-stock-t3093389
Well i wanted to say thankyou so much for such wonderful work you did, Before this i was maintaining the PA-Gapps Packages PA Gapps 4.4.4 and PA-Gapps 5.X everything was going great and i must say i learned a lot by maintaining these Gapps packages and that was very wonderful experience but i was only updating Gapps and Libs, But than i realized that they weren't as perfect as the original gapps packages because the original Gapps package uses two important files which were size and libs list which the packages uses and they were the most important part of PA-Gapps. I tried to update these two files manually but they were two time consuming so i decided to drop this project due to lack of resources but by these scripts it will very easy to create Gapps Packages so i wanted to say thankyou again
Rapper_skull said:
Hi, @provolinoo suggested me to take a look at this topic.
I made my own GApps, derived from PA GApps by TKruzze and without knowing I solved some problems that the people that were trying to continue the PA GApps are having.
I completely removed the sizes.prop file and now I measure the sizes of the apps on the fly using unzip. I took a look at your scripts and they're really useful, and reminded me that the APKs need to be zipaligned, which I forgot.
Maybe we can join your scripts, my changes, and a fast internet connection to bring PA GApps back to life.
If you're interested here's the link to my topic, please take a look at it: http://forum.xda-developers.com/android/general/alpha-gapps-stock-t3093389
Click to expand...
Click to collapse
Hey, I looked at your work. Some things are indeed good improvements and I will try to incorporate them into my work if you don't mind.
I also looked at your sizes.prop solution, but honestly I don't like it that much, because although the calculation will be very exact, I don't think it is a good idea to unzip large files and pipe all this data through on our small little phones . I prefer to keep the sizes.prop estimations on the generating-side rather than on the execution-side.
I really would like you to be involved in the project, somebody else also already PMed on the forum, wanting to be involved. I described which tasks and roles are very welcome to be fulfilled within a joint team effort.
MastahF said:
Hey, I looked at your work. Some things are indeed good improvements and I will try to incorporate them into my work if you don't mind.
I also looked at your sizes.prop solution, but honestly I don't like it that much, because although the calculation will be very exact, I don't think it is a good idea to unzip large files and pipe all this data through on our small little phones . I prefer to keep the sizes.prop estimations on the generating-side rather than on the execution-side.
I really would like you to be involved in the project, somebody else also already PMed on the forum, wanting to be involved. I described which tasks and roles are very welcome to be fulfilled within a joint team effort.
Click to expand...
Click to collapse
Thank you for your appreciation. The files however are not extracted but I used "unzip -l" that lists the content of the archive with the file sizes. Keep me informed about this project.
hellowasif said:
Well i wanted to say thankyou so much for such wonderful work you did, Before this i was maintaining the PA-Gapps Packages PA Gapps 4.4.4 and PA-Gapps 5.X everything was going great and i must say i learned a lot by maintaining these Gapps packages and that was very wonderful experience but i was only updating Gapps and Libs, But than i realized that they weren't as perfect as the original gapps packages because the original Gapps package uses two important files which were size and libs list which the packages uses and they were the most important part of PA-Gapps. I tried to update these two files manually but they were two time consuming so i decided to drop this project due to lack of resources but by these scripts it will very easy to create Gapps Packages so i wanted to say thankyou again
Click to expand...
Click to collapse
Hi hellowasif,
would you be interested in collaborating then together with other people in a team to bring back PA Gapps using these scripts?
MastahF said:
Hi hellowasif,
would you be interested in collaborating then together with other people in a team to bring back PA Gapps using these scripts?
Click to expand...
Click to collapse
Yes that will be wonderful to work as a team and you count me in. :highfive:
Rapper_skull said:
Thank you for your appreciation. The files however are not extracted but I used "unzip -l" that lists the content of the archive with the file sizes. Keep me informed about this project.
Click to expand...
Click to collapse
Ah, then i misread your code, I will take a look at it then again. Anyhow, since the files in the package are static, I think at moment of generation is a good moment to get the file sizes
I have btw a question for you, a problem I was not able to resolve myself yet, even though trying a lot.
When creating the .zip-package to be signed and afterwards flashed, I am at the moment not using any compression (but use the -Z store flag).
If I use *any* kind of compression, the package refuses to flash at my phone (GT-i9300) with the message error executing update binary error.
I tried a lot of combinations, like using a different zip-application, compressing only the files outside META-INF etcetera, but nothing seems to work.
So my question is: how do you generate and sign your zip file? On which platform? With which application? With which parameters?
MastahF said:
Ah, then i misread your code, I will take a look at it then again. Anyhow, since the files in the package are static, I think at moment of generation is a good moment to get the file sizes
I have btw a question for you, a problem I was not able to resolve myself yet, even though trying a lot.
When creating the .zip-package to be signed and afterwards flashed, I am at the moment not using any compression (but use the -Z store flag).
If I use *any* kind of compression, the package refuses to flash at my phone (GT-i9300) with the message error executing update binary error.
I tried a lot of combinations, like using a different zip-application, compressing only the files outside META-INF etcetera, but nothing seems to work.
So my question is: how do you generate and sign your zip file? On which platform? With which application? With which parameters?
Click to expand...
Click to collapse
You will maybe laugh at my reply, but I simply use WinRAR, on Windows, with maximum compression. I do not yet sign the ZIPs because I wanted to generate my own private key instead of using the generic test-key. What you can try to do is update your recovery (if it's not updated) to see if the problem is solved.
dowloaded your GitHub project and ran through the scripts to create the signed zip file. So far everything is running smoothly. Did a full wipe. Great Job!
Question I have. Do you know why the com.android.vending is still installed in the user space (/data/app) vs system space?
Chrome doesn't seem to be working. It crashes every time I try to run it.
DJAlik said:
dowloaded your GitHub project and ran through the scripts to create the signed zip file. So far everything is running smoothly. Did a full wipe. Great Job!
Question I have. Do you know why the com.android.vending is still installed in the user space (/data/app) vs system space?
Click to expand...
Click to collapse
Thank you for your feedback, at the moment I was extracting the Play Store from a Nexus Image under the cryptic name of Phonesky.
I only found out just now that this Phonesky is the same as android.vending (the Play store) and updated this.
DJAlik said:
Chrome doesn't seem to be working. It crashes every time I try to run it.
Click to expand...
Click to collapse
Also thank you for your feedback. Using your feedback I discovered some nasty typos in my script which were breaking Chrome and Talkback.
I fixed those and will be uploading a new package later today. The fixes are already on github.
hellowasif said:
Yes that will be wonderful to work as a team and you count me in. :highfive:
Click to expand...
Click to collapse
Nice
Very happy to hear that! Tomorrow I will be heading for an island for a week, for holidays, so lucky for me, but unlucky for those interested in updates .
After that holiday I will set-up the basic infrastructure for the team.
I also thought about how to make packages for older android versions and I came up with the following solution that I will be then implementing next week:
*Within the SourceAPKs folder we will put some meta-data or an overlay that specifies the minimal SDK/API version that the package requires
*I will update the scripts in such a way that you can target various android versions, including such a sdk/api version.
*When building for a target, the highest version of the APK that still support that sdk/api version will be selected
*A seperate output directory will be created for each target
*All scripts will be executed from a single Makefile
E.g., the directory structure will look like:
SourceAPKs/18/com.google.android.apps.maps-9.8.1-908101124-minAPI18.apk
SourceAPKs/14/com.google.android.apps.books-3.3.41-30341-minAPI14.apk
Scripts/extract_sources.sh
Output/4.4/pa_unsigned.zip
Output/5.0/pa_unsigned.zip
Output/pa_gapps-4.4-20150504.zip
Output/pa_gapps-5.0-20150504.zip
Makefile
If anybody has any feedback on these ideas, you are welcome!
MastahF said:
Nice
Very happy to hear that! Tomorrow I will be heading for an island for a week, for holidays, so lucky for me, but unlucky for those interested in updates .
After that holiday I will set-up the basic infrastructure for the team.
I also thought about how to make packages for older android versions and I came up with the following solution that I will be then implementing next week:
*Within the SourceAPKs folder we will put some meta-data or an overlay that specifies the minimal SDK/API version that the package requires
*I will update the scripts in such a way that you can target various android versions, including such a sdk/api version.
*When building for a target, the highest version of the APK that still support that sdk/api version will be selected
*A seperate output directory will be created for each target
*All scripts will be executed from a single Makefile
E.g., the directory structure will look like:
SourceAPKs/18/com.google.android.apps.maps-9.8.1-908101124-minAPI18.apk
SourceAPKs/14/com.google.android.apps.books-3.3.41-30341-minAPI14.apk
Scripts/extract_sources.sh
Output/4.4/pa_unsigned.zip
Output/5.0/pa_unsigned.zip
Output/pa_gapps-4.4-20150504.zip
Output/pa_gapps-5.0-20150504.zip
Makefile
If anybody has any feedback on these ideas, you are welcome!
Click to expand...
Click to collapse
The idea of automating all the process is certainly good, but we have to keep in mind that things changes really fast. For example in an update Google can add a library to an application that previously didn't have one. We have to check if there's a lib folder inside the APK instead of assuming that all the APKs that now don't have libs will never have them. Also the idea of various app versions is very good, but also error prone. Maybe we can just put all the APKs in one folder and use the filename to determinate the app name and the minAPI (if we don't want to make a step further and read directly from the AndroidManifest.xml, using aapt).
MastahF said:
Thank you for your feedback, at the moment I was extracting the Play Store from a Nexus Image under the cryptic name of Phonesky.
I only found out just now that this Phonesky is the same as android.vending (the Play store) and updated this.
Also thank you for your feedback. Using your feedback I discovered some nasty typos in my script which were breaking Chrome and Talkback.
I fixed those and will be uploading a new package later today. The fixes are already on github.
Click to expand...
Click to collapse
latest commit:
$ sh make_package.sh
rm: missing operand
DJAlik said:
latest commit:
$ sh make_package.sh
rm: missing operand
Click to expand...
Click to collapse
You can ignore that error-message. I just delete all the temporary files of my text editor (ending with a ~) before packaging using this 'rm' command.
Rapper_skull said:
The idea of automating all the process is certainly good, but we have to keep in mind that things changes really fast. For example in an update Google can add a library to an application that previously didn't have one. We have to check if there's a lib folder inside the APK instead of assuming that all the APKs that now don't have libs will never have them. Also the idea of various app versions is very good, but also error prone. Maybe we can just put all the APKs in one folder and use the filename to determinate the app name and the minAPI (if we don't want to make a step further and read directly from the AndroidManifest.xml, using aapt).
Click to expand...
Click to collapse
Very good comments. I have been looking into the possibility of using aapt. With 'aapt d badging file.apk' I will be able to retrieve the API information and app/packagename from any apk.
Using this information I could make an add-script or an 'add-directory' that would scan any new apks and archives them automatically in a storage. This could help to maintain overview, and to automatically discard older versions of the app if the api-level is not changed. It could also allow for adminstation of x86 and arm64 packages in the tree.
So a 'storage' filetree could become in that case like:
/arm/packagename/api14/maps2.apk
/x86/packagename/api11/maps1.apk
/x86/packagename/api14/maps2.apk
My goal is also to indeed in the future to (more) automatically process the existence of a 'lib' and automatically incorporate into the output process. For this I will have (as within my goals set out for next week) to upgrade the process of generating the output folder. Because at this very moment the scripts still assume availability of (most of the) outputfolders to be there already.
MastahF said:
Very good comments. I have been looking into the possibility of using aapt. With 'aapt d badging file.apk' I will be able to retrieve the API information and app/packagename from any apk.
Using this information I could make an add-script or an 'add-directory' that would scan any new apks and archives them automatically in a storage. This could help to maintain overview, and to automatically discard older versions of the app if the api-level is not changed. It could also allow for adminstation of x86 and arm64 packages in the tree.
So a 'storage' filetree could become in that case like:
/arm/packagename/api14/maps2.apk
/x86/packagename/api11/maps1.apk
/x86/packagename/api14/maps2.apk
My goal is also to indeed in the future to (more) automatically process the existence of a 'lib' and automatically incorporate into the output process. For this I will have (as within my goals set out for next week) to upgrade the process of generating the output folder. Because at this very moment the scripts still assume availability of (most of the) outputfolders to be there already.
Click to expand...
Click to collapse
Exactly, in this way it would be possible to easily maintain arm64 and x86 packages too. The scripting is not complicated, it just requires some time. If you want I can contribute too (I had some experience with shell scripting). If the final scripts are not too long or complicated we can plan a Windows port too, if some future team member is not used to Linux.
We can also try to contact AP to request an RSS feed for new APKs on APKMirror.
EDIT: It seems that adding /feed to any URL will give you the corresponding RSS feed. Good to know.
Rapper_skull said:
Exactly, in this way it would be possible to easily maintain arm64 and x86 packages too. The scripting is not complicated, it just requires some time. If you want I can contribute too (I had some experience with shell scripting). If the final scripts are not too long or complicated we can plan a Windows port too, if some future team member is not used to Linux.
We can also try to contact AP to request an RSS feed for new APKs on APKMirror.
Click to expand...
Click to collapse
I've been using Cygwin on Windows and the scripts work great.
AUTOMATEROne command automation for most things
Well, being a ROM developer/maintainer, we spend time in uselessly cloning trees, giving build commands, going through build logs to see where the error is and in the end upload the files and manually get links. And there is no denying that all of this is time consuming. So I made a program to do all of this automagically for you
This program can do:
Trigger the build of your ROM
Send message about the build status
Upload the build and target files to any SFTP client
Send you the direct download links
Send the build log
If the build fails, abort other statements, and send you trimmed build logs
Clean everything and restore the server to the previous state
Can be integrated with Jenkins to provide 1 click builds
Some other miscellanous functionality
Now, moving on to how to use it. This needs you to clone the given source into the scripts folder, which should be located inside the ROM directory. Also, the script assumes that you have clone your ROM sources inside a folder placed in the home directory.
The script is executed by the command :
Code:
devicename="yourdevicebuildname" bash scripts/CI/main.sh
You need to add some stuff in the main.sh file located inside the scripts/CI folder before you can actually use this. They are:
Your telegram CHAT_ID, the place where you will recive all the info
Your BOT API KEY, make a bot through BotFather on Telegram and paste the key
Your SFTP client username, I prefer Sourceforge
Your SFTP client password, once again, I prefer Sourceforge
If you want some other client, you have to alter the rom.py and tgt.py where you can write the client name of your choice.
Also, this script is pre-configured for CygnusOS. So naturally, you will want it for your ROM, to replace the variables, you can just run
Code:
sed -i -e "s/cygnus/your_rom_name/g" $(grep -RI cygnus $(ls)) && sed -i -e "s/Cygnus/Your_rom_name/g" $(grep -RI Cygnus $(ls))
NOTE: CYGNUS GENERATES THE ROM ZIP IN THE ROM DIRECTORY ITSELF INSTEAD OF THE USUAL OUT/TARGET ETC ETC PATH, SO YOU NEED TO CHANGE THE ROM.PY AS FOLLOWS:
Code:
Find the "." and replace it with out/target/product/"+devicename+"/ "
Also, edit the localfilepath to "out/target/product/"+devicename+"/"+name
That's it guys! Now just take a deep breath and leave it all, the script will do everything for you and even remove the generated zip! Yeah, make clean if that's what you wanna hear :laugh:
Hit the thanks button if you find this helpful
Also, if you wanna support me, buy me a cup of coffee
PayPal
XDA:DevDB Information
AUTOMATER for ROMs, Tool/Utility for all devices (see above for details)
Contributors
Dhruvgera
Source Code: [url]https://github.com/Dhruvgera/custom_python_scripts[/URL]
Version Information
Status: Stable
Current Stable Version: v1.0
Stable Release Date: 2020-04-02
Created 2020-04-02
Last Updated 2020-04-02
Congratulations !!!
Great work
Ragy747 said:
Great work
Click to expand...
Click to collapse
:good:
Uwu
PERU
blitzfire3 said:
PERU
Click to expand...
Click to collapse
U Too
great work sir, and thanks.
Peru as heck