I got a uk glow worm on the 27th (2 days before release ) and one of the first things that I tried to do was to root with glownooter. Bad idea! I thought that I had bricked my device. I had to use a lot of trickery to recover from a loading screen lock-up and once I did the first thing that I did was backup my nook (Which i should have done before).
UPDATE!
Please try my new ROOT install pack HERE:
http://forum.xda-developers.com/showthread.php?p=34216660#post34216660
This can be used to root and install the most requested things of this thread in just one zip.
Here is a quick guide to most things you will need to do to get started. I will update this guide as I discover and build new modifications.
To Backup and Restore
Follow this guide. Please do this BEFORE any other tinkering!
http://blog.the-ebook-reader.com/20...-and-restore-nook-glow-and-nook-simple-touch/
Its important to check your backup before proceeding! Please listen to roustabout and dont skip this step... He knows what he's talking about
roustabout said:
I'd like to suggest an addition to the backup method that many folks are using - always test your restore, but dont test it (the first time) on your device.
Your backup file should be about 2 gig.
find a 2 gig or larger flash drive or sdcard and restore your backup image to that drive.
when you're done, there should be 8 partitions, as there were on your Nook to begin with.
If you can't get that working - you're not ready to root yet. Until you're sure you can restore, don't start making changes, please.
People turn up all the time having screwed themselves over by restoring a partial backup and not knowing it, or having restored only one partition from a complete backup and having blown out the partition table.
Click to expand...
Click to collapse
Thanks roustabout
To Root!
Make sure you use the CWM file suitable for your SD Card. I used "2gb_clockwork-rc2" because my card was 2gb+.
http://forum.xda-developers.com/showthread.php?t=1360994 (Thanks mali100)
Use WinImage with admin rights to restore CWM virtual hdd image to your SD.
Download tinynoot-1-of-2 and tinynoot-2-of-2
http://forum.xda-developers.com/showthread.php?t=1650593 (Thanks to eded333 and roustabout)
Put on CWM boot SD.
Install them in CWM back to back (I didn't bother with the restart in the middle as it should not make a difference considering the file content). After a restart you should have root access and an android launcher on your 1.2.0 Nook (among other files). If nook fails to boot one of the tinynoot files may have corrupted. Recover, Re-download and Retry!
To Add Apps
Using ADB to install apps is easy. Extract this to your C drive:
http://dl.dropbox.com/u/13673492/ADB + Fastboot + Drivers.zip
Navigate to the folder in a cmd prompt.
Drop your APK into the same folder and on your nook open the "adbwireless" app and enable ADB
That app will tell you what your nooks IP address is.
Then you can:
Code:
adb connect ip.address.of.nook:portnumber
adb install app_of_your_choice.apk
Setup ADB over USB
OK I have taken the liberty of building a quick driver mod to support your nook through USB. It works for me. First you need to have the android SDK if you don't already (sure you do but just in case ).
http://developer.android.com/sdk/index.html
Make sure you tick to install the android USB driver when the SDK is installed.
Browse to extras\google\usb_driver in your SDK folder (wherever you put it) and replace android_winusb.inf with my file:
http://dl.dropbox.com/u/13673492/android_winusb.inf
Next go to C:\Users\your_user_account\.android and replace adb_usb.ini with my file:
http://dl.dropbox.com/u/13673492/adb_usb.ini
In device manager, point google ADB driver to this and hopefully that should get you set up!
To test type
Code:
adb devices
Its working if you get something like this:
Code:
* daemon not running. starting it now on port ____ *
* daemon started successfully *
List of devices attached
[YOUR NOOK] device
And then try
Code:
adb install app_of_your_choice.apk
UPDATE
Install Multitouch Kernel With Overclocking
Install the CWM zip using your clockworkmod SD card
http://forum.xda-developers.com/showthread.php?t=1906507
:good: Thanks to johnjtaylor for discovering that this kernel works works.
Hopefully this more comprehensive guide will get others with this software to start playing around.
If this helps, be polite and say thankyou
Have you setup ADB yet? If you can connect with ADB and get a shell, you can execute a 'df' at the shell prompt to see how much free space is available in each partition. On my NST (no glowlight) apps seem to be installed in /data/app so see how much free space is there. On the NST, this appears to be the same partition that books purchased from B&N are placed in, so if you have a lot of books from B&N, you may have to archive some to install apps. Of course all this is going on the assumption that the NST Glow is similar to the NST in this regard.
David0226 said:
Have you setup ADB yet? If you can connect with ADB and get a shell, you can execute a 'df' at the shell prompt to see how much free space is available in each partition. On my NST (no glowlight) apps seem to be installed in /data/app so see how much free space is there. On the NST, this appears to be the same partition that books purchased from B&N are placed in, so if you have a lot of books from B&N, you may have to archive some to install apps. Of course all this is going on the assumption that the NST Glow is similar to the NST in this regard.
Click to expand...
Click to collapse
Thanks for replying. I'm actually working setting up ADB now. As for books I only just got my nook so all of my titles are epub format on an sd card so i wouldn't think it would be that. As soon as I get ADB set up I will post back my results incase it helps anyone else with this new software version.
Can you look in the documentation that comes with the reader for any reference to 'third party software' or 'GPL software'. They should list where to download / apply for the source code somewhere. Once we can see the source code we can compare it against the existing versions and identify any significant issues.
I set up ADB.
Plenty of space in all partitions including /data for the apps that I want. Managed to install through "adb install some_app_i_want.apk" so problem resides with the amazon app store. Not really an issue for me because I have a specific set of apps that I want and don't need to browse the app store.
I will try to work out what's wrong for others.
staylo said:
Can you look in the documentation that comes with the reader for any reference to 'third party software' or 'GPL software'. They should list where to download / apply for the source code somewhere. Once we can see the source code we can compare it against the existing versions and identify any significant issues.
Click to expand...
Click to collapse
Thanks. I'm looking for it now
No reference to GPL. Only references to third party software are to tell me that my warranty is no longer valid (no surprise there!)
Is there any other place I can find this info thats not the documentation?
loney01843 said:
No reference to GPL. Only references to third party software are to tell me that my warranty is no longer valid (no surprise there!)
Is there any other place I can find this info thats not the documentation?
Click to expand...
Click to collapse
Nothing obvious from the uk.nook.com website. On the US site the 'support' section links to terms of service which contain the links to the open source code (see http://www.barnesandnoble.com/container/nook_lnav.asp?pid=43307 and search for NOOK 1.1.5 OSS Release ), but I can't see an equivalent on the UK site. It's an oversight, but such things happen with a new product launch.
You can email them at [email protected] . The relevant paragraph from the US site is:
1. Notwithstanding anything to the contrary in this Agreement, certain components of the Software are licensed subject to the General Public License Version 2.0, a copy of which is attached as Exhibit A (the "GPL License"). You may not use these components except in compliance with the GPL License. In addition, you may have additional rights with respect to such components under the GPL License, including, without limitation, the right to obtain the source code for such components from us. You may obtain a copy of such source code by contacting us through the contact information provided on the Web Site. We will provide such source code in accordance with the GPL License.
I don't legally have the right to request the source code myself, because I don't own a UK NOOK yet. (Yeah, thinly veiled excuse for laziness!)
staylo said:
Nothing obvious from the uk.nook.com website. On the US site the 'support' section links to terms of service which contain the links to the open source code (see http://www.barnesandnoble.com/container/nook_lnav.asp?pid=43307 and search for NOOK 1.1.5 OSS Release ), but I can't see an equivalent on the UK site. It's an oversight, but such things happen with a new product launch.
You can email them at [email protected] . The relevant paragraph from the US site is:
1. Notwithstanding anything to the contrary in this Agreement, certain components of the Software are licensed subject to the General Public License Version 2.0, a copy of which is attached as Exhibit A (the "GPL License"). You may not use these components except in compliance with the GPL License. In addition, you may have additional rights with respect to such components under the GPL License, including, without limitation, the right to obtain the source code for such components from us. You may obtain a copy of such source code by contacting us through the contact information provided on the Web Site. We will provide such source code in accordance with the GPL License.
I don't legally have the right to request the source code myself, because I don't own a UK NOOK yet. (Yeah, thinly veiled excuse for laziness!)
Click to expand...
Click to collapse
You're obviously not that lazy. Thanks for looking and gathering all of the extra info I need. I'll send B&N an e-mail and see what they say. I wouldn't be surprised if they didn't want to hand it out considering you can use it for an easy root setup and install the amazon and kobo stores which could financially damage their advance into new territories! What are we to do! Can't even subscribe to a newspaper or magazine through the nook store yet!
I'll let you know when / if I get a response
There is a setting in nook touch tools that you need to "arm," to allow software from unknown sources to be installed before the Amazon appstore can install software on a tinynooted device.
The setting is a tickbox, "Allow non-Market apps"
Untick it if it is ticked by default, then re-tick it to get apps to install.
roustabout said:
There is a setting in nook touch tools that you need to "arm," to allow software from unknown sources to be installed before the Amazon appstore can install software on a tinynooted device.
The setting is a tickbox, "Allow non-Market apps"
Untick it if it is ticked by default, then re-tick it to get apps to install.
Click to expand...
Click to collapse
Thanks for the reply but I actually tried that. No joy. However perhaps it is this that is not working and not amazon app store. Im just installing through ADB instead. I wonder if I can enable unknown sources through ADB. Something to look at I guess!
UK tinynoot attempt failing
loney01843 said:
Thanks for the reply but I actually tried that. No joy. However perhaps it is this that is not working and not amazon app store. Im just installing through ADB instead. I wonder if I can enable unknown sources through ADB. Something to look at I guess!
Click to expand...
Click to collapse
I tried using the tinynoot process from roustabout's thread here http://forum.xda-developers.com/showthread.php?t=1650593 and am stuck on the final reboot with a "Your NOOK is starting up..." message. Could you let me know if you used a different tinynoot method/set of files?
smerrett said:
I tried using the tinynoot process from roustabout's thread here http://forum.xda-developers.com/showthread.php?t=1650593 and am stuck on the final reboot with a "Your NOOK is starting up..." message. Could you let me know if you used a different tinynoot method/set of files?
Click to expand...
Click to collapse
Yep thats what I used. You on 1.2.0 and did you back up?
I didn't backup first and to get out of the starting message I used this:
http://forum.xda-developers.com/showthread.php?t=1289233&highlight=restore
Then I made a backup using this:
http://blog.the-ebook-reader.com/20...-and-restore-nook-glow-and-nook-simple-touch/
I hope this helps!
loney01843 said:
Yep thats what I used. You on 1.2.0 and did you back up?
I didn't backup first and to get out of the starting message I used this:
http://forum.xda-developers.com/showthread.php?t=1289233&highlight=restore
Then I made a backup using this:
http://blog.the-ebook-reader.com/20...-and-restore-nook-glow-and-nook-simple-touch/
I hope this helps!
Click to expand...
Click to collapse
Thanks for the tip on screen freeze but the link to the images on that post don't work for me. I am on 1.2 and made a backup before attempting any rooting - have managed to reinstate my original nook so quite pleased with myself.
Is there any point in rooting until someone can find a way of getting apps onto the 1.2 NSTG?
smerrett said:
Thanks for the tip on screen freeze but the link to the images on that post don't work for me. I am on 1.2 and made a backup before attempting any rooting - have managed to reinstate my original nook so quite pleased with myself.
Is there any point in rooting until someone can find a way of getting apps onto the 1.2 NSTG?
Click to expand...
Click to collapse
Great that you got a backup. If you want custom apps you can either wait for a different root kit or push ahead (since you have a safety net).
As I said, it worked for me.
If you don't mind searching for the .apk files you want you can use this:
Code:
adb connect ip.address.of.nook:portnumber
adb install app_of_your_choice.apk
This has worked fine for me so far. Just don't try for custom kernels yet as they seem to give me troubles.
I will work more on this tomorrow including adjusting framework for gapps.
loney01843 said:
Great that you got a backup. If you want custom apps you can either wait for a different root kit or push ahead (since you have a safety net).
As I said, it worked for me.
If you don't mind searching for the .apk files you want you can use this:
Code:
adb connect ip.address.of.nook:portnumber
adb install app_of_your_choice.apk
This has worked fine for me so far. Just don't try for custom kernels yet as they seem to give me troubles.
I will work more on this tomorrow including adjusting framework for gapps.
Click to expand...
Click to collapse
Thanks also for the code but as this is my first foray into rooting I think I'll hang around and watch for a bit! Perhaps if I start learning some more I may feel confident enough to try it.
Do you have copies of the files needed for the factory reset - the links are still not working for me.
Thanks again and sorry for bothering you. Hope tomorrow is productive for you.
smerrett said:
Thanks also for the code but as this is my first foray into rooting I think I'll hang around and watch for a bit! Perhaps if I start learning some more I may feel confident enough to try it.
Do you have copies of the files needed for the factory reset - the links are still not working for me.
Thanks again and sorry for bothering you. Hope tomorrow is productive for you.
Click to expand...
Click to collapse
http://dl.dropbox.com/u/13673492/n2T-Recovery_0.2.img
This is the file needed to force factory reset. However a quality backup like you have is far more important.
For anyone who wants to give this a go, here is a quick guide for root access and app installs using windows tools until I can make something more complete:
Make sure you use the CWM file suitable for your SD Card. I used "2gb_clockwork-rc2" because my card was 2gb+.
http://forum.xda-developers.com/showthread.php?t=1360994
(Thanks mali100)
Use WinImage with admin rights to restore CWM virtual hdd image to your SD.
Download tinynoot-1-of-2 and tinynoot-2-of-2
http://forum.xda-developers.com/showthread.php?t=1650593
(Thanks to eded333 and roustabout)
Put on CWM boot SD.
Install them in CWM back to back (I didn't bother with the restart in the middle as it should not make a difference considering the file content). After a restart you should have root access and an android launcher on your 1.2.0 Nook (among other files). If nook fails to boot one of the tinynoot files may have corrupted. Recover, Re-download and Retry!
Using ADB to install apps is easy. Extract this to your C drive:
http://dl.dropbox.com/u/13673492/ADB + Fastboot + Drivers.zip
Navigate to the folder in a cmd prompt.
Drop your APK into the same folder and on your nook open the "adbwireless" app and enable ADB
That app will tell you what your nooks IP address is.
Then you can:
Code:
adb connect ip.address.of.nook:portnumber
adb install app_of_your_choice.apk
Hopefully this more comprehensive guide will get others with this software to start playing around.
Click thanks if this guides helpful.
loney01843 said:
If nook fails to boot one of the tinynoot files may have corrupted. Recover, Re-download and Retry!
Navigate to the folder in a cmd prompt.
Drop your APK into the same folder and on your nook open the "adbwireless" app and enable ADB
That app will tell you what your nooks IP address is.
Then you can:
Code:
adb connect ip.address.of.nook:portnumber
adb install app_of_your_choice.apk
Hopefully this more comprehensive guide will get others with this software to start playing around.
Click thanks if this guides helpful.
Click to expand...
Click to collapse
Great - thanks to your more detailed instructions, I have persevered and the second attempt at installing the tinynoot zips worked. Your post gave me the confidence to try installing apks for the first time and for anyone else who is unfamiliar with the processes used, I found these two pages useful for:
navigating within a command prompt (how naive): pcstats.com/articleview.cfm?articleid=1723&page=3
Pasting text into a command prompt (ditto): megaleecher.net/Copy_Paste_Text_Dos_Window
Sorry, as a newb I'm not trusted to post outside links yet. It took a couple of attempts of pasting and pressing enter to work out how to install using the adb code but it's not as hard as I expected.
I have tried installing the 1 Mobile Market which works but it is unable to install apps itself (not enough space).
Also, I have just installed NoRefreshToggle and can't seem to get it to work. Any thoughts - is 1.2 to blame? Really want this to work as Business Calendar Free is just a series of blinks at the moment!
Great! I'm glad you pushed onward and have root.
I am going through possibilities of other ways to install apps and mods.
No refresh is something that I would like as well but I think that the framework may need editing for 1.2.0. For fast mode a new kernel will need to be made or existing one modified.
Be aware that installing kernels not designed for this firmware can give you some serious problems.
Once I can setup app markets, I will work on these other modifications.
Stay tuned :good:
Take a look at the overclock kernel - it's got a lot of the norefresh features baked in, and gives you a nice ability to control both clock and kernel tuning (via the governor control.)
You're right, you can flash the two zips back to back with no ill effect, I was mistaken about what the second zip was doing.
I mistyped when I typed "nook touch tools," I meant nook color tools.
I'm very interested that the amazon store is not working in 1.2. I will see if the software's available for my device, so I can try to see what's happened.
As of now, the us bn site does not mention an os 1.2 for the glowlight.
DERP (Device Environment Replacement Program)
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
initial pre-alpha version 0.001
(Aug 3, 2013)
by fattire (twitter: @fat__tire)
tldr?
Derp is a general-purpose, platform-independent installer, written in python with wxpython, that executes .derp XML-based scripts to walk the user step-by-step through a ROM installation (or do whatever you want.) The idea is to replace text-based walkthroughs, howtos, and installation instructions by requiring a user to do almost nothing but run a .derp script and sit back. Derp walks through a series of scripted steps (as in, say, a ROM installation) and automatically does all the file downloading/adb/fastboot stuff while the user waits and maybe reads what's happening (at the script author's discretion). Derp also pre-installs and keeps Google's Android SDK tools up to date and even gives adb & fastboot a simple UI. .derp scripts are human-readable XML. Embedding bash and python is also supported in Derp, as is restricting scripts or even script parts to certain platforms. Derp runs as root on the local computer (it's an installer after all), and is open source/GPLv3 licensed. It also comes with sample scripts and a built-in tutorial for creating your own.
Still tldr? It's a script-runner thing!
-----
LONGER DISCUSSION...
WARNING: RIGHT NOW, DEVELOPER TYPES ONLY! This is not for end users...yet. Hopefully people will find bugs and help fix them before an end user uses this on a “live” computer with an actual device. Again, because this is a software installer, DERP AND ITS SCRIPTS RUNS AS ROOT. Never run random .derp or .xml scripts you find on the Internet. This could screw up your device AND your computer, so... treat it just like any other script you’d (not) run as root. Also, the discussion in this forum is how Derp is supposed to work, but of course, there may be (probably are?) bugs.
THE "PROBLEM" AS I SEE IT
Working on the CM wiki, I've grown to appreciate how varied firmware install methods can be. Some devices need rooting. Some need firmware downgrades. Sometimes you can use fastboot. Other times you can’t. Some systems need to unlock the bootloader, etc. etc.
Installing this stuff can be hard. Okay, maybe not for you, but how about your mom or dad? Could your grandparents buy a device today and put CM on it themselves? There's been some chatter on the interwebs about how to make rooting and replacing firmwares easier... some kind of graphical installer seems to be the answer. But there are a million devices out there...
So people have been using text-based HOWTOs, walkthroughs, step-by-step instructions, and/or shell scripts and batch-type files to do this. I thought maybe a generic, unified scripting method might work better that gives the users readable instructions but optionally automatically does technical steps for them.
Hoping to avoid creating yet-another-standard-way-to-do-something, back in February, I searched online for generic installation solutions. But they all seemed to be platform-dependent, or weren’t licensed for general use, or looked really ridiculously complicated....
So (and big caveat here-- I'm not a programmer!) I whipped up a proof-of-concept for developers to play with to start thinking about how to address the issue. It took me a few weeks to get going, but ended up sitting collecting dust for months as I worked on other things and occasionally bothered friends to test the latest version.
Derp is not necessarily intended as any kind of final solution per se-- it’s just for further discussion/testing. A totally legit question to be answered: is this in any way even a good idea?
Let's find out.
SO WHAT IS DERP?
It's a general-purpose installer, written in python with wxpython, that executes .derp XML-based scripts to walk the user step-by-step through a ROM installation, optionally doing all the "technical stuff" like downloading files and running commands in the background. Ideally, you wouldn't have to write a long tutorial for every platform on how to do stuff-- a .derp script could BE the walkthrough.
The first thing Derp does is install the latest adb and fastboot from Google. That looks like this:
(Mac version)
Next, when you run a .derp script, it can automatically download and verify ROM files like CyanogenMod or tools or whatever from the Internet, and then install them.
(Linux version)
(Mac version)
As it does all this, the script can provides information and/or feedback to the user via a UI that hopefully looks like a normal installer. What the user sees is written by the script author in simple HTML. As the .derp script runs, the user simply ftaps "Continue" to proceed through the scripted steps.
As mentioned, .derp scripts are written in XML, which is platform-generic and easy for a human to read. The .derp script syntax, explained below, is also very simple. The script author is also free to embed bash shell scripts or python (or both) if advanced stuff is needed.
Worth mentioning too-- sections, steps, text, actions, or entire scripts can be restricted by the type of computer its running on (ie, don't run certain python commands on Mac, but do run them in Linux, or whatever).
And finally-- while my initial thought was to use this for installations of ROMS like CyanogenMod to a device, I'd think Derp can be used for many kind of installations or scripted operations-- even to wrap a UI around a bash or python script to make it easier for users to run without having to open a Terminal and start typing. Derp scripts don't even have to have to be used for anything to do with mobile devices, though it does pre-set up the Google SDK tools for that purpose.
In fact, a Derp script can do NO actions-- simply serve as a click-through set of HTML-based instructional steps for a user to follow by hand. Conversely, it can say nothing to the user but "Stand by, doing everything." and that's it.
FEATURES:
Easy to install. Debian-based Linux just uses sudo dpkg -i derp_0.001-1_all.deb and it's ready to go. Mac users: it’s Derp.app. Done and done.
Derp is GPLv3-licensed and source code is available now. Read license for terms, conditions, and more disclaimers.
Automatically downloads/updates all SDK tools (primarily adb and fastboot) directly from Google at every launch. So the user is always up to date. (also requires users to agree to Google's T&C...)
Uses an XML-based, OS-neutral installation script format that is easy to write and understand. Just about anything you want the user to do-- restart in bootloader mode, unlock the device, etc-- the derp script should be able to do. Even run bash or python scripts from within the script.
XML Tags:
<derp> - the main tag for a derp script.
<section> - a major category for individual steps.
<step> - Put as many of these in a section as you want.
<info> - The stuff the user sees as the script runs. You can add HTML tags to make it look good.
<file> - tell derp a file’s URL, MD5/SHA hash, and local filename. Derp will grab it and verify it for you automatically. These files can be roms, scripts, recovery images, etc. whatever your script needs to do its job.
<action> - valid “action” types include “adb”, “fastboot”, “python”, and “bash”. Future versions of Derp can add more. <action> allows your script to do stuff. Never worry about whether the user installed and set up the latest versions of adb or fastboot properly. They should "just work".
Using the above tags, you can not only have your scripts automatically do full installations, rooting, bootloader unlocks, etc, but simultaneously tell the user what’s happening behind the scenes if you choose. The user feedback is written in standard HTML-formatted text. The user just hits “Continue” whenever you want to move from one step to the next.
Included are example scripts to install CyanogenMod 10.1.2 on stock Nexus 7, Nexus 4, Galaxy Nexus, and HTC One. The latter script, written by Cowmix, demonstrates how to embed python to interact with the user, and they all include bootloader unlocking.
The only things I can think of that can’t be done automatically are steps that requires hands-on (ie, holding down buttons during power-on) or where, say, debugging mode needs to be manually turned on, or the slider needs to be physically unlocked. In the few cases where user involvement can’t be avoided, the <info> tag can be used to walk them in “real time” through that step.
A built-in tutorial on how to write your own .derp scripts explain how the tags work. (The tutorial itself is a .derp script.)
A console window helps you see what derp is doing in real-time...
Also included: a quick-access adb and fastboot text-entry in the console. This lets you start up Derp and type quick adb or fastboot commands without needing a terminal (or to deal with PATH issues)
“Debug Mode” lets you go through the script without invoking the <action> tags. Makes writing scripts easy.
Derp should automatically detect when a device is connected via adb or fastboot and let you know.
You may filter any Derp tag (including <action> tags) by operating system. This means that using a single script, the user can see different text or the script may behave differently depending on the platform. In fact, you can restrict the entire script to a particular operating system(s).
The script doesn’t actually have to “do” anything. It can be used simply to create walkthroughs or tutorials in a much nicer format than a step-by-step text file. Just link to a .derp file and let it walk the user through whatever. Easy to convert a text walkthrough to an interactive click-through just by adding <section> and <step> tags.
WHY MUST DERP RUN AS ROOT?
Remember, Derp is an installer. It needs to do important stuff, and as such it runs as root. I had considered trying to sandbox the parts that "needed" root and only enable it there and ask for permission for a single operation via an "enter your password" type of dialog box. But because the .derp format is so flexible, there were a million potential places where a script author could do varying kinds of trickery-- by breaking out of Derp to execute python code, spoofing directory paths, abusing the embeddable bash scripts, etc. It just didn't seem to make sense to try to anticipate and counteract all that. Playing cat-and-mouse endlessly is pointless. Again, Derp is an installer. Installers get administration permissions. Just like any installation script you'd run with "sudo" would get. Just like the package installer on OS X. Also, it is much easier to run adb and fastboot with root permission-- you can easily kill all running versions of adb for example, and fastboot seems to prefer it. Plus, it avoids the need for playing with udev configuration stuff in Linux.
This means that, like pretty much every other type of installer, .derp scripts will have full access not only to your mobile device, but to the computer Derp is running on. This seems to make the most sense to me, but I invite others to chime in on improving the design if you disagree.
All caveats and considerations apply. Do not run untrusted scripts, and do not run Derp on a "sensitive" computer (however you wish to define that).
WHAT IS MEANT BY "CROSS-PLATFORM"? (IN OTHER WORDS, WHERE'S THE WINDOWS VERSION?!)
I don't have/use Windows. Right now, Mac & Linux builds are currently available. Derp still needs to be ported to Windows, but since it’s wxpython, and I tried to make as little dependent on the underlying operating system, 95% of the work is hopefully done. Anyone with Windows who’d like to help, let me know.
I think it should be some minor changes to the setup.py file and a few definitions. Also, not sure if Windows supports the “bash” shell...
ANYTHING ELSE?
Ummm... That’s it. Remember, this is a work in progress and a proof of concept... Again, I dunno if anyone will see the value here, and maybe it will need a complete rethink. There are likely to be bugs, maybe even really bad ones. But after a few months of playing around, I kinda feel it’s ready for other developers to at least see and even try in a secure environment (as suggested, maybe a VM or something).
SHOW ME THE CODE!
The code is on github-- please submit commits-- fixes, new features, whatever-- as well as bug reports there. And again, figure there will be tons of bugs to be squashed.
Enjoy.
fattire (@fat__tire)
THANKS TO...
Big thanks for helping me test w/different devices: cowmix, hashcode, kornyone, ciwrl, utkanos, verygreen, and jeagoss
DOWNLOADS:
Debian-based Linux (Debian, Ubuntu, Mint)
derp_0.001-1_all.deb
MD5: 6e8eabe94cdfdba649ea41198211bb64
SHA512: 307aed0ad79de17793bb445d2b588388bf66b42716de36a055227f555bfc12ab3e61d5f0e3de804eb4c0c560f140a6318ea6dd1608cc78ee84b50336895cdfc2
Mac OS X (Tested on: Snow Leopard, Lion, Mountain Lion)
Derp-v0.001-mac.zip
MD5: b738e0a270f53d274baec0ce121577fb
SHA512: 3cf7d438c4dfd0c5c5d7c2f29fe19a76dcbb728acfe73a24e28cdb3f21624510c94f1c4224ad31118851f17205e4d7152619c15281c98189cb33ccac82c1505a
Source code on GITHUB is here.
SAMPLE DERP SCRIPTS
Nexus 7 stock to CM 10.1.2 installer - included (written by me)
Nexus 4 stock to CM 10.1.2 installer - included (written by me)
HTC One stock to CM 10.1 nightly installer - included (written by cowmix)
Galaxy Nexus stock to CM 10.1.2 installer - included (written by cowmix)
EXTERNAL SCRIPTS BY OTHERS
None yet...?
DONATIONS?
Not to me, please. If you feel the need to give someone money, consider donating to the EFF or the Software Freedom Law Center. It's really a donation to your digital rights. (I'm not affiliated with them except as a huge fan and occasional donor.)
REMEMBER, DERP IS EXPERIMENTAL AND YOU RUN IT AT THE RISK OF YOUR COMPUTER, YOUR DEVICE, AND YOUR VERY EXISTENCE AS A HUMAN BEING. I TAKE NO RESPONSIBILITY FOR WHAT DOES OR DOESN'T HAPPEN. DON'T RUN DERP SCRIPTS YOU DON'T TRUST COMPLETELY. YOU ARE ADVISED, JUST IN CASE, TO ONLY RUN SCRIPTS IN A SANDBOXED VIRTUAL COMPUTER. And let me know what y'all think.
Script Syntax (Tutorial)
SCRIPT SYNTAX
So you want to write a Derp installation script? It's easier than you might think. Derp isn't too complicated-- it doesn't have a lot of "logic". It just follows a script and does what you tell it.
To start a script file, just get out any text editor (or XML editor) and name it something with the .derp file extension, such as:
sample.derp
Once you write up a sample script, you can load the file with Derp to see if it works.
The <derp> tag
Every script starts with the <derp> tag and ends with the </derp> tag. Within the "<derp>" tag, at least for this pre-alpha version, you need to put at least one required attribute, app_version:
<derp app_version="0.001">
</derp>
This is to identify the version of Derp that your script is for. Future versions may not support your script. You can put other attributes that might be used in the future:
<derp device_codename="mako" os="Linux Darwin" title="CM10.1-M1 for Mako" device_name="Nexus 4" device_vendor="lge" app_version="0.001" script_version=".5" author = "fattire" author_email="[email protected]" author_twitter="@fat__tire" license="GPLv2">
These additional tags may be required in future versions of Derp, so if you are able to supply 'em, it's recommended. They'll simply be ignored if they're not needed.
The title="CM10.1-M1 for Mako" is a general title for the script. VERY briefly explain what it does. It's not required, but recommended.
The one other important attribute, os="Linux Darwin", will be explained later. For now, just know that it is optional, but you can use it to restrict the whole script to only run only in certain operating systems.
The <section> tag
Every set of instructions should be divided into logical sections, such as the ones on the left. The section has its own required attribute, the name:
<derp app_version="0.001" os="Linux Darwin" script_version=".5">
<section name="This is the first section"></section>
</derp>
Notice the name attribute is used with a section to identify what the section is for.
There's not much more to say about sections. It's easy. Let's move on.
The <step> tag
Each Section can be made of (at least one but) an unlimited number of individual steps. And the tag for that is called <step>. Here's how it's used:
<derp app_version="0.001" script_version=".5">
<section name="This is the first section">
<step name="This is step one"></step>
<step name="This is step two"></step>
</section>
<section name="This is the second section">
<step name="This is step three"></step>
<step name="This is step four"></step>
</section>
</derp>
Notice that steps, like sections, need to have a designated name attribute so that Derp knows what to display. The step name will appear to the user at the top on the right as the centered step heading.
The <info> tag
The stuff that appears in the main info area should be wrapped in info tags.
Example:
<derp app_version="0.001" script_version=".5">
<section name="This is the first section">
<step name="This is step one">
<info>This is the text you'll see! It explains what's going on to the user. <b>I'm bolding this part because it's really important for the user to see.</b></info>
</step>
</section>
</derp>
Note: The stuff that you put between the <info> and </info> tags is...HTML!
So you can format it however you want. You can even include images from the Internet.
Here is the list of HTML tags that are recognized:
A NAME=[string]
HREF=
TARGET=[target window spec]
ADDRES... can add os="Linux Darwin" to the <derp> tag.
RESERVED
RESERVED
Derp
Derp is a pretty slick interface for scripting not only device installation, but resources needed for modifications on Android devices (namely the Android SDK). A developer can create a custom script to automate the installation, decreasing one off bad installs, and ensuring the process is completed as intended.
People new to Android customization or developers could find this of use. I am excited to see where it goes.
kornyone said:
Derp is a pretty slick interface for scripting not only device installation, but resources needed for modifications on Android devices (namely the Android SDK). A developer can create a custom script to automate the installation, decreasing one off bad installs, and ensuring the process is completed as intended.
People new to Android customization or developers could find this of use. I am excited to see where it goes.
Click to expand...
Click to collapse
Thanks.. BTW for those asking about the Windows port (in IRC)...
I simply don't have windows, but it was written to be as platform generic as possible. Anyone with a tiny amount of programming skills (again, I have zero myself) should be able to add Windows compatibility pretty quickly... I think it's a matter of just fixing that setup.py file to work with py2exe. See here for more info.
fattire said:
Thanks.. BTW for those asking about the Windows port (in IRC)...
I simply don't have windows, but it was written to be as platform generic as possible. Anyone with a tiny amount of programming skills (again, I have zero myself) should be able to add Windows compatibility pretty quickly... I think it's a matter of just fixing that setup.py file to work with py2exe. See here for more info.
Click to expand...
Click to collapse
Windows dev here, I may be able to help. Also, any interest in a Mono version? Looking for an excuse...
fattire said:
Thanks.. BTW for those asking about the Windows port (in IRC)...
I simply don't have windows, but it was written to be as platform generic as possible. Anyone with a tiny amount of programming skills (again, I have zero myself) should be able to add Windows compatibility pretty quickly... I think it's a matter of just fixing that setup.py file to work with py2exe. See here for more info.
Click to expand...
Click to collapse
Grats BTW, great idea...
I've come across several usages of Linux only Python functions so far and I don't see drop-in alternatives for Windows, so I've just commented out that particular section (line 1183). I managed to get the tool download working. Suggestion, maybe sticking with MD5 hashes would be simpler as the script receives updates to match Android SDK download updates. I can understand why you would want to use SHA512, but google offers MD5 on the site next the downloads for simple copy/paste replacement. There's the potential for lots of hard-coded configuration and for those configurations to be platform specific, such as the download folders for tool updates. I'll see if I can finish up the first bit of win compat this afternoon, but my Android device is at work and is a Dell Streak at that, so my test options are a bit limited.
http://docs.python.org/2/library/platform.html
1183 - os.geteuid()
1196 - os.uname()
fork:
https://github.com/strvmarv/derp
screen:
Windows... already?!!
Holy crap! I don't think it's been 12 hours and there's an early windows port.. amazing job!
The unix-only stuff was from a last second addition I did when I realized that dero would try to run on ARM-based linux machines. The easy fix is to simply indent everything past:
if platform.system() == "Linux"
so that the if not os.geteuid() == 0: and testarch = os.uname() stuff is conditional on it running Linux. (Unless there's a windows ARM version, in which case it also won't work).
In both cases it would work except for the fact that Google doesn't provide libraries for ARM. Interestingly though, debian does. So if we REALLY wanted, we could just apt-get install the tools for ARM Linux users. But that would (1) require a debian-based version of Linux, and (2) we wouldn't know that adb/fastboot/etc are the very latest from Google. But it might be a good version .002 feature, with a preference to turn it on or something.
Again, amazing work. Keep it up!
strvmarv said:
http://docs.python.org/2/library/platform.html
1183 - os.geteuid()
1196 - os.uname()
Click to expand...
Click to collapse
fattire said:
Holy crap! I don't think it's been 12 hours and there's an early windows port.. amazing job!
The unix-only stuff was from a last second addition I did when I realized that dero would try to run on ARM-based linux machines. The easy fix is to simply indent everything past:
if platform.system() == "Linux"
so that the if not os.geteuid() == 0: and testarch = os.uname() stuff is conditional on it running Linux. (Unless there's a windows ARM version, in which case it also won't work).
In both cases it would work except for the fact that Google doesn't provide libraries for ARM. Interestingly though, debian does. So if we REALLY wanted, we could just apt-get install the tools for ARM Linux users. But that would (1) require a debian-based version of Linux, and (2) we wouldn't know that adb/fastboot/etc are the very latest from Google. But it might be a good version .002 feature, with a preference to turn it on or something.
Again, amazing work. Keep it up!
Click to expand...
Click to collapse
Good deal, glad I could help. If you ever want to give a Mono/GTK# port a try just give me a shout. I could do the majority of the leg work code in C# very quickly, lightweight app, which is excellent these days.
I just pushed up my initial changes for the setup.py. I haven't figured it out yet, there are some imports, specifically in derp.py line 23 (platform) that aren't getting consolidated into the build with py2exe. It's most definitely how I've setup the options in the setup.py, hopefully someone is more familiar with py2exe than I and can provide some insight.
strvmarv said:
Good deal, glad I could help. If you ever want to give a Mono/GTK# port a try just give me a shout. I could do the majority of the leg work code in C# very quickly, lightweight app, which is excellent these days.
I just pushed up my initial changes for the setup.py. I haven't figured it out yet, there are some imports, specifically in derp.py line 23 (platform) that aren't getting consolidated into the build with py2exe. It's most definitely how I've setup the options in the setup.py, hopefully someone is more familiar with py2exe than I and can provide some insight.
Click to expand...
Click to collapse
Okay, let me take a second and fix the bug I described above... then-- damn, I wish I could try the setup.py myself. So you're saying that the platform stuff doesn't get imported into the build for some reason?
Standby for the fix.. just gotta test it and stuff.
Update: Pushed. Also added /build, /dist, and one other mac build-related directory to .gitignore to make things a little easier to see...
strvmarv said:
I haven't figured it out yet, there are some imports, specifically in derp.py line 23 (platform) that aren't getting consolidated into the build with py2exe. It's most definitely how I've setup the options in the setup.py, hopefully someone is more familiar with py2exe than I and can provide some insight.
Click to expand...
Click to collapse
Question, would doing something like this on line 52 do anything:
options = {'py2exe': {'bundle_files': 1, 'optimize': 2, 'compressed': 1,}},
I think you can also do something like:
includeList=["a list", "of modules", "to include"]
first, and then replace the line above with something like...
options = {'py2exe': {'bundle_files': 1, 'compressed': 1, 'optimize': 2, 'includes': includeList}},
see more info here and let me know if the above gets those modules in there! I see some option called "unbuffered".. dunno if that needs to be set to true.
bundle_files to 1 means that it hopefully will end up being a self-contained .exe
Let me know! Thanks!
Suggestion, maybe sticking with MD5 hashes would be simpler as the script receives updates to match Android SDK download updates. I can understand why you would want to use SHA512, but google offers MD5 on the site next the downloads for simple copy/paste replacement.
Click to expand...
Click to collapse
Forgot to answer this. You're totally right that MD5 is the one Google provides, and at first I used MD5 for everything-- then sluo reprimanded me, told me how MD5 can't be taken seriously any more, that it's really really easy for anyone to create a MD5 spoofed file these days... So I figured, since this runs as root, it's better to be very extra super-cautious and make absolutely sure the right file is downloaded
Of course, in a user-provided script, you can use md5s or whatever the author wants, but for the Android tools themselves I figured it was better practice to use SHA512 to be more forward/future looking and make sluo (a *real* programmer) happy
More work done by hashcode on a windows port
Okay strvmarv and other windows folk--
Hashcode helped me out by testing on his machine that has Windows.. we did a little debugging, and the result are these two commits:
Pull Request #1
He was able to run derp successfully and do adb/fastboot commands from the Console interface.
But because he's using win64, he couldn't build (apparently only win32 supports building .exe files) all the way.
So, if you have a win32 system-- after applying these, does python setup.py py2exe build an .exe?
Questions:
* on win32 does it build into an .exe?
* If so, does the .exe run properly as the administrator-- right-click and select "Run as Administrator" I am told
* if so, does it install the android tools and ask you to agree to the License?
* if so, does it download/detect your devices?
* if so, can you run scripts (does it work?)
Note: You may also need to manually install Java, since the android sdk updater uses java.
I'm wondering too if the installer installs any drivers, and/or if any were needed.
Thanks!
fattire said:
Okay, let me take a second and fix the bug I described above... then-- damn, I wish I could try the setup.py myself. So you're saying that the platform stuff doesn't get imported into the build for some reason?
Standby for the fix.. just gotta test it and stuff.
Update: Pushed. Also added /build, /dist, and one other mac build-related directory to .gitignore to make things a little easier to see...
Click to expand...
Click to collapse
Awesome, will take a look tonight. It's very likely I'm just not setting the options in the setup.py correctly.
You're running snow leopard, correct? You could grab a copy of the Windows 8.1 Preview (free until Jan something I believe - http://preview.windows.com) and dual-boot, or even just run a VM...if you wanted. I had to install Python 2.7 x86, wxPython x86, python2exe x86, and then run derp.py from source directly (powershell or cmd) to get where I'm at now.
strvmarv said:
Awesome, will take a look tonight. It's very likely I'm just not setting the options in the setup.py correctly.
You're running snow leopard, correct? You could grab a copy of the Windows 8.1 Preview (free until Jan something I believe - http://preview.windows.com) and dual-boot, or even just run a VM...if you wanted. I had to install Python 2.7 x86, wxPython x86, python2exe x86, and then run derp.py from source directly (powershell or cmd) to get where I'm at now.
Click to expand...
Click to collapse
Ideally I'd like to test it on a win32 system because that's the one that py2exe will make a .exe for. But that said, hashcode has it running and adb installs and works and such. It's now a matter of getting it packaged up properly I think. If you can double-check that it works for you, that would be a good start. Then hopefully the .exe can be made. It should also check to make sure java is installed (which is needed by the Google updater) and if not, maybe help the user do it (or even do it for them)...
Also, his version of windows already had drivers on them, so we're not sure whether derp (well, the android tools installer from Google) will take care of that or not.
One last note-- you may have had problems with the looping downloads because the sha512sum seemed to have been off. I did my own sha and it was different.. The new one worked for hashcode.. it's in his commit linked above..
Thanks!
fattire said:
Okay strvmarv and other windows folk--
Hashcode helped me out by testing on his machine that has Windows.. we did a little debugging, and the result are these two commits:
Pull Request #1
He was able to run derp successfully and do adb/fastboot commands from the Console interface.
But because he's using win64, he couldn't build (apparently only win32 supports building .exe files) all the way.
So, if you have a win32 system-- after applying these, does python setup.py py2exe build an .exe?
Questions:
* on win32 does it build into an .exe?
* If so, does the .exe run properly as the administrator-- right-click and select "Run as Administrator" I am told
* if so, does it install the android tools and ask you to agree to the License?
* if so, does it download/detect your devices?
* if so, can you run scripts (does it work?)
Note: You may also need to manually install Java, since the android sdk updater uses java.
I'm wondering too if the installer installs any drivers, and/or if any were needed.
Thanks!
Click to expand...
Click to collapse
* on win32 does it build into an .exe?
- I'm not win32, I'm running 8.1 x64, but it builds/executes just fine if you're using the 32 bit versions of Python, wxPython, and py2exe due to WOW64, long story
- It does build into an exe, see screen
Output
View attachment output.txt
Screen of dist folder
* If so, does the .exe run properly as the administrator-- right-click and select "Run as Administrator" I am told
- I'm running it with Run as Administrator, no, it still seems to blow up and stop running when it get's to __init__, it appears it can't find it for some reason when built with py2exe, likely the need for inclusion, not exactly sure yet...hard to capture error since it flashes by very quickly and then the console closes
* if so, does it install the android tools and ask you to agree to the License?
- If I run derp.py directly in Python it works just fine, android tools, etc...I haven't tried a script yet
* if so, does it download/detect your devices?
- I haven't tried a script yet
* if so, can you run scripts (does it work?)
- Ditto
Note: You may also need to manually install Java, since the android sdk updater uses java.
- Java SDK already installed, I dabble in Android
These missing modules indicated in build output worry me, not certain how to install them...
The following modules appear to be missing
['Carbon', 'Carbon.Files', 'ElementC14N', '_scproxy', '_sysconfigdata', 'win32api', 'win32con', 'win32pipe']
I've pulled a fresh copy of your repo, added Hashcodes changes, and tweaked the setup.py according to what I've found so far. Still blowing up as indicated above, but still moving in the right direction. If you want to go ahead and merge Hashcode's pull and ignore mine I'll reapply my changes so things don't get weird.
Pull request 2: https://github.com/fat-tire/derp/pull/2
I'll look over py2exe documentation and see if I can figure out what we need to change.
Ah, now we're getting somewhere, I changed console=["src/derp.py"] to windows=["src/derp.py"] as indicated here http://www.py2exe.org/index.cgi/ListOfOptions , get an error on execute, which is dumped into a text file, and looks like this...
Traceback (most recent call last):
File "derp.py", line 48, in <module>
NameError: name '__file__' is not defined
Any ideas?
scriptFolder = os.path.join(os.path.dirname(os.path.realpath(__file__)), "..", "scripts/")
UPDATE:
This may help...
http://stackoverflow.com/questions/...e-path-of-the-current-executed-file-in-python
Nice.. thanks!
strvmarv said:
* on win32 does it build into an .exe?
- I'm not win32, I'm running 8.1 x64, but it builds/executes just fine if you're using the 32 bit versions of Python, wxPython, and py2exe due to WOW64, long story
Click to expand...
Click to collapse
Ah, cool.
- It does build into an exe, see screen
Output
View attachment 2167579
Screen of dist folder
View attachment 2167586
* If so, does the .exe run properly as the administrator-- right-click and select "Run as Administrator" I am told
- I'm running it with Run as Administrator, no, it still seems to blow up and stop running when it get's to __init__, it appears it can't find it for some reason when built with py2exe, likely the need for inclusion, not exactly sure yet...hard to capture error since it flashes by very quickly and then the console closes
Click to expand...
Click to collapse
The file size looks tiny... it looks like it doesn't build into it all the stuff it needs...
* if so, does it install the android tools and ask you to agree to the License?
- If I run derp.py directly in Python it works just fine, android tools, etc...I haven't tried a script yet
Click to expand...
Click to collapse
^ This is awesome and a good sign for this working once we get the build finished.
* if so, does it download/detect your devices?
- I haven't tried a script yet
* if so, can you run scripts (does it work?)
- Ditto
Click to expand...
Click to collapse
Now that I think about it-- the "welcome" stuff and auto-download of the tools are all a running .derp script (welcome.derp) so yes, you are running them
These missing modules indicated in build output worry me, not certain how to install them...
The following modules appear to be missing
['Carbon', 'Carbon.Files', 'ElementC14N', '_scproxy', '_sysconfigdata', 'win32api', 'win32con', 'win32pipe']
Click to expand...
Click to collapse
Hmm.. Did you try adding them explicitly in the optionList as I suggested above?
And another way to do it is to use the -p and -i paremeters when you do python setup.py py2exe
Also maybe try adding:
import win32com
after "import py2exe" in setup.py I saw some reference to that somewhere...
What else...
looks like elementc14n is something related to the elementree module of python... win32api is here I think... but I think it would be installed when you installed python to begin with.
I've pulled a fresh copy of your repo, added Hashcodes changes, and tweaked the setup.py according to what I've found so far. Still blowing up as indicated above, but still moving in the right direction. If you want to go ahead and merge Hashcode's pull and ignore mine I'll reapply my changes so things don't get weird.
Pull request 2: https://github.com/fat-tire/derp/pull/2
I'll look over py2exe documentation and see if I can figure out what we need to change.
Click to expand...
Click to collapse
I'm looking too... See this?
For py2exe to work with packages loaded during runtime, the main thing seems to be that u explicitly import the modules needed by your app somewhere in your app. And then give py2exe in setup.py with moudlefinder.AddPackagePath( , ) the hint, where to search for modules it couldn't find by std. introspection. in the app
I won't do a full-on pull to the repo until everything is working and tested against linux/mac just to make sure we're only fixing stuff and not breaking the other platforms in the process
strvmarv said:
Ah, now we're getting somewhere..
UPDATE:
This may help...
http://stackoverflow.com/questions/...e-path-of-the-current-executed-file-in-python
Click to expand...
Click to collapse
Ah yes-- does this help as recommended in the link above..?
http://www.py2exe.org/index.cgi/WhereAmI
fattire said:
Ah yes-- does this help as recommended in the link above..?
http://www.py2exe.org/index.cgi/WhereAmI
Click to expand...
Click to collapse
jpath wouldn't pull in for some reason, despite installing via pip and having an import, so I resorted to logic to assign "." as the path (very hacky)...
So, here it is...running from derp.exe compiled with py2exe...when I get a moment I'll put together a quick summary of how to get a local win environment going..
Here's another pull:
https://github.com/fat-tire/derp/pull/3
UPDATE:
Ack, storing sdk tools in Program Files\Common Files is great and everything, but it needs to be store in Program Files (x86)\Common Files since it's x86 compiled. Pretty sure things may go wrong at some point as it is...