Related
I'm no expert when it comes to the topics of rooting and getting access to the emmc and all of that good stuff. I more specialize in ROMs and themes and stuff, the less complexed stuff lol
Someone has posted an idea in the general forums in relation to permanent root, I'm not sure if he posted it here or not. So here's what he wrote....and is it possible? Or does it have to be done manually first before this idea can happen?
Originally Posted by deliberate187
In order to unlock the phone, we have to figure out what the protected sectors are first and all related flags. If an Android app could be made to have direct read access to the eMMC filesystems (including write protect flags) and save a log to the SD card detailing these items, this would be ideal.
Then all that would remain is a program to undo the write protection (and re-do it if necessary to unvoid warranty)
If anyone is willing to create these programs, I would be more than happy to test them out on my own G2.
However, I think the keys to the mystery may lie in the recovery image, and/or in the bootloader itself. Has anyone disassembled these yet?
Click to expand...
Click to collapse
Sorry to have to tell you but this is all old information stuff we already know just are unable to do anything about it. Its harder then just coming up with an idea of something. Now if we knew a person that programed the g2 in htc factory then all would be good but as of now we just dont have the information we need to do anything
thanks
Thanks for the idea. Some people will be mad you didn't post in the root thread though.
File under "I'm no expert but..."
Here is one observation I have noted in my exploration. The root filesystem and system partition are mounted with the flags "-o ro,relatime" but in addition the /system partition has ",errors=continue" leading me to believe that this change is in fact written to the release configuration rather than to the eMMC itself. Can anyone try to get a permanent write to the fstab and see if this can net us permanent root? Possibly take a temp root session and remount the system and / filesystems read/write to see if writes stick... just an idea.
The errors=continue flag allows the ext3 filesystem to continue working even if there was a read/write error.
I've been able to get the system to change to r/w a couple times while wandering through root explorer. I have made subtle changes to certain folders such as moving txt files but nothing has ever been permanent. I can't really tell you how I did it either seeing as I can't replicate it on demand...I'm assuming it still gets written to cache despite being in the /system
Sent from my T-Mobile G2 using XDA App
heyy, I'm not punchie, I've got what the doctor calls a relaxed brain
I am thinking there should be a set of adb commands to unlock the nand. I am definitely thinking a nand dump and full disassembly of the bootloader and recovery image could be absolutely crucial in discovering what needs to be done. Just a thought, has anyone done a nandroid backup of the G2 yet? I'm pretty sure TMob doesn't have HTC encrypt its bootloaders...
deliberate187 said:
I am thinking there should be a set of adb commands to unlock the nand. I am definitely thinking a nand dump and full disassembly of the bootloader and recovery image could be absolutely crucial in discovering what needs to be done. Just a thought, has anyone done a nandroid backup of the G2 yet? I'm pretty sure TMob doesn't have HTC encrypt its bootloaders...
Click to expand...
Click to collapse
if you can figure it out, go for it and i wish you luck
deliberate187 said:
Here is one observation I have noted in my exploration. The root filesystem and system partition are mounted with the flags "-o ro,relatime" but in addition the /system partition has ",errors=continue" leading me to believe that this change is in fact written to the release configuration rather than to the eMMC itself. Can anyone try to get a permanent write to the fstab and see if this can net us permanent root? Possibly take a temp root session and remount the system and / filesystems read/write to see if writes stick... just an idea.
The errors=continue flag allows the ext3 filesystem to continue working even if there was a read/write error.
Click to expand...
Click to collapse
If it were only this easy.
Re-mounting /system as r/w is part of the rooting process. This does not result in changes written to eMMC. In fact, the controller "lies" to Linux that the change has been synced. From then on, Linux holds the changes in its cache which, when dropped or rebooted, reverts changed files to their original state (because they were never written in the first place.)
The ext3 continue on errors thing is merely a way to skip fsck in the event that the read-only system has issues in the journal (very unlikely to happen, since nothing can write to it.) Presumably, this only covers an oversight in OTA updates (where the journal of the image provided by the OEM is dirty for some odd reason.) Again, since nothing can write to /system, it's all but an impossible scenario (nothing can write to the journal either...)
As for marking "sectors" as write-protected or not, that's also easier said than done. Entire partitions are locked, and half of the space is mysteriously "missing." It's difficult to see what's really going on from userland, as the device is deceptive as to what is and is not being written, or what is even stored on the eMMC in the first place.
The real solution is to exploit either the boot-loader or eMMC (re)/initialization somehow to allow a) unsigned firmware to be loaded and/or b) allow booting without write protection, allowing us to c) flash rooted rom to the phone and/or d) disable said protection. The unlock procedure will likely be similar to Unrevoked, as that is essentially the same situation (aside from the controller issue.)
All of this is covered in the wiki and various threads - check those out, if you find a way around it everyone would be glad to hear it.
HamNCheese said:
If it were only this easy.
Re-mounting /system as r/w is part of the rooting process. This does not result in changes written to eMMC. In fact, the controller "lies" to Linux that the change has been synced. From then on, Linux holds the changes in its cache which, when dropped or rebooted, reverts changed files to their original state (because they were never written in the first place.)
The ext3 continue on errors thing is merely a way to skip fsck in the event that the read-only system has issues in the journal (very unlikely to happen, since nothing can write to it.) Presumably, this only covers an oversight in OTA updates (where the journal of the image provided by the OEM is dirty for some odd reason.) Again, since nothing can write to /system, it's all but an impossible scenario (nothing can write to the journal either...)
As for marking "sectors" as write-protected or not, that's also easier said than done. Entire partitions are locked, and half of the space is mysteriously "missing." It's difficult to see what's really going on from userland, as the device is deceptive as to what is and is not being written, or what is even stored on the eMMC in the first place.
The real solution is to exploit either the boot-loader or eMMC (re)/initialization somehow to allow a) unsigned firmware to be loaded and/or b) allow booting without write protection, allowing us to c) flash rooted rom to the phone and/or d) disable said protection. The unlock procedure will likely be similar to Unrevoked, as that is essentially the same situation (aside from the controller issue.)
All of this is covered in the wiki and various threads - check those out, if you find a way around it everyone would be glad to hear it.
Click to expand...
Click to collapse
Listen to this dude. Absolutely correct.
If you ever used CWM, CWMT or other non factory recoveries to wipe your data, you probably noticed that you lost the ability to encrypt your phone. Or maybe you did not even realize this is why encryption does not work.
For the Android phone encryption to work, it needs the /data (usrdata) partition to have a little bit of unused space between the end of the filesystem and the end of the partition. And as soon as you use CWM to wipe, it actually reformats using all space, and encryption does not work anymore.
User lolo250612 brought this to my attention, and together we created a update.zip that shrinks the /data filesystem by 1MB
In fact, we created 2 patches: One to shrink, and one to first repair the filesystem. The first will refuse to shrink if the file system is not clean and healthy. They will automatically find the correct usrdata partition device and its size. The shrink will then resize to 1MB less then the partition size (which means it could also be used to grow if you somehow had a filesystem a lot smaller, for example because you restored an smaller image from somewhere).
Both patches are created with statically linked e2fsprogs binaries and its own static copy of busybox shell interpreter. So they should work on all Android devices that use ext file system (probably all V2.3.1 Gingerbread and higher androids), and you should not lose any data because of this. But it is always good to make a backup.
We tested this on 2 phones, both ICS phones, and with both CWM and TWRP type recoveries, and are fairly certain it is safe to use. But to repeat, you should always take a backup of your phone.
Both patches can be found on my shared drive:
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
Procedure:
- Make backup of your phone
- Place files on SD card
- Boot into recovery
- Apply the shrink update
- If it tells you the filesystem is damaged apply the fix-fs update first
The patch only shrinks the filesystem, nothing is actually installed or removed on the phone. But if you use encryption, you could leave this patch on your SD card so that every time you wipe data, you can run the shrink patch again afterward to enable encryption again.
If you do use this, please report back in this thread, possibly mentioning your phone model and ROM you are using.
Quick encryption guide (and more)
I won't go deep into useless details as everything has already been described about Android phone protection somewhere on the internet. I will just give some meaningful links and tips by illustrating how I have protected my phone. Really nothing new or innovative, just a compilation of a few hints that I have put in practice to protect the numerous pieces of information that are on my phone.
Step 0: awareness
----------------------------
Why bother with phone security?
In short, I am clearly paranoid. Well, in fact, I don’t really feel at ease when I know all the information, both personal and professional I have on my phone. Over the month, my Androphone has become a real digital Swiss-knife and personal secretary. This includes:
Personal and professional contacts
Personal and professional agendas
Personal and professional digital exchanges (SMS and email)
Personal and professional photos
Banking account information
Trails where I run
Etc… etc…
Don't want someone looks at them. Not you?
Fist step: on-line protection
----------------------------------------
The first step in protecting your data consists in making hard to access indirectly the data that lay on your phone memory. This access consists in using the system when the phone is on, either via the GUI and the phone controls, or remotely (essentially by network connections, or phone basic functionalities like sms). So, basically, you need to lock efficiently your phone from preventing someone else to unlock the user interface that allows interactions with the system, and protect all communication channels.
To lock efficiently your phone, you must use a pin code of at least 4 digits (6 is better) or a pass-phrase. The latter is much less practical without improving online security that much. Above all, you must avoid those silly locking solutions like face recognition unlocking, or pattern lock. Those are toys for naive young boys. Not for those concerned seriously by security.
For protecting remote access to your phone, I would suggest:
1) Double check that USB debugging is disabled. This a major security hole.
2) Turn on data connections (bluetooth, wifi and 2/G/3G/4G) only when required (email checking, web-surfing session, data synchronising), and off rest of the time.
3) Avoid install cracked unofficial apks, or applications that asks for permission far beyond their obvious and principal utility
4) Install a software security app, if possible, open source and recognised by xda members. Once an adept of Droiwall, I have switched to Avast mobile security because of its extra features. But it is not opensource and it is a question of taste. But do this carefully, see that for instance before making a choice: http://download.cnet.com/8301-2007_4-57391170-12/dont-get-faked-by-android-antivirus-apps/ and http://www.av-test.org/fileadmin/pdf/avtest_2012-02_android_anti-malware_report_english.pdf.
But, you must be rooted (which is in itself a security hole if not mastered) and one must have a kernel with netfilter functionalities activated. This is the case with the stock kernel of the phone I use at the present time (Lenovo A789). But was not the case of 2 Samsung phones I used before. You have to either install a custom kernel adapted to your phone, or make your own if you have access to its sources (see tutorials as: http://forum.xda-developers.com/showpost.php?p=22941057&postcount=1)
5) Personally, I would feel more at ease if I could find an easy to use firewall solution that could close, and better, make stealth all the local ports of my phone, especially when I am not behind a wifi router. But I haven’t found one yet. Droidwall, nor Avast, addresses this functionality, whereas it would be fairly easy to implement it with the netfilter system layer underneath.
Second step: offline protection
-------------------------------------------
Here we are. Now your phone is protected when it is on. But, what if you switch it off, or remove its sdcard? The data lay on the internal memory, unprotected (at best obfuscated). Really easy to find a custom recovery for almost all phones, write a script to dump /data on a sdcard and then make whatever you want with the copy.
Don’t like that? The only solution to prevent /data from being read by someone else is to encrypt the /data partition. To do that, your phone or tablet internal storage partitions must be seen by your system as block devices. This is the case with eMMC but not with Yaffs. So beware, if you want encryption you need to buy a device that answers this requirement. This is not always true and almost never documented. Notes on the implementation of Android encryption are there: http://source.android.com/tech/encryption/android_crypto_implementation.html
Now, as me, if you are reading these lines, you are certainly looking for extra information about your Android device and probably extra functionalities.
Certainly, the most frequent way to install extra functionalities and custom ROMs to your phone is to use an update zip file. With stock recovery, this zip file needs to be signed, otherwise it is rejected. For maximum flexibility and ease of use, alternative boot recovery have been developed, of which CWRP is certainly the most famous.
Usually, for 99% of users and operations, CWRP operates great. Sometimes, as nothing is perfect, a bug may occur. This is the case for built in ICS encryption process. As Cybermaus indicates in the first post, to be able to perform this encryption the /data filesystem must be slightly smaller than the underlying partition. But CWRP, at least up to the version 5.5, formats all the corresponding partition leaving no place for Android to store the required information to be able to start the encryption process. This is clearly described in the following links: http://forum.xda-developers.com/showthread.php?t=1792101 and http://rootzwiki.com/topic/25652-fixing-galaxy-tab-2-encryption/
I have discovered that by using aLogcat to track down the origin of the failure. The interesting part revealed to be: E/Cryptfs ( 87): Orig filesystem overlaps crypto footer region. Cannot encrypt in place.
To circumvent this problem, you will find in Cybermaus first post, two CWM update zip files that will do the trick in a simple and secure way. After flashing your ROM and wiping data with CWM, apply them, go to system encryption as described here:http://support.google.com/android/bin/answer.py?hl=en&answer=1663755, and after waiting one or two minutes (not more), the system should restart automagically to encrypt your /data partition.
Third step: making your phone even more secure and practical at the same time
-------------------------------------------------------------------------------------------------------------------
Android built-in encryption is in fact more or less Linux LUKS (http://en.wikipedia.org/wiki/Linux_Unified_Key_Setup). Plus, it is open-source so that everyone with the required skills can make an audit of the code to see if no security hole is present in the Android implementation. The underlying mechanism is strong and secure, as long as you use a strong password. I mean by strong, at least 12 characters that includes at the same time lower-case letters, upper-case letters, numbers and symbols. And it must be something impossible to guess for others while easy to remember for yourself. You will find a lot of resources on the internet on how to create such a password. For instance: https://help.ubuntu.com/community/StrongPasswords .
The problem with Android, in its attempt to keep the system not too complicated to use, is that the GUI (I insist: only the GUI, not the system) does not distinguish between the PIN or passphrase that you use to lock your phone when it is on, and the password used to encrypt the data that lay physically on your phone storage. So the casual user is in front of a paradigm: either he chooses a strong password for its data, but this will rapidly become tedious to type at least 12 characters to unlock his device several times a day; or he decides to use a PIN code, which is more practical to unlock the phone, and consequently uses a really weak password to encrypt its data which contains only digits, and thus may be cracked in a breath by any PC.
Fortunately, this paradigm is addressed and solved by small tools like EncPassChanger or Cryptfs Password (both requiring that your phone be rooted, which is by the way, paradoxically, a security hole if not used with caution ). See: http://nelenkov.blogspot.fr/2012/08/changing-androids-disk-encryption.html for complete notes about that. So for me, the only way, both secure and practical, to secure your phone is by using a PIN code of at least 4 numbers (6 is better). Then use a handy tool like EncPassChanger to have a true complex password for decryption at boot time.
Fourth step: increase security, without sacrifying practicability
-----------------------------------------------------------------------------------------
As I am paranoid, but at the same time don’t want my phone to become a source of annoyances, the previous “basic” steps were not enough for me.
So I decided to improve security in two ways:
1) By following the following tip, which I find great and is itself self-explaining: http://forum.xda-developers.com/showpost.php?p=26730989&postcount=2
2) By encrypting the photos I take with my phone, because these are linked with my private life and I won’t like that somebody gain access to them.
3) By encrypting documents I scan with CamScanner, for home and work, which may be sensitive.
4) By automating the action that disables USB debugging in case I forget to put it off after using it .
For point 2 and 3, documents lay on your sd card uncrypted. Android built-in encryption does not deal with both internal and external sdcard (just to be clear, by sdcards I mean partitions mounted as /mnt/scard or /mnt/scard2). To encrypt them you have to use once again an external tool. As I am an opensource fanatic for all that deal with security, I would recommend to use LUKS Manager (https://play.google.com/store/apps/details?id=com.nemesis2.luksmanager&feature=search_result and http://forum.xda-developers.com/showthread.php?t=1141467) which is based on dm-crypt module (yes, the same that Android uses for its build-in encryption), or Cryptonite (https://play.google.com/store/apps/details?id=csh.cryptonite&feature=search_result) which is completely open-source and implements the rock-solid Linux encfs on Android.
The latter is my personal choice. I do not use Crytonite in itself, except for creating the initial .encfs6.xml file. For everyday use, I use directly the Android port of the binary file encfs that comes with Cryptonite, and embed it into shell scripts. Up to now, no flaw, no problem. The password to open my encfs encrypted volumes is stored in a text file located on the /data partition. It is thus encrypted by Android and made accessible on boot when you decrypt this partition. So nothing more to remember.
To make things usable and practical, I use Tasker to automate the following things:
- Mount encfs volumes on start-up, by reading directly the password in the file located on /data
- Umount encfs volumes when usb is plugged
- Copy photos on a regular basis from the unencrypted /mnt/sdcard/DCIM to the safe place I created with encfs, delete AND wipe the original ones
Fifth step: be coherent about security
-----------------------------------------------------
Some people, torn apart by the paradigm described in Third step, by negligence or by lack of knowledge, strongly secure one part of the system, but make other parts big security holes.
Concretely, I am thinking about two examples: mixing encryption with pattern lock (or, even worse, with face unlock), or mixing encryption with usb debugging. Face recognition is just a jock. It is not reliable and fails very often. Moreover it is really easy to crack, with a photo for example. One of my colleague even achieved to unlock my phone with its own face, just because we are morphologically close enough. Pattern lock is not much better. (See: http://forum.xda-developers.com/showpost.php?p=37649447&postcount=6 and https://www.google.fr/search?q=smudge+attack).
So always ponder over (two times rather than one) each action you take that may touch system security.
Thanks lolo
I'm trying to use this on my VZW Galaxy S3 16Gb and this is what I'm seeing in TWRP v2.2.0:
Mounting System
Extracting system fixes
Update script starting...
Update script started
Disk /dev/block/mmcblk0p15: 13.1GB, 13140754432
4 heads, 10 sectors/track, 401024 cylinders
Units = cylinders of 64 * 512 = 32768 bytes
ERROR: unlikely size of KB
aborting operation!
Update script ended
Unmounting system...
Update Complete
Click to expand...
Click to collapse
edit: The same thing happens with both scripts.
I need to enable device encryption because my employer requires it for email and other Google Apps for Business apps. Thank you for your help!
Anyone know why full disk encryption isn't available on some (if not most roms)? Is it something that needs to be added with intent aside in the building process, or dependent on how the stock rom was set-up to work with?
I was hoping this would help get encryption working on an EVO ics rom which has encryption available, but when you click "encrypt phone" it just hangs on an android screen and doesn't actually do anything.
i was really happy to find your solution to enable encryption on my HTC desire S (ICS, rooted), but unfotunately it doesn't work. the same thing happen to me as it happened to mushu13, only different numbers in lines 5 and 6. same result whichever script i choose. please help, i really need system encryption.
thanky you very much!
First thing you should know, I am not an Android Guru. And unfortunately, if your phone is not an A789, I won't be able to help you deep in technical details. Cybermaus is the most skilled of the two of us, technically speaking, and he may lack time to answer correctly every request he is regurlarly faced with.
Okay, I do not know your phones and don't own them. So, distant debugging is much harder in these conditions. But the first things you should check, before applying Cybermaus' patches, are :
1) if encryption works with stock rom
2) follow thoroughly all steps I described in "Second step: offline protection" of the second post of this thread :
- your phone or tablet internal storage partitions must be seen by your system as block devices. This is the case with eMMC but not with Yaffs. If you don't have this information from the manufacturer, install Terminal Emulator from the Play Store and type 'mount' in it. You should see lines beginning with /[email protected] and /[email protected] If this is not the case, I fear encryption won't be able to work on your device.
- use aLogcat to track down the origin of the failure (see resources on the internet to learn how to use it, and links I have put in the second post)
3) Be sure that required modules are built into the kernel you use, especially dm-crypt
4) Post your results and cross your fingers that either this is a problem I have already encountered (in this case I may help you further), or Cybermaus see your posts.
While this script did allow me to encrypt my phone, it also shrunk my /data partition to roughly 1.1 GB.
Any ideas on how to expand it back to a reasonable size? I supposedly have 4 GB of ROM, and I assume more than 1 GB ought to be available for data.
Sent from my HTC Sensation using xda app-developers app
Thank you for your nice guide.
Only one thing is missing: baseband security.
Attacks on the baseband system requires very skilled people. Such as government agencies. It is believed they use baseband attacks to break into almost every mobile device. And there is only little you can do. Some vendors like Cryptophone have mobile devices with a hardened Android system. All others have no way to protect their device against baseband attacks.
Is this patch and reasoning still valid for newer android releases.
I am running a custom kitkat rom and twrp on a note 3 and can't encrypt so im looking for a fix.
I have been looking around for fixes but different posts blame different things.
Sometimes its the fact its a custom recovery, sometimes its that root is on the device and then there is this reasoning
Is there a way to find out the cause and fix for kitkat?
Virus
Hi, i tried to download your files
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
But the file are exe files with viruses.
Any ideas?
u2funker said:
Hi, i tried to download your files
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
But the file are exe files with viruses.
Any ideas?
Click to expand...
Click to collapse
Maybe false alarm.
Lossyx said:
Maybe false alarm.
Click to expand...
Click to collapse
no, but if you search for these file, you will find some which work and which are without viruses. Check the link..it is not an zip file..it is an exe file
@cybermaus: just tried flashing the two *.zips on my Galaxy S 4 Mini running CM 12 (Android Lollipop) because my logcat tells me I'm getting the described cryptfs error. It seems my /data partition doesn't have that 1 MB of unused space needed for encryption. Now I would love to encrypt my phone using CM's integrated function without having to completely format the internal storage (because that's the other workaround I found: flash stock rom, wipe data (factory reset), flash Custom Recovery, flash CM again)
Do you have the time and device to update your script so it works with Android Lollipop as well? I see a lot of people come across this issue recently so there would be definetly use for such a nice script like yours!
Thanks for sharing this with us!
-Teutone
no available for download any mirror ?
Or write the script on the thread.
Thanks
Can you post the scripts? links are dead!
---------- Post added at 16:33 ---------- Previous post was at 16:32 ----------
cybermaus said:
If you ever used CWM, CWMT or other non factory recoveries to wipe your data, you probably noticed that you lost the ability to encrypt your phone. Or maybe you did not even realize this is why encryption does not work.
For the Android phone encryption to work, it needs the /data (usrdata) partition to have a little bit of unused space between the end of the filesystem and the end of the partition. And as soon as you use CWM to wipe, it actually reformats using all space, and encryption does not work anymore.
User lolo250612 brought this to my attention, and together we created a update.zip that shrinks the /data filesystem by 1MB
In fact, we created 2 patches: One to shrink, and one to first repair the filesystem. The first will refuse to shrink if the file system is not clean and healthy. They will automatically find the correct usrdata partition device and its size. The shrink will then resize to 1MB less then the partition size (which means it could also be used to grow if you somehow had a filesystem a lot smaller, for example because you restored an smaller image from somewhere).
Both patches are created with statically linked e2fsprogs binaries and its own static copy of busybox shell interpreter. So they should work on all Android devices that use ext file system (probably all V2.3.1 Gingerbread and higher androids), and you should not lose any data because of this. But it is always good to make a backup.
We tested this on 2 phones, both ICS phones, and with both CWM and TWRP type recoveries, and are fairly certain it is safe to use. But to repeat, you should always take a backup of your phone.
Both patches can be found on my shared drive:
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
Procedure:
- Make backup of your phone
- Place files on SD card
- Boot into recovery
- Apply the shrink update
- If it tells you the filesystem is damaged apply the fix-fs update first
The patch only shrinks the filesystem, nothing is actually installed or removed on the phone. But if you use encryption, you could leave this patch on your SD card so that every time you wipe data, you can run the shrink patch again afterward to enable encryption again.
If you do use this, please report back in this thread, possibly mentioning your phone model and ROM you are using.
Click to expand...
Click to collapse
links are dead. Can you post the scripts?
★★★★[INFO]ANDROID ROM & How they Work★★★★
Parts of a ROM
i. The kernel.
Android (like many other Smartphone operating systems) runs on the Linux kernel. The Linux kernel was created in the early 1990’s by a gentleman named Linus Torvalds in Helsinki, Finland. It’s incredibly stable, incredibly friendly, and incredibly difficult for the layman to understand and modify. Thankfully it’s also very popular so it has been ported on to a multitude of hardware, including our Android devices.
Think of the kernel as an interface layer between the hardware and software on your device. The kernel decides when things happen, such as the LED indicator gets lit or when the soft button's LED gets lit. An application sends a request to the operating system to blink the LED. The operating system then sends the request to the kernel, which makes the light flash for the amount of time requested by the OS.
What sounds like a round-about way to get things done is also what makes the system so scalable and robust. Application developers only have to code in a way the operating system understands and the kernel makes it work on the hardware. This also keeps the application running in it’s own user-space and separate from the kernel. That means when you run the latest uber-cool app that wasn’t designed for your particular OS version, or is still very beta and it crashes, the kernel gives you the option to Force Close the application and the kernel can run untouched.
In a standard Android ROM (we will leave developer images and the like for another discussion) the kernel is bundled along with a set of instructions that tell the device how to load the kernel and the OS during boot. This is the boot.img that you see inside a zipped ROM that your not able to easily open. The device knows to extract this image to internal memory (the ramdisk) and follow a series of scripts (init scripts) to load the kernel and then the other portions of the OS. That’s what’s happening while you’re watching the boot animation. Interestingly enough this is done the same way for a PC, your smartphone, an Android tablet, or even a smart Linux powered toaster. If you’re feeling exceptionally geeky, plug your Android phone into the USB port on your PC and let the PC boot from the USB device. No, it doesn’t actually load, but you can watch the animation while it tries to match up the hardware support with what’s inside your PC. As I said, Linux is amazingly scalable and as a result so is Android.
What is a kernel? If you spend any time reading Android forums, blogs, how-to posts or online discussion you'll soon hear people talking about the kernel. A kernel isn't something unique to Android -- iOS and MacOS have one, Windows has one, BlackBerry's QNX has one, in fact all high level operating systems have one. The one we're interested in is Linux, as it's the one Android uses. Let's try to break down what it is and what it does.
Android devices use the Linux kernel, but it's not the exact same kernel other Linux-based operating systems use. There's a lot of Android specific code built in, and Google's Android kernel maintainers have their work cut out for them. OEMs have to contribute as well, because they need to develop hardware drivers for the parts they're using for the kernel version they're using. This is why it takes a while for independent Android developers and hackers to port new versions to older devices and get everything working. Drivers written to work with the Gingerbread kernel on a phone won't necessarily work with the Ice Cream Sandwich kernel. And that's important, because one of the kernel's main functions is to control the hardware. It's a whole lot of source code, with more options while building it than you can imagine, but in the end it's just the intermediary between the hardware and the software.
When software needs the hardware to do anything, it sends a request to the kernel. And when we say anything, we mean anything. From the brightness of the screen, to the volume level, to initiating a call through the radio, even what's drawn on the display is ultimately controlled by the kernel. For example -- when you tap the search button on your phone, you tell the software to open the search application. What happens is that you touched a certain point on the digitizer, which tells the software that you've touched the screen at those coordinates. The software knows that when that particular spot is touched, the search dialog is supposed to open. The kernel is what tells the digitizer to look (or listen, events are "listened" for) for touches, helps figure out where you touched, and tells the system you touched it. In turn, when the system receives a touch event at a specific point from the kernel (through the driver) it knows what to draw on your screen. Both the hardware and the software communicate both ways with the kernel, and that's how your phone knows when to do something. Input from one side is sent as output to the other, whether it's you playing Angry Birds, or connecting to your car's Bluetooth.
It sounds complicated, and it is. But it's also pretty standard computer logic -- there's an action of some sort generated for every event. Without the kernel to accept and send information, developers would have to write code for every single event for every single piece of hardware in your device. With the kernel, all they have to do is communicate with it through the Android system API's, and hardware developers only have to make the device hardware communicate with the kernel. The good thing is that you don't need to know exactly how or why the kernel does what it does, just understanding that it's the go-between from software to hardware gives you a pretty good grasp of what's happening under the glass. Sort of gives a whole new outlook towards those fellows who stay up all night to work on kernels for your phone, doesn't it?
Click to expand...
Click to collapse
ii. The operating system.
Once the kernel is loaded, the init scripts tell the Operating System to load. Android is the user interface for a custom built Java virtual machine called Dalvik. Dalvik was written by Dan Bornstein, who named it after the fishing village of Dalvik in Iceland, where his family originated from. The debate of which Java VM is superior is best left for another discussion, so I’ll simply say that DalvikVM is a register-based machine versus true JavaVMs which are stack based.
The Dalvik machine creates executable files (.dex files) which can be interpreted by the OS and run by the end user. These .dex files are OS version dependant. That simply means that applications and core functions built to work with one version of Android may or may not work well with other versions. Google provides the tools through it’s Software Development Kit (SDK) for applications to communicate with the OS.
Click to expand...
Click to collapse
iii. Core functions.
No smartphone would be complete without a set of functions that allow the device to be used as intended. Things like the phone and dialer interface, the calendar, the messaging system are core functions of the Operating System. In Android, these are run on top of the kernel as separate applications. The merits (or lack of) of providing these needed functions as separate applications is once again best left for another discussion, but this is what allows developers like HTC or Motorola to replace the standard functions with alternatives that provide a different look and feel from stock. HTC’s onscreen keyboard or Motorola’s MotoBlur contact list are great examples of this. The “little guy” isn’t left out of the mix either. Handcent SMS or Chomp SMS can integrate into the OS very well, as most of us already know.
An additional set of Core Functions are provided by Google. Popularly called GoogleBits, things like Gmail, sync, Gtalk and the Android Market are applications written by Google that give an extra set of useful functions to the OS. You’ll find these on all smartphones, as well as many other Android devices.
Click to expand...
Click to collapse
iv. Optional applications.
These are applications provided by the manufacturer to give the device even more usability. Things like the Amazon MP3 store, PDF readers, Corporate Calendar etc. allow you to do even more with your device. Remember - Droid Does
Click to expand...
Click to collapse
B. How is a ROM packaged?
In most cases a ROM will come packaged in a .zip file. The recovery image’s kernel (yes, it has one too!) has the ability to unzip and copy the contents into the correct place. Inside this zip file is a folder (META-INF\com\google\android\) that contains a script prepared by the ROM “cooker” (another of those techie terms - it means the person(s) who developed the ROM) that tells the system what to format, what to copy and where, and any file operations that need to be done. Each device does things a bit differently, but this script is where it all gets done. More on this folder later.
You’ll also see a /system folder. This is the meat of the ROM. It has the necessary OS files, the Core functions, and any optional applications the cooker decided to include. The folder is structured the same way it is on your device - /system/app, /system/framework, etc. The whole tree is usually copied over and the existing /system folder is overwritten. The cooker uses the script to tell the kernel to erase the existing system folder, copy the new folder over, and set the file permissions.
Sometimes you will also see a data folder. This usually is space set up for optional applications, including optional system tools like busybox or SuperUser white list. These applications could be placed in the /system folder, but placing them in the data folder makes it easier for the end user (you and I) to remove or update them as needed.
You’ll also notice a META-INF folder. This contains the update script we talked about earlier, as well as secure keys that need to be provided so the device knows the update can be trusted. A special note needs made here. Trusted means that the update is trusted to be in the correct form to load the device. It in no way means the ROM is safe from malicious code. Anyone is able to use a set of test keys and create a ROM that will flash and run your device - even those people with bad intentions. Flashing and running a custom 3rd party ROM is putting faith in the cooker that he or she not only knows what they are doing, but are honest as well. Also, some Motorola custom ROMs will have a small update.zip stored inside this folder to be run on first boot of the device.
Finally we are left with the boot.img file. This is the kernel and ramdisk image we discussed earlier. Your phone copies this over to be decompressed and run when the device boots.
Click to expand...
Click to collapse
2. How do I install a ROM?
In this section we’re discussing how to install a custom 3rd party ROM. ROMs from the manufacturer usually have a utility that runs on your PC to flash and load the new image.
A. Got Root???
Yes ?:good:!!!
Custom ROM’s simply will not load on devices that aren’t rooted. In theory, it may be possible to sign a 3rd party ROM with the keys that the stock recovery image will flash, but for the most part you need to have flashed a custom recovery image before you can change your device’s ROM. Instructions and tutorials on how to root your device are all over the internet. Some are good, some are bad. The hacking forum is a great place to go and learn more about rooting and how to successfully get it done on your device.
Click to expand...
Click to collapse
B. Recovery
Most Android devices have had a custom recovery image written for them. This will overwrite the stock recovery image, allowing you to flash 3rd party ROMs as well as giving extra functionality. Help with finding and flashing the custom recovery image for your device can also be found in the hacking forum. The installation of a custom recovery image also allows for a very important function. Backup and restore.
Click to expand...
Click to collapse
.C. Nandroid
Nandroid is a set of bash scripts and code written by that copies the state of your system and stores it in a folder on your SD card. You can then use the restore function of Nandroid to restore to this point at any time. This is a priceless feature and reason enough to root your phone. It’s included by default in most custom recovery images, and the code is freely available to use if you’re inclined to write your own recovery image.
Click to expand...
Click to collapse
In most situations, using Nandroid to back everything up is easy:
1. Verify you have a memory card with enough free space (~300MB to backup, ~500MB to restore).
2. Reboot your device into recovery. It’s slightly different for each device, once again hacking forum FTW!
3. Navigate through the menu and select the Nandroid Backup function.
4. Apply your choice and wait for the device to tell you it’s finished.
It’s always good practice to copy the entire nandroid folder from your SD card to a safe place. You can then copy it back to the SD card if the card is ever damaged, lost or erased.
D. Copy and Flash
You’re rooted, have downloaded a custom ROM, have your system backed up and are now ready to flash your device. This is not nearly as scary as it sounds.
1. Mount your SD card to your PC, and copy the .zip file to the root folder of the card. Don’t unzip the file, and don’t look for a folder called root. The root folder in this case means the base folder, what you will see when you mount your card to a PC or the device.
2. Reboot your phone into recovery.
3. Navigate through the recovery menu and select the flash update option. Depending on your recovery image, the file may need to be named update.zip, or you may be able to select any zip file on your card as long as it’s the correct format. The cooker knows this as well and if the ROM needs to be named update.zip it will be.
4. Apply your choice and wait for your device to tell you it’s finished.
5. Reboot.
Click to expand...
Click to collapse
It’s worth noting that many times a new ROM will require that you wipe and factory reset your devices data. While inconvenient, it’s often necessary to get rid of the old data as it may be incompatible. As long as you’re using the cloud for calendar and contacts, they will be re- downloaded and stored back on your device automatically.
Dirty flash and Clean flash
A dirty flash is only wiping cache and davlik then flashing your ROM....
a Clean flash is at LEAST factory reset/data wipe + wiping davlik(factory wipe takes care of /cache also)... Maybe doing a format /system also.
***Odin***
Odin is the ROM Flashing Tool for SAMSUNG smartphones. ROM files flashable with Odin come with .tar extension.
Most of the ROMs you are going to flash with Odin are the official stock Samsung ROMs (or leaked stock ROMs). Custom ROMs are rerely flashable by Odin because they come with .zip extension that Odin does not recognize (it recognizes .tar files).
Custom kernels, however, are sometimes provided in .tar format by their developers (e.g. CF-Root kernels), so that they can be flashed by Odin. When your phone is new and running official firmware you most often cannot flash a custom ROM to it because a Samsung phone often requires a custom recovery and root rights that are included in a custom kernel to be able to flash custom ROMs. That's why Odin often comes in handy in rooting and flashing a custom firmware to your phone because you (often) can flash a custom kernel with it that already includes root and custom recovery and enables you to flash custom firmware (custom ROMs). I use the word "often" very frequently in the previous sentence because every Samsung smartphone is different and requires various procedures for rooting it and flashing custom ROMs (see the section about using Odin below).
If it comes to stock ROMs, the best source of stock (official) Samsung ROM files is located at this excellent website: SamMobile.com/firmwares (link). It requires registration (it's free) and I encourage you to set up an account there because you will most likely use this site several times during your stay at XDA. You will most likely come across 1 .tar or 3 .tar file ROMs there, flashable by Odin. Refer to the Odin flashing guide below for more info.
Click to expand...
Click to collapse
****Heimdall****
What is Heimdall?
Heimdall is a cross-platform open-source tool suite used to flash ROMs onto Samsung Galaxy S devices.
How does it work?
Heimdall uses the same protocol as Odin to interact with a device in download mode. USB communication in Heimdall is handled by the popular open-source USB library, libusb-1.0.
Why “Heimdall”?
The flashing software Odin is named after the king of gods in Norse mythology. Loke, the software component on the Galaxy S that provides functionality to flash, may also to be named after an important character in Norse mythology, often translated as Loki. As such I have named my flashing software Heimdall, after the Norse god, and guardian of the Bifrost Bridge.
What platforms does Heimdall run on?
Linux, OS X and Windows (XP, Vista, 7 etc.)
Why use Heimdall when we can use Odin?
Odin is generally unreliable and only runs on Windows systems. Furthermore, Odin is leaked Samsung software that is not freely available or well understood by the community.
Is Heimdall safe?
No matter what method you chose, flashing firmware onto your phone has a lot of potential for disaster. We have tested Heimdall with a variety of phones flashing several different firmware versions resulting in a 100% success rate. As such we believe that Heimdall is generally reliable. However keep in mind, just like any flashing software, Heimdall has the potential to brick your phone if not used correctly.
How do Galaxy S phones get bricked when flashing?
Besides the inherent risks like power outs, accidental removal of the USB cable etc. The Galaxy S appears to be running extremely unreliable USB control software.
A failure to flash does not automatically equate to a bricked phone. However if you're extremely unlucky and the flash fails whilst transferring the primary boot-loader, secondary boot-loader or params.lfs (all quite small) than you've got yourself a paper weight that you're hoping Samsung will replace.
Please be extremely careful mixing files from different firmware releases. Don't do so unless you're certain it will work!
What Galaxy S variants has Heimdall been tested with?
We’ve tested Heimdall with a Galaxy S GT-I9000 (8 GB) from the United Kingdom and Galaxy S GT-I9000 (16 GB) from Australia. We don’t personally have access to any other devices to test with, however users have confirmed Heimdall functions correctly with the AT&T Captivate, Bell Vibrant, Telstra GT-I9000T, Epic 4G and the Galaxy Tab.
Click to expand...
Click to collapse
^
CWM Errors and Solutions
ERRORS encountered in CWM Recovery
.
What is CWM Recovery ?
ClockworkMod Recovery is a custom recovery for many Android devices. It is considered to be the most popular recovery for Android due to its easily-ported nature, and integration with ClockworkMod ROM Manager by Koush(Koushik Dutta). The easiest way to recognize it is by the printed name when it first starts, and the background logo of a gear and hat.
Click to expand...
Click to collapse
ERROR STATUS 6
This is usually caused by CR/LF EOL(Windows style End Of Line) in updater-script. Change it to LF EOL(Unix Style EOL) using Linux command: dos2unix updater-script, then re-signing the ZIP, will usually fix this error.
Click to expand...
Click to collapse
ERROR STATUS 7
This is usually caused by a corrupt download, or bad file signature. Re-downloading (or re-signing) the ZIP will usually fix this.
Click to expand...
Click to collapse
We have been consistently seen and heard people facing error “Status 7″ error while trying to flash or install
custom ROMs or firmware packages on their Android smart phones or tablets with ClockworkMod Recovery. Many
of the users are nowadays facing this problem with CWM Recovery while flashing .zip files of modded or custom
Ice Cream Sandwich (ICS) or Jelly Bean (JB) ROMs on their devices. So, you have also downloaded a custom ROM,
placed its .zip file in your phone’s or tablet’s SD card, booted into ClockworkMod Recovery, selected – “install zip
from sdcard” and then chosen the .zip file of the ROM to get it installed on your device. But instead of getting
flashed successfully, if you are facing the issue mentioned below, then just keep reading this article to find out
what’s wrong and fix up the problem :
Finding update package…
Opening update package…
Installing update…
Error in /sdcard/custom-jelly-bean-rom.zip (Status 7)
Installation aborted
Click to expand...
Click to collapse
or the following error right after CWM recovery shows –
Installing update…
assert failed: getprop(“ro.product.device”) == “I9103″ || getprop(“ro.build.product”) == “I9103″ || getprop
(“ro.product.board”) == “I9103″
Error in /sdcard/android-4-1-1-ics-rom-latest.zip (status 7)
Click to expand...
Click to collapse
So, if you are facing any of these errors while trying to install the desired custom ROM package on your Android
phone or tab, then you may try a various things or steps which may turn out to be the workaround of this
problem. Here are a few tips to get this “Status 7” error fixed in ClockworkMod Recovery and flash the ROM
successfully on your device :
(1) First of all, make sure your device’s bootloader is unlocked. If it is already unlocked but you are still
not able to flash the ROM, then just extract the .zip file of the ROM into a new folder, find the boot.img file from
that directory and flash it up on your phone or tablet via fastboot on your PC.
(2) Make sure that you are having the appropriate Radio or Baseband version installed on your device which is
supported by the custom ROM you are trying to flash. Most of the ROMs requires the latest version of Baseband, so
just update or upgrade your device to the latest Baseband version and then try to install the ROM once again.
(3) Update your device to the supported / latest build of official firmware before trying to install the ROM. You can
do it from – Settings > About Phone / Device > Software Update.
(4) Make sure you are having the supported or required kernel installed on your phone or tab. If it’s not, then flash
a new kernel right away and try to install your custom ROM once again.
(5) Is the ROM which you are trying to flash really works ? Find out whether it is working for other users or not.
Click to expand...
Click to collapse
Error Status 0
Well sometimes while flashing some ROMs especially the cooked ones we get Error status 0 in the CWM Recovery
this error is an indicator of Wrong Update Binary.This is usually caused by an incompatible update-binary in edify ZIPs. Replacing it with a compatible one, then re-signing the ZIP, will usually fix this error.
Click to expand...
Click to collapse
Partitions
Now it's time for the partitions :good:
Let’s start with a list of standard internal memory partitions on Android phones and tablets. These are:
/boot
/system
/recovery
/data
/cache
/misc
In addition, there are the SD card partitions.
/sdcard
/sd-ext
Note that only /sdcard is found in all Android devices and the rest are present only in select devices. Let’s now take a look at the purpose and contents of each of these partitions.
/boot
This is the partition that enables the phone to boot, as the name suggests. It includes the kernel and the ramdisk. Without this partition, the device will simply not be able to boot. Wiping this partition from recovery should only be done if absolutely required and once done, the device must NOT be rebooted before installing a new one, which can be done by installing a ROM that includes a /boot partition.
/system
This partition basically contains the entire operating system, other than the kernel and the ramdisk. This includes the Android user interface as well as all the system applications that come pre-installed on the device. Wiping this partition will remove Android from the device without rendering it unbootable, and you will still be able to put the phone into recovery or bootloader mode to install a new ROM.
/recovery
The recovery partition can be considered as an alternative boot partition that lets you boot the device into a recovery console for performing advanced recovery and maintenance operations on it. To learn more about this partition and its contents, see the ‘About Android Recovery’ section of our guide to ClockworkMod recovery.
/data
Also called userdata, the data partition contains the user’s data – this is where your contacts, messages, settings and apps that you have installed go. Wiping this partition essentially performs a factory reset on your device, restoring it to the way it was when you first booted it, or the way it was after the last official or custom ROM installation. When you perform a wipe data/factory reset from recovery, it is this partition that you are wiping.
/cache
This is the partition where Android stores frequently accessed data and app components. Wiping the cache doesn’t effect your personal data but simply gets rid of the existing data there, which gets automatically rebuilt as you continue using the device.
/misc
This partition contains miscellaneous system settings in form of on/off switches. These settings may include CID (Carrier or Region ID), USB configuration and certain hardware settings etc. This is an important partition and if it is corrupt or missing, several of the device’s features will will not function normally.
/sdcard
This is not a partition on the internal memory of the device but rather the SD card. In terms of usage, this is your storage space to use as you see fit, to store your media, documents, ROMs etc. on it. Wiping it is perfectly safe as long as you backup all the data you require from it, to your computer first. Though several user-installed apps save their data and settings on the SD card and wiping this partition will make you lose all that data.
On devices with both an internal and an external SD card – devices like the Samsung Galaxy S and several tablets – the /sdcard partition is always used to refer to the internal SD card. For the external SD card – if present – an alternative partition is used, which differs from device to device. In case of Samsung Galaxy S series devices, it is /sdcard/sd while in many other devices, it is /sdcard2. Unlike /sdcard, no system or app data whatsoever is stored automatically on this external SD card and everything present on it has been added there by the user. You can safely wipe it after backing up any data from it that you need to save.
/sd-ext
This is not a standard Android partition, but has become popular in the custom ROM scene. It is basically an additional partition on your SD card that acts as the /data partition when used with certain ROMs that have special features called APP2SD+ or data2ext enabled. It is especially useful on devices with little internal memory allotted to the /data partition. Thus, users who want to install more programs than the internal memory allows can make this partition and use it with a custom ROM that supports this feature, to get additional storage for installing their apps. Wiping this partition is essentially the same as wiping the /data partition – you lose your contacts, SMS, market apps and settings.
With this, we conclude our tour of Android partitions. Now whenever you install a ROM or mod that requires you to wipe certain partitions before the installation, you should be in a better position to know what you’re losing and what not and thus, you’ll know what to backup and what not.
You should at least post the source of such a large copy paste post.
Sent from my GT-N7100 using Tapatalk 2
Source? How do you post a source for an article which is compiled from 10+ sites? Plus my own addition?
Started from the bottom
Good job man, this saves me the time to do all this researches.
Keep it up
Best regards
Sifou
Using a Samsung N7100
sos_sifou said:
Good job man, this saves me the time to do all this researches.
Keep it up
Best regards
Sifou
Using a Samsung N7100
Click to expand...
Click to collapse
DO tell me if you have some suggestions for the thread.
"Thanks button is just to avoid "THANKS" posts in threads. Nothing more than that. Don't ask in signature or post for it and defeat the purpose why it was introduced"
I think that this is a pretty good summary of the basics. I even converted it to epub and stocked it on my e-reader for reference
You can get to the details if you want? Adding some info about flashing softwares like odin and the Linux based one (i don't remember it name)
The different recoveries available and their advantages vs désavantages
How to protect yourself from malicious applications, starting from knowing what are permissions...
Keep it up mate
Best regards
Sifou
Using a Samsung N7100
sos_sifou said:
I think that this is a pretty good summary of the basics. I even converted it to epub and stocked it on my e-reader for reference
You can get to the details if you want? Adding some info about flashing softwares like odin and the Linux based one (i don't remember it name)
The different recoveries available and their advantages vs désavantages
How to protect yourself from malicious applications, starting from knowing what are permissions...
Keep it up mate
Best regards
Sifou
Using a Samsung N7100
Click to expand...
Click to collapse
Heimdall?
"Thanks button is just to avoid "THANKS" posts in threads. Nothing more than that. Don't ask in signature or post for it and defeat the purpose why it was introduced"
Tha TechnoCrat said:
Source? How do you post a source for an article which is compiled from 10+ sites? Plus my own addition?
Started from the bottom
Click to expand...
Click to collapse
I guess you have a point, it's just the scientist in me with source-referral-ocd.
Sent from my GT-N7100 using Tapatalk 2
adytum said:
I guess you have a point, it's just the scientist in me with source-referral-ocd.
Sent from my GT-N7100 using Tapatalk 2
Click to expand...
Click to collapse
DO tell me if you have any problems or if you want something to be added.
"Thanks button is just to avoid "THANKS" posts in threads. Nothing more than that. Don't ask in signature or post for it and defeat the purpose why it was introduced"
Thread updated with Odin and Heimdall information.
"Thanks button is just to avoid "THANKS" posts in threads. Nothing more than that. Don't ask in signature or post for it and defeat the purpose why it was introduced"
Tha TechnoCrat said:
Source? How do you post a source for an article which is compiled from 10+ sites? Plus my own addition?
Started from the bottom
Click to expand...
Click to collapse
By listing ALL the different sources? And obviously crediting yourself with bits you've added.
Sent from my GT-N7100 using xda premium
You should make the title of the thread more presentable though.
Simone said:
You should make the title of the thread more presentable though.
Click to expand...
Click to collapse
Would like some suggestions.
"Thanks button is just to avoid "THANKS" posts in threads. Nothing more than that. Don't ask in signature or post for it and defeat the purpose why it was introduced"
Tha TechnoCrat said:
Would like some suggestions.
"Thanks button is just to avoid "THANKS" posts in threads. Nothing more than that. Don't ask in signature or post for it and defeat the purpose why it was introduced"
Click to expand...
Click to collapse
You should think of your own. That would be the best
Make it more professional looking, though.
Everything else is good.
Guys I have got my Note 2 finally. Will compile some guides for it too.
Sent from my GT-N7100 using xda app-developers app
You bought a note 2? Congrats mate !
Best regards
Sifou
Using a Samsung N7100
sos_sifou said:
You bought a note 2? Congrats mate !
Best regards
Sifou
Using a Samsung N7100
Click to expand...
Click to collapse
Thanks buddy. Get ready for more guides
Sent from my GT-N7100 using xda app-developers app
Hello everyone,
Please, just a question about repartitioning a PIT file via Odin. I´m a bit confused about information I have read about the result of the repartitioning operation.
In some forums appear to say the repartitioning operation via Odin is, first, a values rewriting of GPT entries, and after that any kind of formatting/wipe of all partitions.
In other words, if i repartition a PIT file exactly with the same values I have in my GPT table, does nothing occurs and the content of partitions keeps, or partitions data are lost?
Sorry about my english, and thank you for any answer you can give me.
D,
hi,
if i understand correctly, you would still be performing a repartition operation , which will destroy data during the process.
Although, if you have some experience with data/forensic recovery and can get the tools ported to your tab, it's likely
you may be able to recover a fair amount of what you lose without the data being corrupted.
That being said your repartitioning would need to succeed first.
The better approach for rescuing your data would be to pull the mmcblocks/partitions off of the device and onto
your pc [Linux] as img files through ADB [android debug bridge], BEFORE YOU PERFORM THE OPERATION.
That way if you fail in repartitioning [ which is highly likely]
your data will still be preserved on your pc. To be able to pull the information/data from the device, the device must be rooted
or have a custom recovery available with properly functioning access to/through adb for root functions and adb shell as root.
questions belong in q&a by the way.
m
hi,
if i understand correctly, you would still be performing a repartition operation , which will destroy data during the process.
Click to expand...
Click to collapse
Thank you for you answer. Well, what i´m trying to do is just an "experiment" resizing partitions. Of course there are other ways to do it, but what i´m thinking about is this:
Imagine the recovery partition was located in the first memory addresses, so, in one of the first GPT entries, and what i want to do is just modifiy size of last three partitions, so, three last entries in GPT table. If i create a new PIT file exactly with the same values that i have in the GPT table of the device but only modified values for the three last GPT entries, after perform a repartitioning via Odin and restart the device the recovery will be still there?
So, that is the sense of my question: Repartitioning does only write the values in the GPT table, or besides performs any kind of data lose in partitions (wipe,formatting...) ?
questions belong in q&a by the way.
m
Click to expand...
Click to collapse
Sorry and thank you, next time i´ll pay attention to it.
On older devices I had some success resizing partitions using parted in recovery mode via adb.
parted doesn't support ext4 as far as it's useful functions goes, you would have to create/resize any partition
as ext2 and reformat from there, cute approach but more hassle/trouble than it's worth.
for your experiment, be sure you can afford a new tab ! :silly:
i'm pretty sure your block layout is hardcoded in the bootloader. So you will probably end up creating
a very fashionable serving tray.
Meaning your device won't be able to find recovery partition, also there is likely a set amount of partitions allowed
by way of kernel if i remember correctly.
If your trying to get rid of that annoying no-execute permission in data thing, getting rid of FUSE would maybe get that done.
m
Thanks again for your answer. Experiment cancelled, currently no budget for a new tab.
Just a last -and sure a stupid- question, please: In your opinion, what would happen if i extract the PIT file from my device, and use itself to repartitioning via Odin? I mean, just specifying the PIT file and marking Re-Partition, other options (PDA,Phone,etc) unmarked? After restart, could work the tablet or I´ll get a brick?
Thanks again.
Regards.
D,
hi,
That's not a stupid question at all. I would suggest you read this thread all the way through
http://forum.xda-developers.com/tab-4/help/t530nu-pit-file-t2968498
also search via your preferred engine and XDA for terms/variations
samsung pit file signed odin heimdall
so far, the chances of repairing/modifying partition table on these newer devices [samsung] is slim/grim.
However maybe utilizing external/usb-otg storages to suit your needs would be a way to go. :good:
m
What it ain't: it's not a "how-to" manual. There are lots of threads out there that explain how to do things.
What it is: a fairly simple explanation of how things are on this phone written for those who, like me, have come from a phone with no A/B partitions (a far simpler world).
"Toto, I've a feeling we're not in Kansas anymore"The OnePlus 8T is an AB device. This means that a large number (but not all) of its hardware partitions have an A and a B version. The A set of partitions are called slot a, the B set are slot b.
The reason for this is to support seamless over-the-air (OTA) updates. These are updates to the operating system (OS) that can be installed while the phone is being used! Once the update has been installed the phone is rebooted (it automatically switches slots) and comes up on the newly updated version of the OS. If the update fails, the phone will simply revert to the original slot.
To implement this a number of things have changed:
The AB partitions have a currently active/running version (called the active slot) and an inactive version (the inactive slot) where the updates can be written to.
An update module (update_engine) is included in the OS to process the updates. Previously this would have been handled by code running in the recovery partition.
To avoid doubling up on large system partitions (system, system_ext, odm, product, vendor) and to can cater for increases in these partions, these are now carved out dynamically from a large super partition. This allows more effective use of the hardware space.
This has some important implications:
While it looks like you could have two versions of the operating system installed on the phone (one on slot a, one on slot b), this is not in fact the case. And that's because the super partition isn't large enough to hold two sets of system partitions. Just like there's only one version of the data partition, so there's only one version of the super partition (containing the system partitions).
You can write to hardware partitions in bootloader/fastboot mode using fastboot flash <partition> <file>, but you can't use this mode to write a specific file within a partition. And, therefore, you can't use this mode to write to the system partitions because they are files within the super partition. For these, you need to use fastbootd which is part of recovery.
Flashing/installing a ROM using the update engine (as done by System update of OOS or a recovery from LineageOS or TWRP) will alwaysbe written to the inactive slot. So, if you are flashing a custom ROM and want it to use the latest OOS firmware installed on the phone, then you will have to either:
Copy the firmware from the currently active slot to the inactive one, or
Swap slots before you do the install so that it gets installed on what was the active slot containing the correct firmware.
If you are used to using TWRP to backup your system partitions, then the way to do it now is to backup the super partition and not the system, system_ext, odm, product, or vendor partitions.
"It's complicated""Delta variants"Updates (whether Over-the-air (OTA) or from the OnePlus download site) come in two flavours:
Full update: the files for each updated partition contain the full image. These are quite large, typically about 3-4GB.
Delta update: the files contain only the changes to be applied to each partition and can be quite small in size (MB rather than GB).
Since a delta update only contains changes, it is imperative that the update process check that the current partition contains the expected data that will be updated. The update_engine will check that the current partition hash is what the update is expecting. If it isn't, the update will fail.
If a partition has been modified (you have installed Magisk which changes the boot partition, or installed TWRP which changes the recovery partition), then you will have to restore the changed partition back to their original stock version before starting the update process.
That's why it is best practice to take a copy of the stock partition before you modify it.
"Good memories"The 8T came out originally with LPDDR4X memory, but OnePlus then decided to improve it by using LPDDR5 memory. As a result of this change, the xbl and xbl_config partitions now hold different data depending on the type of memory used in the phone. Note: you can find out what type of memory your phone has using:
on OOS 11: getprop ro.boot.ddr_type - 0 means LPDDR4X, 1 means LPDDR5.
on OOS 12 (and later): cat /proc/devinfo/ddr_type - you need to run this as root.
The way this is implemented is by including files for both LPDDR4X (files named xbl, xbl_config) and LPDDR5 (files named xbl_lp5, xbl_config_lp5) in updates/downloads. OnePlus had to make custom changes to update_engine to cater for this. It is only as of 2021-12-22 that a version of TWRP was produced that could handle these files correctly.
"Hitting the books"Google has some great documentation about AB devices. If you are using your phone to read this, make sure you use your browser in desktop mode so that you can see the Google documentation's navigation pane on the left hand side.
A/B (Seamless) System Updates: https://source.android.com/devices/tech/ota/ab
Dynamic Partitions: https://source.android.com/devices/tech/ota/dynamic_partitions
Fastbootd: https://source.android.com/devices/tech/ota/dynamic_partitions/implement#fastbootd and https://source.android.com/devices/bootloader/fastbootd
Virtual A/B Overview: this covers how the system partitions in super get updated and it's fairly complex https://source.android.com/devices/tech/ota/virtual_ab
Thanks Bill. Very helpful and that is why I'm still on OOS for now. I'm sure I am missing something but I have several questions.
1. How is the 8T different from the 8 or 8Pro as far as OOS is concerned? Is it a different OS architecture?
2. When you use another OS like LineageOS, how does it compare to OOS as far as the internals go (super partition, etc.). Is the OS completely different in the way it uses the hardware and the way updates occur?
3. When we refer to firmware what do we actually mean (code designed exactly for the specific harware interface or what)? Does the firmware stay the same for any OS that is used and if so how is that accomplished? Are there certain areas of the system that are not touched by any new OS for that phone except the OEM?
I'm trying to wrap my head around how this phone seems to be so different from even other OP phones.
Thanks John
zzjea said:
Thanks Bill. Very helpful and that is why I'm still on OOS for now. I'm sure I am missing something but I have several questions.
1. How is the 8T different from the 8 or 8Pro as far as OOS is concerned? Is it a different OS architecture?
2. When you use another OS like LineageOS, how does it compare to OOS as far as the internals go (super partition, etc.). Is the OS completely different in the way it uses the hardware and the way updates occur?
3. When we refer to firmware what do we actually mean (code designed exactly for the specific harware interface or what)? Does the firmware stay the same for any OS that is used and if so how is that accomplished? Are there certain areas of the system that are not touched by any new OS for that phone except the OEM?
I'm trying to wrap my head around how this phone seems to be so different from even other OP phones.
Thanks John
Click to expand...
Click to collapse
I am not Bill, but I will still attempt to answer your questions.
1. There are no real difference between OOS for 8T compared to the 8/8 Pro. They all use the same SOC (Snapdragon 865). For the last few OOS updates the changelog has been the same between 8/8Pro and 8T. Only thing that is different compared to the 8/8Pro is that the 8T device can have either LPDDR4X or LPDDR5 ram. While the 8 and 8 Pro only have either of them depending on the model. There are also plenty of unified custom kernels that work for all 3 devices as well.
2. I am no custom rom expert but I think they still handle the super partition and updates in the same way, but it is best to check the thread for the rom and read the update instructions there.
3. The FAQ in this thread should answer your questions. It also contain flashable firmware packages. These packages are extracted from the stock OOS rom and can be used to update firmware if you are on a custom rom https://forum.xda-developers.com/t/firmware-oneplus-8t-kebab-ddr0-ddr1-updated.4299231/
zzjea said:
Thanks Bill. Very helpful and that is why I'm still on OOS for now. I'm sure I am missing something but I have several questions.
1. How is the 8T different from the 8 or 8Pro as far as OOS is concerned? Is it a different OS architecture?
2. When you use another OS like LineageOS, how does it compare to OOS as far as the internals go (super partition, etc.). Is the OS completely different in the way it uses the hardware and the way updates occur?
3. When we refer to firmware what do we actually mean (code designed exactly for the specific harware interface or what)? Does the firmware stay the same for any OS that is used and if so how is that accomplished? Are there certain areas of the system that are not touched by any new OS for that phone except the OEM?
I'm trying to wrap my head around how this phone seems to be so different from even other OP phones.
Thanks John
Click to expand...
Click to collapse
1. I don't know anything about the other OP 8 devices. I'm only interested in the 8T because that's what I have.
2. The hardware determines how the OS must be structured. So all ROMs have to support the A/B structure and dynamic system partitions.
3. I think of Firmware as phone specific software/data related to the underlying vendor-specific hardware components/chip set. It's required by the OS to access the hardware, but is not included in custom ROMs. It's vendor-specific.
BillGoss said:
What it ain't: it's not a "how-to" manual. There are lots of threads out there that explain how to do things.
What it is: a fairly simple explanation of how things are on this phone written for for those who, like me, have come from a phone with no A/B partitions (a far simpler world).
"Toto, I've a feeling we're not in Kansas anymore"The OnePlus 8T is an AB device. This means that a large number (but not all) of its hardware partitions have an A and a B version. The A set of partitions are called slot a, the B set are slot b.
The reason for this is to support seamless over-the-air (OTA) updates. These are updates to the operating system (OS) that can be installed while the phone is being used! Once the update has been installed the phone is rebooted (it automatically switches slots) and comes up on the newly updated version of the OS. If the update fails, the phone will simply revert to the original slot.
To implement this a number of things have changed:
The AB partitions have a currently active/running version (called the active slot) and an inactive version (the inactive slot) where the updates can be written to.
An update module (update_engine) is included in the OS to process the updates. Previously this would have been handled by code running in the recovery partition.
To avoid doubling up on large system partitions (system, system_ext, odm, product, vendor) and to can cater for increases in these partions, these are now carved out dynamically from a large super partition. This allows more effective use of the hardware space.
This has some important implications:
While it looks like you could have two versions of the operating system installed on the phone (one on slot a, one on slot b), this is not in fact the case. And that's because the super partition isn't large enough to hold two sets of system partitions. Just like there's only one version of the data partition, so there's only one version of the super partition (containing the system partitions).
You can write to hardware partitions in bootloader/fastboot mode using fastboot flash <partition> <file>, but you can't use this mode to write a specific file within a partition. And, therefore, you can't use this mode to write to the system partitions because they are files within the super partition. For these, you need to use fastbootd which is part of recovery.
Flashing/installing a ROM using the update engine (as done by System update of OOS or a recovery from LineageOS or TWRP) will alwaysbe written to the inactive slot. So, if you are flashing a custom ROM and want it to use the latest OOS firmware installed on the phone, then you will have to either:
Copy the firmware from the currently active slot to the inactive one, or
Swap slots before you do the install so that it gets installed on what was the active slot containing the correct firmware.
If you are used to using TWRP to backup your system partitions, then the way to do it now is to backup the super partition and not the system, system_ext, odm, product, or vendor partitions.
"It's complicated""Delta variants"Updates (whether Over-the-air (OTA) or from the OnePlus download site) come in two flavours:
Full update: the files for each updated partition contain the full image. These are quite large, typically about 2.8GB.
Delta update: the files contain only the changes to be applied to each partition and can be quite small in size (MB rather than GB).
Since a delta update only contains changes, it is imperative that the update process check that the current partition contains the expected data that will be updated. The update_engine will check that the current partition hash is what the update is expecting. If it isn't, the update will fail.
If a partition has been modified (you have installed Magisk which changes the boot partition, or installed TWRP which changes the recovery partition), then you will have to restore the changed partition back to their original stock version before starting the update process.
That's why it is best practice to take a copy of the stock partition before you modify it.
"Good memories"The 8T came out originally with LPDDR4X memory, but OnePlus then decided to improve it by using LPDDR5 memory. As a result of this change, the xbl and xbl_config partitions now hold different data depending on the type of memory used in the phone. Note: you can find out what type of memory your phone has using getprop ro.boot.ddr_type - 0 means LPDDR4X, 1 means LPDDR5.
The way this is implemented is by including files for both LPDDR4X (files named xbl, xbl_config) and LPDDR5 (files named xbl_lp5, xbl_config_lp5) in updates/downloads. OnePlus had to make custom changes to update_engine to cater for this. It is only as of 2021-12-22 that a version of TWRP was produced that could handle these files correctly.
"Hitting the books"Google has some great documentation about AB devices. If you are using your phone to read this, make sure you use your browser in desktop mode so that you can see the Google documentation's navigation pane on the left hand side.
A/B (Seamless) System Updates: https://source.android.com/devices/tech/ota/ab
Dynamic Partitions: https://source.android.com/devices/tech/ota/dynamic_partitions
Fastbootd: https://source.android.com/devices/tech/ota/dynamic_partitions/implement#fastbootd and https://source.android.com/devices/bootloader/fastbootd
Virtual A/B Overview: this covers how the system partitions in super get updated and it's fairly complex https://source.android.com/devices/tech/ota/virtual_ab
Click to expand...
Click to collapse
Device OnePlus 8 Pro
OOS 11.0.10.10 IN11BA
TWRP 3.6
Root Magisk 23.0
Hello
Hello
I noted in your information above
“If you are used to using TWRP to backup your system partitions, then the way to do it now is to backup the super partition and not the system, system_ext, odm, product, or vendor partitions. »
It is precisely the use of TWRP that I took to back up my old LG or Samsung devices (S7, EDG or Tab 4 tablet)
Before the appearance of "dynamic partitions", there were no questions to ask to make a "Nandroid Backup" of your smartphone.
Now, since the arrival of OOS 10 and 11, things are different.
So here is my question for you:
If I check all the boxes by doing the backup and restore with TWRP (version 3.6.0), will the smartphone return to its general and precise state at the time of the backup? (See attached screenshots from TWRP)
If I understood correctly, TWRP performs a global image (super partition including A and B ) of all partitions at the time of backup, similar to an image of an SSD or HDD with several partitions… That's good that ?
I thank you in advance for your information and explanations and your instructions so that I can perform a "Nandroid Backup" and a possible restoration in complete safety.
Best regards
Jean-Noel
nadnan said:
Device OnePlus 8 Pro
OOS 11.0.10.10 IN11BA
TWRP 3.6
Root Magisk 23.0
Hello
Hello
I noted in your information above
“If you are used to using TWRP to backup your system partitions, then the way to do it now is to backup the super partition and not the system, system_ext, odm, product, or vendor partitions. »
It is precisely the use of TWRP that I took to back up my old LG or Samsung devices (S7, EDG or Tab 4 tablet)
Before the appearance of "dynamic partitions", there were no questions to ask to make a "Nandroid Backup" of your smartphone.
Now, since the arrival of OOS 10 and 11, things are different.
So here is my question for you:
If I check all the boxes by doing the backup and restore with TWRP (version 3.6.0), will the smartphone return to its general and precise state at the time of the backup? (See attached screenshots from TWRP)
If I understood correctly, TWRP performs a global image (super partition including A and B ) of all partitions at the time of backup, similar to an image of an SSD or HDD with several partitions… That's good that ?
I thank you in advance for your information and explanations and your instructions so that I can perform a "Nandroid Backup" and a possible restoration in complete safety.
Best regards
Jean-Noel
Click to expand...
Click to collapse
TWRP does image backups generally, but does file-based backups for data and some others (if I remember correctly, persist and EFS).
The "system" partitions on the OP8T are, as I mentioned, boot, dtbo, and super. This means that a "full" backup like in the "old days" is data plus the "system" partitions.
You could backup the other items in the TWRP backup list, but I'm going to ask you: Why do you want you do that?
BillGoss said:
TWRP does image backups generally, but does file-based backups for data and some others (if I remember correctly, persist and EFS).
The "system" partitions on the OP8T are, as I mentioned, boot, dtbo, and super. This means that a "full" backup like in the "old days" is data plus the "system" partitions.
You could backup the other items in the TWRP backup list, but I'm going to ask you: Why do you want you do that?
Click to expand...
Click to collapse
Hello
Thank you for your feedback.
I have taken note of the three system partitions you indicate.
With devices without a dynamic partition, I was "used" to saving all the data "as a precaution" on an external memory card and I was not limited in storage.
On the OnePlus 8 Pro, I have 256 GB of storage (without an external SD Card) and I could effectively limit the backup to the “Boot”, “DTBO” and “Super (System Syystem_ext product vendor)” partitions that you specify.
But I think that if I want to protect myself from any malfunction of the device after the restoration, it is better that I select all the partitions and then copy to PC the whole TWRP folder created by the " Nandroid”.
I could also buy a USB C / OTG adapter / SD card reader to store the backup…
Do you think I'm wrong?
Thanks again for your advice….
Best regards,
nadnan said:
Hello
Thank you for your feedback.
I have taken note of the three system partitions you indicate.
With devices without a dynamic partition, I was "used" to saving all the data "as a precaution" on an external memory card and I was not limited in storage.
On the OnePlus 8 Pro, I have 256 GB of storage (without an external SD Card) and I could effectively limit the backup to the “Boot”, “DTBO” and “Super (System Syystem_ext product vendor)” partitions that you specify.
But I think that if I want to protect myself from any malfunction of the device after the restoration, it is better that I select all the partitions and then copy to PC the whole TWRP folder created by the " Nandroid”.
I could also buy a USB C / OTG adapter / SD card reader to store the backup…
Do you think I'm wrong?
Thanks again for your advice….
Best regards,
Click to expand...
Click to collapse
Backing up partitions that you don't understand how they are used is rather pointless in my mind.
Restoring a partition that you don't understand how it's used is asking for trouble.
So I would never do that. But this is your phone and you can do whatever you want with it. Though learning more about how your device works and how the various partitions are used is a really good idea.
PS: you say you have an 8 Pro. You do realise that this is an 8T forum and that I'm writing about the 8T, not any other phone?
BillGoss said:
Backing up partitions that you don't understand how they are used is rather pointless in my mind.
Restoring a partition that you don't understand how it's used is asking for trouble.
So I would never do that. But this is your phone and you can do whatever you want with it. Though learning more about how your device works and how the various partitions are used is a really good idea.
PS: you say you have an 8 Pro. You do realise that this is an 8T forum and that I'm writing about the 8T, not any other phone?
Click to expand...
Click to collapse
Thank you for your remarks
Yes, I know about the 8T forum but I got here while researching the web and read that the OOS build is the same for both devices...
On the other hand, I have retained your words to know what each partition was used for and I will also direct my research on this subject.
Thanks again for your availability...
Sincerely...
Very intersting topic, and great explenation so far. I've came from old-school smartphone (Pocophone F1, without A/B partitions) and everything was so simple back then... I was about to switch from ROM to ROM on my brand new OP 8T , but I came into your topic and I realised that's something different. I have to search for some more info, because I'm looking forward to switch to colorOS (Or just other ROM in general. I just like switching and modifying my phone xD), but I'm wondering if, and how can I restore everything to stock, will I loose some widevine, mobile payment etc. I think everything is possible, I just have to make proper research. Anyway thank you again for that very interesting article!