But... why?
Well, many people like and buy the Xperia line of devices currently on the market who previously owned a Samsung, LG or HTC. There are many differences in hardware, but most certainly in how Sony perceives the Android ecosystem and how it differs from the other major brands. This (noob) guide is meant to help people on the path to the Sony side I'm sure it contains lots of info which is even useful for the old timers
Things which differ a LOT from the other brands:
First and foremost: no recovery partition;
Second but not least: no download mode.
Sony has replaced the Android recovery partition with the FOTA kernel, which is meant to aid the device in rolling out OTA updates, which allow kernel updates without the risk of bricking the device. I hear you scream: "But wait, what about the recovery partition announcement by Sony themselves!?", well the answer is simple: that is meant for unlocked bootloader devices ONLY, as a part of their "Open Devices" program.
What is Flashmode, Flashtool and what are these FTF and SIN files I am reading about?
I'm going to quote @Androxyde here (it's a straight copy of his index page), as he is the maintainer of the tool:
Flashtool is a S1 flashing software that works for all Sony phones from X10 to Xperia Z Ultra. They all use the S1 protocol for flashing firmwares.
This program was originally made to flash sin files downloaded by SEUS/SUS or PC Companion.
Based on a command line tool written by @Bin4ry (Andreas Makris), I brought a user interface to sin files flashing.
We worked together to add more features to the tool such as rooting methods implementation or TA backup / restore.
Then I took the lead and got some advice and help from him occasionally on some features like rom cleaner or bootloader unlocking.
From time to time, sin files have been bundled into what is now well known FTF (Flash Tool Firmwares) and more features have been implemented.
But flashing firmwares is still the core of Flashtool (that is updated at least to follow Sony improvements around sin files) and the reason of its name.
Flashtool can also easily unlock the bootloader of the phone using the BLU icon as far as the bootloader of your phone is unlockable
The flashing feature as well as bootloader unlock feature are available whatever the phone is recognized or not by the application. What is only mandatory for flashing is to own the FTF file according to the device you want to flash it on.
Why should I use Flashtool?
Once bootloader unlocked, official sony tools do not work anymore.
Using official sony tools, you can only upgrade. No downgrade possible.
Using flashtool, you can choose what to flash and what not to flash. This said, many rooting scenarios are available implying kernel only downgrade to retrieve a patched rooting exploit and then flash back the right kernel.
You said unlocking bootloader?
This process gives you the opportunity to flash custom roms such as CyanogenMod ROMs.
I invite you to visit the FXP Project that brings CM and AOSP to xperia devices.
Click to expand...
Click to collapse
To complete this explanation, Sony devices know 2 bootloader based flash modes:
Flashmode (This is the S1 flash protocol @Androxyde wrote about)
Fastboot (This is the original, unmodified fastboot mode from Google)
In the past there was the Sony-Ericsson Update Software/Service (SEUS, later named SUS because Ericsson got removed from the name) which could update your device to the latest software or recover it from an inoperable state. PC Companion was less of a tool for updating and more of making backups, installing applications and managing the device storage. Sometime the past 3-4 years Sony merged the 2 programs, so if people on XDA or anywhere on the web talk about using PC Companion to restore their device, they are not wrong, they are simply using the latest version
Sony recently released their own package called 'flash tool' (to add to the confusion of noobs in the community), which deep down is a little easier to use and stripped down version of EMMA, which is a tool we mere mortals will probably never use as EMMA is the flash tool for Sony's repair shops and tech support. It's primary function is to allow owners of an unlocked bootloader device (or, 'open device' as Sony named it) to still use Sony firmwares and update their device, because as soon as you unlock the bootloader, the OTA updates will stop.
Things to remember on the files used by these tools:
An FTF file is basically not more then a ZIP archive containing multiple SIN files, you can open the FTF using 7z/WinZip/WinRar and look inside it.
A SIN file is a disk/partition image, which is encrypted by Sony. S1 (the bootloader) will check this encryption to make sure the image was not tampered with before it accepts it for flashing.
Fastboot flashing will always fail when the device is still in a locked bootloader state. For some models it is even disabled entirely or non-functional until you unlock the bootloader.
I am reading about the TA, what is it, why should I make a backup?
The TA partition (Trim Area) is a signed partition which holds various things which are unique to your device, like the device's IMEI, DRM keys and bootloader settings and configuration options. This partition can not be exchanged between devices, because it really is unique. If you would flash the TA backup from someone else it will cause a hard-brick rendering your device only useful as a paperweight...
When you unlock your bootloader you will lose all the DRM features on your device, this makes it valuable to have a backup of the unmodified version stowed away somewhere safe. You will need root level access to create that backup before unlocking. There is a tool called Backup TA which is widely used to create and restore backups of the TA partition. TWRP in XZDualRecovery can do the same.
When you restore the backup TA partition you made before unlocking the bootloader you will essentially re-lock the bootloader and restores the DRM keys. This process is (as far as we know up to now) undetectable by Sony's support staff, which makes it easy to restore the phone to stock for warranty driven support issues as unlocking your bootloader will void your warranty on the device (it is subject to local law though). That is why, for a lot of owners of a Sony device at least, it is considered to be the "Holy Grail" and is usually the reason for a lot of users to wait for a root exploit to be found before unlocking their bootloader.
Okay, I get it now. I would like to unlock my bootloader, how to proceed?
I'm not here to rewrite everything other people or Sony themselves can write just as well or even better, so I have a link for you:
http://developer.sonymobile.com/unlockbootloader/
Read it, it will teach you just about everything you need to know.
Once your bootloader is unlocked, your device will be much like a Nexus device when it comes to rooting, excluding the recovery partition, so that's why we flash or hotboot a custom kernel with a recovery, by using fastboot. If you are afraid of a terminal and typing commands, you can use a tool like QuickIMG or Flashtool to make your life easier.
Right, now I want root!
Well, if you have an 'open device', this is a lot easier then you think. Just remember that using root exploit kits is unnecessary and in some cases even risky as some packages do funky things or jeopardize your privacy.
Try to find a 'stock based' custom kernel. These are custom kernels built by the community to add features to the kernel but are meant to work with Sony's stock firmwares. I'm the maintainer of XZDualRecovery myself and created the Kernel Builder for the supported devices.
These custom kernels will NOT root your device (unless otherwise stated by the creator), but introduce a recovery to the boot process and with that you will be able to flash SuperSU to root your ROM.
But you just said Sony devices don't have a recovery partition, please... UN-confuse me!!
Yes, I did, and I'm right: there have been bright minds in the community who included a recovery in the boot image (the kernel partition) in the past and that way included a recovery on our Sony devices.
With the current 'open devices' policy from Sony, we now have:
Recovery stored inside the system partition, which is meant for locked bootloader devices (closed devices) because they can not run custom kernels;
Recovery stored in the boot image (for open devices);
Recovery stored on the FOTA partition, but with a trigger from the regular boot image at boot (also for open devices);
Recovery stored on the FOTA partition -renamed to recovery- together with an updated bootloader (for open devices, of course).
Hmm, okay... it's still confusing, but OK. My service menu says I'm rooted, but none of the root apps work properly, what gives?!
If you open the phone dialer app and on the keys see the letters below the digits, you can spell the word SERVICE. Type *#*#SERVICE#*#* and a service menu will pop up. Tap 'Service Info' and then 'Configuration'. Then you will see one of these lines there almost on the bottom of the list:
"Rooting status: unknown": it's probably unlocked, but it was unable to verify that;
"Rooting status: rooted": you have unlocked the bootloader;
"Unlock bootloader allowed: YES/NO": this tells you if the bootloader is (vendor-) locked or not, if it says NO, you're out of luck.
The rooting status there is not telling your system is rooted, it tells you your bootloader is and will allow custom rom/kernel flashing. Don't confuse these two.
I'm not allowed to unlock my bootloader But I still want root, can I?
In some cases you can. It depends on the bugs found in specific firmware versions which allow a root exploit to be developed.
From the 2015 range of Xperia devices Sony started using dm-verity, which causes a bootloop once the system partition is modified. This modification of the system partition will be required to include a SU binary in the system to obtain root, so until a dm-verity defeating option is found, locked bootloader root or recovery will not be possible.
For older models, check the device forums and the cross device development forums to check out the community rootkits available. Usually it will tell you what ROM version it is intended for. Be careful with rootkits/roottools though, some are also found to be introducing malware to your device or sending privacy sensitive data to the creators. Use common sense, if you have no valid use for the root user level, keep it off your phone. If you already have recovery, you can use that to modify or clean your device instead.
I have rooted my phone, but whenever I try to modify something on it it spontaneously reboots or I get a message 'Permission denied" when trying to remount the system partition R/W! Why is that?
Like all manufacturers, Sony tries to make it difficult (or downright impossible) to modify the Android base system they created. Because if you can, anyone or anything which obtains root access can. This is a serious security risk, because if it's malware which puts itself on the system partition and locks up your phone, the only way around this is to wipe your entire device and restore a stock ROM using PC Companion or Flashtool. Of course, they have their own proprietary software to protect as well, but security is the main objective here. The really sensitive bits are stored in the TA partition as I explained earlier.
Sony (-Ericsson) had a service called RIC, which in time moved partially in to a kernel feature. What it does is monitor if system is remounted writeable. This usually is a situation you want to avoid at all costs so RIC will deny you permission, cause a kernel panic OR simply reboots your device to get out of that state.
"Remount-Reboot fix", RICKiller, RICDefeat, and XZDualRecovery all (attempt to) disable this service or stop the kernel from acting on a remount of system.
Hard-bricks, Soft-bricks, bootloops??
They are simple to understand, really:
Hard-brick, TYPICALLY NOT RECOVERABLE: The bootloader stopped functioning, this can be caused by a bad flash/update or by restoring the wrong TA backup.
Soft-Brick, ALWAYS RECOVERABLE: the system partition is corrupted or just simply empty, this causes the device to stall at boot. A soft-brick can also make the screen remain off, because of a bad or missing kernel image.
Bootloops, ALWAYS RECOVERABLE: If the system gets powered up and then reboots during the start. This can be at the kernel splash screen or during the boot animation.
In case of a Soft-brick or Bootloop:
Use the installed recovery (if it still works), PC Companion, QuickIMG or Flashtool to restore your device to working order.
In case of a Hard-brick:
You can never recover from that state without physically opening your device and do some heavy duty engineering (JTagging) on it to flash back the correct bootloader/TA (read that link to see what it would take!). This is way too difficult for 98% of the community, which means that hard-bricking your device is typically the creation of a very expensive paper weight.
Please, be extremely careful when dealing with the TA partition.
*********************************************
I will be updating the above text for sure, if you feel anything is missing, please write a post in this thread with the text you wish to include. I want this to be a community driven guide and I know a lot, but I can't know everything
*********************************************
Extended the text some more to include ideas from:
@Klaos3000
@Yenkazu
Thanks for the suggestions/additions guys! :highfive:
As it concern the recovery, i think you can create a partition with EMMA.
Sent from Greece
kos25k said:
As it concern the recovery, i think you can create a partition with EMMA.
Sent from Greece
Click to expand...
Click to collapse
Please, re-read the first part...
Can I root my iPhone 6 with this guide?
That was a bad joke.
Very useful guide. We should probably educate people around here about what an unlocked bootloader actually means and what it let us do. I'm shocked by the number of people using Kingroot and other risky closed-sourced tools, especially the ones with an unlocked bootloader.
People of earth, if your bootloader is unlocked, it means that your device will be much like a Nexus device when it comes to rooting, sans the recovery partition, so that's why we flash or hotboot a custom kernel with a recovery, by using fastboot. If you're afraid of a terminal and commands, then you can use a tool like QuickIMG. After that, you simply flash SuperSU. That's it!
You can add something about TA Partition and RIC server. :3
Good thread btw
Yenkazu said:
You can add something about TA Partition and RIC server. :3
Good thread btw
Click to expand...
Click to collapse
I'd say, give me a piece of text on the subject for the OP and I'll include it :good:
[NUT] said:
I'd say, give me a piece of text on the subject for the OP and I'll include it :good:
Click to expand...
Click to collapse
RIC, from your thread :3
http://forum.xda-developers.com/xpe...b-definitive-root-remount-reboot-fix-t2317432
But, it's kinda useless if people already use your DualRec, since it's already integrated xD
But more info didn't hurt (?)
TA Partition
http://forum.xda-developers.com/xperia-z/help/ta-partition-t2451186
Not really details, but user should know the impact of unlocking bootloader
Updated the OP to include info on the TA partition and RIC protection. Also included a part of the post by @Klaos3000, because it contained some useful info
Thanks guys :highfive:
Very usefull! :good: But for me 2-3 Weeks to late. I'm still quite new on Z3C and I collect all those info the old style
Without this thread you would need days to catch all dependencies - With this you would need approx. 10 min!
Very helpfull and good to link new user to...
And yes - I came form the Sammy side (of the moon ) and was a bit shocked what sony did with "open source android".
Not because of SystemUI ( I love it...) but because of all this "anti modding" stuff they build in.
Sticky? Sure - must be!
Updated the OP to include info on Hard-bricks, Soft-bricks and bootloops.
Please people, if you have anything to add to the OP, let me know!
As I said, I know a lot, but I can't know everything there is to know about Sony devices...
Good work (...as usual from your side) :good:
Very useful tutorial.
No need to write long explanations to Sony beginners anymore - just add a link from here.
Really Helpful
Brilliant.....Really it deserves place at (Sticky Threads).I think if you add minimum One Custom Kernel(for stock firmware)
of every devices..That would b very helpful to recover from Soft bricks.Then this thread will be an "ALL IN 1" thread.Its my Opinion after all...Brilliant work.
Need a little advice.
Hi,
I have a ZL with Locked bootloader and your ZL-lockeddualrecovery2.8.22 installed
I am on stock 5.0.2 now Rooted thanks to your awesome recovery.
Question is now I have your dual recovery would I be able to simply flash crDroid CM zip Thread Here and others like it or would I still need to unlock the bootloader Edit OP of ROM says it is required
So if I flashed the above would it replace your recovery with the boot.img in the zip
Sorry for the noob questions but I am new to Sony devices and still taking baby steps with this phone
I can unlock the bootloader no problem but I am more concerned about your recovery been replaced.
Thanks in advance :good:
bigrammy said:
Hi,
I have a ZL with Locked bootloader and your ZL-lockeddualrecovery2.8.22 installed
I am on stock 5.0.2 now Rooted thanks to your awesome recovery.
Question is now I have your dual recovery would I be able to simply flash crDroid CM zip Thread Here and others like it or would I still need to unlock the bootloader Edit OP of ROM says it is required
So if I flashed the above would it replace your recovery with the boot.img in the zip
Sorry for the noob questions but I am new to Sony devices and still taking baby steps with this phone
I can unlock the bootloader no problem but I am more concerned about your recovery been replaced.
Thanks in advance :good:
Click to expand...
Click to collapse
As long as your phone is locked, you only can flash stockroms and stock-kernels.
You also have to use recovery for stockrom, because you kernel is "untouchable" and recovery have to put in /data and /system partition and can't be put in kernel.
All other roms/kernels will end in errors while flashing.
If you have unlocked your BL, you can flash any rom and kernel you want, as long as your phone is supporting it.
Yes, by flashing a zip, your kernel will be replaced and in most cases they have a recovery in it.
By flashing a rom from another version or changing from stock-based roms to i.e. CM-roms or Omni and vice versa, you have to unpack the kernel (boot.img) by hand and flash (fastboot) this first, before you flash (after a reboot in recovery) the whole zip.
Otherwise it could end in bootloop.
Because of (i most cases) wiping /system, /data while installation, your stock-recovery will deleted too. This depends of the work of the installer in the zip file.
Btw... before unlocking your phone, backup your TA ( with FlashTool). This TA is unique and you may use it to lock your phone again later.
And... if your phone is unlocked - no fear of losing recovery. You always can flash another one with fastboot again.
Someone correct me, if i was wrong or forgot something.
@bigrammi, you can always try yo repack the CM kernel using my kernel builder, that way you will still have XZDualRecovery but then included in the boot image, so no risk of losing it...
akkufix said:
As long as your phone is locked, you only can flash stockroms and stock-kernels.
You also have to use recovery for stockrom, because you kernel is "untouchable" and recovery have to put in /data and /system partition and can't be put in kernel.
All other roms/kernels will end in errors while flashing.
If you have unlocked your BL, you can flash any rom and kernel you want, as long as your phone is supporting it.
Yes, by flashing a zip, your kernel will be replaced and in most cases they have a recovery in it.
By flashing a rom from another version or changing from stock-based roms to i.e. CM-roms or Omni and vice versa, you have to unpack the kernel (boot.img) by hand and flash (fastboot) this first, before you flash (after a reboot in recovery) the whole zip.
Otherwise it could end in bootloop.
Because of (i most cases) wiping /system, /data while installation, your stock-recovery will deleted too. This depends of the work of the installer in the zip file.
Btw... before unlocking your phone, backup your TA ( with FlashTool). This TA is unique and you may use it to lock your phone again later.
And... if your phone is unlocked - no fear of losing recovery. You always can flash another one with fastboot again.
Someone correct me, if i was wrong or forgot something.
Click to expand...
Click to collapse
Thanks yet again bro :highfive:
Wow these Sony Xperia's take some figuring out :laugh:
I think I have just about got my head around it all now
I have managed to get the TA backed up with TWRP and Flashtool so I should be safe now
I will have to unlock the bootloader just to stop it nagging me to upgrade :laugh:
@Nut Thanks bro I will take a look at your suggestion it's a little more complicated than what I am used to or should I say different.
bigrammy said:
[...]
@Nut Thanks bro I will take a look at your suggestion it's a little more complicated than what I am used to or should I say different.
Click to expand...
Click to collapse
Well, i saw HTC M7 in your signature. If you were able to unlocked, s-off-ed, re-flashed firmware and rooted this beast - you don't need to have any fear about a Sony device.
[NUT] said:
@bigrammymi, you can always try yo repack the CM kernel using my kernel builder, that way you will still have XZDualRecovery but then included in the boot image, so no risk of losing it...
Click to expand...
Click to collapse
Hi @Nut,
I thought I would take your advice and expected to download a tool to unpack everything and pick through the files and start editing init.rc etc etc :silly:
I had no idea this was a fully automated Online tool
I still can't quite believe it you're a genius!! :angel:
A BIG THANKS to All the Xperia dev's helpful community members and especially the tool creators XZDualRecovery, Flashtool, XperiFirm and PRFCreator etc you're all Awesome :highfive:
I fear I will become lazy with such great dev's :laugh:
bigrammy said:
Hi @Nut,
I thought I would take your advice and expected to download a tool to unpack everything and pick through the files and start editing init.rc etc etc :silly:
I had no idea this was a fully automated Online tool
I still can't quite believe it you're a genius!! :angel:
A BIG THANKS to All the Xperia dev's helpful community members and especially the tool creators XZDualRecovery, Flashtool, XperiFirm and PRFCreator etc you're all Awesome :highfive:
I fear I will become lazy with such great dev's :laugh:
Click to expand...
Click to collapse
Thanks, glad to have been of help to you
Well, I have investigated a way to root and backup the drm keys before unlock the bootloader, and i found this guide to root an impossible tablet SGP21 with a system of a z3 phone d6603 vulnerable firmware and kernel:
http://forum.xda-developers.com/z3-...t/root-root-locked-bootloader-sgp621-t3013534
So maybe we can adapt this guide to work with our device and root with giefroot or with kingroot, what do you think guys?
Cause i wanna try to flash the system of a z3 first and see if it boots but i don´t know if is possible to damage my phone or i can revert back without problem?
Thanks for reading my thoughts.....
Your idea is similar to this http://forum.xda-developers.com/xperia-z5/general/flash-z3-firmware-to-root-flash-z5-t3310611
The problem is, apart from the risk of bricking, that we don't have a rootable stock rom to flash for the Z5 line: Z4 has no locked bl root and Z3/Z3c have a completely different hardware and their firmware would probably not boot on a Z5.
I think that noone can tell you that you'll brick or not your phone, if you really wish, you have to try yourself. But, in my opinion, it's stupid to do so as it won't boot properly.
You are right , and how about this guide?
http://forum.xda-developers.com/att-galaxy-s6/general/root-5-1-1-qa-oj7-root-snag-t3334546
It looks like a tethered root is possible in a locked environment, it requires that each reboot you have to manually set selinux to permissive via adb in order to get your phone to boot, so may be we can implement in that way only to backup the DRM Keys and then unlock the bootloader and the phones with locked bootloader can achieve root with a tethered solution, is this possible in this phone? :fingers-crossed:
We need someone to help us
unfortunately this in not possible, because of dm-verity protection we have on Z5 devices, look here
http://www.xda-developers.com/a-look-at-marshmallow-root-verity-complications/
and my answer here:
http://forum.xda-developers.com/xperia-z5/general/root-using-vulnerabilities-snapdragon-t3338173
The only available option seems temporary root/system priviledges and backup of the TA partition, then unlock the bootloader
Hi all,
I have two questions:
1 What data is being stored in the drm-protected partition on the xc? I know that it was (among othets) camera related stuff on the z5c. Do I need to worry about just unlocking the bootloader without doing a tedious backup process (for which there's only good documentation for the z5c anyway)?
Do I need a custom kernel if I want to flash twrp after unlocking my bootloader? Or is update to latest software > unlock bootloader > fastboot flash recovery.img ?
Thanks a lot!
ApplepieFTW said:
1 What data is being stored in the drm-protected partition on the xc? I know that it was (among othets) camera related stuff on the z5c. Do I need to worry about just unlocking the bootloader without doing a tedious backup process (for which there's only good documentation for the z5c anyway)?
Click to expand...
Click to collapse
Only the keys to unlock the functions are stored there. So yeah, without the DRM-patch or DRM-fix there's no denoising on the camera etc. You don't have to backup the keys get full camera functionality back, check the thread from next answer.
ApplepieFTW said:
Do I need a custom kernel if I want to flash twrp after unlocking my bootloader? Or is update to latest software > unlock bootloader > fastboot flash recovery.img ?
Click to expand...
Click to collapse
If you want to go to Nougat (Android 7), check the steps here:
https://forum.xda-developers.com/showpost.php?p=70657390&postcount=60
You can just skip the TA keys flashing part as it's a bit complicated, but I would recommend doing the backup so that you can restore the phone to factory settings in case you need to use the warranty services.
Me personally I'm sticking with Android 6 as I can't imagine using any new phone without XPrivacy.
1. you will loose the same functions x-reality, denoise, all that 'special' sony stuff.
2. you can flash twrp with stock kernel.
you need to be on MM to backup the TA partition (device key) but if you are on MM a backup will take less than 2 mins and all you need is click a icon, you can also use the latest version of flashtool to back it up. but yeah if you upgraded to nougat the downgrade is a bit tedious. :/
realtuxen said:
1. you will loose the same functions x-reality, denoise, all that 'special' sony stuff.
2. you can flash twrp with stock kernel.
you need to be on MM to backup the TA partition (device key) but if you are on MM a backup will take less than 2 mins and all you need is click a icon, you can also use the latest version of flashtool to back it up. but yeah if you upgraded to nougat the downgrade is a bit tedious. :/
Click to expand...
Click to collapse
Alright thanks! It's nice that I don't have to flash the kernel since iirc that's the only way you can actually break things.
I also discovered the easy dirtycow backup, I definitely didn't want to go through the z5c backup process. Let's hope the xc I'm going to buy isn't on 7.x yet
Hello
I own a clean X Compact phone and I want to unlock bootloader and root it ( Magisk) before install anything on it. Actually it is on Android 8 v34.4 and I can't backup DRM with indicated tools (backup TA), so do I need to backup DRMs if I don't use Sony services except the camera to take pictures ?
If yes, which tool can do this on Android 8 ?
Thanks
Feedback on this.
After done the android 8 update to the last release from sept 2018, I was able to activate the OEM unlock and run fastboot unlock command.
Installation of TWRP is ok
Magisk cause Unsupported/Unknown image format with the v21 . Investigate to know why
Trying to install DRM-Fix-system_Mode.zip and the phone boot loop on fastboot mode.. I'm looking to a way to uninstall this DRM Fix or flash another file,if someone can help.
Finally install Lineage OS 16 after failed to install stock firmware with flashtool. Added OpenGApps and Magisk (v21) to disable non desired apps ( 95%)
It is better to backup the TA in case you would need to get back to the stock firmware or sell device in factory conditions (on stock ROM).
If you have already unlocked the bootloader without backing up the TA first then there is no way to bring the device back to the completely factory state.
mp107 said:
It is better to backup the TA in case you would need to get back to the stock firmware or sell device in factory conditions (on stock ROM).
If you have already unlocked the bootloader without backing up the TA first then there is no way to bring the device back to the completely factory state.
Click to expand...
Click to collapse
Thanks for your precisions.