[Q] TA partition and DRM fix - Sony Xperia X Compact Questions & Answers

Sorry for a noob question.
I rooted my previous device, Xperia Z3 compact, without unlocking bootloader, so I am completely new to this stuff.
I am looking into buying a used X Compact and worried that the device no longer has its TA intact.
I've searched the forum but didn't quite find an answer to my question.
So, for X compact and many other xperia models, there is DRM fix that restore DRM keys on a bootloader-unlocked device, even if you didn't backup your TA and your TA is completely wiped by unlocking, correct?
Then, how can you tell the difference between a device that has its original TA (either still intact or TA properly restored from backup) and a device without TA but DRM keys restored?
And, if you can't tell the difference, what is the point of having TA backup and restoring it later?
Sorry it seems pretty basic stuff but I can't find any direct answer to it.
Hope someone can help. Thanks in advance!

tapa_t said:
So, for X compact and many other xperia models, there is DRM fix that restore DRM keys on a bootloader-unlocked device, even if you didn't backup your TA and your TA is completely wiped by unlocking, correct?
Click to expand...
Click to collapse
Not exactly - the drm fix is a mod that attempts to replicate the features and functions that are lost when wiping ta. I've never used it, so can't say myself, but it seems like people who have are happy with it. The main issue with wiping ta is the loss of Sony camera features and quality. I believe there are also other things attached to the ta, but most people probably wouldn't ever miss them.
If you wipe the ta without backing it up, then it is gone, and your only option is drm fix. Most people would probably rather have the ta intact...

levone1 said:
Not exactly - the drm fix is a mod that attempts to replicate the features and functions that are lost when wiping ta. I've never used it, so can't say myself, but it seems like people who have are happy with it. The main issue with wiping ta is the loss of Sony camera features and quality. I believe there are also other things attached to the ta, but most people probably wouldn't ever miss them.
If you wipe the ta without backing it up, then it is gone, and your only option is drm fix. Most people would probably rather have the ta intact...
Click to expand...
Click to collapse
Thanks -- but that doesn't really answer my question. Question is *HOW* do you tell apart a phone with drm fix and no TA from a phone with an intact TA. If you can't tell the difference, people can't be sure they've got the TA intact whether they prefer it or not.

tapa_t said:
Thanks -- but that doesn't really answer my question. Question is *HOW* do you tell apart a phone with drm fix and no TA from a phone with an intact TA. If you can't tell the difference, people can't be sure they've got the TA intact whether they prefer it or not.
Click to expand...
Click to collapse
Service menu > service tests > security
See in screen shot :
- Widevine... "OK"
- ckb... "OK"
- Fido keys provisioned

with drm fix

zokkii said:
with drm fix
Click to expand...
Click to collapse
Interesting - and thats with drm keys missing (TA wiped)?

yes,without drm keys

That's only half of the story. The DRM Fix library has two working modes, and what defines which mode it'll pick depends if you have your unique device key (that get wiped when you unlock the bootloader) flashed in an alternate TA Unit. You can extract your device key and then flash it on an alternate unit as long as you have a TA backup taken before unlocking the bootloader (the RootKernel utility from @tobias.waldvogel ships with the required tools to extract and flash your device key).
On the "normal" mode (DRM Fix + device key flashed on alternate TA unit) you have exactly the same functionality you would get on a bootloader locked device with its TA intact, everything works. But, in case you don't have your device key flashed into an alternate TA unit (e.g. unlocked the bootloader without backing up the TA first, lost backup, etc), then DRM Fix will work in emulation mode. On emulation mode, most features that check for DRM will still work (e.g. camera algorithms, X-Reality Engine, etc) but some specific DRM features that requires the device key (e.g. Widevine L1 decryption, HDCP on Screen Mirroring, etc.) won't work. As it was increasingly more difficult to find exploits for newer devices and firmwares, that's the mode most DRM Fix users utilize, I guess.
Now, how to tell a DRM Fix restored device apart a device with intact TA is a little tricky, but you basically have 3 situations:
• Bootloader locked device, security test in Service Menu shows both Widevine and CKB as [OK] [Active] => stock device with TA partition intact
• Security test in Service Menu shows [unknown] or [generic error] on Widevine or CKB (or both) => TA partition wiped, DRM Fix not applied
• Bootloader unlocked device, security test in Service Menu shows both Widevine and CKB as [OK] [Active] => TA partition wiped, DRM Fix applied
When a device has the DRM Fix applied, it's not possible to tell if it's working in normal or emulation mode until you try something that requires the device keys (e.g. Widevine L1 or HDCP on Screen Mirroring). If it works, DRM Fix is in normal mode, otherwise it's in emulation mode. Also, you can't trust the "Bootloader Status" info on the Security Test of the Service menu to tell whether the bootloader was unlocked or not, as it'll always report "locked" if you have the DRM Fix applied. A safe way to tell the bootloader status is (re)booting the device and observing if the "Your device has been unlocked and can't be trusted" message appears before the Sony logo. If that message appears, the bootloader was unlocked.
You'll also find many people debating whether FIDO Keys being provisioned or not actually indicates the status of the DRM Fix, but from my experience, they aren't related. Fact is that I had FIDO Keys not provisioned on my Xperia X right after I turned it on for the first time (brand new device, had just taken it from the box), but upon registering a fingerprint, FIDO Keys became Provisioned again. Same thing occurred after I backed up my TA with the dirtycow exploit then unlocked the bootloader, FIDO Keys became not provisioned but after enrolling a fingerprint they became Provisioned again, even through I hadn't installed DRM Fix yet at that point.
And to finish, keep in mind all I said on this post is only valid for Xperia devices that launched before Xperia XZ Premium. With XZ Premium launch, Sony reworked many parts of the DRM protection and the original DRM Fix from previous devices didn't work at all. Another user eventually developed a new DRM Fix solution for those newer devices, but it works completely different and I have absolutely zero experience with it.

Does this work on the Android 9.0 custom roms? DRM Camera fix for 9.0?

sheeplings said:
Does this work on the Android 9.0 custom roms? DRM Camera fix for 9.0?
Click to expand...
Click to collapse
No

mbc07 said:
That's only half of the story. ...
Click to expand...
Click to collapse
Thanks A LOT for this information!
I have searched for a long time for something similar but everyone seems to just know this and expecting everyone else too, so it's never thoroughly described, until now!
mbc07 said:
On the "normal" mode (DRM Fix + device key flashed on alternate TA unit)
Click to expand...
Click to collapse
Do you have any information that would help a noob accomplish this? Link to a good, noob adopted, step-by-step guide? So far I'm out of luck finding anything that will give me confidence enough to try it.
Thanks again!

Holton181 said:
Do you have any information that would help a noob accomplish this? Link to a good, noob adopted, step-by-step guide? So far I'm out of luck finding anything that will give me confidence enough to try it.
Click to expand...
Click to collapse
Sorry, nope. The process is "half" streamlined if your phone happens to be supported by the original Root Kernel tool or the modded version posted on the last pages of that thread, but you would still need knowledge on how to get a TA backup on your particular device and on unlocking the bootloader / flashing a modded kernel. If your phone isn't supported then the process also involves manually modding the kernel to include the DRM Fix library too, and none of that is noob-friendly, at all...
If you can't/don't want to deal with it, your best bet is searching at your device's subforum, perhaps somebody already did the work for your particular phone model and published a ready to flash, DRM Fixed kernel there...

mbc07 said:
Sorry, nope.
Click to expand...
Click to collapse
Okay, thanks anyways. My understanding of this is much better already after your description.
Backing up TA, unlocking bootloader and rooting is the easy part, pretty good understanding how its done. It's how to make good use of the DRM:s that's all new to me.
My device is the full size X (not Compact as in this sub-forum), my wife's old working phone. I'll try to search the forum for it.

Related

[Q] Questions about bootloader unlocking

Hey Guys, i'm having a xperia PLAY for a long time and i've been unlocking the bootloader of it. On xperia devices you're losing DRM keys, when you're unlockoing it.
1.Happens something similar to the N7 when unlocking it?
2.And if you'd relock it, woukd it be in any way noticable that ialready had it unlocked?
3.What else are you losing besides the sdcard data?
Thanks
Spatinator said:
Hey Guys, i'm having a xperia PLAY for a long time and i've been unlocking the bootloader of it. On xperia devices you're losing DRM keys, when you're unlockoing it.
1.Happens something similar to the N7 when unlocking it?
2.And if you'd relock it, woukd it be in any way noticable that ialready had it unlocked?
3.What else are you losing besides the sdcard data?
Thanks
Click to expand...
Click to collapse
I can't answer question 1 for you, but for the other questions:
2: after unlocking, an unlocked symbol will be displayed while booting the device. After relocking, this symbol will disappear. As far as I know, there are no ways to find out if a device has been unlocked.
3: everything. Your device will return to factory settings.

"The camera may be in use" and other camera problems after bootloader unlocking

"The camera may be in use" and other camera problems after bootloader unlocking
Long story short, my camera app is giving me the "The camera may be in use by another application." error even after a factory reset.
Third party camera apps cannot focus and most of the time only produce a green picture when trying to save the image, or fail completely to do so.
The hardware works, as I can see the image on third party apps, and some third party apps manage to save the image as well.
The dirty details:
After getting my brand new X Compact, it quickly updated itself to 6.0.1 (34.1.A.3.49) and since rooting it with some random one-click app failed, I have haphazardly decided to unlock my bootloader, following the Sony guides.
The camera was definitely working OK, before attempting these steps.
I was aware, that some camera functionality would be lost, and I was like sure, whatever, but the end result was not exactly what I've expected.
Bootloader unlocking went fine (except of course I did not read the small text, so was rather surprised rebooting to a factory reset ), but since I still was unable to root the phone with stock firmware, I decided to relock the bootloader with Flashtool and leave it at that. (note, phone still on stock everything at this point).
Flashtool said relocking successful, but it did sort of stop half-way through the progress bar and didn;t move on from there.
Next reboot and the phone didn't accept my PIN or any other thing I could come up with, so after 30 tries it factory reset itself and the next morning I realized my camera is FUBAR.
I am honestly not sure whether everything was ok between unlocking and relocking.
now the Sony camera app is completely hopeless. I have tried stopping everything, including system apps, I've tried a factory reset, I've tried PC companion repair, nothing worked.
third party camera apps are a mixed bag. They all show the live preview image, but modifying anything or saving the image fails:
open camera doesn't seem to do anything (doesn't save the image or do anything when shutter pressed)
FV5 lite sometimes works sometimes not (perhaps depending on some settings? no idea tbh), when it works, can take pictures of the main camera, but cannot move focus. Front camera results in green image.
facebook, instagram etc work, but there's not much control in those. Also they do not auto focus at all.
The generated "green image" has a correct looking EXIF, but then where the image should be there is mostly just a long repeat of the following four bytes: 0A 28 A2 80
One possible hint is that in the file I can see the follwing string: "Qualcomm Camera Debug"
Anyone seen anything remotely familiar?
As I've already did the bootloader unlock, I chose not to send it back even if I have relocked the bootloader and I have no replacement phones available, so I kind of prefer not to be phoneless until it would get clarified.
I'm not sure if it's working, but try this:
unlock bootloader again. then flash your stock firmware again via flashtool. You know how to do that?
I did mark all wipes in flashtool. Then you load kernelpatcher V5.0 tool here:
http://forum.xda-developers.com/xpe...-kernel-dm-t3301605/post64990566#post64990566
follow all steps to patch you stock kernel with DRM-Fix. Did you backup yor TA befor unlocking bootloader?
So you can extract DRM keys and make .ftf in order to flash your DRM keys. If not, DRM Fix should work as well.
You can also read here:
http://forum.xda-developers.com/showpost.php?p=69993876&postcount=86
of course no TA backup, as I wasn't aware of this and didn't prepare for this properly (I just assumed following the Sony guides would be OK to follow, even if I loose some features)
To be fair, I may have screwed this up while trying to "relock" with flashtool, so there. Lesson learned.
Now that I start reading up on what even TA is, first hit on google: on sony's own forums about an unlocked Xperia-Z3-Compact and lost TA partition is eerily familiar and not promising much good (sorry, can't post links yet. Anyway, the guy's camera also stopped focusing.)
thanks for the tips, I'll look into these in a more relaxed moment.
Question: could the potentially damaged TA partition impact the functionality even on third party roms like CM?
I never used a custom rom, but i think if you did a clean flash of stock firmware, you should be able to flash a custom rom and everything shold be fine, except functions which need DRM keys.

Factory reset after unlock?

Hello,
I bought some months ago my XC to install SailfishOS. So I recently unlocked the phone and installed SFOS but...I totally forgot to save the DRM keys required to enable the camera full resolution.
Now, I'm not ready to use this XC + SFOS as my daily driver. I plan to sell it, but I need to restore Android back. I just would like to know if there is a way to "fully restore" it, understand with full camera capabilities?
Thanks.
romut said:
Hello,
I bought some months ago my XC to install SailfishOS. So I recently unlocked the phone and installed SFOS but...I totally forgot to save the DRM keys required to enable the camera full resolution.
Now, I'm not ready to use this XC + SFOS as my daily driver. I plan to sell it, but I need to restore Android back. I just would like to know if there is a way to "fully restore" it, understand with full camera capabilities?
Thanks.
Click to expand...
Click to collapse
Pretty sure drm keys are unrecoverable. There is a 'drm fix', which mimics the functions lost by wiping ta, but it's a recovery-flashable zip. I guess you could factory reset, root, and install drm fix, and sell it that way...
Problem with that approach is, the buyer of the phone won't be able to do software update for security patch or android p(if by some luck we'll get it). Another problem is, unless you install magisk, they won't be able to download netflix (due to saftey net check). And when safety net is updated, they might need to update magisk.
But overall i agree with above, that is the best way of getting all functionality "back".

Possible way to backup TA/DRM keys, maybe?

EDIT 07/27/2021: Though this method didn't end up panning out (different signing keys on stock XZ Marshmallow! who would've thought!), I found a way to get temp root on the XZs and thus pull the TA partition off the device. Details in the bottom half of my latest post.
From reading through the forums here, it's clear that nobody has yet devised a way to perform a temporary root exploit on the XZs prior to bootloader unlock so that the TA partition (and thus the DRM keys) can be backed up on this model before bootloader unlock erases them forever.
Most pre-SD835 Xperia phones rely on the "dirtycow" exploit to do so, which is only available pre-Nougat. The XZs is somewhat unique in that it's an SD820-based model that shipped with Nougat from the start. The other two Xperia models that share the SD820, and are even considered to be part of the same underlying Xperia platform ("tone"), are the X Performance and the XZ, and both originally shipped with Marshmallow, which is "dirtycow"-exploitable. The X Performance and XZ are even supported by backupTA v2, even though XZs is not due ONLY (as far as I can tell) to the lack of Marshmallow firmware for the XZs.
I also have found a few posts on here from people who considered trying what occurred to me: if the XZs is so similar to the XZ, what about trying to flash the XZ Marshmallow firmware to an XZs? It would only be temporary, long enough to run the backupTA script on it, so even if not all of the hardware is 100% the same (and thus not all of the drivers work), you would only need enough things to work (main CPU, eMMC, USB port) in order to run backupTA, and then flash back to actual XZs firmware.
Unfortunately, it sounds like the few people brave enough to try this were met in the end with bricked phones. I have so far not run into anybody who claims to have tried this who managed to "un-brick" their device. :crying:
But, I suspect what may be going on is that, sure: if you flash the WHOLE XZ firmware to an XZs, yeah, you probably risk a brick, because there are likely enough low-level components not in common between the two getting flashed. My guess is that what's actually going on is that when the XZ bootloader gets flashed to the XZs, either there is some hardware-level thing on the XZs it doesn't know how to deal with, or perhaps Sony used different encryption keys on the XZ vs. XZs, so the (signed by Sony) XZ bootloader freaks out when on the XZs and refuses to start the phone.
Result: brick. Maybe even hard brick (unless you want to unsolder the eMMC and reflash with an external programmer).
But what I don't think anybody has tried yet is trying to only flash enough software components necessary to get a Sony-signed Marshmallow release running on the XZs. In other words, DON'T flash bootloader, modem/baseband, and all that stuff.
Hypothesis: it MIGHT be possible to get a barebones "dirtycow"-exploitable Marshmallow release running on the XZs by flashing ONLY the "boot" (kernel) and "system" SIN files from the XZ ftf. Sure, go ahead and wipe userdata and cache too, but don't touch anything else: leave modem/baseband alone, do NOT flash any TA files, etc. Has anybody tried that?
At worst, I can imagine that it simply won't work, and it's possible that it won't because I imagine that Sony probably *did* use separate encryption/signing keys for XZ vs. XZs. But it's also possible that they did, given how similar the devices are. If the keys aren't a match, then the XZs bootloader (which will still be intact, since you didn't try flashing the XZ bootloader!) will simply refuse to boot the XZ kernel. And in that case, you should only have a "soft" brick, easily recoverable by going back into flash mode and re-flashing official XZs boot.sin and system.sin files to it.
Thoughts?
-- Nathan
My experience with swapping xzs with xz firmware is an unresponsive camera. TA can be recovered/restored, but not backed up (what's the point if it's recoverable?). Would be great to see temp root exploit imported to the 820; unlikely, as all the attention are for newer models that run p and q. Only real benefit is for those running locked bootloaders and it's pretty minimal what could be accomplished on after. Would rather see someone spoof signatures on custom roms.
&(*) said:
My experience with swapping xzs with xz firmware is an unresponsive camera. TA can be recovered/restored, but not backed up (what's the point if it's recoverable?).
Click to expand...
Click to collapse
I'm not 100% sure I understand you, or if maybe you misunderstood me.
Are you saying that you have successfully tried flashing XZ firmware to an XZs, and it worked? In all of the threads I have read on XDA from others who have tried this, the result has been a bricked phone...
It doesn't matter if the camera is unresponsive when running XZ firmware on an XZs. Running the XZ firmware would be *temporary*. You would only have it on your phone long enough to extract a complete copy of the TA partition, with all DRM keys intact. Once you have extracted a TA backup, you would re-flash back to XZs firmware. So, you maybe will run your phone with XZ Marshmallow firmware for 5 minutes, tops.
You can't "recover" or "restore" TA if you don't have a backup to begin with, so I'm not sure what you are talking about there. If you unlock your bootloader without first making a backup of TA, then your DRM keys are gone. Forever. NO recovery is possible. So, of course, if you have already unlocked your bootloader, it is too late, and putting XZ Marshmallow on your phone for 5 minutes to make a backup of TA would be pointless. My proposal is for people who have XZs phones still with LOCKED bootloader, who want to make a pristine backup of their DRM keys *before* unlocking for the first time.
Up until now, XZs users have had no means of making a backup of TA before unlocking their bootloader. So only solution for XZs users who unlock is "drmfix", which does some tricks to make *some* of the Sony software think that DRM keys are present even though they are not.
This is not as good of a solution as actually keeping your DRM keys, however. Most of the Xperia models before XZs have a more comprehensive "drmfix", where you extract the DRM keys for your phone from your TA backup that you made *before* unlocking your bootloader, flash them *back* to the TA (*just* the DRM part, and nothing else), and then you have the perfect solution: DRM keys AND unlocked bootloader!
Plus, with a copy of the original, pre-unlock TA partition, you have the option to flash the whole TA backup back to your phone at any time in order to re-lock your bootloader and put things back to stock.
All of this is only possible with SD820 or older Xperia models. The ones that came with SD835 and newer (XZ Premium, XZ1, XZ2, etc.) have additional security. It's still possible to extract DRM keys from TA on some of these phones before unlocking the bootloader, but you can never go back to locked bootloader on those phone models, even with a complete TA backup.
-- Nathan
You're right, my mistake - I meant DRM is recoverable (sort of) not TA . What you are suggesting with a locked bootloader will not work, as it will not contain the verification keys to allow a rewrite of it - in short.
&(*) said:
You're right, my mistake - I meant DRM is recoverable (sort of) not TA . What you are suggesting with a locked bootloader will not work, as it will not contain the verification keys to allow a rewrite of it - in short.
Click to expand...
Click to collapse
I think we are still talking past each other. I'm not talking about writing anything to a "locked bootloader". What I'm describing is exactly how things already work for the vast majority of pre-SD835 AArch64 Xperia models when it comes to unlocking the bootloader while simultaneously preserving the DRM keys. Almost all pre-SD835 AArch64 Xperia models, that is, except for XZs, simply because the XZs lacks an exploitable stock firmware version...or does it?
The TA partition contains all sorts of Xperia-specific data in it, not JUST DRM keys. In pre-SD835 models, the TA partition also happens to contain the bootloader lock status. So if you unlock your bootloader and then later decide you want to re-lock it, simply take the TA snapshot you made while the phone was locked, and re-flash that back to your phone. This is no longer true in SD835 and beyond, where Sony is now seemingly taking advantage of the TrustZone feature in newer Snapdragons.
I have a couple of Z5 Compacts myself, and I've tested this flow many, many times on both of them, and it works...what follows is not conjecture or hypothesis:
Start with a phone that is running stock, untouched firmware, and a locked bootloader
If said phone is running Nougat, the downgrade to a stock Marshmallow or Lollipop firmware; this is possible to do without unlocking the bootloader since such images are signed by Sony
Use backupTA v2 to achieve temporary root, and use temporary root to do a bit-for-bit copy of the TA partition to a backup file that you pull back to your computer for safe keeping/a rainy day
Go ahead and upgrade back to Nougat or later after that, if you so desire...you only had to boot Marshmallow once so that you could achieve temporary root, in turn so that you could image the TA partition
Unlock the bootloader, which in turn wipes the DRM keys from the TA partition
Use the flash_dk script from rootkernel suite to extract the DRM keys from your TA partition backup image made in step 3
Flash the DRM keys (and ONLY the DRM keys) back to the TA partition using Flashtool, pointing it at the FTF that flash_dk outputted for you in the previous step
Have rootkernel bundle up for you a modified kernel image with an initramfs that contains the drmfix, and flash that to the phone
Bam: you now have an unlocked bootloader AND you have managed to preserve your unique Sony DRM keys
If at any time you want to return to completely stock software and re-lock your bootloader, simply push the TA backup image you made at the start of this guide to the phone, and then bit-for-bit write the contents of it back to the TA partition (using 'dd' or your weapon of choice)
Of course, after you do this, if you had previously modified either the system partition or the kernel, you will need to re-flash stock, unmodified, Sony-signed kernel and system images before the phone will boot again
You can always re-unlock your bootloader again using the same code that you got from the Sony website in order to unlock it the first time, so keep that code around
If you do NOT back up your TA/DRM keys prior to unlocking, they are lost forever, but in that case "drmfix" can employ a sort-of workaround, where it "fakes out" having DRM keys for some Sony apps. This does not restore 100% of functionality, though, UNLIKE if you actively work to preserve the DRM keys prior to unlocking, in which case you DO retain 100% of functionality post-unlock (plus you're always free to go back to being locked + still having 100% functionality, which is not possible if you don't make a TA backup first prior to unlocking!)
As the rootkernel post explains: "First [drmfix] tries to use the device key which you flashed with flash_dk. If it does not exist it uses an alternative method which cannot fix everything (e.g. Widevine will not work, but X-reality, Camera denoise etc. will work)." This is also very well summarized/explained in this post by another forum member.
Therefore, it is within every Xperia owner's interest to accomplish a TA backup before performing a bootloader unlock, even though "drmfix" exists, because "drmfix" is able to "fix" more things if it has your actual DRM keys to work with. "drmfix" + your actual DRM keys > "drmfix" by itself.
Up until now, XZs owners have had to be satisfied with drmfix by itself, because there has been no known way of accomplishing temporary root with a locked bootloader on this model. I'm not the first to think about maybe trying to flash XZ Marshmallow to an XZs in order to achieve temporary root for the purpose of making a TA backup, but I don't think I've seen anyone else suggest flashing ONLY the XZ Marshmallow 'system.sin' and 'boot.sin' partitions to an XZs, while leaving every other partition alone (except maybe wiping userdata and cache, of course...but don't touch bootloader, modem, etc.).
I don't have an XZs myself or I would try this. I do, however, have a friend with one which still has a locked bootloader, and who is mildly interested, but is understandably nervous given the other attempts people have made at flashing XZ Marshmallow to their XZs only to result in a brick (because, as I theorized before, they're flashing EVERYTHING, including bootloader). I'm pretty sure, however, that the worst that could happen if you made sure to only flash system and boot/kernel is that the bootloader simply would refuse to boot the XZ kernel. At that point you should be able to re-flash the XZs kernel and system to get back to status quo.
Boot.sin would not suffice, boot is now a folder with a number of files to choose from based on what is read from the bootloader status during flash. There are keys stored and keys appended to the enrolled verified signatures during an upgrade, a downgrade would attempt the same with keys for that version or possibly based on how it parses the list insert the keys to the top of the list. In order for the bootloader to accept a write to the table and enroll the keys for signed images of the sort, the bootloader requires a key itself to allow a write to it. The key for the XZs bootloader will more than likely not match is my thought at which point the loader is flagged for being corrupt and will no longer boot or simply not write and abort (which apparently other's state is not the case). I'm not missing the point here, it's just not likely that the bootloader or the TA where the flags are set will allow a write to the loader itself considering how it handles OEM signed images and how it verifies each partition before it writes to it with keys (signatures). My understanding might not be descriptively accurate in terms of technicality with the order and operations of how the bootloader protects itself, best bet here is to coach your friend on the flashing process and report the results. I don't think the loader will allow a downgrade to a system it didn't support during it's lifecycle.
&(*) said:
Boot.sin would not suffice, boot is now a folder with a number of files to choose from based on what is read from the bootloader status during flash.
Click to expand...
Click to collapse
Sorry, my mistake...I meant kernel.sin, not boot.sin. I misspoke since I had in mind that fastboot itself calls the kernel partition 'boot', so when you reflash a new kernel (in uImage format, not .sin) with fastboot, you use 'fastboot flash boot'. So, a brain fart moment on my part.
Yes, I'm aware the bootloader itself is made up of a number of files in a folder, though until your explanation I did not know what the point of the diversity of files was.
&(*) said:
There are keys stored and keys appended to the enrolled verified signatures during an upgrade, a downgrade would attempt the same with keys for that version or possibly based on how it parses the list insert the keys to the top of the list. In order for the bootloader to accept a write to the table and enroll the keys for signed images of the sort, the bootloader requires a key itself to allow a write to it.
Click to expand...
Click to collapse
Got it; thanks. I was not aware of this.
But this sounds like the bootchain keys in the TA only get touched/written to when the bootloader itself is being flashed. What if only main application processor code (kernel, system) partitions are being touched while in flashmode?
&(*) said:
The key for the XZs bootloader will more than likely not match is my thought at which point the loader is flagged for being corrupt and will no longer boot or simply not write and abort (which apparently other's state is not the case). I'm not missing the point here, it's just not likely that the bootloader or the TA where the flags are set will allow a write to the loader itself considering how it handles OEM signed images and how it verifies each partition before it writes to it with keys (signatures).
Click to expand...
Click to collapse
Right; my mistake was in misleading you by writing "boot.sin", when I was trying to reference the Linux kernel. It seems to me that if you attempt to only flash kernel and/or system from a different model with a locked bootloader, at most you risk the system not booting because bootloader refuses to chain off to the kernel on account of mismatched keys or whatever, but this is a state that can be recovered from by going back into flashmode (which still works, since XZs bootloader hasn't been touched and is still intact) and reflashing actual stock XZs kernel and system again. So, on the off-chance that it works, my thought is why not give it a try, since the only thing you potentially have to lose/waste is time...
This is all predicated on the theory that pre-SD835, Sony largely re-used the same set of bootchain signing keys across all models. Which admittedly could be entirely wrong. The main reason why it seems probable is that there doesn't seem to be any problem with cross-flashing different firmware customizations within a model, nor with cross-flashing between variants of a model (e.g., any Z5c E5803 firmware of any version can be flashed to E5823, or vice-versa), all while running a locked bootloader. It's also never been a problem to upgrade or downgrade Android version at-will with a locked bootloader, suggesting (to my simple mind) that the keys haven't changed over the lifetime of that platform.
I suppose one test that would be interesting would be to see if I could flash and boot a Marshmallow kernel and userland on a Z5c with a bootloader from either a Lollipop or Nougat firmware bundle, while the bootloader is in a locked state. If a mismatched but locked bootloader willingly passes off to a signed kernel from a different firmware version than what it itself shipped with, that's an indication the signing keys didn't change. Right?
What I have heard almost no chatter about, other than in XZs-land, is attempting to cross-flash between any models that are part of a larger shared platform (e.g., flash any official Kitakami platform firmware to any other Kitakami model phone with locked BL). XZ and XZs are both part of such a shared platform (Tone).
-- Nathan
Ran some tests on my Z5 Compact with a locked bootloader, the results of which are promising.
I wanted to see what would happen if I booted various levels of mismatched bootloader + kernel sets, increasing the amount of differences between them as I went:
...where the bootloader and kernel were for the same hardware model but from different firmware bundles for different Android versions,
...where the bootloader and kernel were not only from different firmware versions but also from different but related hardware models within the same immediate family, and
...where the bootloader and kernel were from different models taken from different families within the same common platform.
First up, different firmware versions:
I flashed the bootloader that comes with latest official Nougat release for this platform (32.4.A.1.54) to my phone, and then I flashed kernel and system from an old Lollipop release (32.0.A.6.200). Bootloader from the Nougat bundle booted the entire Lollipop system without any complaint.
While in this state, I went ahead and enabled USB Debugging under Developer Tools, and permanently authorized my PC for adb access over USB.
So, success!
Next up, different models from same device family:
Next I grabbed the same Lollipop release (32.0.A.6.200) for different but related device (Z5), and flashed just the kernel from it to the Z5c. The locked Z5c Nougat bootloader also booted the Z5 Lollipop kernel, no problem. Granted, I couldn't see anything on the display which remained black the whole time (probably because the display is one obvious major difference between Z5 and Z5c), but the system still came up, which I knew because I was able to get an adb shell after it booted. So USB was working. Audio was also working, as it would make chime noises whenever I plugged the USB cable in.
Looking good. And now we know that at least within Z5 family, the boot signing keys are the same across major Android system versions, and also the same across the entire family ("sibling" models).
What about the real test: entire shared platforms (which, in Sony's world, is usually any set of phones that use the same SoC)?:
I decided to try flashing a Z3+ kernel to my Z5c. Both are based on Snapdragon 810, and both are internally categorized as "Kitakami" platform devices. So not only do Z3+ and Z5 share SoC, but firmware updates for both are released with the same version #. I was unable to find a Z3+ Lollipop firmware available for direct download from Sony via XperiFirm, and didn't feel like searching around for one elsewhere, so I switched gears and settled on Marshmallow instead.
So first, as a sanity test & control, I flashed Z5c Marshmallow 32.2.A.5.11 kernel and system while leaving Nougat bootloader still in place, cleared userdata and cache, and as expected based on the earlier Lollipop test, phone booted up fine. I set it up as new, enabled Developer Tools, enabled USB Debugging again, and whitelisted my PC, just as before.
I grabbed Z3+ Marshmallow 32.2.A.5.11 firmware, flashed JUST the kernel from it, and the results exactly match what I experienced with the Z5 kernel: the bootloader booted it just fine, and most everything worked except for the screen itself! So I adb shell'd in, watched logcat scroll by, heard the speakers chime whenever I plugged the USB cable in, etc.
So what I believe I did here was approximate/proof-of-concept on my Z5c what I am hypothesizing is probably also doable on XZs: I used a locked Nougat bootloader to boot a signed-by-Sony Marshmallow kernel that was built and intended for an older device that shares the same SoC.
The test would probably have been more "fair" if I had used a Z5 instead of a Z5c, since the Z3+ and Z5 displays are also identical, but I don't have a Z5. The XZ is much more similar to the XZs than the Z3+ is to the Z5c, and yet despite the differences, it still worked.
In conclusion: we know that Sony finally "got wise" (or at least wiser) for their next set of flagship phones (XZ Premium and XZ1 family), but unless they made moves that aren't public knowledge to tighten down the screws within the SD820 "Tone" family of devices (X Performance, XZ, XZs) compared to the SD810 "Kitakami" ones, a bootloader from any one of these phones within that platform family can likely boot a kernel from any other one of these phones, regardless of which Android version either the bootloader or the kernel was intended for. Thus, there is a fairly high degree of likelihood that you can flash and boot just the XZ Marshmallow kernel and system on a XZs, and that the hardware support in the XZ kernel may be enough that it's at least possible to perform a temporary root and TA extraction by this means.
What I haven't yet done is taken the time to do a system flash from the Z3+ to the Z5c, too. I intend to also try that shortly.
-- Nathan
&(*) said:
[...]
Click to expand...
Click to collapse
Z5 Compact + locked bootloader + Z5c bootloader from Nougat + Z3plus kernel + Z3plus system:
It works.
A stock, Sony-signed kernel and system image for Android 6.0 from a KITAKAMI device (Z3+) can be booted just fine by a locked bootloader (from the Android 7.1.1 firmware, no less) on a KITAKAMI-r2 device (Z5c). I have an ADB shell and a logcat running and everything.
Now, naturally, things don't run *well*. I can't see anything on the screen because the Z3+ display is higher resolution than the Z5c display, and so the Z5c display isn't initialized properly by the Z3+ kernel. (Touch inputs work, however! I'm hitting targets blind, but every once in a while I'll get it to vibrate or make a sound like I managed to tap something.) logcat reveals that basically all radio-related processes are going berserk: the NFC server is crashing and relaunching itself every second or so, and almost just as often, the system is trying to communicate with the cellular baseband and failing to do so. Also, the phone runs HOT. Probably because these processes are freaking respawning themselves every 1-2 seconds. I imagine CPU load is through the roof. (Also, as a fun test, if I try to flash the Z5c kernel back to the phone while leaving the Z3+ system image in place, the display initializes properly, but everything form text to images is all rendered hilariously huge, given both the resolution and DPI difference between the Z3+ and the Z5c.)
But none of this matters. And that's because the only reason why you would theoretically want to do this anyway is so that you can run the dirtycow exploit against the phone in order to gain temporary root and then do a dump of the TA partition. So the only things that need to be working are the USB port and eMMC access. And since both are working, I was able to successfully do exactly this to this phone while it was in this state.
I would be shocked if a similar thing were not possible with XZs (TONE-r3) using the Marshmallow kernel + system from XZ (TONE-r2)...
You would need an oem (vendor), kernel, and system .sin along with the partition layout to theoretically accomplish what you are proposing.
&(*) said:
You would need an oem (vendor), kernel, and system .sin along with the partition layout to theoretically accomplish what you are proposing.
Click to expand...
Click to collapse
It only remains theoretical on the XZs...I believe that I demonstrated that the concept is sound by flashing and booting Z3+ kernel and system on a locked Z5c.
Vast, vast majority of Snapdragon-based devices -- including SD-based Xperias -- utilize a GPT partition table, not static LBA offsets hard-coded into the kernel (or passed to the kernel by the bootloader). And then all partitions are looked up by name, not sequence#, UUID/GUID, or whatever. Partitioning should be a non-issue, and it most definitely was not an issue with Z3+ code running on Z5. I would definitely be very nervous about flashing partition.sin from XZ to an XZs (or from Z3+ to a Z5). Chances are good anyway that the partition layout changed either very little or not at all between the two models, but GPT should abstract away any subtle differences.
oem, maybe. If XZs oem partition contains stuff that happens to confuse the XZ system software, and causes it to behave badly or go into a crash/restart loop, then yeah, I could see trying to flash oem along with kernel and system. Should be no biggie. It's likely, though, that even if you left XZs oem contents in place, the base of the system would work "enough" to be able to do a TA partition dump, which is all we really care about. Only thing I believe that's needed to accomplish that is an ADB shell over USB. ADB is started up early enough by the system during boot (well before the Android user environment has even rendered a single pixel on-screen) that I just don't see this being a problem.
The tricky part is actually enabling ADB while running the "wrong" kernel and system. I have war stories on this topic with the Z3+ > Z5c test. I accomplished it on there, but the XZs may be a different story. If I am given a chance to try it, I'll let y'all know.
You need oem, as it pertains to what software the kernel loads for system I believe. Partition more than likely is necessary due to the shift from legacy (M) to HAL (>=N); whether or not the architecture was implemented in M for XZ is questionable, having a look through the developer binaries at the repo from Sony might indicate it. What is more likely the case, you will need to make sure your paths for everything are in order for a mostly clean boot so that the system will accept what you propose without much fuss (like not losing adb ie flashmode).
Long time no post.
Good news / bad news:
Bad news first, as per tradition. I finally got hold of my friend's XZs to test this theory on, and I can't even get as far as flashing a signed, stock XZ Marshmallow kernel to it, because the phone while in flashmode fails to validate the SIN header of the XZ Marshmallow kernel.sin. So it would seem that perhaps Sony finally got wise and used different signing keys for firmwares issued to different devices within the same "family" (in this case, the Tone family, which includes X Performance, XZ, and XZs, all of which share the same SoC and thus kernel source tree in common with each other)...and since of course my whole plan was predicated on Sony sharing the same software signing keys between all models from the same family, this is officially kaput.
...though I have another theory about the signing key difference. By their end-of-life, all Tone phone models were running builds of Oreo that were kept in-step with each other: before support for a given model finally dropped off, it was getting firmware releases identical in version number to what the other models were getting within the 41.3.A.x.y range. This is very similar to how they treated the Kitakami family (Z3+/Z4 and all Z5 submodels): they all ended with Nougat releases versioned identically (32.4.A.x.y). It's not clear to me what all of the various components of the Xperia firmware version numbers represent, but it seemed clear that the second number would get +1'd every time the base Android version changed (so for Kitakami, 32.2.A.x.y was Marshmallow 6.0.x, 32.3.A.x.y was Nougat 7.0.x, and 32.4.A.x.y was Nougat 7.1.x), and the first number *seemed* like it might represent either a family, SoC, or major kernel revision change, since it would often stay consistent within a given Xperia family.
Well, interestingly, while X Performance and XZ went from 39.0.A.x.y for Marshmallow to 39.2.A.x.y for Nougat 7.0, when XZs was released with Nougat 7.1, it was versioned as 41.2.A.x.y, and then when X Performance and XZ got their Nougat 7.1 updates, they too jumped from 39.blah to 41.blah. I am now wondering if that first number perhaps also indicates a change in signing keys. If so, then probably the only way to make this work would be to try to force the bootloader from XZ 39.blah firmware onto the XZs...but of course assuming that was even possible, this is in no way wise as there's an *extremely* high likelihood of ending up with a brick. I could, however, as a fun test, take a Nougat kernel from an XZ 41.2.A.x.y bundle and try to flash it to the XZs: if the locked phone accepts it and it even happens to boot, that would provide more evidence for this theory.
But of course this is all academic at this point, which leads to my...
GOOD NEWS, everyone! I stumbled across a way to successfully make a TA backup from a locked XZs anyway despite this setback.
Turns out the Tone-family Oreo kernels are all vulnerable to CVE-2019-2215, which at this point has been exploited on many phones of this vintage. j4nn famously exploits this vulnerability in his bindershell temp root tool for Yoshino family devices, for example.
I was sure that I was going to have to end up doing some kernel spelunking on the XZs in order to identify the right offsets for the various things in kernel memory that need to be manipulated to get full root (SELinux policies and toggles, etc.), but it ended up being waaaaay easier than that: turns out that this pre-compiled binary "just works" on (at least) the very last XZs firmware release, out-of-the-box. Upload to your phone where you have write access (e.g., /sdcard), copy over to a place where it's possible to set execute permissions (e.g., /data/local/tmp), run the thing, bingo: instant temp root with permissive SELinux. At that point, dd your TA partition over to a file on /sdcard, pull it off the device, and voila.
(You do have to be running Oreo...the Nougat kernels for XZs use an older version of the Android binder driver that is *not* vulnerable to the exploit.)
Theory about the "major" version number of Xperia firmware versions having a uniform set of signing keys seemingly confirmed: I was able to successfully flash an XZ (F8331) Nougat 7.1.1 kernel.sin to this locked XZs (G8232) that's running Oreo and the phone took it without a complaint.
It didn't fully boot, but that's likely at least in part due to the fact I didn't bother flashing system.sin, just the kernel. Unlike when I've had an unsigned kernel flashed to an Xperia with a locked bootloader, though, the bootloader did display the normal Sony boot logo, so I'm guessing it successfully verified the signature and then chained off to the kernel, but that due to initrd and booting differences between Nougat & Oreo, it eventually hung at some point in the boot process. (Actually, just occurred to me that there would be massive mismatch between Nougat kernel and any kernel modules included with Oreo...)
nlra said:
but it ended up being waaaaay easier than that: turns out that this pre-compiled binary "just works" on (at least) the very last XZs firmware release, out-of-the-box.
Click to expand...
Click to collapse
@nlra ,
With the binary provided I was able to backup/restore the TA and relock the bootloader after unlocking it (no more exclamation posting on boot)!

[CLOSED][HELP] Quest of IMEI repair on Sony Xperia models

Since IMEI is a sensitive subject with xda, I'll preface by specifying that this topic concerns only the repair/restoration of original IMEI number, not its change or replacement for another one with nefarious purposes.
I'll get right to it: the IMEI of my Sony Xperia XZ1 Compact has become accidentally damaged at some point in the process of flashing different ROMs to figure out which one works best. Now, internet is chuck full of IMEI-related guides but I tried a good dozen of solutions and apps, often multiple times, to no avail. That's probably because every IMEI guide and app out there seems to be for phones from brands other than Sony. But I'll share what I learned, and perhaps with help of other interested users we could figure this out together, not just for me, but for other Xperia owners in the similar situation.
Programs which troubleshoot IMEI reach the phone through its COM port(s), accessible when it is put in Diag mode. Through a lot of experimentation and ROM-flashing, I was able to piece together that in order to put Xperia XZ1 Compact in Diag mode, the following is required:
- Latest stock ROM. In case of SO-02K (Japanese version of XZ1c) it's 47.2.B.5.38. I've tried four other stock and custom ROMs, and for whatever reason couldn't access Diag mode from them; FYI stock ROMs for Japanese Xperias are available from ftf.andro.plus and can be flashed with newflasher v20 (other versions and flashing apps failed on me)
- Root for adb (or a terminal apk) and the following commands:
adb shell
su
setprop persist.sys.usb.config diag,serial_cdev,rmnet,adb
With that, Diag mode becomes enabled, no reboot is needed, and the phone's COM port becomes visible in Device Manager. Enabling Diag and backing up the existing IMEI is as far as I got, because running IMEI restoration after editing the backup .qcn file to input the proper IMEI simply doesn't work, the proper IMEI is never there after reboot even when the program notifies of successful operation; I tried QPST, QFIL, EFS' NVTools, Miracle PowerTool, WriteIMEI, Qualcomm Smartphone Write IMEI, Octoplus and Chimera.
One of the guides suggested that to be able to write in the IMEI, partitions modemst1, modemst2 and fsg must first be emptied, while another suggested reflashing modem and persist partitions. Doing either yielded nothing; however both erasing the former from TWRP and reflashing the latter with newflasher removed IMEI completely... but then QPST with QFIL refused to restore the backup and NVTools kept showing success messages without any actual effect, and also wifi stopped working. So I restored modemst1, modemst2 and fsg from backup via twrp, corrupt IMEI instead of no IMEI but at least the wifi is working.
There's has to be more to Xperia IMEI repair than guides suggest, so if you have some suggestions or ideas, I'd appreciate those.
I'm guessing that this topic will have even less comments from people, because of the fact that Sony cell phones are not very popular. This is probably a lost cause, I would try to return the phone somehow, or sell it for parts on ebay or something. good luck
4qx said:
some suggestions or ideas
Click to expand...
Click to collapse
Hello 4qx
IMEI restoration may be helpful here.
My SO-02K (Havoc rom) consumes a lot of battery (10% / H).
NFC is disabled.
It may be the performance deterioration of the battery itself, but it seems that it was not so much at the time of stock rom.
You don't know any solution?
norabitox said:
IMEI restoration may be helpful here.
Click to expand...
Click to collapse
Thank for the suggestion, but that tutorial is for LG Nexus 5X. The method described in it does not apply to Xperia XZ1c.
norabitox said:
My SO-02K (Havoc rom) consumes a lot of battery (10% / H).
NFC is disabled.
It may be the performance deterioration of the battery itself, but it seems that it was not so much at the time of stock rom.
You don't know any solution?
Click to expand...
Click to collapse
You could use apps such as Battery Guru or/and AccuBattery to see which apps consume the most battery and to learn how healthy your battery is; in case of Battery Guru you can apply some tweaks to extend battery use.
It could also be a flaw of HavocOS. You could try latest LineageOS 19 with Android 12 or Lineage 18 with Android 11 to see if it's better for the battery.
4qx said:
method described
Click to expand...
Click to collapse
Thank you
I remember changing the IMEI of the LT26W a long time ago.
4qx said:
Since IMEI is a sensitive subject with xda, I'll preface by specifying that this topic concerns only the repair/restoration of original IMEI number, not its change or replacement for another one with nefarious purposes.
Click to expand...
Click to collapse
Thread closed! Based on the post history it can be validly assumed that this thread has been started to retrieve information about an IMEI change.
[CLOSED] Changing IMEI
Does anyone know how to change IMEI on XZ1c, preferably without root?
forum.xda-developers.com
[CLOSED] IMEI repair on Xperia devices?
Hi there. I've been trying to find a way to repair IMEI on my Xperia XZ1 Compact and could use some guidance. The methods I came across describe putting the phone into Diagnostics mode before the following steps can be taken, but no explanations...
forum.xda-developers.com
[CLOSED] del
del
forum.xda-developers.com

Categories

Resources