So, I've been reading ad nauseum on this topic with regards to flashing new ROMs on my 6t and getting locked out due to decryption.
Among other threads and places across the interweb, I've read all of these:
https://forum.xda-developers.com/search/forum/8259?query=Encryption
During my travels I've discovered that the s**t doesn't really hit the fan until "data" is wiped via TWRP. I see that the recommend process for flashing new ROMs goes like this:
Boot on twrp
Flash ROM
Flash twrp installer
Reboot to twrp
Factory reset
Reboot to system
Am I understanding this correctly that if I transpose steps 4 and 5, then I shall plan on losing all of my data, but if I reboot from TWRP (right back into TWRP) before step 5 (aka data wipe), I shouldn't loose my data?
Assuming that is correct, does it matter if I leave a lock screen password enabled before rebooting to TWRP from the soon-to-be replaced ROM? I ask this because removing the lock screen password did not seem to reduce my chances of encountering permananly encrypted data.
Thanks!
notorious.dds said:
So, I've been reading ad nauseum on this topic with regards to flashing new ROMs on my 6t and getting locked out due to decryption.
Among other threads and places across the interweb, I've read all of these:
https://forum.xda-developers.com/search/forum/8259?query=Encryption
During my travels I've discovered that the s**t doesn't really hit the fan until "data" is wiped via TWRP. I see that the recommend process for flashing new ROMs goes like this:
Boot on twrp
Flash ROM
Flash twrp installer
Reboot to twrp
Factory reset
Reboot to system
Am I understanding this correctly that if I transpose steps 4 and 5, then I shall plan on losing all of my data, but if I reboot from TWRP (right back into TWRP) before step 5 (aka data wipe), I shouldn't loose my data?
Assuming that is correct, does it matter if I leave a lock screen password enabled before rebooting to TWRP from the soon-to-be replaced ROM? I ask this because removing the lock screen password did not seem to reduce my chances of encountering permananly encrypted data.
Thanks!
Click to expand...
Click to collapse
Yes. For the love of god. Someone please clear up how we can flash on the go. I don't always have access to a computer with Adb/fastboot.
Every time I try to switch roms, upon rebooting to TWRP, my folders encrypt. Then I have to format data and voila, no fricking ROM to flash and I'm stuck
idkwhothatis123 said:
Yes. For the love of god. Someone please clear up how we can flash on the go. I don't always have access to a computer with Adb/fastboot.
Every time I try to switch roms, upon rebooting to TWRP, my folders encrypt. Then I have to format data and voila, no fricking ROM to flash and I'm stuck
Click to expand...
Click to collapse
Yeah A/B partition are a nuisance when it comes to flashing. I'd recommend you to do a clean flash of the ROM. Follow these steps.
1. Download the Latest Stable OOS from OnePlus's Website. Download the ROM ZIP of your Choice as well as the latest TWRP Installer. Transfer these to your phone and also copy them to a Laptop as you might have to Format Data.
2. Now Reboot to Recovery and flash OOS ZIP and TWRP Installer. Let it finish. Once it's done Reboot to Recovery from Within TWRP.
3. Again flash OOS and TWRP Installer. If for some reason your folders are messed up (as you posted in the image earlier) just go to Wipe and Format Data. And transfer the OOS and TWRP Installer to Internal and Flash them. Let it finish.
4. Once that's done, now again Reboot to Recovery and now flash ROM and TWRP Installer. After that's done, again Reboot to Recovery.
5. Again, flash ROM and TWRP Installer. Once done, Reboot to Recovery.
6. Now flash Gapps of your Choice. Stock are Recommend while anything above Nano will work. AROMA won't work. Once Gapps are flashed now go to Wipe and do a Factory Reset (Swipe to Factory Reset). Once that's Done, hit Reboot System and wait for the ROM to Load.
7. After the Initial Setup, Reboot to Recovery and flash Magisk and Custom Kernel if you want.
Personally I Format Data after flashing Gapps to get a "clean install". But that's not necessary. Also if you want to flash ROMs often I'd suggest investing in Swift Backup. It's an excellent app for Backups and can Backup almost Anything. Hope this helps.
This is what I do. Occasionally I get the Encrypted Folders but if followed correctly all is smooth. I can flash any ROM without Encryption. Except maybe stock OOS.
Thanks Mannan.
However, what I'm really looking for is someone to explain the following:
1. Which action or actions is it that triggers the phone to be encypted without a way to decrypt when flashing a new rom? My suspicion is that if the phone was encrypted while having been boot from slot A, then wiping data while in slot A results in data loss. By extension, rebooting into slot B and then wiping data allows slot B to now hold the encryption key. I'm sure this theory has got some errors, but it's the best I can come up with having no intrinsic knowledge on the topic.
2. Are there any means of mitigating data loss should the phone become encrypted? I.e. If possible, can I back up data (minus /data/media) and then restore that when I can't get access to /data/media?
With regards to question #1, I developed my "suspicion" after lossing ambition to test it. When I get my ambition back to fight this issue, I'll try again. I'm just getting sick of transfering 25+ gigs of data via adb every time the data gets encrypted and I can't get it decrypted.
notorious.dds said:
Thanks Mannan.
However, what I'm really looking for is someone to explain the following:
1. Which action or actions is it that triggers the phone to be encypted without a way to decrypt when flashing a new rom? My suspicion is that if the phone was encrypted while having been boot from slot A, then wiping data while in slot A results in data loss. By extension, rebooting into slot B and then wiping data allows slot B to now hold the encryption key. I'm sure this theory has got some errors, but it's the best I can come up with having no intrinsic knowledge on the topic.
2. Are there any means of mitigating data loss should the phone become encrypted? I.e. If possible, can I back up data (minus /data/media) and then restore that when I can't get access to /data/media?
With regards to question #1, I developed my "suspicion" after lossing ambition to test it. When I get my ambition back to fight this issue, I'll try again. I'm just getting sick of transfering 25+ gigs of data via adb every time the data encryption kicks in.
Click to expand...
Click to collapse
You're not that far off, actually. And while I'm no developer I suspect that Encryption kicks in when
a). You flash stock OOS. No matter what ROM you are on, when you flash OOS it's possible you can get encrypted. I'm not sure about this but if a developer could confirm that'd be great. This one time, I flashed OOS Stable while on Beta and it Encrypted my Storage. So I had to retransfer with a computer to flash it the required two times. So basically avoid flashing OOS when on a Custom ROM. Even when switching ROMs.
b). Just as you said, when you Wipe Data within TWRP and then Reboot to TWRP it also Encrypts the Device. So I usually Wipe Data after flashing ROM & Gapps. Otherwise if you Wipe Data after flashing ROM it will Encrypt you.
And to answer that last Question the app I personally use is called Swift Backup. It's an amazing app and although it costs $5.49 it can Backup Apps and Data. It can also backup the Files in Android/obb. Give it a go.
Mannan Qamar said:
You're not that far off, actually. And while I'm no developer I suspect that Encryption kicks in when
a). You flash stock OOS. No matter what ROM you are on, when you flash OOS it's possible you can get encrypted. I'm not sure about this but if a developer could confirm that'd be great. This one time, I flashed OOS Stable while on Beta and it Encrypted my Storage. So I had to retransfer with a computer to flash it the required two times. So basically avoid flashing OOS when on a Custom ROM. Even when switching ROMs.
Click to expand...
Click to collapse
I've been fiddling around with OOS and The Pixel Experience (aka TPE) ROM. I've yet to need to flash OOS in order to loose my ability to decrypt. Flashing TPE screws everything up quite nicely as well. That said, I have gotten into the situation where TWRP (booted from either slot) has got everything encrypted. However, in one case, I was able to get the data back by recreating the boot_a partition as it existed before I wiped data. I think there may be something to be learned here. However, subsequent attempts to use this method have not been successful. In other words, I'm not sure what I actually learned.
Mannan Qamar said:
And to answer that last Question the app I personally use is called Swift Backup. It's an amazing app and although it costs $5.49 it can Backup Apps and Data. It can also backup the Files in Android/obb. Give it a go.
Click to expand...
Click to collapse
I'm still using Titanium Backup (paid version as well). It works quite well and I'm happy with it. That said, it's still a much bigger pain in the butt to restore vs performing a nandroid restore of the data. It's apples and oranges though. In order for the nandroid to provide any real value, you pretty much have to do right before need it... unless you never do anything on your phone. It also only works with the ROM from which it was created... obviously. Since my current nandroid backup of /data is > 22 gb, its fairly cumbersome.
notorious.dds said:
I've been fiddling around with OOS and The Pixel Experience (aka TPE) ROM. I've yet to need to flash OOS in order to loose my ability to decrypt. Flashing TPE screws everything up quite nicely as well. That said, I have gotten into the situation where TWRP (booted from either slot) has got everything encrypted. However, in one case, I was able to get the data back by recreating the boot_a partition as it existed before I wiped data. I think there may be something to be learned here. However, subsequent attempts to use this method have not been successful. In other words, I'm not sure what I actually learned.
I'm still using Titanium Backup (paid version as well). It works quite well and I'm happy with it. That said, it's still a much bigger pain in the butt to restore vs performing a nandroid restore of the data. It's apples and oranges though. In order for the nandroid to provide any real value, you pretty much have to do right before need it... unless you never do anything on your phone. It also only works with the ROM from which it was created... obviously. Since my current nandroid backup of /data is > 22 gb, its fairly cumbersome.
Click to expand...
Click to collapse
I dunno if it will work but when you get Encrypted try booting the TWRP image. Maybe that'll work.
Doesn't this problem occur with backups and restore from twrp as well?.... This A/B stuff I'm not used to but I'll keep reading and hopefully something in my brain will kick in lol...
Mannan Qamar said:
I dunno if it will work but when you get Encrypted try booting the TWRP image. Maybe that'll work.
Click to expand...
Click to collapse
Yeah, that I defintitely tried. No dice. However, I just backed up everthing and I'm about to start blowing the thing up with ROM flashes. Consider it a stress test. I'll report back.
What I've got so far...
Coming from OOS 9.0.14 running on slot B with a lock screen pattern enabled, I boot into TWRP on slot B.
I then flashed The Pixel Experiance ROM via it's .zip file. (The flash is then applied to slot A because it goes to the inactive slot).
Flashed the TWRP install .zip
Changed active slot to A
Reboot to recovery (aka TWRP) ... now in slot A.
wiped data (minus storage)
Flashed magisk
Reboot system
This got me into the new ROM with data intact. However, when rebooting to recovery (still slot A), it would ask for a pattern but yet wouldn't accept the pattern to decrypt. Rebooting back into Pixel Experience the data was decrypted. So, even the data would decrypt when booted into system, I could no longer get to the data from within TWRP. I then changed the lock pattern from within Pixel Experience and reboot to TWRP, it still couldn't decrypt the data. Rebooting back to system succeeded in that it actually boot, but I could no longer unlock the phone (stuck on "phone is starting"). My presumption at this point was that Pixel Experience could no longer decrypt the data.
I then:
Reboot to TWRP (slot A still)
Flashed OOS
Flashed TWRP
Set active slot to B
Reboot to recovery (aka TWRP)
wiped data (minus storage)
reboot to system
At this point OOS failed to boot and I was returned to TWRP. Data was still not able to be decrypted. I then did a factory reset plus wiped storage (aka data, dalvik, and internal storage) and tried to boot to system... still failed and sent me back to TWRP. This time, although data was empty, it was decrypted. I tried to reboot system again. It failed again and sent me back to TWRP.
So, at this point , I've wiped data and internal storage but I cannot get stock OOS to boot. So, I reboot to bootloader and executed:
Code:
fastboot -w
My understanding is that this should do the same this as performing a factory reset from within TWRP. However, rebooting to system succeeded this time.
So, the new questions are:
1. How is it that I can decrypt data when booted into Pixel Experience on slot A, but I cannot decrypt the data via TWRP?
2. If I removed the lock screen pattern from OOS before flashing PixelExperience, would I have been able to decrypt the data in both the ROM and within TWRP?
3. Why is factory resetting via fastboot effective when doing so in TWRP is not?
notorious.dds said:
What I've got so far...
Coming from OOS 9.0.14 running on slot B with a lock screen pattern enabled, I boot into TWRP on slot B.
I then flashed The Pixel Experiance ROM via it's .zip file. (The flash is then applied to slot A because it goes to the inactive slot).
Flashed the TWRP install .zip
Changed active slot to A
Reboot to recovery (aka TWRP) ... now in slot A.
wiped data (minus storage)
Flashed magisk
Reboot system
This got me into the new ROM with data intact. However, when rebooting to recovery (still slot A), it would ask for a pattern but yet wouldn't accept the pattern to decrypt. Rebooting back into Pixel Experience the data was decrypted. So, even the data would decrypt when booted into system, I could no longer get to the data from within TWRP. I then changed the lock pattern from within Pixel Experience and reboot to TWRP, it still couldn't decrypt the data. Rebooting back to system succeeded in that it actually boot, but I could no longer unlock the phone (stuck on "phone is starting"). My presumption at this point was that Pixel Experience could no longer decrypt the data.
I then:
Reboot to TWRP (slot A still)
Flashed OOS
Flashed TWRP
Set active slot to B
Reboot to recovery (aka TWRP)
wiped data (minus storage)
reboot to system
At this point OOS failed to boot and I was returned to TWRP. Data was still not able to be decrypted. I then did a factory reset plus wiped storage (aka data, dalvik, and internal storage) and tried to boot to system... still failed and sent me back to TWRP. This time, although data was empty, it was decrypted. I tried to reboot system again. It failed again and sent me back to TWRP.
So, at this point , I've wiped data and internal storage but I cannot get stock OOS to boot. So, I reboot to bootloader and executed:
My understanding is that this should do the same this as performing a factory reset from within TWRP. However, rebooting to system succeeded this time.
So, the new questions are:
1. How is it that I can decrypt data when booted into Pixel Experience on slot A, but I cannot decrypt the data via TWRP?
2. If I removed the lock screen pattern from OOS before flashing PixelExperience, would I have been able to decrypt the data in both the ROM and within TWRP?
3. Why is factory resetting via fastboot effective when doing so in TWRP is not?
Click to expand...
Click to collapse
Well starting from the way you flashed the ROM, the rule of thumb is that you NEVER manually change slots. Now since you are on stock follow the instructions I posted earlier to flash PE or any other ROM for that matter. I think when you manually set the slot it somehow messed up Decryption. Next, after flashing OOS from TWRP when you are on a Custom ROM, you must always Format Data. The command you ran via Fastboot (fastboot -w) does just that.
So I just flashed Bootleggers from Stock OpenBeta 11. These are the steps I followed. I was successfully able to flash and was able to keep my Data intact. These are the steps I followed.
Starting from OpenBeta 11 I flashed ROM (Bootleggers) and then TWRP Installer. Then go to Reboot and Select Recovery. Once in Recovery, again flash ROM and TWRP Installer. Once done, reboot to Recovery. Flash Gapps and then go to Wipe and do a Swipe to Fa Tory Reset. This will Delete all your Data except Internal Storage. This is a necessary step when flashing a ROM. Once done, reboot to System. After this I was able to boot up Successfully with my Internal Storage as it was before flashing. After that I restored my backup. Everything is working and I can enter and Decrypt TWRP without error.
This thread should be pined as a guide because instalation notes in ROM threads are so basic.
A couple of things come to mind reading this thread in reference to encryption
1) if security patches dont match on A/B, it seems to trigger a lockout with encryption. i may be wrong.
2) if internal storage isnt wiped, i.e.-if you use the "factory reset' option in twrp, your data is still there and that in itself post-flash can trigger encryption error as the data is still there.
I think about it like this, despite it being A/B partitions, the data is like a middle layer that isnt individualized to one partition or the other. so a trigger/failure for secure boot encrypts it all.
kitcostantino said:
A couple of things come to mind reading this thread in reference to encryption
1) if security patches dont match on A/B, it seems to trigger a lockout with encryption. i may be wrong.
2) if internal storage isnt wiped, i.e.-if you use the "factory reset' option in twrp, your data is still there and that in itself post-flash can trigger encryption error as the data is still there.
I think about it like this, despite it being A/B partitions, the data is like a middle layer that isnt individualized to one partition or the other. so a trigger/failure for secure boot encrypts it all.
Click to expand...
Click to collapse
I'm pretty sure, that if you flash anything with a security patch earlier than the one you're currently using your data will get encrypted.
Which is why it happens with going back to OOS from custom, because they're always late with security patches compared to custom roms.
The hardest thing for.me coming from an A only device (Axon 7) has been learning order of operations. as long as one flashes rom followed by twrp and then a reboot into recovery, followed by installing magisk, things usually go okay. Going from aosp to aosp went okay, but like you said moving from OOS to AOSP or vice versa always yielded encryption lock. maybe we could make a merged security patch or something of the sort to bridge the gap. im no dev, so im sure someone who knows more than i can tell us why that wouldnt work. it would be really cool for One Plus to gain a better foothold in custom OS before the majority of crack flashers and devs swear off. Dont get me wrong, OOS is amazing and i feel with the inherent features, is superior to any other stock rom, but android is all about choice.
i really and truly wish someone would make a version of TWRP that had a dual boot set up vs A/B. I have had devices (looking at you, Droid Bionic) that never had proper root/bl unlock and had amazing rom communities bc of safestrap/dual boot/etc. i am more than willing to give up internal storage space to duplicate/clone /data and anything else that is on both systems. i also wish recovery had its own partition again, but that one is beyond our control at this point as it resides in boot now.
Maybe its conceivable. Who knows.
I have no issues. I don't lose anything when I flash ROMs. I boot to twrp, factory reset(not wipe storage), flash ROM, flash twrp installer....boot ROM, reboot twrp, flash gapps, custom kernel. Then I factory reset again (not wipe storage) and then install magisk..done....no issues. It will fail boot once and then boot fine because of this process but only after you do this. So if you reboot later you are fine...I keep all my stuff
First off, I want to thank all of you who contributed to this thread. I'm defintely gaining a better understanding of some of the pitfalls associated with A/B devices and encryption. Thanks!
Mannan Qamar said:
Well starting from the way you flashed the ROM, the rule of thumb is that you NEVER manually change slots. Now since you are on stock follow the instructions I posted earlier to flash PE or any other ROM for that matter. I think when you manually set the slot it somehow messed up Decryption.
Click to expand...
Click to collapse
So, my understand is that flashing a new ROM from within TWRP flashes it to the inactive slot. Therefore, my assumptions as to the reasoning behind rebooting from TWRP back into TWRP before wiping data were that:
Any modifiations made to the boot partition intended to affect the new ROM need to be made to the boot partition that shares the same slot as that of the new ROM, and
Wiping data while booted into image of TWRP which shares the same slot as the new ROM has some magical effect on preserving the ability to decrypt data vs wiping data while booted into the image of TWRP that resides in the slot of the ROM to be replaced.
It is these assumptions (combined with my execution of the basic recipe failing to prevent encryption lock-out) which led me to manually changing slots. I will say this... after flashing PE and TWRP.zip from within TWRP on slot B, simply rebooting to recovery brought me right back to TWRP on slot B. If PE is now on slot A, how does installing magisk, etc. do me any good while in slot B? Also, are my assumptions misguided as to the "why" rebooting to TWRP before installing magisk, wiping data, etc is necessary?
Mannan Qamar said:
Next, after flashing OOS from TWRP when you are on a Custom ROM, you must always Format Data. The command you ran via Fastboot (fastboot -w) does just that.
Click to expand...
Click to collapse
Lightbulb status: on
Thanks!
kitcostantino said:
If security patches dont match on A/B, it seems to trigger a lockout with encryption. i may be wrong.
Click to expand...
Click to collapse
Is this why in Mannan Qamar's earlier post he appears to be flashing the new ROM to BOTH slots before trying to boot into system?
ebproject said:
I'm pretty sure, that if you flash anything with a security patch earlier than the one you're currently using your data will get encrypted.
Which is why it happens with going back to OOS from custom, because they're always late with security patches compared to custom roms.
Click to expand...
Click to collapse
I'm assuming that flashing OOS to BOTH slots as is mentioned earlier with regards to flashing a custom ROM won't help when going back to OOS given the old vs new issue. Has anyone verified that yet?
It's my understanding that the sure security patch is applied to the system partition, correct? Is part of that patch included in boot, or no?
jamescable said:
I have no issues. I don't lose anything when I flash ROMs. I boot to twrp, factory reset(not wipe storage), flash ROM, flash twrp installer....boot ROM, reboot twrp, flash gapps, custom kernel. Then I factory reset again (not wipe storage) and then install magisk..done....no issues. It will fail boot once and then boot fine because of this process but only after you do this. So if you reboot later you are fine...I keep all my stuff
Click to expand...
Click to collapse
I notice that the FIRST thing you do is "factory reset". That's definitely not standard with the install threads I've read. Hmmmmm, interesting.
Also, why do you boot the ROM before flashing gapps, and kernel? It seems unnecessary since you're just factory resetting again. I'm sure I'm missing something on this one.
notorious.dds said:
I notice that the FIRST thing you do is "factory reset". That's definitely not standard with the install threads I've read. Hmmmmm, interesting.
Also, why do you boot the ROM before flashing gapps, and kernel? It seems unnecessary since you're just factory resetting again. I'm sure I'm missing something on this one.
Click to expand...
Click to collapse
Booting to ROM solved the encryption issues
idkwhothatis123 said:
Yes. For the love of god. Someone please clear up how we can flash on the go. I don't always have access to a computer with Adb/fastboot.
Every time I try to switch roms, upon rebooting to TWRP, my folders encrypt. Then I have to format data and voila, no fricking ROM to flash and I'm stuck
Click to expand...
Click to collapse
If you stuck on encrypted storage ever, reboot to system and after you see the setup screen, reboot to recovery again. Voila, your storage is decrypted now.
It happened to me all the time when I flash OOS and this way I am able to decrypt my internal storage.