Database Users

Apps Cannot Access Internal Memory after TWRP Recover

Background:
Trying out ROMs here and there, making backups along the way in TWRP. Turn out I had my favorite ROM from the start, so clean all the slates and performed a restore to Hybrid X Series 1.07. Everything is as it was...almost.
Apps are complaining about not being able to download to internal memory, there not being enough space, etc (there's plenty). Switching the download directory of these apps to the extSdCard is my current workaround.
What I've tried:
Fix Permissions from TWRP
Repair partition /data from TWRP using e2fsck
recursive busybox chown to 1001 on /data/media
restorecon on /data/media/0
This Fix Script Also found a similar flashable ZIP which did the same thing, but to no avail
None of these seem to have any effect. Also, trying to change permissions from Root Browser works on /sdcard, but not on any subfolders. Error returned is "Changing permissions was not successful. Please note that some file systems do not allow permission changes."
Any thoughts

funkeywoookey said:
Background:
Trying out ROMs here and there, making backups along the way in TWRP. Turn out I had my favorite ROM from the start, so clean all the slates and performed a restore to Hybrid X Series 1.07. Everything is as it was...almost.
Apps are complaining about not being able to download to internal memory, there not being enough space, etc (there's plenty). Switching the download directory of these apps to the extSdCard is my current workaround.
What I've tried:
Fix Permissions from TWRP
Repair partition /data from TWRP using e2fsck
recursive busybox chown to 1001 on /data/media
restorecon on /data/media/0
This Fix Script Also found a similar flashable ZIP which did the same thing, but to no avail
None of these seem to have any effect. Also, trying to change permissions from Root Browser works on /sdcard, but not on any subfolders. Error returned is "Changing permissions was not successful. Please note that some file systems do not allow permission changes."
Any thoughts
Click to expand...
Click to collapse
First off, this is an ownership issue, not a permissions issue
Second, chown 1001 gives ownership to Radio, not System. Chown to 1000 for System ownership
I never did find a solution to this problem. In my attempts to correct the problem, I ran chown on the entire /data partition by accident. The ROM booted, but everything crashed. After this, I just decided to salvage what little I might be able to save and move on with life.
The problem even persisted through wipe/reinstall of the ROM.
Finally, as a last resort, I had flash the stock version through Odin and start over from scratch.

TWRP Nandroid backups of /data won't restore

None of my /data backups from the past month will restore. Instead, each restore will end with "extractTarFork() process ended with ERROR=255" (more from log below).
Code:
file: //data/data/com.android.externalstorage/lib
==> extracting: //data/data/com.android.externalstorage/lib (symlink to )
symlink(): No such file or directory
FAILED RESTORE OF FILE i: //data/data/com.android.externalstorage/lib
E:Unable to extract tar archive '/data/media/0/TWRP/BACKUPS/[serial number]/2015-10-22--16-35-10/data.f2fs.win000'
I:Error extracting '/data/media/0/TWRP/BACKUPS/[serial number]/2015-10-22--16-35-10/data.f2fs.win000' in thread ID 0
E:Error extracting split archive.
E:extractTarFork() process ended with ERROR=255
Here's the setup:
Recovery: Multirom TWRP (2015-06-30 2.8.7)
ROM: CM12.1 nightlies and more recently crDroid (CM12.1 based)
F2FS formatted /data, /cache, /sdcard
Here's what I've done to investigate:
1. Thinking the problem had to do with compressed backups, I uncompressed using Nandroid Manager. No difference.
2. Thinking the problem had to do with the backup being spread across multiple .win tar files, I concatenated them into one .win tar file. No difference.
3. Thinking that perhaps I could unarchive the tar and adb push the /data partition. It pushed fine and booted, however the davlik cache wasn't rebuilt when booting and the OS was unusable due to FC.
4. Thinking that I just needed to remove the files/directories that were throwing errors during restore, I used tar --delete to remove files/directory from the nandroid backup. This was somewhat successful in that the restore would progress further before failing, but it would just throw a similar error for another directory.
I haven't figured out why these backups won't restore, but I've thus far resorted to just restoring my most recent backup that will restore. The problem with this is that my future backups seem to have this same problem of not being restorable.
What to do?

Is there a way to extract the /data nandroid and adb push it? When I attempted that, the booted OS was filled with force closes. Was it a permissions or selinux permissions problem? How to resolve it?

How do I persist changes to the /oem folder?

I have the Moto G7 International variation (INDIA/RETIN, it turns out; I'm in the US and thought I was buying the RETUS version, but I didn't pay enough attention to the store page). I have rooted the device easily, thanks to the assistance of these forums. I have also installed TWRP 3.3.1-2-river for the purpose of backups and restores, though there seems to be problem with it. However TWRP is not the focus of my question right now.
I have become quite annoyed with the boot animation on this phone and would like to change it. Putting new boot animations in the /system/media folder has not been working and it seems to be because this international model has a boot animation override (that trumps all other boot animation locations) in the /oem/retin/media folder. The /oem folder is initially read-only, but can be remounted with read-write without error (using root). Using MiXplorer or the adb shell, I can delete stuff from there without error as well. However, when I reboot, it all comes back. I suspect that the changes I make to the /oem folder are not actually persisted to whereever in the first place. When I look at the mounts on the phone, I see /oem is mounted from /dev/block/dm-1. /dev/block/dm-1 is listed in my partitions list when the phone is booted up normally. From what I can tell though (and I'm not that knowledgable on this topic), that is just a "wrapper" around some data elsewhere. How can I tell where this dm-1 partition's data is really coming from?
As additional information, the block list by-name lists the oem image as belonging to block mmcblk0p58. When I boot into TWRP and mount that block, I do see the usual /oem stuff in there. I deleted the boot animation from there and tested unmounting the block, rebooting into TWRP and remounting the block and can see that the boot animation is still properly deleted from there. However, when I boot the phone back up normally, I see the usual stuff (including the boot animation) in the /oem folder still. That might make some sense, since it is listed as being mounted from /dev/block/dm-1 and not from /dev/block/mmcblk0p58 (though I was hoping dm-1 was a wrapper for mmcblk0p58).
This is the only dm mount on the phone; I don't have the problem with /system being uneditable because it is mounted to dm-0 like I see in many other posts for other phones. In turn, I don't think I have the "base data is encrypted" problem from those cases either (it is "just" an oem setup and not personal data). This still seems somewhat like the dm-verity problem I read about with other phones though (although this one doesn't give the dm-verity warning when I try to remount /oem as read-write). Running the commands:
Code:
adb root
adb disable-verity
just tells me
Code:
disable-verity only works for userdebug builds
verity cannot be disabled/enabled - USER build
I'm not sure how to check if there is dm-verity on in the first place though.
As a more direct question (though I would still like to know how to trace this dm-1 partition backwards), how can I make changes to this /oem folder actually persist between reboots?

@Spaceminer described the exact same problem for the Moto G6 in the Universal DM-Verity remover thread. In the end he said that it wasn't solvable without a system root.
https://forum.xda-developers.com/an...rceencrypt-t3817389/post81974407#post81974407
(I have since returned the phone because of this.)

1equals2 said:
@Spaceminer described the exact same problem for the Moto G6 in the Universal DM-Verity remover thread. In the end he said that it wasn't solvable without a system root.
https://forum.xda-developers.com/an...rceencrypt-t3817389/post81974407#post81974407
(I have since returned the phone because of this.)
Click to expand...
Click to collapse
i was unable to get this to persist either. i ended up using magisk for my work instead.

@1equals2 @HT123 To persist changes to your /oem partition the HAB (high assurance boot) verification must be deactivated. I've already successfully tested it on my device, a Moto G6 plus.
Here are the required steps: (only for Stock ROM!!)
1.) Start TWRP and backup /vendor + /system
2.) Mount /vendor + /system and delete the following
files:
- /vendor/etc/init/hw/init.mmi.hab.rc
- /vendor/bin/init.mmi.hab.sh
- /system/verity_key
3.) Go to /vendor/etc/init/hw and adb pull init.mmi.rc in order to edit the file.
4.) Open it and delete the lines 19+20:
Code:
19 # Moto verified boot extension
20 import /vendor/etc/init/hw/init.mmi.hab.rc
5.) Save the changes, copy it back to its path and set permissions to: -rw-r--r-- root:root
==> After the next reboot /vendor and /oem are mounted as ../vendor_a and ../oem_a (depending on your active slot).
As the last step you have to customize fstab.qcom to be able to access the files within the /oem folder.
Adb pull /vendor/etc/fstab.qcom and open it. The last line defines how /oem is mounted during the init process. You should change it like this:
before:​
Code:
/dev/block [...] /oem ext4 ro,nosuid,nodev,[STRIKE]context=u:object_r:oemfs:s0[/STRIKE] wait,slotselect,[STRIKE]verify[/STRIKE]
after:​
Code:
/dev/block [...] /oem ext4 ro,nosuid,nodev,[COLOR="red"]barrier=1[/COLOR] wait,slotselect
Save it and copy it back. Permissions: -rw-r--r-- root:root
>>> REBOOT AND ENJOY!!

OP 5T - FBE /dev/block/sda13 or /data partition - file restoration / residues scan

Hello,
unfortunately, my Google Keep app had just decided to wipe its db file up. (/data/data/com.google.android.keep/databases/keep.db)
It is not (and never was) synchronized to Google.
When it happened I just copied (dd'ed over adb) /dev/block/sda13.
From mount command output, I could recognized /dev/block/sda13 as /data backing block device.
Then I did some searches in the output image file and noticed it actually encrypted per file.
I learned from it OP uses FBE (file-based-encryption). I wanted to try and run some ext4 recovery tools (autopsy, undelete ...).
FBE encrypts also filenames so I can't use the generated image as is.
I checked what caused FBE on /data partition and it's the encryption setting. I read it uses the fingerprint/pattern/password to decrypt the encrypted /data.
I didn't power-off the phone since then to avoid any more losses.
My questions:
1. Given /dev/block/sda13 as an image file, how can one decrypt it and use forensics/restoration tools on it?
2. Another option - how to acquire the unencrypted /dev/block/sda13 from the running device?
3. Maybe there is some backup partition for files in /data?
4. Can I run the mentioned tools straight on some unencrypted device file? (Assuming cross-compilation to android)
5. Any caches/other places I can search for residues? (I tried dumping CursorWindow ashmem the Keep app used but the app has restarted since then)
6. OP compiled the kernel without DEVMEM. Maybe there is other option to acquire the whole RAM to try to scan for residues?
More side-notes:
- I tried other files in the Keep app directory to try to recover some residues, pictures where saved outside the db file so I was able to restore them (but what is important are the notes stored in the db itself(
- I was able to notice keep.db was replaced with an empty sqlite3 db file (for some strange reason ...). My guess was it didn't overridden the original db file so if I had the ext4 /data partition, I will be able to scan it with the tools mentioned above.
References:
* https://forum.xda-developers.com/t/...-encrypt-data-partition-on-oneplus-5.3642144/
* https://www.cs1.tf.fau.de/research/system-security-group/one-key-to-rule/
If some details are missing, please let me know!

https://www.cs1.tf.fau.de/research/system-security-group/one-key-to-rule/ is really interesting.
The problem now is how to get a full memory dump without /proc/mem and signed kernel modules.

Any help? I'm out of ideas but it seems like a solvable problem.

[SOLVED] Restore decrypted nandroid backup of FBE

Tried restoring a nandroid backup of the data partition with twrp.
also copied the /data/media partition back from an external copy.
when booting up the phone immediately reboots back into twrp with an error message:
Android Rescue Party...
The reported problem is:
'--reason=set_policy_failed_:/data/vendor'
the vendor partition seems to be intact and i do have a backup of it taken at the same time as the data backup, restoring it doesn't yield results..
i'm wondering if FBE is throwing it off, as the backup was taken when the phone was decrypted (within twrp) however the data on the partitions is referencing some sort of encryption key?

you may also exhibit the following error upon bootup of a restored nandroid backup.
immediately after booting, the phone reboots back into recovery.
viewing the log in twrp will show:
Android Rescue Party...
The reported problem is:
'--reason=set_policy_failed_:/data/bootchart'
1. the solution to this is editing fstab.​​under twrp or other recovery​mount /vendor from the mount icon.​​in twrp: Advanced > File Manager > /vendor/etc/fstab.qcom​select edit file under userdata, find where it says fileencryption=ice​rename fileencryption to encryptable.​​Original​
Code:
/dev/block/bootdevice/by-name/userdata /data ext4 noatime,nosuid,nodev,barrier=1,noauto_da_alloc,discard wait,check,fileencryption=ice,quota,reservedsize=512M
​​Modified​
Code:
/dev/block/bootdevice/by-name/userdata /data ext4 noatime,nosuid,nodev,barrier=1,noauto_da_alloc,discard wait,check,encryptable=ice,quota,reservedsize=512M
​save file.​
2. next delete the following directories:​
/data/unencrypted
/data/misc/vold/user_keys
3. Lastly delete any of the existing files from /data/system/ :​
locksettings.db
Gatekeeper.password.key
gatekeeper.pattern.key
locksettings.db-shm
locksettings.db-wal
recoverablekeystore.db
password.key
pattern.key
4. Reboot and re-encrypt​​
At this point rebooting from recovery will result in a running and successfully recovery backup.
one thing to note is the data and data/media partitions are at this point unencrypted
TRYING TO REINCRYPT NOW WILL FAIL to reencrypt got to settings > security > re set your pin or password for the phone
(optional) then select encryption and there will be an orange button to encrypt device.
the encryption process will take quite a while as it will reencrypt your entire phone.
The above doesn't work as it's trying to accomplish FDE, and the fstab line for encryptable=ice, isn't compatible with this.
i could not find an fstab string to follow the same option but for FBE.
​

Thanks for sharing, nice guide to disable forced FBE encryption! I think this applies to Android 12+ in general, not just OnePlus devices.
I ended up with the same problem on my Mi 10 Ultra with MIUI 13 after a /data partition restore and it was a real pain to solve ("set_policy_failed:..." rescue party error for different directories). It's strange though why it fails to set the fscrypt policy for existing directories with no policy, correct permissions and SELinux context...
(Btw: whether a fscrypt policy is applied to a directory ("is this directory encrypted?") can be checked with fscryptpolicyget in terminal.)
Unfortunately, this didn't directly solve my TWRP backup restore problem and I still had to do a manual restore, but now I can at least disable FBE and it's always nice to have actual control over the device you paid money for (you should really have control by default, but oh well...)
(Some of) the troubleshooting I did:
Like I mentioned, I first thought the issue might be with the SE linux context, so I tried running restorecon, but this didn't help - I eventually found that in init.rc, restorecon is usually already automatically run during each boot for directories under /data/... so running it manually makes no difference.
To edit /vendor/etc/fstab.qcom (or /system) on my device, I had to first disable the shared blocks EXT4 optional feature. I followed this nice guide to unpack/repack super.img. But this is missing the step for disabling shared blocks: when I tried to mount any of the unpacked images (e.g. vendor.img) as R/W, it failed with the useless generic error:
wrong fs type, bad option, bad superblock on ...
Click to expand...
Click to collapse
Then dmesg gave me another clue, but at the same time was still cryptic and not immediately helpful:
EXT4-fs (loop*): couldn't mount RDWR because of unsupported optional features (4000)`.
Click to expand...
Click to collapse
So I guess 4000 is the code for shared blocks and you can disable these with e2fsck -E unshare_blocks <your .img file or loop device> (and probably need a filesystem check with e2fsck -yf <file>). Again very annoying that these numerical feature codes are not mentioned anywhere in the e2fsck manual pages for example.
Anyway, I was finally able to either:
1. mount vendor.img on my PC (mount -o loop vendor.img /mnt/vendor) and edit the /mnt/vendor/etc/fstab.qcom right there before repacking the .img and flashing the new edited super.img to my device
or
2. just repacking the vendor.img with shared blocks disabled and size increased (resize2fs vendor.img <new size>) and flashing the new super.img without other modifications - this way /vendor can also be mounted as r/w in Android and changes made later (mount -o remount,rw /vendor).
The worst part is that in the end, even with decryption disabled and the keys deleted, the device still wouldn't boot after a /data restore from TWRP (and after multiple days spent on debugging )... I still had to manually extract the TWRP backup and move directories/files individually - thankfully no issues with app/ or data/ - I think the problem was with some files in either system/ or misc/, but idk for sure. I just manually went through and kept only what seemed important (saved wifi APs, BT devices, SMSs etc, but not saved accounts). And after this it finally booted with all my apps and (most of) my settings!
(Btw2: a TWRP/nandroid backup is apparently just a bunch of separate tar.gz files, not a split archive, so you can just extract them with for file in ../data.f2fs.win*; do echo "extracting $file..."; busybox tar -xzf $file; done)