I have a complete setup for the Nexus 7, part of a product we are working on, that I need to easily clone on "virgin" tablets for production. The app requires a rooted OS.
I want to write an installation script using fastboot to unlock the bootloader, erase partitions, then flash them with .img files for each partition (kernel, system, cache, etc.).
How do I extract .img files from my "master" tablet? I have an understanding from some where that these are simple byte-for-byte dumps of the partition -- is this true? As such can I create a .img file by simple doing 'cat blkfile >file.img' where "blkfile" is the appropriate block device for the partition in question?
Or do I need to use 'dd'? Or something else?
I have searched and searched, and can't find an anwer. I've found other answers using some tools to create these files from a build on a PC, but nothing about creating them from an existing tablet.
Thanks in advance!
Use the dd command. You can use it both to dump and write a partition. It's how I install recovery programs like TWRP
Sent from my Nexus 7
You can use dd for the boot partition and recovery partition - they are raw binary blobs. (Don't use dd on other Android devices, esp. those that have MTD flash devices, though - it only works most of the time there)
If you want to use the same fastboot-based scenario that Google uses for factory image sets, then for the system & userdata image files you will need to find out about "sparse ext4 filesystem images"
If you took a raw block-device based dump of any of your tablet ext4 partitions, you could actually take those image files and mount them on any other linux machine (using a loopback mount procedure).
But you will find that if you attempt to do that with the Google factory ".img" files (for system & userdata partitions), they will not mount. It's not a simple matter of a offset superblock, either.
Since these are the formats that the stock recovery expects, I suppose you ought to use those formats if you want to do the "all at once all partitions" fastboot flashing if you plan on using the stock recovery.
Note that there is absolutely nothing that prevents you from unpacking whatever you want from whatever archive format you want - so long as the recovery's busybox supports the archive format correctly - you could use cpio or pax or tar archives for that matter. (The stock recovery's "toolbox" has very little functionality, so this comment applies to custom recoveries, which typically have more robust functionality in their busybox) You will be writing your own scripts to do those things though, typically either in one of two ways:
1.A mount target filesystem partition
1.B do a deep recursive remove at that mountpoint ( rm -rf * )
1.C unpack your archive into same mount point ( tar xf archive.tar, etc)
1.D unmount the mount point
OR
2.A unmount target partition and zero it out (dd if=/dev/zero, flash_erase, etc)
2.B recreate filesystem in partition (mke2fs -t ext4 etc)
2.C mount target filesystem
2.D unpack your archive into the same mount point (tar xf archive, pax, cpio, unyaffs2, etc)
2.E unmount that mountpoint
Even though this post is for the Samsung Galaxy S II, the same thing applies to the factory Nexus 7 images from Google:
http://forum.xda-developers.com/showthread.php?t=1081239
As that thread mentions, the simg2img and mkuserimg.sh programs are part of the Android project.
Here's a Nexus 7 thread where the contributor built the tools for both x86 linux and arm linux
Finally, I should note that because /system is typically mounted read-only, imaging /system from the live OS is no big deal. Trying to do the same thing with /data is an extremely dopey idea, however. Accurate backups are rarely made from live read-write filesystems.
cheers
Thank you so much for all the great information! I hit thanks for both of you.
The link to the nexus 7 thread is what I need... This is for my company, and I need a simple cloning solution that can be performed by a non-technical assembly person. The fastboot install procedure is about as simple as it gets.
Thanks again!
All-F2FS Partitions for Oneplus One
I am not responsible for any damage to your phone.
Make always a full nandroid backup.
Read before downloading or posting in the thread.
Proceed with caution if you have no experience.
Again, make always a full (including /data/media) nandroid backup.
[ What is F2FS ]
http://www.cnx-software.com/2013/01/15/f2fs-a-new-flash-file-system-for-mobile-devices-elce-2012/
http://www.xda-developers.com/android/f2fs-put-to-the-test-against-ext4/
http://en.wikipedia.org/wiki/F2FS
[ TESTED WITH ROMs ]
CM12 nightly
CM11 nightly
Temasek
Slim Saber (tested by @Setting.Out)
CM11S stock using complete guide with ak boot.img from CM11S kernel. (tested by @Leicxan)
Calkulin's_OPO_ROM (tested by @DarkAngel-81)
Madhi
Gummy (tested by @Setting.Out)
EOS (tested by @Setting.Out)
Slim (tested by @Setting.Out)
For CM12 use this guide:
F2FS for CM12.X
(if you test other roms, please report here, so we can track working roms)
Use these recoverys to work with F2FS partitions:
CWM Advanced Edition - PhilZ Touch
or
TWRP
this guide for All-F2FS on OPO is based on guide of ronedogg.
Original thread: http://forum.xda-developers.com/showthread.php?t=2694201
[ Prebuild ROM ]
Download a F2FS ready ROM:
CM-11-20141203-NIGHTLY-bacon-F2FS
CM-11S-XNPH33R-bacon-signed-F2FS
Calkulin's stock rooted XNPH33R-F2FS (thanks to @LookedPath)
Madhi 8-30 build-F2FS (thanks to @SchmilK)
SlimSaber-F2FS (thanks to @dexxa05)
Droidkang-F2FS (thanks to @SchmilK)
Carbon-F2FS (thanks to @Setting.Out)
1. Flash Philz Recovery with F2FS support.
WARNING make a FULL nandroid backup, and place it on external storage, also SD partition will be formatted
2. Boot to the Philz Recovery, format /system /cache and "/data and /data/media" to F2FS
3. Flash the downloaded modified rom, (install the gapps.zip if needed) and AK Kernel for your rom.
4. Reboot & enjoy the new speed!
[ Easy Way First time format in F2FS]
[ without using a prebuild F2FS enabled rom ]
1. Download latest CM nightly or CM11S
2. Download:
F2FS-ALL-CM11.zip for CM11
F2FS-ALL-CM11S.zip for CM11S ( tanks to @Setting.Out) and extract boot.img and updater-script
3. Copy the extracted boot.img in rom zipfile overwriting original one (tested with 7zip)
4. Copy the extracted updater-script in rom zipfile overwriting original one, for cm11 is located under .\META-INF\com\google\android\.
5. Flash Philz Recovery with F2FS support.
WARNING make a FULL nandroid backup, and place it on external storage, also SD partition will be formatted
6. Boot to the Philz Recovery, format /system /cache and "/data and /data/media" to F2FS
7. Flash the modified CM nightly zipfile, the gapps.zip and AK Kernel ( for this step, i've used an OTG cable and an external hdd, because the phone now is completely empty, or use ADB sideloading)
8. Reboot & enjoy the new speed!
[ Easy Way Upgrading a rom with partition already on F2FS ]
[ without using a prebuild F2FS enabled rom ]
1. Download latest CM nightly (https://download.cyanogenmod.org/?device=bacon)
2. Download F2FS-ALL-CM11.zip and extract boot.img and fstab.grouper
3. Copy the extracted boot.img in the CM nightly zipfile overwriting original one
4. Copy the extracted fstab.grouper in the CM nightly zipfile overwriting original one, located under .\META-INF\com\google\android\
5. Flash the modified CM nightly zipfile, and AK Kernel
6. Reboot
[ Complete guide to manually modify the rom ]
This is the exact procedure i've used for CM11 (OPO ready)
to unpack/repack the boot.img i've used http://boot-repacker.blogspot.it/
Oneplus ONE - CM Nightly with All-F2FS
1. Download latest CM nightly
2. Download AK kernel zipfile
3. Open CM nightly zipfile, extract the boot.img
4. Open AK kernel zipfile, extract the zImage and rename it boot.img-zImage
5. Unpack the boot.img in step3
6. Replace the boot.img-zImage in step5 with AK boot.img-zImage (step4)
7. Replace fstab.grouper content with this one
Code:
# Android fstab file.
#
# The filesystem that contains the filesystem checker binary (typically /system) cannot
# specify MF_CHECK, and must come before any filesystems that do specify MF_CHECK
/dev/block/platform/msm_sdcc.1/by-name/system /system f2fs ro,noatime,nosuid,nodev,discard,nodiratime,inline_xattr,errors=recover wait
/dev/block/platform/msm_sdcc.1/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,nodiratime,inline_xattr,errors=recover wait,nonremovable,encryptable=/dev/block/platform/msm_sdcc.1/by-name/reserve4
/dev/block/platform/msm_sdcc.1/by-name/cache /cache f2fs noatime,nosuid,nodev,discard,nodiratime,inline_xattr,errors=recover wait
/dev/block/platform/msm_sdcc.1/by-name/persist /persist ext4 nosuid,nodev,barrier=1,data=ordered,nodelalloc,nomblk_io_submit,errors=panic wait,check
/dev/block/platform/msm_sdcc.1/by-name/boot /boot emmc defaults defaults
/dev/block/platform/msm_sdcc.1/by-name/recovery /recovery emmc defaults defaults
/dev/block/platform/msm_sdcc.1/by-name/misc /misc emmc defaults defaults
/dev/block/platform/msm_sdcc.1/by-name/modem /firmware vfat ro,shortname=lower,uid=1000,gid=1026,dmask=227,fmask=337,context=u:object_r:firmware_file:s0 wait
/devices/platform/xhci-hcd* auto auto defaults voldmanaged=usbdisk:auto
8. Repack boot.img place it back in the CM nightly zipfile
9. On the CM nightly zipfile, edit updater-script and edit these lines:
Code:
mount("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/userdata", "/data");
replace it with
Code:
run_program("/sbin/busybox", "mount", "/data");
this
Code:
mount("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "/system");
replace it with
Code:
run_program("/sbin/busybox", "mount", "/system");
and these
Code:
format("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "0", "/system");
mount("ext4", "EMMC", "/dev/block/platform/msm_sdcc.1/by-name/system", "/system");
replace with
Code:
run_program("/sbin/mkfs.f2fs", "/dev/block/platform/msm_sdcc.1/by-name/system");
run_program("/sbin/busybox", "mount", "/system");
10. Save the updater-script and put it back to the zipfile. CM nightly zipfile is complete and ready for flashing.
11. Flash Philz Recovery with F2FS support.
WARNING make a FULL nandroid backup, and place it on external storage, also SD partition will be formatted
12. Boot to the Philz Recovery, format /system /cache and "/data and /data/media" to F2FS
13. Flash the modified CM nightly zipfile, the gapps.zip and AK Kernel ( for this step, i've used an OTG cable and an external hdd, because the phone now is completely empty, or use ADB sideloading)
14. Reboot
15. Enjoy the new speed!
thanks to:
@ak for the kernel
@Phil3759 for recovery
@ronedogg for original guide
@katinatez & @johnnyslt for OPO F2FS enabled Philz Recovery
reserved
Tested,work with stock cm11s XNPH33R
I just followed step 1~7 then i restored my nandroid backup
Here's my method
Step 8,repack boot.img at first,then i use fastboot command to flash boot.img.
Step 9, Boot in to Philz Recovery, flash AK kernel ,reboot your phone
Step 10, enjoy your phone with Full F2FS
Using Complete guide from OP sholud work, you can use this rom, Calkulin's_Stock_OPO_XNPH33R_ROM_v1.0
sorry for my english
Any noticeable performance gains? Is it worth the effort?
Hi gulp79, do you have problem with power off charging? I stucked at oneplus logo , battery logo wasn't show up,I'm using AK kernel with full f2fs.
STOCK XNPH33R is good work, too
Thanks to @gulp79, @Leicxan
I do Leicxan's Guide, good work now.
XNPH30O-bacon-signed-fastboot.zip flash -> XNPH33R Update -> nandroid full backup
kernel : cm11s/stock: ak.031.opo.cm11s
recovery : bacon_philz_touch_6.57.5_katinatez.zip
sorry for my english
sublimaze said:
Any noticeable performance gains? Is it worth the effort?
Click to expand...
Click to collapse
i didn't have time to run a benchmark before and after the conversion, btw the boot time is a lot better now.
zzori said:
Thanks to @gulp79, @Leicxan
I do Leicxan's Guide, good work now.
XNPH30O-bacon-signed-fastboot.zip flash -> XNPH33R Update -> nandroid full backup
kernel : cm11s/stock: ak.031.opo.cm11s
recovery : bacon_philz_touch_6.57.5_katinatez.zip
sorry for my english
Click to expand...
Click to collapse
Can you conform if power off charging works after convert to f2fs? For me it's not working.
Leicxan said:
Hi gulp79, do you have problem with power off charging? I stucked at oneplus logo , battery logo wasn't show up,I'm using AK kernel with full f2fs.
Click to expand...
Click to collapse
with phone off i too have this problem, but i don't know if is related to FS.
Result
Leicxan said:
Can you conform if power off charging works after convert to f2fs? For me it's not working.
Click to expand...
Click to collapse
1. Power Off (83%)
2. Connect Charge cable => 1+1 Logo .............. continue
after 3min
3. Power On => (85%) +2%
I'll go home retesting.
※ Do not use "Settings - Backup & reset - Factory data reset" !!
Philz recovery unlimit loop and reset all data T.T
Sorry for my english T.T
I should make a repacker zip if no one has already. It would repack the boot.img and could be flashed after s nightly. It would save a few steps.
tiny4579 said:
I should make a repacker zip if no one has already. It would repack the boot.img and could be flashed after s nightly. It would save a few steps.
Click to expand...
Click to collapse
would be great!
Result 2
Leicxan said:
Can you conform if power off charging works after convert to f2fs? For me it's not working.
Click to expand...
Click to collapse
19:19 : Power Off (89%)
Connect Charge cable (samsung galaxy S5 LTE-A 5.3v 2.0A) => 1+1 Logo .............. continue
after 9min
19:28 : Power On => +5% (94%)
Can i only use /system as f2fs partition?
It hangs at boot and only with /cache partition at PA bootlogo?
Or is just PA incompatible?
_Beni_ said:
Can i only use /system as f2fs partition?
It hangs at boot and only with /cache partition at PA bootlogo?
Or is just PA incompatible?
Click to expand...
Click to collapse
You can use only system, but use the complete guide and don't edit these two lines in fstab.grouper
Code:
/dev/block/platform/msm_sdcc.1/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,nodiratime,inline_xattr,errors=recover wait,nonremovable,encryptable=/dev/block/platform/msm_sdcc.1/by-name/reserve4
/dev/block/platform/msm_sdcc.1/by-name/cache /cache f2fs noatime,nosuid,nodev,discard,nodiratime,inline_xattr,errors=recover wait
if you want to use only system in F2FS, /data and /cache had to stay in ext4
gulp79 said:
with phone off i too have this problem, but i don't know if is related to FS.
Click to expand...
Click to collapse
zzori said:
1. Power Off (83%)
2. Connect Charge cable => 1+1 Logo .............. continue
after 3min
3. Power On => (85%) +2%
I'll go home retesting.
※ Do not use "Settings - Backup & reset - Factory data reset" !!
Philz recovery unlimit loop and reset all data T.T
Sorry for my english T.T
Click to expand...
Click to collapse
zzori said:
19:19 : Power Off (89%)
Connect Charge cable (samsung galaxy S5 LTE-A 5.3v 2.0A) => 1+1 Logo .............. continue
after 9min
19:28 : Power On => +5% (94%)
Click to expand...
Click to collapse
:good: Thanks for your test , it seems power off charging was working, but charge animation was not load
the charge animation was located to /res/images/charger, so i think may be it's a kernel problem.
Leicxan said:
:good: Thanks for your test , it seems power off charging was working, but charge animation was not load
the charge animation was located to /res/images/charger, so i think may be it's a kernel problem.
Click to expand...
Click to collapse
Do you have Quickboot enabled? If so it won't show the charging icon when it's off because it's in hibernate.
Saythis said:
Do you have Quickboot enabled? If so it won't show the charging icon when it's off because it's in hibernate.
Click to expand...
Click to collapse
No, i disabled Quickboot.
I'm attempting to convert Slim Saber.. (Fingers Crossed)
---------- Post added at 06:09 PM ---------- Previous post was at 05:39 PM ----------
Quick question, does Philz mount the usb?.. ( I know the option is there, just if it actually works) I would love to be able to just transfer the files and flash and not use fastboot comands..
Thanks..
gulp79 said:
You can use only system, but use the complete guide and don't edit these two lines in fstab.grouper
Code:
/dev/block/platform/msm_sdcc.1/by-name/userdata /data f2fs noatime,nosuid,nodev,discard,nodiratime,inline_xattr,errors=recover wait,nonremovable,encryptable=/dev/block/platform/msm_sdcc.1/by-name/reserve4
/dev/block/platform/msm_sdcc.1/by-name/cache /cache f2fs noatime,nosuid,nodev,discard,nodiratime,inline_xattr,errors=recover wait
if you want to use only system in F2FS, /data and /cache had to stay in ext4
Click to expand...
Click to collapse
FYI it's fstab.bacon not fstab.grouper.
Repartition tool for A5 and A7 2017
Disclaimer
Code:
I am NOT responsible for bricked devices or any damage that may happen to your device.
Repartitioning is a dangerous process and may brick your device in the worst case.
Features
This script will set 64MB for your /cache partition and ~4.6GB for your /system partition (The size of /system may be smaller or bigger depending on which variation of A5 and A7 is used)
The script will create a new partition called "vendor" with a size of 300MB
Only /system and /cache will be deleted during the flashing process, your /data partition keeps untouched
Flashing the script for a second time after already having it flashed will delete the /vendor partition and restore your stock partitions
You will still be able to install any recent ROM (Note that A720S may be unable to install any recent ROM. Contact me if you have problems)
TWRP, SHRP and Orangefox are supported
Installation
Make a backup of all your partitions first in case something goes wrong. Repartitioning is a dangerous process!
Make sure to have the latest version of TWRP/SHRP/Orangefox installed before installing this script.
Download the Repartitioner from post #2
If you only want to patch your recovery but already have the /vendor partition, download the RecoveryPatch from post #2
Flash the installer zip and wait until it is finished. It will add the necessary partition, patch your recovery and automatically reboot you to recovery if the installation is successful.
After reboot, make sure that /system, /cache and /vendor are mountable. If not, format them. (See a How-To in FAQ in post #3)
Congratulations, your device now supports treble!
Downloads
Find them in post #2
Support and FAQ
Telegram Group
FAQ (Frequently asked questions)
Credits
Me for the time I spent on this
@corsicanu for his template script
@Reverant for the original repartition script
@osm0sis for AnyKernel
Telegram users for testing
XDA:DevDB Information
[TOOL][SCRIPT][A520X|A720X] Repartition tool for treble support, Tool/Utility for the Samsung Galaxy A Series 2017 A3, A5, A7
Contributors
Simon1511
Version Information
Status: Stable
Created 2020-09-08
Last Updated 2020-09-09
Downloads
v1.2
Repartitioner: Google Drive | Androidfilehost (Will resize your partitions AND patch recovery)
RecoveryPatcher: Google Drive | Androidfilehost(Will ONLY patch your recovery for treble support. Flash this if you upgrade from an older version.)
Changelog
Spoiler
v1.2
Spoiler
Removed TWRP version check
Added support for SHRP and Orangefox
v1.1
Spoiler
Fixed a typo that prevented the script from creating new partitions on A720F and korean A5 variants
v1.0
Spoiler
Initial release
Reserved
FAQ (Frequently asked questions)
I cant mount /system, /cache or/and /vendor. What do I do?
->In recovery, go to Wipe, Advanced Wipe, select the partition you have problems with and tap "Repair or Change File System". Choose "Change File System", choose EXT4 and confirm the process. Reboot to recovery again to complete the process.This may be slightly different depending on which recovery you are using.
Restoring my /system backup always fails! What do I do?
->This seems to be a general TWRP problem. To fix, wipe /system, reboot to recovery, install the ROMs zip you had installed before, reboot to recovery again and restore your backup.
I have an A720S and cant install any recent ROM! Why?
->A720S has the smallest /system partition of all A5 and A7 variants and Lineage and every other ROM currently has its stock partition sizes set. Just flash the repartition ZIP again to revert the changes and contact me so we can resolve this problem.
How do I upgrade from an older version to the latest?
->The latest version doesn't change anything in terms of partition size. Therefore, you don't need to upgrade unless you have any issues.
How do I patch my recovery to support treble?
->Download the "RecoveryPatch" ZIP from the download section above and flash it in recovery. It will patch your recovery to support treble. If you want to switch to non-treble recovery, flash the recovery IMG/ZIP or the RecoveryPatch ZIP again.
reserved
in a720f can't format or mount /system, /cache or/and /vendor
says unable to wipe them
error changing file system
After second time installing "repartitioner-v1.0-a57y17lte.zip" reverse process fails to restore /cache partition to first /cache
matkali said:
in a720f can't format or mount /system, /cache or/and /vendor
says unable to wipe them
error changing file system
After second time installing "repartitioner-v1.0-a57y17lte.zip" reverse process fails to restore /cache partition to first /cache
Click to expand...
Click to collapse
Did you have TWRP 3.4 installed before installing the repartition script?
If not, thats likely to be the cause.
Also, dont forget to format /system, /vendor and /cache after installation.
Simon1511 said:
Did you have TWRP 3.4 installed before installing the repartition script?
If not, thats likely to be the cause.
Also, dont forget to format /system, /vendor and /cache after installation.
Click to expand...
Click to collapse
yes I do all of them
matkali said:
in a720f can't format or mount /system, /cache or/and /vendor
says unable to wipe them
error changing file system
After second time installing "repartitioner-v1.0-a57y17lte.zip" reverse process fails to restore /cache partition to first /cache
Click to expand...
Click to collapse
Yeah same issue here,. A720F
deleted
Updated the script to fix A720F and korean A5 support.
Sorry for the circumstances.
Downloads can be found in post #2
UPDATE
repartitioner v1.2: Google Drive | Androidfilehost (Will resize your partitions and patch recovery)
RecoveryPatcher: Google Drive | Androidfilehost (Will ONLY patch your recovery for treble support)
Changelog
Removed TWRP version check
Added support for SHRP and Orangefox
Tried restoring a nandroid backup of the data partition with twrp.
also copied the /data/media partition back from an external copy.
when booting up the phone immediately reboots back into twrp with an error message:
Android Rescue Party...
The reported problem is:
'--reason=set_policy_failed_:/data/vendor'
the vendor partition seems to be intact and i do have a backup of it taken at the same time as the data backup, restoring it doesn't yield results..
i'm wondering if FBE is throwing it off, as the backup was taken when the phone was decrypted (within twrp) however the data on the partitions is referencing some sort of encryption key?
you may also exhibit the following error upon bootup of a restored nandroid backup.
immediately after booting, the phone reboots back into recovery.
viewing the log in twrp will show:
Android Rescue Party...
The reported problem is:
'--reason=set_policy_failed_:/data/bootchart'
1. the solution to this is editing fstab.under twrp or other recoverymount /vendor from the mount icon.in twrp: Advanced > File Manager > /vendor/etc/fstab.qcomselect edit file under userdata, find where it says fileencryption=icerename fileencryption to encryptable.Original
Code:
/dev/block/bootdevice/by-name/userdata /data ext4 noatime,nosuid,nodev,barrier=1,noauto_da_alloc,discard wait,check,fileencryption=ice,quota,reservedsize=512M
Modified
Code:
/dev/block/bootdevice/by-name/userdata /data ext4 noatime,nosuid,nodev,barrier=1,noauto_da_alloc,discard wait,check,encryptable=ice,quota,reservedsize=512M
save file.
2. next delete the following directories:
/data/unencrypted
/data/misc/vold/user_keys
3. Lastly delete any of the existing files from /data/system/ :
locksettings.db
Gatekeeper.password.key
gatekeeper.pattern.key
locksettings.db-shm
locksettings.db-wal
recoverablekeystore.db
password.key
pattern.key
4. Reboot and re-encrypt
At this point rebooting from recovery will result in a running and successfully recovery backup.
one thing to note is the data and data/media partitions are at this point unencrypted
TRYING TO REINCRYPT NOW WILL FAIL to reencrypt got to settings > security > re set your pin or password for the phone
(optional) then select encryption and there will be an orange button to encrypt device.
the encryption process will take quite a while as it will reencrypt your entire phone.
The above doesn't work as it's trying to accomplish FDE, and the fstab line for encryptable=ice, isn't compatible with this.
i could not find an fstab string to follow the same option but for FBE.
Thanks for sharing, nice guide to disable forced FBE encryption! I think this applies to Android 12+ in general, not just OnePlus devices.
I ended up with the same problem on my Mi 10 Ultra with MIUI 13 after a /data partition restore and it was a real pain to solve ("set_policy_failed:..." rescue party error for different directories). It's strange though why it fails to set the fscrypt policy for existing directories with no policy, correct permissions and SELinux context...
(Btw: whether a fscrypt policy is applied to a directory ("is this directory encrypted?") can be checked with fscryptpolicyget in terminal.)
Unfortunately, this didn't directly solve my TWRP backup restore problem and I still had to do a manual restore, but now I can at least disable FBE and it's always nice to have actual control over the device you paid money for (you should really have control by default, but oh well...)
(Some of) the troubleshooting I did:
Like I mentioned, I first thought the issue might be with the SE linux context, so I tried running restorecon, but this didn't help - I eventually found that in init.rc, restorecon is usually already automatically run during each boot for directories under /data/... so running it manually makes no difference.
To edit /vendor/etc/fstab.qcom (or /system) on my device, I had to first disable the shared blocks EXT4 optional feature. I followed this nice guide to unpack/repack super.img. But this is missing the step for disabling shared blocks: when I tried to mount any of the unpacked images (e.g. vendor.img) as R/W, it failed with the useless generic error:
wrong fs type, bad option, bad superblock on ...
Click to expand...
Click to collapse
Then dmesg gave me another clue, but at the same time was still cryptic and not immediately helpful:
EXT4-fs (loop*): couldn't mount RDWR because of unsupported optional features (4000)`.
Click to expand...
Click to collapse
So I guess 4000 is the code for shared blocks and you can disable these with e2fsck -E unshare_blocks <your .img file or loop device> (and probably need a filesystem check with e2fsck -yf <file>). Again very annoying that these numerical feature codes are not mentioned anywhere in the e2fsck manual pages for example.
Anyway, I was finally able to either:
1. mount vendor.img on my PC (mount -o loop vendor.img /mnt/vendor) and edit the /mnt/vendor/etc/fstab.qcom right there before repacking the .img and flashing the new edited super.img to my device
or
2. just repacking the vendor.img with shared blocks disabled and size increased (resize2fs vendor.img <new size>) and flashing the new super.img without other modifications - this way /vendor can also be mounted as r/w in Android and changes made later (mount -o remount,rw /vendor).
The worst part is that in the end, even with decryption disabled and the keys deleted, the device still wouldn't boot after a /data restore from TWRP (and after multiple days spent on debugging )... I still had to manually extract the TWRP backup and move directories/files individually - thankfully no issues with app/ or data/ - I think the problem was with some files in either system/ or misc/, but idk for sure. I just manually went through and kept only what seemed important (saved wifi APs, BT devices, SMSs etc, but not saved accounts). And after this it finally booted with all my apps and (most of) my settings!
(Btw2: a TWRP/nandroid backup is apparently just a bunch of separate tar.gz files, not a split archive, so you can just extract them with for file in ../data.f2fs.win*; do echo "extracting $file..."; busybox tar -xzf $file; done)