Database Users

[Q] TWRP backup just erased everything in my /sdcard folder

I'm running stock/unrooted with my bootloader unlocked.
I wanted to try out the PA beta with the PIE navigation so I used adb fastboot to boot into TWRP 2.3.3.3 (not flashed, booted the image directly. I have no custom recovery flashed) to make a nandroid backup. This is the second one I've made with this method but this one went tits up. It reported success and I rebooted. All of my apps are in place and everything seems to be in order other than that my /sdcard folder was completely wiped. All my of pictures and everything. The only folders it contains are the ones the OS and apps are creating as they need them. I also have the /TWRP folder but it only contains .twrps and .version.
I get the same results from ES File Explorer. I went back in TWRP to use the built-in root file explorer to see if it was a permissions issue but it is not. The recovery file explorer shows the same results. The only folders under /sdcard are newly created by Android or by my apps.
Anyone have any idea what happened here and how I can prevent it from happening again?
EDIT: Oddly enough I just noticed if I go back into the TWRP backup screen it's reporting about 2GB less free space (approx backup size) than it did when I made the backup that apparently broke everything. I can't seem to find any backup actually stored though. The Restore option doesn't list any backups available to restore.
EDIT2: I found the backup. It is in /data/media/TWRP. /data/media/0 has the contents of the newly created /sdcard folder but none of my old data is anywhere to be found. Did TWRP writing this folder break my /sdcard folder somehow?

[GUIDE] Decrypt your OnePlus One

So you thought that encrypting your OnePlus One was a good idea? So did I. I had so many issues after having encrypted my One that I simply wanted to get rid of it. After a lot of searching, I didn't come across a simple guide to decrypt it. And here I am
First thing first. Did you know that TWRP doesn't do a _full_ nandroid backup? It backs up everything _except_ /data/media. That was basically the reason why I'm writing this.
What you need:
An unlocked bootloader
TWRP recovery (I was using 2.8.6.1 but any version might just work)
An USB-OTG dongle with a memory stick that is large enough to hold all the data
A full battery
Here are the steps:
Go into TWRP recovery and make a backup. Be sure to select the usb_otg as the target location for your backup
Go into TWRPs terminal and make a backup of your /data/media folder. When starting the terminal, you will be asked in which folder you want to start. Navigate to "/data/media" and ckicl "select". I used tar to achieve this. Simply type in "tar cvpzf /usb_otg/datamedia.gz ." (Note: there is a dot a the end!) and it will create an archive with everything in it including permissions.
Now wipe your device. *scary* I know... Still in TWRP go to "Wipe", "Format Data" and type in "yes". This won't just wipe your data but also wipe the encryption
Since our device is basically empty, we just have to restore eveything. Let's start by restoring the nandroid backup
After the nandroid backup, let's restore /data/media. For this go back into TWRP terminal. Starting folder will be "/data/media" again. Type in "tar xvpzf /usb_otg/datamedia.gz ." (Note: there is a dot a the end!) to restore everything.
There you go... OnePlus One decrypted!

Simple and clear guide. Just what I was looking for. A couple of questions before I begin:
File size limit. Most USB drives are FAT32 (4GB file limit) so how do you split the tarball into <4GB files?
Will TWRP (or Android or whatever recognizes filesystems) recognize exFAT, NTFS or other filesystems that don't have file size limitations?
If you do split the tarball into smaller sizes, how do you restore them?
Thanks again for the guide and info

LG G2, TWRP 2.8.6.1, Restoring data partion failed

Hello everyone,
I encounter a problem when trying to restore backup in TWRP. All md5 checksums are successfully verified, backup restores "Boot" and "System" partitions, then proceeds to "Restoring Data". After about 85% the process stops, and "Restore Complete Failed" message appears. The phone reboots OK but all data is missing. Essentially, that's like a fresh install: the basic setup has to performed. All apps are there but all settings are gone.
I see that people have similar problems but unfortunately I could not find a solution how to restore data partition. Is there a way to successfully restore data? What is causing the problem?
Here is just a bit more info about the device:
1. Rooted LG G2 D801 (lollipop, Android version 5.0.2)
2. Custom recovery TWRP 2.8.6.1 installed with AutoRec.
3. After rooting and installing the device was fully functional.
4. The initial backup was done in TWRP. The process completed successfully with no warning or errors.
Hope someone could help. Thanks.

Try to update TWRP to 3.0.2-1 and check will it help. Also... do you have enough of free space on your device? TWRP zip backups so it need to have a place to unzip it.

Try to update TWRP to 3.0.2-1 and check will it help.
Click to expand...
Click to collapse
Where do you get TWRP 3.0.2-1 for LG G2? The TWRP website says that LG G2 is no longer updated and the latest version for D801 is 2.8.6.0.
Also... do you have enough of free space on your device? TWRP zip backups so it need to have a place to unzip it.
Click to expand...
Click to collapse
That might be the issue.
I was trying many different ways to restore the data partition. The only way I was able to successfully do it if I wiped or formatted the internal storage (/data/media) in addition to TWRP defaults (data and cache; TWRP excludes /data/media). Somehow, the internal storage interferes with the restore process. I had to backup the internal storage separately to my computer before all the wipes.
The cause might be insufficient disk space (there is less than 1GB available). However, TWRP backups are uncompressed and kept on external storage via USB-OTG. The backups for data, cache and system seem to be regular TAR archives:
Code:
$ file data.ext4.win000 cache.ext4.win system.ext4.win000
data.ext4.win000: POSIX tar archive (GNU)
cache.ext4.win: POSIX tar archive (GNU)
system.ext4.win000: POSIX tar archive (GNU)
One would think the data is directly extracted from TAR archives to the data partition. Even with compressed TAR archives, decompression can be be done "on the fly" (eg. tar xzvf) without decompressing to disk first.
Not sure what exactly happens. The restore sort of works now (I do have to backup internal storage prior to this, which is a big inconvenience). I should have examined recovery.log after failed restore.

sakej said:
Also... do you have enough of free space on your device? TWRP zip backups so it need to have a place to unzip it.
Click to expand...
Click to collapse
Just did few more experiments and insufficient disk space is certainly the issue. This time I did nothing to the internal storage (/data/media). I have two backups: a larger one (includes an initial state of the device) and a smaller one (in which a lot of pre-installed LG G2 stuff was purged from the device). The larger backup consistently fails, the smaller one works like a charm. Of course, it would be great to know in advance whether the backup is going to fail during restore
Thanks for the help!

I've had bad backups in the past, despite completing successfuly.

ursus107 said:
Where do you get TWRP 3.0.2-1 for LG G2?
Click to expand...
Click to collapse
You will find it here on XDA
https://forum.xda-developers.com/lg-g2/development/twrp-twrp-2-8-0-0-kernel-f2fs-tools-t2898705
But if you using it, KEEP IN MIND THAT: https://forum.xda-developers.com/showpost.php?p=70091412&postcount=9
ursus107 said:
One would think the data is directly extracted from TAR archives to the data partition. Even with compressed TAR archives, decompression can be be done "on the fly" (eg. tar xzvf) without decompressing to disk first.
Click to expand...
Click to collapse
It's more the case of how recovery works I think, it cannot directly override files from backup in case if anything went wrong, so rollback is possible.

Turbine1991 said:
I've had bad backups in the past, despite completing successfuly.
Click to expand...
Click to collapse
Restoring from the larger backup only fails if I do not wipe '/data' entirely (including '/data/media'). Otherwise, everything completes successfully. So it has something to do with the available disk space.

[GUIDE] How to properly backup your partitions using TWRP

Hi XDA, as most of you are aware, my phone's efs partition was corrupted through too many rom installations, I have recived a new Pixel XL as part of google's warranty and want to teach you how to properly backup your partitions using TWRP.
General steps (REQUIRED):
Download TWRP 3.2.0-0 image (link coming soon, for now just check any ROM forum)
fastboot boot it (fastboot boot *file path*)
Without Storage:
Skip the copying of /data/media steps and you will be fine!
With Storage:
Go into the file manager
navigate to /data/media
make a copy in /data
There should be a copied /data/media folder
click backup and refresh sizes, you will see that the data partition has increased significantly
backup all partitions ONE BY ONE (if one partition goes wrong, you don't have to erase everything)
Use a file transfer program (I use Air droid) to transfer this backup to your computer

Stock Oreo backup file ddmmyy on my pc...how to restore it?

Ihave taken Oreo backup after bootloader unlock..and I copied it in my pc too... unfortunately I formatted data from internal storage...how to restore that backup which I have backup in pc...I copied it in internal but it does not showing there... please help me...

[email protected] said:
Ihave taken Oreo backup after bootloader unlock..and I copied it in my pc too... unfortunately I formatted data from internal storage...how to restore that backup which I have backup in pc...I copied it in internal but it does not showing there... please help me...
Click to expand...
Click to collapse
A TWRP backup? Put it on the root of an SD card, insert it in the phone, start TWRP and restore from the SD card.

Actually I just dragged it in to the pc...now though it have quite 1 or 2gb size...it is showing empty...I think I forgot to follow ADB fastboot pull push command..if still possible to restore then it would be great...

[email protected] said:
Actually I just dragged it in to the pc...now though it have quite 1 or 2gb size...it is showing empty...I think I forgot to follow ADB fastboot pull push command..if still possible to restore then it would be great...
Click to expand...
Click to collapse
Sorry, I have no idea what you are saying...
If you backed up your TWRP directory (XXX storage/TWRP/BACKUP), copy it back to the "new" TWRP directory on the device... it is not a single file, you need the whole backup directory for your device to restore.

When I took backup of boot efs persist...then I copied it in my pc...and formatted Moto g5 plus...but now since I have that backup file in pc I copied it in mobile ..then I tried to restore but it is not showing there... although it has stock persist boot efs inside that file...my question is how can I restore it...

[email protected] said:
When I took backup of boot efs persist...then I copied it in my pc...and formatted Moto g5 plus...but now since I have that backup file in pc I copied it in mobile ..then I tried to restore but it is not showing there... although it has stock persist boot efs inside that file...my question is how can I restore it...
Click to expand...
Click to collapse
You have to put it back in the TWRP/BACKUP folder