Database Users

Backing up p2 (/rom) prior to rooting: Additional precautions?

There seem to be a lot of posts lately from people who have failed to make a proper backup prior to rooting their NST or NSTG. The first reaction to a failed root seems to be trying to restore the image, and of course, a bad image results in lots of heartbreak. I've read Renate's posts patiently talking people through the process of attempting to recover /rom.
I understand that it's the contents of /rom that are unique to each device. In addition to using a disk imager to make a copy of the device, might it also be a good idea to copy the files within /rom to a safe place prior to embarking on any rooting adventure? At least this way, they'd be able to *see* the files and hopefully increase the odds of a successful backup. Also, depending on what gets destroyed during the failed restore, are there other backups on the device?
1. While the new device is mounted using Noogie, is making and extra copy of the contents of /rom a good idea?
2. Are the contents of /boot/romrestore.zip (/system/rom) sufficient? (/rom/BCB has the same md5 checksum, but the only bootcount shows in devconf).
3. The contents of /factory/rombackup.zip appear at quick glance to provide the missing files from devconf). Do /boot/romrestore.zip + /factory/rombackup.zip provide a good backup of the device-unique files?
And finally, is making an image of the device the best way to backup if there's no way to verify it? Would copying the files and doing checksums against the pre-rooting originals be safer? Recreating partitions is not much fun, but could be done with a cut & paste script.

I like playing it safe.
I have full image backups and copies of /rom and rombackup.zip from /factory
/rom is owned by system so you can't read it from a regular app without superuser.
I've thought of making some easy way to back things up.
If you're rooted and have ADB, you just have to adb pull /rom
I've tried to use noogie and give the p2 (/rom) partition a drive letter in Windows Disk Management but it didn't seem to work.

Renate NST said:
I like playing it safe.
I have full image backups and copies of /rom and rombackup.zip from /factory
/rom is owned by system so you can't read it from a regular app without superuser.
I've thought of making some easy way to back things up.
If you're rooted and have ADB, you just have to adb pull /rom
I've tried to use noogie and give the p2 (/rom) partition a drive letter in Windows Disk Management but it didn't seem to work.
Click to expand...
Click to collapse
I'm thinking that it might be possible to pull backups during the rooting itself. I've looked at the scripts GabrialDestruir uses in TouchNooter 2.1.31 and I believe it could be done there. The current 2.1.31 nooter script looks like straightforward shell scripting, so I can hack something together there. I've already modified my TN card to add a few files during the boot process to suit my needs, so I'm comfortable with that process.
1. Would there be a major downside to writing the backup files to the TN uSD filesystem?
2. Are /rom and rombackup.zip sufficient? Is /factory/rombackup.zip needed as well, if /rom is bad for whatever reason?
I believe I read that GabrialDestruir wants to migrate TN to a CWM configuration, and I think that's been done for the NSTG already. I'm not sure about the CWM scripts, or even what they're called to search for syntax. Can anyone point me to info on the syntax for the scripts in /META-INF/com/google/android/updater-script as used by ClockWorkMod when installing .zip files? I don't want to try something that will be obsolescent in a few weeks!
I do wonder if a single bootable uSD image could be used to back up critical info from both the NST and the NSTG though. The same files and partitions apply to both, correct? A bootable image that writes copies of these critical files to a VFAT partition on the uSD card would allow newbies to boot, run and then easily copy these files somewhere safe on their PC.

bobstro said:
I'm not sure about the CWM scripts, or even what they're called to search for syntax.
Click to expand...
Click to collapse
See: Edify script language

Renate NST said:
See: Edify script language
Click to expand...
Click to collapse
Perfect! Thank you, Renate. Now that I know what to look for, I did find a tutorial thread that I'm reading through.
Looking at the Edify options, I don't see any way to copy files, nor to write to a .zip or tarball. It looks like I can use run_program(program, arg0, ...) to execute busybox commands, but I'd need to look into that to make sure everything needed is provided by busybox.
My initial impression is that doing this via a bootable image with a shell script like TouchNooter for NST uses today might be the best option, especially if I want to add options to recreate partitions. Using CWM .zip files allows different scripts to be run by selecting a different .zip (e.g. backuppartitions, restorepartitions).
I did compare the md5sums of the files from /rom and from both /boot/romrestore.zip (/system/rom) and /factory/rombackup.zip (/system/rom). They do match, so it looks like recovery of both /boot/romrestore.zip and /factory/rombackup.zip will allow restoration if /rom is not recoverable. Do we need both?

[Q] how to make .img files from existing tablet?

I have a complete setup for the Nexus 7, part of a product we are working on, that I need to easily clone on "virgin" tablets for production. The app requires a rooted OS.
I want to write an installation script using fastboot to unlock the bootloader, erase partitions, then flash them with .img files for each partition (kernel, system, cache, etc.).
How do I extract .img files from my "master" tablet? I have an understanding from some where that these are simple byte-for-byte dumps of the partition -- is this true? As such can I create a .img file by simple doing 'cat blkfile >file.img' where "blkfile" is the appropriate block device for the partition in question?
Or do I need to use 'dd'? Or something else?
I have searched and searched, and can't find an anwer. I've found other answers using some tools to create these files from a build on a PC, but nothing about creating them from an existing tablet.
Thanks in advance!

Use the dd command. You can use it both to dump and write a partition. It's how I install recovery programs like TWRP
Sent from my Nexus 7

You can use dd for the boot partition and recovery partition - they are raw binary blobs. (Don't use dd on other Android devices, esp. those that have MTD flash devices, though - it only works most of the time there)
If you want to use the same fastboot-based scenario that Google uses for factory image sets, then for the system & userdata image files you will need to find out about "sparse ext4 filesystem images"
If you took a raw block-device based dump of any of your tablet ext4 partitions, you could actually take those image files and mount them on any other linux machine (using a loopback mount procedure).
But you will find that if you attempt to do that with the Google factory ".img" files (for system & userdata partitions), they will not mount. It's not a simple matter of a offset superblock, either.
Since these are the formats that the stock recovery expects, I suppose you ought to use those formats if you want to do the "all at once all partitions" fastboot flashing if you plan on using the stock recovery.
Note that there is absolutely nothing that prevents you from unpacking whatever you want from whatever archive format you want - so long as the recovery's busybox supports the archive format correctly - you could use cpio or pax or tar archives for that matter. (The stock recovery's "toolbox" has very little functionality, so this comment applies to custom recoveries, which typically have more robust functionality in their busybox) You will be writing your own scripts to do those things though, typically either in one of two ways:
1.A mount target filesystem partition
1.B do a deep recursive remove at that mountpoint ( rm -rf * )
1.C unpack your archive into same mount point ( tar xf archive.tar, etc)
1.D unmount the mount point
OR
2.A unmount target partition and zero it out (dd if=/dev/zero, flash_erase, etc)
2.B recreate filesystem in partition (mke2fs -t ext4 etc)
2.C mount target filesystem
2.D unpack your archive into the same mount point (tar xf archive, pax, cpio, unyaffs2, etc)
2.E unmount that mountpoint
Even though this post is for the Samsung Galaxy S II, the same thing applies to the factory Nexus 7 images from Google:
http://forum.xda-developers.com/showthread.php?t=1081239
As that thread mentions, the simg2img and mkuserimg.sh programs are part of the Android project.
Here's a Nexus 7 thread where the contributor built the tools for both x86 linux and arm linux
Finally, I should note that because /system is typically mounted read-only, imaging /system from the live OS is no big deal. Trying to do the same thing with /data is an extremely dopey idea, however. Accurate backups are rarely made from live read-write filesystems.
cheers

Thank you so much for all the great information! I hit thanks for both of you.
The link to the nexus 7 thread is what I need... This is for my company, and I need a simple cloning solution that can be performed by a non-technical assembly person. The fastboot install procedure is about as simple as it gets.
Thanks again!

[ROOT]Root shw-m250S/K (and others?) through hidden partition in JB

Not revolutionary (especially since zips work on stock recovery), maybe not even new on other variations, but still I thought worth bring up as at least it's new for for this variant in JB:
Now in JB for korean GSII we finally have a hidden partition(for better or worse) and it can be used to get root. I have no idea if this can apply at all to other variants of the gs2
On the SK ROM this partition contains nothing but apps (apks) which are all readable (I think they must be to work, but anyway they are). Most are probably arguably bloatware anyway, but it looks like some might be desirable or even fairly fundamental, I'm not sure yet.
It also turns out that it is possible to execute setuid-root files from this partition but of course it's not writable without flashing it.
So it's easy to copy all the files off the so called "hidden" partition through adb without root access... add an su binary, repack with make_ext4fs and tar and reflash with odin. Then you can adb in, run /preload/su to get root, and then copy/install su/supersu into the more normal place to make it more readily available to apps.
Of course the only thing preventing this method with the /system partition was that a few files in /system were not readable without root access and copying all the file permissions, links etc correctly could be a minor pain using only toolbox or whatever. For the hidden partition, for now at least, the directory layout is very simple and all readable.
If hemidall actually worked right in linux on this device for me I could do this with one linux script.
I have not tested a straight through trial of this because I got root already, but I've tested all steps.
In the past I got stock root without flashing unsigned kernels by hijacking the ROM through KIES (freeze it right after it's decrypted), unpacking the factoryfs, adding su/supersu, repacking and flashing. This allows some other customizations anyway so is at least sort of useful, not sure this hidden partition method has any added value. Maybe it will be a useful idea at point in time though.

[GUIDE]How to extract /system from Samsung S7 OTA/FW in Windows without a Phone

This is a complete working guide on how to extract the /system partition from a Samsung S7 FW/OTA package using Windows (This is the same package one might flash to their phone with ODIN.) You can get this FW/OTA package from sites like Samsung-Firmware.org & SamMobile.com. It is very likely this works (or parts of it duct-taped together ) on other Phone models,etc, but I vouch that this guide works on the Samsung S7. Please chime in if it works on other models & brands!
Intro:
I found a LOT of guides similar to this, but couldn't get any to work with the Samsung S7 packages! And there are MANY different versions of the tools I mention below, many not working! So be sure to use the tool versions I post below. Over much time, it was a tiny step with each new attempt until I finally got it...& wanted to share!
Purpose:
Why would anyone want to do this? If you're reading this thread and don't know the answer to that, then I'm confused But I'll answer anyway - What is the purpose of this thread?
You would want to do this because you're a ROM Developer and don't want to have to go through the time & trouble of installing an OTA, then do a dd/cat to get the system image, etc. (With this method you don't even need a phone, just a PC!)
You are using a custom ROM, but want a stock app; for example the custom ROM you're using has the Google Dialer/Phone app builtin, but you prefer the stock Samsung Phone. Doing the steps outlined here will yield a "system" folder in Windows that you can simply navigate to /system/app or /system/priv-app and copy over the apk to your phone and install it! (via ADB or phone File manager app, etc). Obviously not all apks will work. Or maybe you want the libraries from another phone OTA package in order for an apk to work, and so on...
You are just curious what's in the /system partition for an OTA package!
Tools:
7-Zip
LZ4
simg2img: "Clone or download"->Download ZIP
Ext2Explore (Same as Ext2Read)
Guide:
Download FW/OTA (TMB-G930TUVU4CRI2.zip) from SamMobile website (or whereever)
Use 7zip to extract TMB-G930TUVU4CRI2.zip to a folder
Use 7zip to extract AP_G930TUVU4CRI2*.tar.md5 to a folder (ignore "There is no correct record at end of archive" error)
Use lz4 to extract system.img.ext4.lz4 -> system.img.ext4
lz4 system.img.ext4.lz4​
Extract simg2img_win-master.zip -> \simg2img_win-master\
Copy system.img.ext4 to \simg2img_win-master\ folder
Rename system.img.ext4 -> system.img
Use simg2img_win-master to convert system.img -> system.ext4.img
Double-click convert.bat (or run in cmd prmpt) (This will take a few minutes)​
Create new folder to save contents in, eg: "C:\System"
Use ext2explore to mount system.ext4.img: Open ext2explore->File->Open Image->Select system.ext4.img
Click Save icon->Save to your new folder, eg: "C:\System"
Wait for it to extract. Once complete, enjoy!
Links & Useful Resources:
Tmobile Versions: https://support.t-mobile.com/docs/DOC-30276
Search Keywords:
(This section is here so this thread comes up in searches for the many errors I came across while trying to get this to work in both Windows & Linux. That's right! I tried in both OSs and actually got it to work first in Windows... & yet to get it to work in Linux!)
- losetup /dev/loop2 /media/sf_Share/system.img.ext4 ->warning file does not fit into 512-byte sector; the end of the file will be ignored
- mount /dev/loop2 /mnt/mysystem2 -> mount: /mnt/mysystem2: cant read superblock on /dev/loop2
- mount -t ext4 /media/system.img /mnt/mysystem6 -> wrong fs type, bad option, bad superblock on /dev/loop6, missing codepage or helper program, or other error
fsck /media/system.img -> ext2fs_open2: Bad magic number in super-block
fsck.ext2: Superblock invalid, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open ...
The superblock could not be read or does not describe a valid ext2/ext3/ext4 filesystem. If the device is valid and it really contains an ext2/ext3/ext4 filesystem (and not swap or ufs or something else), then the superblock is corrupt, and you might try running e2fsck with an alternate suberblock: ...

Reserved

Just happened to see this thread.
Dropping in the let you know for linux just do this
Code:
simg2img system.img.ext4 system.img
mkdir system
sudo mount -t ext4 system.img system/
Of course all work is done in the current working directory.
You can just copy whatever you want from this mounted loop device of the system.img or whatever.
This is what i do because it seems to be the fastest in terms of work.
Hope it helps. I prefer all android work on linux

kevin71246 said:
Reserved
Click to expand...
Click to collapse
oh man you saved my day thanks man none of the old method worked but this did wonders

OP 5T - FBE /dev/block/sda13 or /data partition - file restoration / residues scan

Hello,
unfortunately, my Google Keep app had just decided to wipe its db file up. (/data/data/com.google.android.keep/databases/keep.db)
It is not (and never was) synchronized to Google.
When it happened I just copied (dd'ed over adb) /dev/block/sda13.
From mount command output, I could recognized /dev/block/sda13 as /data backing block device.
Then I did some searches in the output image file and noticed it actually encrypted per file.
I learned from it OP uses FBE (file-based-encryption). I wanted to try and run some ext4 recovery tools (autopsy, undelete ...).
FBE encrypts also filenames so I can't use the generated image as is.
I checked what caused FBE on /data partition and it's the encryption setting. I read it uses the fingerprint/pattern/password to decrypt the encrypted /data.
I didn't power-off the phone since then to avoid any more losses.
My questions:
1. Given /dev/block/sda13 as an image file, how can one decrypt it and use forensics/restoration tools on it?
2. Another option - how to acquire the unencrypted /dev/block/sda13 from the running device?
3. Maybe there is some backup partition for files in /data?
4. Can I run the mentioned tools straight on some unencrypted device file? (Assuming cross-compilation to android)
5. Any caches/other places I can search for residues? (I tried dumping CursorWindow ashmem the Keep app used but the app has restarted since then)
6. OP compiled the kernel without DEVMEM. Maybe there is other option to acquire the whole RAM to try to scan for residues?
More side-notes:
- I tried other files in the Keep app directory to try to recover some residues, pictures where saved outside the db file so I was able to restore them (but what is important are the notes stored in the db itself(
- I was able to notice keep.db was replaced with an empty sqlite3 db file (for some strange reason ...). My guess was it didn't overridden the original db file so if I had the ext4 /data partition, I will be able to scan it with the tools mentioned above.
References:
* https://forum.xda-developers.com/t/...-encrypt-data-partition-on-oneplus-5.3642144/
* https://www.cs1.tf.fau.de/research/system-security-group/one-key-to-rule/
If some details are missing, please let me know!

https://www.cs1.tf.fau.de/research/system-security-group/one-key-to-rule/ is really interesting.
The problem now is how to get a full memory dump without /proc/mem and signed kernel modules.

Any help? I'm out of ideas but it seems like a solvable problem.