Related
I've been digging around trying to find out the benefits and drawbacks of using swap on android devices, and the general consensus is 'not worth it since SD card is slow', and 'android is good at killing programs when necessary and starting them up where they left off'
Well, that got me thinking, since I have the MoDaCo ROM installed, and I'm utilizing Apps2SD - i've got all this free space on the (relatively) fast internal flash memory that's going unused.
It seems to me that a swap file (not partition) in /data would potentially be a boon to system stability for people using *lots* of applications simultaneously. My reasoning:
1) Internal memory is much faster than the SD card interface, even at class 6.
2) Using Apps2SD in combo with android's default mem management, if an app is auto-killed and needs to be restored into memory, it's reloading the apk from the SD card. Due to this behavior, there's little difference between using a swap partition on the card vs android reloading apps from the card and restoring state.
A swap partition or file on internal memory would avoid touching the SD card interface alltogether except when inititally starting up a new app. Storage space benefits of apps2sd is retained, while potentially speeding access to a sleeping app.
So, to test out the theory, I tried to create a 64mb swap file in /data/swap/swapfile, using the following steps (again, this is on the MoDaCo ROM):
Code:
./adb shell
# mkdir /data/swap
# dd if=/dev/zero of=/data/swap/swapfile bs=1024 count=65536
65536+0 records in
65536+0 records out
# mkswap /data/swap/swapfile
Setting up swapspace version 1, size = 67104768 bytes
UUID=ca4e84af-2533-4072-8c66-7bf6f494dc34
Awesome so far. Now I hit a snag:
Code:
# swapon /data/swap/swapfile
swapon: /data/swap/swapfile: Invalid argument
It appears that the swapon function in BusyBox doesn't support swap files, unless I'm overlooking a step in here.
Has anyone tried this sort of thing before (on previous android phones, or other roms that is)? A quick google search showed lots of people doing this exact thing to enable swap files on their G1's, rooted with BusyBox installed, nad having it work just fine.
For reference, the MoDaCo ROM has this version of BusyBox:
BusyBox v1.15.2 (2009-10-08 09:47:18 BST) multi-call binary
Any hints or help would be greatly appreciated.
could you possibly use a symlink to trick it into thinking it's not a file, but a partition instead maybe?
EDIT: Just saw this thread too, don't know if you've seen it or not, but it seems like an app that could do this for you. Again, you'd have to trick it by using a symlink or something I bet, since you're wanting to do it on the internal memory.
http://forum.xda-developers.com/showthread.php?p=3329186#post3329186
symlink trickery didn't work for me unfortunately, but I think I have another approach.
swapon appears to be looking for a device node, so I'm going to try to trick the system into giving it one by mounting a fake disk image with loopback, and assigning it's 'partition' as swap space.
not really sure if android will let me do that sort of crazyness, or if loopback is even compiled into the fs drivers, but will find out shortly.
[edit]
So, it turned out to be far simpler than all that nonsense above. This kind of thing is why I love linux:
Code:
# * made 8mb swapfile using steps in previous post, to /data/swap/swapfile *
# losetup /dev/block/loop0 /data/swap/swapfile
# swapon /dev/block/loop0
magic!
Code:
# free
total used free shared buffers
Mem: 196824 191320 5504 0 24
Swap: 8184 0 8184
Total: 205008 191320 13688
Now, just need to figure out how to mount it up at boot and set a reasonable swappiness.
Well, it was a good experiment. One that I *do not recommend* for others to try.
After getting the swap file setup and running for a few days, most things work really well - the SenseUI interface is always snappy, applications running in the foreground are generally very responsive and smooth.
Unfortunately, *anything* that gets swapped out of main memory will hang your phone for a good 30-40 seconds at least trying to read it back in to main mem (therefore swapping something else out simultaneously). Things like the web browser, the notification pulldown, the apps menu, just become completely unusable. And this was with 'swappiness' set down to 10 - so theoretically there should not have been excessive swapping.
Good to know we have these tools available in times of need, however I again do not recommend swap of any kind, SD card or otherwise.
Too bad we can't upgrade the RAM on these devices
hi everyone,
I am using mattc evoleo build and it is really great.
but I am almost out of space so I need to increase the interior phone storage from 1 GB to 2 GB.
can you show me the proper way to increase it or a data file I can replace to do so ????!!!
Kev007 posted this a while back
Manually edit data.img if you want a different size or use a different build!
I wrote this tutorial using a European HD2, 8GB microSD card, DarkStone's Froyo_v1 and a laptop running Ubuntu 10.04, your experience may vary.
Requirements:
* Desktop/Laptop running some form of Linux.
IMPORTANT NOTE: Both, a PC and a Mac, can resize the .img file but not modify (specifically - run resize2fs) the ext2 file system that Android uses. I was in a hurry to post the resizing instructions and didn't fully test the procedure on a PC. Currently, this procedure is only possible on a Linux based operating system. I apologize if your time was wasted.
* data.img file
* HD2, microSD card, microUSB cable etc, etc (you might be better off using a card reader)
--------------------------------------------(Running Linux)----------------------------------------------
Procedure:
1. UnZip Android or your present data.img file to your Home Folder. Or a folder of your liking (or even on your memory card!), just remember to cd before you do the following:
2. Open Terminal and Copy&Paste (Ctrl+C, Ctrl+Shift+V) this:
Code:
dd if=/dev/zero bs=1M count=XXX >> data.img
where XXX is the amount, in MB, by which data.img should be increased by.
My filesize started out as 256MB and I wanted a total of 512MB. That would mean I needed a extra 256MB, so I executed this:
Code:
dd if=/dev/zero bs=1M count=256 >> data.img
3. Run a file system check and file system resizer
Code:
e2fsck -f data.img
resize2fs data.img
e2fsck -f data.img
if prompted, press "y" for "yes"
4. Copy all of the Android files onto your SD card, put it into your phone, and run CLRCAD.exe and HARET.exe!
-----------------------------------------------TIPS!-------------------------------------------------
Don't know how to cd?
If you're running a modern build of Linux you can just mount your SD card (phone or cardreader), open File Browser and paste "dd if=/dev/zero bs=1M count=XXX >> " into your open Terminal. This way you can just drag and drop the file into Terminal rather than typing out the file location!
Mounting:
If you're switching between builds and need to copy more than just your apps (apps are easily backed up by ASTRO File Manager - found on the Android Market), mount your data.img file and copy/backup the relevant data before moving on!
Linux:
Code:
mount -o loop data.img /mnt/data
Apart from that, I've nothing else to suggest, except maybe moving most of your apps etc from the "internal storage" to the sd card.
Maybe someone with a lot more knowledge than I can help
You could try this program too, not tried it myself, but others report it works
http://forum.xda-developers.com/showthread.php?t=824154
Okay so i was just doing a bit of file management. I removed a 1.4gb psp iso i had on my nexus 7 and then i checked how much space i had left using storage option in settings. I checked it and it said i had 25gb free when before i removed my file i had only 13gb free. Tried rebooting device and then used es file explorer to see all files. Opened es file explorer and then noticed everything was gone. Only stock items were folders were left eg Android, Download etc. All my msuic and some game data was gone. Opened up asphalt 7 to see if it would still work but it doesnt anymore. Tried opening real racing 3 And it asks me to redownload game data. My widgets are still in homescreen and work and some of the other apps i have still work as well eg plague inc, subway surfer and most of the other ones.
Any idea on how this happened and how i can recover my files? I have a bugsense file as well that was left on my device.
As to recovery of lost files your options are not good. (And if we're talking a non-rooted device, the odds are approximately equal to 0%) Recovery in ext4 filesystems is technically much more challenging than in simple filesystems such as FAT. And this pessimistic outlook presumes that the filesystem is healthy/clean. If the reason for the problem occurring in the first place was a corrupted filesystem, then the odds go from simply bad to pathetically poor.
Sorry, dude... got any Nandroid or TiBu backups stored on your PC?
If you had Putin's top-secret files on your N7 and the CIA got hold of it, the first thing a forensic analyst would do is try to take a raw (block) device dump off of the "cold" device. (If you are still running the N7 with the regular OS the /data partition is being continually written to, and this further reduces the chances of file recovery every second the device is booted).
In the case of an analyst with less resources, this might mean using a custom recovery boot to get the raw device copy; unfortunately, the /data partition is huge - nearly 30 GB - so you would have to mount a extN filesystem via OTG... and doing so thus precludes using adb, so you would need to use a recovery with a touch interface and command-line entry (e.g. TWRP)
# mkdir /mnt/myOTGdisk
# mount -t ext2 -o rw /dev/sda1 /mnt/myOTGdisk/
# dd bs=8196 if=/dev/block/platform/sdhci-tegra.3/by-name/UDA of=/mnt/myOTGdisk/userdata.img
Doing such a thing would allow you to examine that huge image file with forensic file recovery tools from a PC (probably running Linux) as in principle you captured the entire ext4 filesystem.
The thing is, efforts spent in file recovery should be proportional to the value of the files being recovered. I'm not sure if your saved gaming history rises to that occasion. For sure the dude at the CIA won't want to help you with that.
As to the source of your troubles, it's hard to say. With TWRP booted, you can run the "e2fsck" program to see if the /data ext4 filesystem is corrupted, e.g.
# mount | grep /data ( see which mmcblk0 partition is /data, on grouper it is mmcblk0p9 )
# umount /sdcard
# umount /data
# e2fsck -f -n /dev/block/platform/sdhci-tegra.3/by-name/UDA
(For the last command above, you might need to use the block device name /dev/block/mmcblk0p?? instead of the UDA symlink )
If the above command shows that you have a corrupted /data filesystem, I would re-initialize that filesystem ( "fastboot format userdata" ) - note this wipes all userdata including the psuedo-SD card.
Finally, I should point out that some type of hardware failure might have occurred somewhere in that huge 30 GB partition - if that is the case then there will be problems down the road again. If that is the case, the only way to detect this will be a write test which nearly fills that partition, followed by a filesystem sanity check as shown above.
Probably that would need to be done in the recovery rather than in the normal OS, as a nearly full /data filesystem will probably wedge the device.
Phew, I've said enough.
Good luck
I never tire of reading your posts, bftb0, ("...the odds are approximately equal to 0%")...genius.
But don't the CIA have access to Cray, 'Kasparov' DeepBlue beating SuperComputers that could make mincemeat out of the kind of thing your alluding to... in less time than it takes to flash a ROM... or have I been watching too many James Bond movies?
Vaguely rhetorical question - think I already know the answer...
Still... what a great post.
Rgrds,
Ged.
---------- Post added at 01:22 AM ---------- Previous post was at 12:50 AM ----------
Hi, leont1280...
You could try running this ...
Disk Usage - http://play.google.com/store/apps/details?id=com.google.android.diskusage&hl=en
It gives a graphical 'map' or overview of your storage, and you can visually see where everything is (or should be), great for tracking down missing stuff... but as bftb0 has mentioned, it doesn't look promising.
Rgrds,
Ged.
Use astro file manager u can check it out
Sent from my Nexus 7 using Tapatalk 2
Right bftb0 i did what you said in twrp and i recieved the following summary information.
/dev/block/mmcblk0p10: 28133/1835008 files (3.2% non-contiguous) , 1571673/733977
Any idea on what that means?
Also under the mount option in Twrp i can Unmount and mount the System, Data and its Cache. However i cant Mount the SD Card. Should that be a concern?
bftb0 said:
As to recovery of lost files your options are not good. (And if we're talking a non-rooted device, the odds are approximately equal to 0%) Recovery in ext4 filesystems is technically much more challenging than in simple filesystems such as FAT. And this pessimistic outlook presumes that the filesystem is healthy/clean. If the reason for the problem occurring in the first place was a corrupted filesystem, then the odds go from simply bad to pathetically poor.
Sorry, dude... got any Nandroid or TiBu backups stored on your PC?
If you had Putin's top-secret files on your N7 and the CIA got hold of it, the first thing a forensic analyst would do is try to take a raw (block) device dump off of the "cold" device. (If you are still running the N7 with the regular OS the /data partition is being continually written to, and this further reduces the chances of file recovery every second the device is booted).
In the case of an analyst with less resources, this might mean using a custom recovery boot to get the raw device copy; unfortunately, the /data partition is huge - nearly 30 GB - so you would have to mount a extN filesystem via OTG... and doing so thus precludes using adb, so you would need to use a recovery with a touch interface and command-line entry (e.g. TWRP)
# mkdir /mnt/myOTGdisk
# mount -t ext2 -o rw /dev/sda1 /mnt/myOTGdisk/
# dd bs=8196 if=/dev/block/platform/sdhci-tegra.3/by-name/UDA of=/mnt/myOTGdisk/userdata.img
Doing such a thing would allow you to examine that huge image file with forensic file recovery tools from a PC (probably running Linux) as in principle you captured the entire ext4 filesystem.
The thing is, efforts spent in file recovery should be proportional to the value of the files being recovered. I'm not sure if your saved gaming history rises to that occasion. For sure the dude at the CIA won't want to help you with that.
As to the source of your troubles, it's hard to say. With TWRP booted, you can run the "e2fsck" program to see if the /data ext4 filesystem is corrupted, e.g.
# mount | grep /data ( see which mmcblk0 partition is /data, on grouper it is mmcblk0p9 )
# umount /sdcard
# umount /data
# e2fsck -f -n /dev/block/platform/sdhci-tegra.3/by-name/UDA
(For the last command above, you might need to use the block device name /dev/block/mmcblk0p?? instead of the UDA symlink )
If the above command shows that you have a corrupted /data filesystem, I would re-initialize that filesystem ( "fastboot format userdata" ) - note this wipes all userdata including the psuedo-SD card.
Finally, I should point out that some type of hardware failure might have occurred somewhere in that huge 30 GB partition - if that is the case then there will be problems down the road again. If that is the case, the only way to detect this will be a write test which nearly fills that partition, followed by a filesystem sanity check as shown above.
Probably that would need to be done in the recovery rather than in the normal OS, as a nearly full /data filesystem will probably wedge the device.
Phew, I've said enough.
Good luck
Click to expand...
Click to collapse
Right bftb0 i did what you said in twrp and i recieved the following summary information.
/dev/block/mmcblk0p10: 28133/1835008 files (3.2% non-contiguous) , 1571673/733977
Any idea on what that means?
Also under the mount option in Twrp i can Unmount and mount the System, Data and its Cache. However i cant Mount the SD Card. Should that be a concern?
leont1280 said:
Right bftb0 i did what you said in twrp and i recieved the following summary information.
/dev/block/mmcblk0p10: 28133/1835008 files (3.2% non-contiguous) , 1571673/733977
Click to expand...
Click to collapse
Hmmm, mmcblk0p10 - you must have a tilapia (3G N7) device, yes?
If you had any filesystem errors, that e2fsck run would have produced copious reams of output. If a filesystem is clean, it produces only 5 or 6 lines of summary output.
leont1280 said:
Any idea on what that means?
Also under the mount option in Twrp i can Unmount and mount the System, Data and its Cache. However i cant Mount the SD Card. Should that be a concern?
Click to expand...
Click to collapse
I've seen the same business with TWRP and the /sdcard mount - I wouldn't worry about it. (It is not a "normal" mount in the sense of extN or FAT device partition mount - it behaves sort of like a strange symlink where the target directory and descendants all appear to have different file ownership and permissions than what exist in the true (underlying) filesystem. No doubt this is all performed in the kernel... I don't know whether a command-line invocation of "mount" can create this mount point, or whether a specific syscall/ ioctl is needed)
But back to your N7 - the lack of any errors in the filesystem check is good news, but also suggests that your files didn't disappear through a hardware failure. Are you sure you didn't fat-finger things when using the file manager? (I suppose it is possible that the file manager has a bug...)
I didn't look into what tools are available for extN forensic/recovery work. I can guess that the effort would be non-trivial, though.
bftb0 said:
Hmmm, mmcblk0p10 - you must have a tilapia (3G N7) device, yes?
If you had any filesystem errors, that e2fsck run would have produced copious reams of output. If a filesystem is clean, it produces only 5 or 6 lines of summary output.
I've seen the same business with TWRP and the /sdcard mount - I wouldn't worry about it. (It is not a "normal" mount in the sense of extN or FAT device partition mount - it behaves sort of like a strange symlink where the target directory and descendants all appear to have different file ownership and permissions than what exist in the true (underlying) filesystem. No doubt this is all performed in the kernel... I don't know whether a command-line invocation of "mount" can create this mount point, or whether a specific syscall/ ioctl is needed)
But back to your N7 - the lack of any errors in the filesystem check is good news, but also suggests that your files didn't disappear through a hardware failure. Are you sure you didn't fat-finger things when using the file manager? (I suppose it is possible that the file manager has a bug...)
I didn't look into what tools are available for extN forensic/recovery work. I can guess that the effort would be non-trivial, though.
Click to expand...
Click to collapse
Im pretty sure i didnt accidently delete it myself. When i was doing file management i was using my laptop with the N7 3g connected to it via MTP. Once i deleted the iso file the N7 started acting strange. I did notice a bit of lag that was usually out lf the ordinary and when i checked available space left it increased to 25gb rather than saying 13gb
aaah, having the same problem here, i was cleaning my n7 using my laptop, found a strange folder on my sd card, looked inside and it has a back up of some of my deleted files! interesting! i deleted the folder, then i started cheking other folders, like my ebooks, audio books and etc, but all my folders were empty!
so i disconnected my tablet, and after reconnecting, bam, all files gone.
So, the Note 8.0 is a nice, fast, expensive device that had a big shortcoming for me: as shipped, the SD card is not useful as a place where application data easily lands - it's only intended for storing music and video files, or those chunks of data you manually target to it.
I fixed it, after a fashion, by mounting the card at boot time as sdcard0, which Samsung normally assigns to the (very limited) internal storage. The internal /data/media pool is treated as the external sdcard and very little installation data defaults to it. Apps2SD is still broken, but quite a lot of things simply default to using /sdcard0 as their preferred storage, which I find helpful.
You have to mount it at start time (init.d), and you have to mount it correctly (bind)
I figured out and set my device up this way by setting init.d scripts I learned about from threads by Ryuinferno (init.d) and mattiadj (rebinding in the Note 2) for this. Translation: none of this is my work, I just sounded out how others had done it and am assembling it here, since I know it will help other Note 8 folks.
I think that the init.d script support is very interesting - there are a lot of ROM tweaks that rely on init.d. You do not need a custom kernel for init.d to work the note 8 - which is nice, since we have no custom kernel
The outcome of this mod is that with an sdcard inserted, I get:
/storage/sdcard0 29.7G 3.68G 26.0G 32768
and
/storage/extSdCard 9.87G 1.78G 8.09G 4096
If I remove the card at boot, I still have apps (installed at /data/data) and the media
pool reverts to normal:
/storage/sdcard0 9.78G 1.78G 7.99G 4096
I wrote none of the scripts I'm using and will be the first to admit that I may have set them up stupidly.
Doing this voids your warranty and gives you pimples and a moon tan. But it does mean your sd card expands your storage and I hope that if there are better ways to do any of this, folks will chime in.
tools needed:
- knowledge of ADB and working ADB
- helps if you know how Unix and windows terminate lines differently, and can get your copy of Notepad++ to help you with the script, if you're trying this from Windows
- a note 8 running the 4.1.2 software - this approach is only tested there
- Kies
- Possibly Odin as a replacement for Kies (untested)
- understanding of how to put your device into recovery mode and trigger the Samsung recovery
- the US stock firmware for your device, for use with Odin (unless waiting on 1+ gig downloads if you screw something up is ok by you - the Kies method works, slooooowly.)
- Framaroot, to root your Note 8 (see the thread by tweebee)
- Busybox installed
Steps:
Install Framaroot
Root your device
Install busybox
Easy part done.
Next, you need to be able to run init.d scripts. At first I thought this might require a custom kernel, until I ran into Ryuinferno's excellent tool for enabling init.d without a custom kernel.
the thread I learned this from is by Ryuinferno at http://forum.xda-developers.com/showthread.php?t=1933849
I used term-init.sh from an ADB command line, but the thread has an APK in it called Uni-Init.apk that I would expect to work.
What you're doing is creating an install-recovery.sh script and telling it to go read /etc/init.d and run scripts there at boot.
Next, you want to create a simple script and drop it into /etc/init.d You can do this with an adb push; if you create the file in windows, though, you need to deal with the line endings correctly. You need the script to be executable and to be owned by root.
The one I'm using is this, from Mattiadj of the Note 2 community
in this thread: http://forum.xda-developers.com/showthread.php?t=2036796:
I call the script 07mount on my device, and a copy is attached to this post.
-----------------------script starts next line
#!/system/bin/sh
#extsd2internalsd is a modification that allows to switch internal sd to external sd
#and viceversa. With this you can use default internal sd only for app storage
#and the external sd to store all apps resource and all others stuff. The resut is a very
#big increase of installable apps on gnote2 and note8
#All credits to Mattiadj of xda forum for the idea and script and to mike1986 for
#the cmw zip. xda thread url
# at http://forum.xda-developers.com/showthread.php?t=2036796:
sleep 10
mount -o remount,rw /
mount -t vfat -o umask=0000 /dev/block/vold/179:17 /storage/sdcard0
sleep 30
mount -o bind /data/media /storage/extSdCard
chmod 777 /mnt/extSdCard
sleep 10
chown 1023:1023 /storage/extSdCard
chown 1000:1000 /storage/sdcard0
------------- end script on blank line above
You need /system remounted read/write, either in your favorite file explorer or via adb shell:
#mount -o rw,remount /system
to put the file in and
#chown root:root
the script itself
Now, put a fat32 formatted card into your sdcard, and reboot. When done, you should be able to see that your data storage has been remapped.
If you ever wanted to install a Samsung update, by the way, you'd need to completely unroot the device. I think the following would probably work:
- remove the su binary
- remove superuser from /system
- remove the busybox binaries
- remove install-recovery.sh from /etc and the /etc/init.d folder
If there was someting in an update you really wanted, you might do better to start by using the Samsung firmware downloaded from samfirmware.com, set your device all the way back to a clean install, then check for the OTA - Samsung does look at modifications and blocks updates to devices with changes to /system. My device is ineligible for OTA at this time.
That said, it appears that using Kies in emergency recovery mode can be used to reset your device to an as-shipped condition (I learned this when an early experiment resulted in my device being weirdly screwed up - bootable, but the network was down for the browser, etc. DNS and ping worked from an ADB prompt, though...)
You can download a copy of the script at http://www.mediafire.com/?2wbm439vlapb6om
I'm gonna try this when I have a full afternoon off as I tend to mess things up the first time I do them. But this will be super useful if I get it working. Hopefully when people start developing custom roms they can build this script in and save us all the trouble :highfive:
Haven't tried it yet, but it sounds like a custom recovery is very, very close. Once that happens, making this into a flashable modification will be trivial.
The very active Note II development is incredibly helpful, because our hardware and software are so similar to theirs.
I know this was something that users on the other Galaxy Note 2 threads were doing, and was wondering if at some point this will be an issue with say JB 4.2.2. I recall that Apps2SD would be an issue later on and wouldn't work, but I never got to that point. For me I am really in need of a 32GB tablet and just can't compromise even with a 64GB microSD card. I don't know why this tablet isn't supporting a 128GB microSD card, that would be awesome.
Upgrading internal storage by blending in sdcard
Just to be clear - this mod does not allow app2sd style migration of apps (in /data/app) to the sdcard.
Rather, it puts /data/media on the sdcard. A lot of apps use /data/media as their default for storage, though, so it's a helpful middle ground. Also, your apps are installed whether or not the sdcard is inserted, which is helpful.
I notice that the update includes a script called
install-recovery.sh - the same script that this method is using.
I have tested replacing a backed up copy of install-recovery and also
appending the command
run-parts /system/etc/init.d/ to the new file.
Using just a script to call run-parts or the stock install-recovery.sh works just fine.
I found a much easier way of doing this.
http://forum.xda-developers.com/showthread.php?t=2276193
hi friends & master
please help me
Is there any way to transfer games and apps to the internal storage with app2sd (moving apps & games in from system Rom to SD card(internal) no memory card)?
my system rom There is almost full
1.44 GB (148 MB free)
internal 12.6 GB (5.8 GB free)
SD card 12.5 GB (5.6 GB free)
I do not need to memory card
Note from the Author -
I am moving on to the N5 now and ditching my S3. I will continue to maintain this thread, however - please do PM me if you think that something needs to be changed or updated in this thread as I doubt I will be answering questions within the thread as much. Please don't PM support questions to me. Only PM updates that need to be made in the thread.
It's been a blast!
Regards
Dan
S3 Storage (Data Loss Recovery / Prevention / Info)
This thread is intended to give you an overview of some of the Storage of the S3 from a Data Loss and recovery perspective. It is not intended to cover USB sticks or mods to Swap / Mount other storage. It is solely to cover day-to-day data concerns and give a background to how these things work
Please note, if you have recently swapped between Android 4.1 and 4.2 and cannot find your sdcard data, you need to read [Info] Flashed 4.2? Can't find your /sdcard data?
{
"lightbox_close": "Close",
"lightbox_next": "Next",
"lightbox_previous": "Previous",
"lightbox_error": "The requested content cannot be loaded. Please try again later.",
"lightbox_start_slideshow": "Start slideshow",
"lightbox_stop_slideshow": "Stop slideshow",
"lightbox_full_screen": "Full screen",
"lightbox_thumbnails": "Thumbnails",
"lightbox_download": "Download",
"lightbox_share": "Share",
"lightbox_zoom": "Zoom",
"lightbox_new_window": "New window",
"lightbox_toggle_sidebar": "Toggle sidebar"
}
Before we get started...
Here are a couple of threads you should get familiar with before posting on XDA.
Forum Rules - use Search before posting
Post Questions or Support queries in Q&A, NOT General
Backround of Android storage (Pre-S3)
Firstly, I think we need to understand how Android worked historically as this will help us to understand how the S3 works now.
A legacy android device, lets pretend the HTC desire on Android 2.2 as this was a standard configuration at the time. It had 2 major partitions (and several minor ones which are irrelevant to this topic). It has the /system partition and the /data partition. These were partitions of the internal NAND flash memory.
/system is where the Android operating system lives. The user could not delete or change anything in here (unless you were rooted). All the software that came with the phone was installed in the /system partition
/data is where all the userdata goes. Whenever you installed an app from the market, it installed to /data/app and it put all it's important data into /data/data. Also any system settings you changed (Wallpaper, ringtones etc) all were stored in /data/data. When you did a factory reset, it wiped /data and everything in it.
Of course, having these 2 partitions was not enough for everyday use. There was no where to store your music, photos, documents etc. /data is just for app data and settings. So this is where /sdcard comes in
/sdcard is the Android mount point for the External SD card in this legacy android device. This means that when you inserted a Micro SD card, Android used /sdcard as it's internal reference to where the card's storage is. The /sdcard was a necessity before you could take photos. Over time, bigger apps started to put other data here. For example, a GPS / Map application would store its apk (application package) in /data/app and store your personal configuration settings in /data/data but it may download and store offline maps somewhere on the /sdcard. In older devices, the internal Memory (NAND Flash) was usually too small to allow much data on it. Many users would have to root to get more storage space or keep uninstalling apps to keep the "low on space" warnings at bay
How the S3 is different
Well, the S3 is substantially different. There are of course SOME similarities. For example, the S3 still has internal NAND Flash Memory. This is often referred to as the eMMC (Embedded MultiMedia Card) - which still contains the /system and /data partitions, used in exactly the same way.
The main difference is /sdcard. The S3 is designed specifically so using an external micro sd card is NOT a necessity. It has a larger NAND Flash internal memory (eMMC) so it can also have an "internal SD card". This is where people start to get confused. The entire internal memory is an eMMC which is essentially an internal SD card, however a partition of that internal memory is /sdcard.
OK, I know - let me explain. /sdcard is a mount point that Android uses to know where to store /sdcard data. But on the S3, instead of storing it on a required external sd card, it points to an internal memory partition. Now here is the clever bit. The /sdcard actually points to /data/media (or /data/media/0 in Android 4.2 onwards). So you continue to have your /data partition, but within /data you have:
/data/app
/data/data
/data/media
The clever thing is that the file system that android uses for /system and /data is a Linux file system called Extended FS. In our case, we use the Ext 4 file system. This is important to understand because these file systems do not work with Windows so an external SD card would usually be Fat 32 file system, or exFAT so we could plug it into our windows computers and read the contents. What Samsung have had to do is use the FUSE file system to allow /sdcard (or /data/media) to exist as a FAT file system within the EXT 4 file system. Clever stuff. But it has it's pro's and cons...
You lost me at file system
All electronic systems that have an operating system and store data use a file system. Think of it in it's simplest form. Imagine a school text book. It has lots of chapters about different things. It has a "Table of Contents" in the first few pages, telling you where each chapter in the book is so if you want to know what page chapter 13 is on, you look in the contents and find the page and go straight there - The alternative is looking through each page individually to find the chapter. Not a quick process.
Well data storage works the same. When you put a file on a hard drive, sdcard, USB stick (or whatever) it is written to a specific location. When it is written to this location, the location is added to the file system. So when you put word.doc onto the drive, The file system is informed of the (very complicated) location of the file. When you tell Windows, Android (or whatever) that you want to open word.doc, the operating system consults the File system and goes to retrieve the data from its true and real (yet very complicated) location on the drive.
There are many file system types still in use today. Usually they are operating system specific. For example, Ext 4 is a Linux file system (and Android by Proxy as it is Linux kernel based) and Windows cannot read Ext file systems. Similarly, exFAT is a Microsoft file system (also used for sdcards on the S3) and cannot be used (easily) on Linux machines. Since most everyday users of the S3 are Windows users, you can see now hopefully why it was important for Sasmung to use "magic" and implement FUSE to allow an exFAT file system to be used for /sdcard, within the Ext 4 partition of /data
Did I REALLY need to know ALL that?
You know what? Probably not but it may go some way to help understand the limitations we will cover later on.
A bit more info for the S3
Obviously, the internal memory supplied with the S3 may not be enough for all users so they added the ability to add expanded memory in the form of the Micro SD card. Android uses the Mount point of /extSdCard now, instead of /sdcard like it used for legacy devices - because /sdcard is already in use elsewhere.
One thing many of you have probably noticed is that with the S3, there is no option to mount the /sdcard or /extSdCard as USB Mass storage on your computer. You must use MTP or PTP.
PTP - Photo Transfer Protocol. When you connect your S3 to your computer using PTP, Your computer sees it as a camera. It will show photos on your "camera" and will set about implementing the default camera options (such as suggesting you import your photos) etc. It won't show documents or other media necessarily.
MTP - Media Transfer Protocol. When you connect your S3 to your computer, it will be seen as a media player. This should allow you full access to all the files on there, including word documents and the such like.
One of the reasons for this is that because /sdcard points to something using the FUSE file system and is not a true partition, it would be difficult to allow it to be used as USB Mass Storage. It may or may not be possible but the biggest advantage of using MTP / PTP is that the computer and the S3 can both access the internal memory at the same time. With USB Mass Storage (UMS) this is quite awkward and can result in errors.
Deleting data
This is partially why we needed to understand a little about file systems. So I could explain to you how data is handled when it is deleted.
As I explained earlier, when you write a file to memory, a corresponding entry is written to the file system to advise the OS where the data is. Sure, you may think you are writing the file to /sdcard/documents/work directory on the internal memory, but in reality these directories or folders do not actually exist at a memory hardware level. The data is written to a block and the file system is informed where that block is, how big the file is, what directory it should appear in to the OS etc. When a file is written the memory, the OS see's the available space go down and the used space go up. All this information comes from the file system.
When you delete the file, the actual data is NOT deleted. It remains where it is on the memory. The block is not overwritten. When the OS is told by the user to delete the file, the File system entry is deleted. This changes the free/used space as the file system is no longer accounting for the data, however the truth is the data still exists. When the next request to write a file to the memory comes from the OS, the file system will think the block where the old data was is empty and will overwrite it.
It is this difference between the file system and reality that allows data to be recovered by external software. if you do not write any data to the memory, external software can scan the memory for data whilst bypassing the file system all together. Ff course the window is small. You only have a very limited time to recover data before the file system allows the data to be overwritten with a new entry.
This is not just true of a deleted file. Even formatting the memory (which is actually just re-creating a new, blank file system) leaves all the data in tact behind-the-scenes and can all still be recovered until you start writing data to the memory. Cool huh?
Wow, all this time I've been stressing, is it really that simple?
Awwww snap! You got me. No it is not that simple. All this PC software, example: Piriform recuva only works on a computer drive. In windows, imagine this is anything with a Drive Letter. C: drive etc.
The only way to get a drive letter on your sdcard is to use USB Mass Storage mode, which as previously discussed - is not possible on your S3 (unless you are rooted, you can mount USB mass storage in custom recovery or use a UMS app from Play). The alternative is to use a card reader on your PC and put the sdcard in it.
There are also apps like Undelete for root users - which again, you guessed it - requires root. So if you're not rooted, it's simplest to use a card reader which can be bought for peanuts.
It's worth mentioning, NONE OF THE ABOVE will work with /sdcard on internal memory. It is not possible to get your data back once deleted from internal memory. Once gone, it's gone forever. You can only restore from /extSdCard (removable, external SDcard)
Phone won't boot, can I get my data back from internal memory?
Let's start by saying, it depends why your phone won't boot. If it's an SDS (Sudden Death Syndrome) type issue, where your internal eMMC (NAND FLash memory) has failed, then no. However, if you believe this is not the case then you can get your /sdcard data using adb BUT you need a custom recovery to be flashed via Odin before you do this. Read [REF] Understanding the basics before rooting your S3
However, if out of curiosity - you do still want to get your data off, using adb , read below:
Pre requisite is having adb "installed" on your windows PC. Download THIS file and follow the instructions in the readme.
You need to observe the following. For android 4.1.x and earlier, /data/media for android 4.2.x and newer, /data/media/0 - I will assume 4.2.2 for this guide,.
1) Boot into recovery, connect usb and go to "mounts and storage". Toggle the "mount data" options to mount these partitions. Tip, when mounted, the option then becomes "unmount data"
2) Open "cmd" in Windows and type the below code, which will copy all your data to a folder called sdcard on your windows desktop
Code:
adb pull /data/media/0/ c:\users\rootsu\desktop\sdcard
Also note, this assumes you have windows vista or newer. Also, it assumes your windows username is rootsu.
That's it, simple.
Display and Digitiser won't work, can I get my data back from internal memory?
You can use adb and a custom recovery to pull data from your /sdcard or even app data from /data/data
Pre requisite is having adb "installed" on your windows PC. Download THIS file and follow the instructions in the readme.
You need to observe the following. For android 4.1.x and earlier, /data/media for android 4.2.x and newer, /data/media/0 - I will assume 4.2.2 for this guide,.
1) Boot into recovery, connect usb and go to "mounts and storage". Toggle the "mount data" options to mount these partitions. Tip, when mounted, the option then becomes "unmount data"
2) Open "cmd" in Windows and type the below code, which will copy all your data to a folder called sdcard on your windows desktop
Code:
adb pull /data/media/0/ c:\users\rootsu\desktop\sdcard
Other things you may want to pull.....
Code:
adb pull /data/data/com.android.providers.telephony/databases/mmssms.db c:\users\rootsu\desktop\sdcard
Code:
adb pull /data/data/com.android.providers.contacts/databases/contacts2.db c:\users\rootsu\desktop\sdcard
Also note, this assumes you have windows vista or newer. Also, it assumes your windows username is rootsu.
That's it, simple.
Data corruption
When data becomes corrupt, there's really not a lot you can do. The file system knows where the data is already. If it's corrupt, you're stuck. Most common causes of corruption are:
1) Dirty unmount of /sdcard. SD card pulled out whilst it is being written to / phone shuts off whilst being written to. SOMETIMES - Plugging the card into a card reader in windows, Windows will ask to fix it and MAY fix it.
2) Fake SD card. This is really the MOST common. Get a card reader ans use:
h2testw.exe for windows to test your card in a card reader. Set it to read the full size of the card, which will take hours but well worth it.
If you get a result like this:
Code:
Warning: Only 63995 of 63996 MByte tested.
The media is likely to be defective.
3.8 GByte OK (8072512 sectors)
58.6 GByte DATA LOST (122989248 sectors)
Details:2 MByte overwritten (4096 sectors)
...Then you have a fake card, that is really 4 GB. I'll explain this.
Commonly, fake cards are reprogrammed to "think" they are high capacity cards, such as 32 GB or 64 GB to defraud buyers out of money. This is common on eBay (Never buy cards from eBay).
When these cards are formatted, the file system also thinks it is this fake capacity. Normally, when a card is full, the file system will report to the OS there is no more space and this prevents additional writes to the card. However, in the case where the card is 4GB and the File system thinks it is 64 GB, the tricked file system doesn't know the card is full. The file system keeps allowing data to be written to the card, over writing the existing data but without replacing the file system entries. The file system thinks data that has been overwritten hasn't been overwritten so when you try to open one of these files, it is essentially "corrupt" or non-existent.
Preventing data loss
Time to wise up guys. It is possible to recover data off your removable media, but internal memory - very unlikely. No apps on your PC or Android will help with deleted data. So you need to backup.
Dropbox - Use dropbox to automatically upload your photos to online storage.
Foldersync - Use FolderSync to upload important sdcard files to your dropbox account, or better yet - got a computer thats always on at home? Set foldersync to schedule a sync over wifi whilst you're asleep.
Other info
Interesting tidbits
Quite an exhaustive reference guide you got here rootSU thanks this will sure come handy for all of us :good:
Cheers
Thank you very much for taking the time to write this. It's a non academic approach to a sum of keywords and all of them are explained in such a manner that it would be almost impossible to misunderstand
Nice!
Nice post!
There are a few other interesting tidbits of info that might be worth mentioning:
- eMMC has an internal micro-controller that runs very specific firmware (and SDS was mainly caused by a bug in that firmware)
- eMMC (just like SSD) has specific writing/erasing limits and commands to deal with that - as a very general idea it can write about 4k at a time but can only erase in much larger blocks - like 64k (at least, but a 16GB model could have a much bigger block); normally on the same erase-size block there is very special list maintained, and based on that list wear leveling is implemented;
- all flash-based memory AGES - there is only a limited amount of erase/writes cycles possible before a point where the info is no longer reliably-stored; in some models that value can be incredibly small! to avoid writing more to some regions than other a mechanism call wear leveling is implemented; that one can have a big impact on both speed and reliability (but really don't expect it to create miracles)
- since it is very important for the speed and reliability of the flash memory to return unused blocks to this internal lists there are special TRIM commands that informs the firmware that the block can be garbage-collected; with an OS that supports TRIM, when a file is erased the blocks are also TRIMmed; this is one extra level that makes recovery basically impossible under normal circumstances
- this does not really mean that things are completely impossible to recover, just that you might need to spend so much on it that recovery would be impractical for any item worth less than 100000 US$ to 1 million US$
EDIT
- also just as with SSD it is not a bad idea to keep a good percentage of the flash memory free - IMHO at least 4GB for 16GB models, 6-8GB for 32GB models - that will improve performance since fragmentation (CLARIFICATION - free-space fragmentation) will grow much slower
- unfortunately there is no program for eMMC similar to smartctl (or any other SMART-data reading program) on normal SATA/IDE/SCSI disks - there seem to be some proprietary commands that are somehow similar but those are generally undocumented.
xclub_101 said:
Nice post!
Click to expand...
Click to collapse
I've put a link to your post in post 1. Where as it's not strictly relevant to my point, it is interesting stuff....
Fragmentation isn't an issue on ssd's. Its an issue on hdd because the head must physically move to another area of the Platter to get the data. That's the slow down. Defrag of a hdd moves all the used blocks (data) together so the actuator doesn't need to move much.
Performance degrades over time on ssds because every write, if data already exists must be erased too. But this hasn't really been an issue so much since TRIM became widely available.
-----------------------
Sent via tapatalk.
I do NOT reply to support queries over PM. Please keep support queries to the Q&A section, so that others may benefit
rootSU said:
...
Fragmentation isn't an issue on ssd's.
...
Click to expand...
Click to collapse
Thank you.
Also sorry for the misunderstanding with my contraction - what I wanted to say was free space fragmentation - that one does matter a lot on solid-state memory because of the garbage collection and some controllers have been famous for having a huge drop in performance with little free space - I will try to also correct that post.
xclub_101 said:
Sorry, I used a misleading contraction - what I wanted to say was free space fragmentation - that one does matter a lot on solid-state memory because of the garbage collection and some controllers have been famous for having a huge drop in performance with little free space - I will try to also correct that post.
Click to expand...
Click to collapse
Yep, it's true about Garbage collection, but TRIM *should* handle this nicely as should "over provisioning" although probably, some cheap SSD's may not over provision.
EDIT> Actually (sorry everyone for off topic) if you're interested in SSD's, these articles are a "fun" read... (I put fun in speech marks as it depends how geeky you are )
http://en.wikipedia.org/wiki/Trim_(computing)
http://www.pcworld.com/article/2038...-ssds-what-makes-these-speedy-drives-hum.html
Update to post 1:
Note from the Author -
I am moving on to the N5 now and ditching my S3. I will continue to maintain this thread, however - please do PM me if you think that something needs to be changed or updated in this thread as I doubt I will be answering questions within the thread as much. Please don't PM support questions to me. Only PM updates that need to be made in the thread.
It's been a blast!
Regards
Dan
Awesome bits of info. This is the game changer. I learned a whole lot just by reading here in XDA. I've only been using Android for a few weeks but thanks to XDA, I've already rooted, installed a bunch of apps and kept my OCD in check.
my device memory has corrupted and when i start recovery mode i get "E: faild to mount /cash (invalid argument) "