Related
:shock:
I was using the XDArit utility to write to the SD card and selected Disk0:Memory Card Adapter.. got a message stating it was my boot disk but I ignored it since it listed as the memory card adapter... guess what.. now my PC won't boot, says no operating system found.. I tried booting the win XP disc and doing a repair but it doesn't see anything on C:
anyway to recover my drive? I am sure the stuff is there somewhere right? Sh*t.. I wish I had done this on my home PC and not the office PC.
any help appreciated
Haha!
you overwrote your harddrive!
Since I don't have any idea what the tool you used does, I will (randomly) suggest http://www.partition-recovery.com and their program PR.exe. If the tool does a lazy format or bootsector write (just rewrites the 1st few sectors) PR may work. You can download a test version which won't operate fully but which will say if it finds anything to recover. If the XDA tool actually writes substantial amounts then this won't help.
There are quite a few tools which will scan the disk for anything that looks like data and recover what they can. Of course, if the tool you used actually overwrote the whole disk then you'd have to go a specialist company - and data receovery might still be possible - but this is likely to cost.
Good luck.
PS
I don't think "Haha" was a very kind comment. Not my idea of a "solutions provider" to go to - no matter how expert they might think they are.
Still, it did make me decide to sign up to the forum to post this in case it helps.
I downloaded the demo of PR.EXE and after scanning my drive, it appears that all data is still there... so I have purchased the full version of PR.EXE and it appears promising that I will be able to recover the most important stuff and simply restore my system (needed a restore anyway).
Thanks for the tip!
baldgyt said:
PS
I don't think "Haha" was a very kind comment. Not my idea of a "solutions provider" to go to - no matter how expert they might think they are.
Still, it did make me decide to sign up to the forum to post this in case it helps.
Click to expand...
Click to collapse
thx mate ... I appreciate such attitude ... hope others will learn someday! ... welcome on board
The SMT5600 is app unlocked and, I think, Super CID (via lokiwiz02_173 but how verify?) but no ROM changes as of yet as I want to make a backup of the original ROM before proceeding further.
After problems getting a term program to work (now using nueTTYConsole on Vista) I am able to get what appear to be complete ROM backups.
Procedure summary:
WinHex zero fill 64MB SD
USB bootloader SMT5600 with 64MB SD
r2sd all (via nueTTYConsole-12-v0.1-spackr)
SD back to PC [no to format query]
psdread E: 0 31328768 ipl.bin (using itsutl050119)
Status messages from the r2sd all command appear to be good and complete but no two backups, using the exact same procedure, are ever identical when binary compared with WinMerge. Size is, of course, the same but WinMerge always reports 'two' differences in what seems to be the same general area of the images: The first is very near the front of the image (WinMerge reports as 'lines', line 3) and the other at the very tail end.
Is that normal (maybe because TIME, or some other dynamic variable, changes or scratch storage?), is there a better backup procedure, and how can I verify the backups are good before I flash a new one and forever lose the original?
Thanks in advance for any enlightenment offered.
To check if it works - just restore the backups before doing anything else.
Follow the whole procedure (including psdread and - after reformatting the card - psdwrite again) to restore your device via the card. As a first try leave out the device external activities and restore immediately afterwards from the card just written.
For me it works well (on the SDA 2 - where no official update exists, a Hurricane device - but this generic handling is identical afaik) and the difference in the backups are normal.
Mind that the size of the read/write to card includes the bootsector, so don't miss the last 512 bytes. As far I remember there were two different size readings with two methods to verify the image size. The r2sd size is smaller than the size of bytes different to null on card.
To check for SuperCID enter "info 2" in the terminalprogram, it should report HTCSuperCID at the end.
tobbbie said:
To check if it works - just restore the backups before doing anything else.
Follow the whole procedure (including psdread and - after reformatting the card - psdwrite again) to restore your device via the card. As a first try leave out the device external activities and restore immediately afterwards from the card just written.
Click to expand...
Click to collapse
Thanks for the reply
Yes, I thought about doing a test restore, but, considering the problems I'd already had, wasn't sure if it might do something like not mention there being a 'problem' till it was half way through, leaving me with a scrambled ROM.
I take it you're saying it'll checksum first and no even start if things don't look good?
tobbbie said:
For me it works well (on the SDA 2 - where no official update exists, a Hurricane device - but this generic handling is identical afaik) and the difference in the backups are normal.
Mind that the size of the read/write to card includes the bootsector, so don't miss the last 512 bytes. As far I remember there were two different size readings with two methods to verify the image size. The r2sd size is smaller than the size of bytes different to null on card.
Click to expand...
Click to collapse
Hmm. I saw the confusion about SMT5600 image size but I'm not sure what you're saying here about the bootsector and "different to null."
Speaking of which, what would be wrong with just making a 64M save and, ok, you've save a pile of extraneous 0's along with it but, so what? Might be irritating if I were putting it on rapidshare but for a personal backup is there any down side to it?
tobbbie said:
To check for SuperCID enter "info 2" in the terminalprogram, it should report HTCSuperCID at the end.
Click to expand...
Click to collapse
Thanks. Good to know.
Something apparently went wrong somewhere because I didn't get that report but I'll try again.
The r2sd is a command that HTC has implemented in the SPL (Secondary Program Loader). I am not aware of checksums or other safety measures - it will as I noticed following the procedure detect if there is an image on the card, which type of image and if you want to restore.
The difference in size is that r2sd reports one size "x" after the image was taken, but if you count the bytes until when the card shows the zeros you will notice that this offset on card is 512 bytes larger than the r2sd reported size. So when using psdread you have to take the larger size. Indeed it is no problem to write more to the file and restore more as well with psdwrite. The restore procedure in the SPL will anyway know how much to restore - it just needs to find ALL bytes, including the last 512
I think there is no risk attached to the procedure, go ahead!
The only danger is if something goes wrong with the IPL (Initial Program Loader) or SPL because they open the door to the device handling.
Sadly you MUST deal with SPL to upgrade to WM5+ afaik, so be very sure to select the right IPL and SPL that matches your device HW (OMAP 730, 750 or 850) and intended OS Version. Also take care not to enter any command in the SPL except the ones you are supposed to enter - it may kill your device as well. Do never use "format all" or "doctest" - you have a brick then.
tobbbie said:
The r2sd is a command that HTC has implemented in the SPL (Secondary Program Loader). I am not aware of checksums or other safety measures - it will as I noticed following the procedure detect if there is an image on the card, which type of image and if you want to restore.
Click to expand...
Click to collapse
Well, I am certainly no expert on this thing but r2sd spits out a wealth of information, including checksums, and I was sort of guessing based on what I'd do if I'd made it. Just that, if you're going to calculate them, it seems a shame to not use them. But, hey, I've seen stranger things done.
tobbbie said:
The difference in size is that r2sd reports one size "x" after the image was taken, but if you count the bytes until when the card shows the zeros you will notice that this offset on card is 512 bytes larger than the r2sd reported size. So when using psdread you have to take the larger size. Indeed it is no problem to write more to the file and restore more as well with psdwrite. The restore procedure in the SPL will anyway know how much to restore - it just needs to find ALL bytes, including the last 512
Click to expand...
Click to collapse
Oh, OK. I wasn't going by r2sd. I opened it up in WinHex, found the end of data, and compared that to the size mentioned on "Backup your Typhoon ROM - WinMo @ MoDaCo." The 'corrected' number there matched well enough.
But now that I think of it, I did that because I *did* look at r2sd and it seemed too small. So you've explained it. Good.
tobbbie said:
I think there is no risk attached to the procedure, go ahead!
Click to expand...
Click to collapse
How can there be no risk if it doesn't check anything?
tobbbie said:
The only danger is if something goes wrong with the IPL (Initial Program Loader) or SPL because they open the door to the device handling.
Click to expand...
Click to collapse
Oh, I think I see what you mean. You're suggesting that if I've cut the ROM image short then only that part will fail but the loader should still be good so I could recover by burning another (good) ROM image.
Well, perhaps, but that would mean I don't have a valid backup and couldn't make one since it would be trashed in the bad flash. Or so it seems to me.
tobbbie said:
Sadly you MUST deal with SPL to upgrade to WM5+ afaik, so be very sure to select the right IPL and SPL that matches your device HW (OMAP 730, 750 or 850) and intended OS Version. Also take care not to enter any command in the SPL except the ones you are supposed to enter - it may kill your device as well. Do never use "format all" or "doctest" - you have a brick then.
Click to expand...
Click to collapse
I was thinking of going straight to WM6.x per
karhoe.net/guide-upgrading-htc-feelertyphoonamadeus-to-windows-mobile-6-update-september-06-2008.html
which involves changing the loader first via Patched_RUU
Do you think going to WM5 first is a safer procedure?
I said I was not aware of any checking - but as I have not written the SPL, I simply do not know it. You are right that reporting stuff like this makes it highly probable that upon restore a check on the image should be done before restoring. Try it out, if you like
WM5 or WM6 does not make a difference what the SPL is concerned. Afaik you have to use the same anyway. The device is tight in memory anyway, so don't expect miracles.
Go ahead, either dare it or leave it...
tobbbie said:
I said I was not aware of any checking - but as I have not written the SPL, I simply do not know it. You are right that reporting stuff like this makes it highly probable that upon restore a check on the image should be done before restoring. Try it out, if you like
Click to expand...
Click to collapse
Hehe. Yeah.
I was sort of hoping someone else had already stepped off that cliff and could tell me what the ground was like before I dove in
tobbbie said:
WM5 or WM6 does not make a difference what the SPL is concerned. Afaik you have to use the same anyway. The device is tight in memory anyway, so don't expect miracles.
Go ahead, either dare it or leave it...
Click to expand...
Click to collapse
The primary aim was to get bluetooth a2dp but the incentive may have diminshed, depending on how another project works out.
Thanks again for the help.
I would not bet on A2DP - I have it in the Tornado and the CPU use is much higher due to additional compression on the BT interface. Player + BT overhead is getting to average above 80% CPU (depending no the settings, but for good quality is like this) - it will also drain your battery much faster.
The Typhoon, Hurricane and Tornado have identical good analog Audio capabilities (I measured them with RMAA - see www.rightmark.org) and make a perfect music player as they are.
If your device is SuperCID you can take any other Typhoon ROM - you must just be sure that r2sd will save your bootloader + OS if you want to go back to WM2k3. I have done this already on my Amadeus (and went back to WM2k3) and this can still serve as a nice musicplayer.
tobbbie said:
I would not bet on A2DP - I have it in the Tornado and the CPU use is much higher due to additional compression on the BT interface. Player + BT overhead is getting to average above 80% CPU (depending no the settings, but for good quality is like this) - it will also drain your battery much faster.
The Typhoon, Hurricane and Tornado have identical good analog Audio capabilities (I measured them with RMAA - see www.rightmark.org) and make a perfect music player as they are.
If your device is SuperCID you can take any other Typhoon ROM - you must just be sure that r2sd will save your bootloader + OS if you want to go back to WM2k3. I have done this already on my Amadeus (and went back to WM2k3) and this can still serve as a nice musicplayer.
Click to expand...
Click to collapse
I admire people who can make these flash things work because it never does for me. I've now got an SMT5600 that will do nothing but display a rainbow boot screen and error out regardless of what ROM I try.
That's why I didn't try this till I had a new phone.
Hey that thread has a long history - what happened in the meantime?
3 colour screen does not mean the device is dead yet. You still have a bootloader that works and this is the thing to start from in any case.
What do the lines tell in the 3 color bars?
Did you already upload the changed SPL (I think it was 1.09) that allows to flash ROMs of WM5 or WM6 on that original WM2k3 device? If so, the you need to revert back to old SPL first before you can upload the original ROMs again.
tobbbie said:
Hey that thread has a long history - what happened in the meantime?
Click to expand...
Click to collapse
I put it on hold pending a new phone and other things cropped up.
Frankly, I had 2003 pretty well tricked out with SmartToolkit and gStart.
tobbbie said:
3 colour screen does not mean the device is dead yet. You still have a bootloader that works and this is the thing to start from in any case. What do the lines tell in the 3 color bars?
Click to expand...
Click to collapse
I swear it wasn't a troll but no sooner than I posted it wouldn't flash I managed a flash and I'm not sure why this worked when the others failed.
I was trying to verify the hard spl, getting info, etc. To make that easier I turned 'ui' on during boot and, just for chuckles expecting nothing, I tried flash again. You know, the definition of 'insanity'. Low and hold the dern thing flew.
As far as I know nothing was different other than 'ui' on. Same tools, same wm6.5 bin file, etc.
tobbbie said:
Did you already upload the changed SPL (I think it was 1.09) that allows to flash ROMs of WM5 or WM6 on that original WM2k3 device? If so, the you need to revert back to old SPL first before you can upload the original ROMs again.
Click to expand...
Click to collapse
You have no idea how helpful mentioning "1.09" is. The SPL flash program opines something like changing to v 5.000 but that number shows up no where and no where does it tell you to look for '1.09'. There are other confusions, like saying the existing device was 'Orlando' (I think it was), but I guess that's moot now.
Anyway, it's now running WM6.5 and I have a new toy to fiddle with inbetween playing with Android on my Tilt 2.
Thank you for the help.
Glad it worked now
The older (wm2k3) devices could only be updated with a binary transfer protocol (the .BIN file - which can be confused with other ".bin" in the scope of cooking in general). To enable the reception of the MTTY command "l" (for Load) and the execution of the related actions, the SPL must be in "UI" (User Interface) mode - this is the key for further flashing - and it must be mentioned in all such upgrade manuals. Also mind that other terminal programs (like TerraTerm) have not implemented that protocol. So only MTTY works for that purpose! As I am struggling currently with porting a Tornado ROM to the Hurricane I have come quite deep into that topic recently.
Are you having the WM65 from aleut now on the device? I think it is very tight on RAM now, so what are the memory key-data from settings->about after a reboot? You should repeat that with the standard home screen (Windows default) which is less memory greedy.
The way back to WM2k3 is not so easy as you must replace the SPL with the original one first before you can get back to the original OS. Whenever you mess with SPL it is a potentially dangerous action as failure doing that right will result in a bricked device.
tobbbie said:
Glad it worked now
The older (wm2k3) devices could only be updated with a binary transfer protocol (the .BIN file - which can be confused with other ".bin" in the scope of cooking in general). To enable the reception of the MTTY command "l" (for Load) and the execution of the related actions, the SPL must be in "UI" (User Interface) mode - this is the key for further flashing - and it must be mentioned in all such upgrade manuals. Also mind that other terminal programs (like TerraTerm) have not implemented that protocol. So only MTTY works for that purpose! As I am struggling currently with porting a Tornado ROM to the Hurricane I have come quite deep into that topic recently.
Click to expand...
Click to collapse
So I discovered after missing the little '0' in the instructions.
tobbbie said:
Are you having the WM65 from aleut now on the device? I think it is very tight on RAM now, so what are the memory key-data from settings->about after a reboot? You should repeat that with the standard home screen (Windows default) which is less memory greedy.
Click to expand...
Click to collapse
Yes, I originally flashed Aleuts 6.5 but I've since reflashed with his 6.1.
tobbbie said:
The way back to WM2k3 is not so easy as you must replace the SPL with the original one first before you can get back to the original OS. Whenever you mess with SPL it is a potentially dangerous action as failure doing that right will result in a bricked device.
Click to expand...
Click to collapse
Yep, flashing SPL is the most vulnerable but I don't think I'll be going back to 2003. Although, I might try WM5 if that has more free memory.
With most things I plan on using installed there's 8.5Meg free at boot and while that sounds laughable by today's standards there's only 22Meg total for a more impressive sounding '38% free' Although, as soon as you touch the thing almost half of that is gone.
I'm no expert when it comes to the topics of rooting and getting access to the emmc and all of that good stuff. I more specialize in ROMs and themes and stuff, the less complexed stuff lol
Someone has posted an idea in the general forums in relation to permanent root, I'm not sure if he posted it here or not. So here's what he wrote....and is it possible? Or does it have to be done manually first before this idea can happen?
Originally Posted by deliberate187
In order to unlock the phone, we have to figure out what the protected sectors are first and all related flags. If an Android app could be made to have direct read access to the eMMC filesystems (including write protect flags) and save a log to the SD card detailing these items, this would be ideal.
Then all that would remain is a program to undo the write protection (and re-do it if necessary to unvoid warranty)
If anyone is willing to create these programs, I would be more than happy to test them out on my own G2.
However, I think the keys to the mystery may lie in the recovery image, and/or in the bootloader itself. Has anyone disassembled these yet?
Click to expand...
Click to collapse
Sorry to have to tell you but this is all old information stuff we already know just are unable to do anything about it. Its harder then just coming up with an idea of something. Now if we knew a person that programed the g2 in htc factory then all would be good but as of now we just dont have the information we need to do anything
thanks
Thanks for the idea. Some people will be mad you didn't post in the root thread though.
File under "I'm no expert but..."
Here is one observation I have noted in my exploration. The root filesystem and system partition are mounted with the flags "-o ro,relatime" but in addition the /system partition has ",errors=continue" leading me to believe that this change is in fact written to the release configuration rather than to the eMMC itself. Can anyone try to get a permanent write to the fstab and see if this can net us permanent root? Possibly take a temp root session and remount the system and / filesystems read/write to see if writes stick... just an idea.
The errors=continue flag allows the ext3 filesystem to continue working even if there was a read/write error.
I've been able to get the system to change to r/w a couple times while wandering through root explorer. I have made subtle changes to certain folders such as moving txt files but nothing has ever been permanent. I can't really tell you how I did it either seeing as I can't replicate it on demand...I'm assuming it still gets written to cache despite being in the /system
Sent from my T-Mobile G2 using XDA App
heyy, I'm not punchie, I've got what the doctor calls a relaxed brain
I am thinking there should be a set of adb commands to unlock the nand. I am definitely thinking a nand dump and full disassembly of the bootloader and recovery image could be absolutely crucial in discovering what needs to be done. Just a thought, has anyone done a nandroid backup of the G2 yet? I'm pretty sure TMob doesn't have HTC encrypt its bootloaders...
deliberate187 said:
I am thinking there should be a set of adb commands to unlock the nand. I am definitely thinking a nand dump and full disassembly of the bootloader and recovery image could be absolutely crucial in discovering what needs to be done. Just a thought, has anyone done a nandroid backup of the G2 yet? I'm pretty sure TMob doesn't have HTC encrypt its bootloaders...
Click to expand...
Click to collapse
if you can figure it out, go for it and i wish you luck
deliberate187 said:
Here is one observation I have noted in my exploration. The root filesystem and system partition are mounted with the flags "-o ro,relatime" but in addition the /system partition has ",errors=continue" leading me to believe that this change is in fact written to the release configuration rather than to the eMMC itself. Can anyone try to get a permanent write to the fstab and see if this can net us permanent root? Possibly take a temp root session and remount the system and / filesystems read/write to see if writes stick... just an idea.
The errors=continue flag allows the ext3 filesystem to continue working even if there was a read/write error.
Click to expand...
Click to collapse
If it were only this easy.
Re-mounting /system as r/w is part of the rooting process. This does not result in changes written to eMMC. In fact, the controller "lies" to Linux that the change has been synced. From then on, Linux holds the changes in its cache which, when dropped or rebooted, reverts changed files to their original state (because they were never written in the first place.)
The ext3 continue on errors thing is merely a way to skip fsck in the event that the read-only system has issues in the journal (very unlikely to happen, since nothing can write to it.) Presumably, this only covers an oversight in OTA updates (where the journal of the image provided by the OEM is dirty for some odd reason.) Again, since nothing can write to /system, it's all but an impossible scenario (nothing can write to the journal either...)
As for marking "sectors" as write-protected or not, that's also easier said than done. Entire partitions are locked, and half of the space is mysteriously "missing." It's difficult to see what's really going on from userland, as the device is deceptive as to what is and is not being written, or what is even stored on the eMMC in the first place.
The real solution is to exploit either the boot-loader or eMMC (re)/initialization somehow to allow a) unsigned firmware to be loaded and/or b) allow booting without write protection, allowing us to c) flash rooted rom to the phone and/or d) disable said protection. The unlock procedure will likely be similar to Unrevoked, as that is essentially the same situation (aside from the controller issue.)
All of this is covered in the wiki and various threads - check those out, if you find a way around it everyone would be glad to hear it.
HamNCheese said:
If it were only this easy.
Re-mounting /system as r/w is part of the rooting process. This does not result in changes written to eMMC. In fact, the controller "lies" to Linux that the change has been synced. From then on, Linux holds the changes in its cache which, when dropped or rebooted, reverts changed files to their original state (because they were never written in the first place.)
The ext3 continue on errors thing is merely a way to skip fsck in the event that the read-only system has issues in the journal (very unlikely to happen, since nothing can write to it.) Presumably, this only covers an oversight in OTA updates (where the journal of the image provided by the OEM is dirty for some odd reason.) Again, since nothing can write to /system, it's all but an impossible scenario (nothing can write to the journal either...)
As for marking "sectors" as write-protected or not, that's also easier said than done. Entire partitions are locked, and half of the space is mysteriously "missing." It's difficult to see what's really going on from userland, as the device is deceptive as to what is and is not being written, or what is even stored on the eMMC in the first place.
The real solution is to exploit either the boot-loader or eMMC (re)/initialization somehow to allow a) unsigned firmware to be loaded and/or b) allow booting without write protection, allowing us to c) flash rooted rom to the phone and/or d) disable said protection. The unlock procedure will likely be similar to Unrevoked, as that is essentially the same situation (aside from the controller issue.)
All of this is covered in the wiki and various threads - check those out, if you find a way around it everyone would be glad to hear it.
Click to expand...
Click to collapse
Listen to this dude. Absolutely correct.
If you ever used CWM, CWMT or other non factory recoveries to wipe your data, you probably noticed that you lost the ability to encrypt your phone. Or maybe you did not even realize this is why encryption does not work.
For the Android phone encryption to work, it needs the /data (usrdata) partition to have a little bit of unused space between the end of the filesystem and the end of the partition. And as soon as you use CWM to wipe, it actually reformats using all space, and encryption does not work anymore.
User lolo250612 brought this to my attention, and together we created a update.zip that shrinks the /data filesystem by 1MB
In fact, we created 2 patches: One to shrink, and one to first repair the filesystem. The first will refuse to shrink if the file system is not clean and healthy. They will automatically find the correct usrdata partition device and its size. The shrink will then resize to 1MB less then the partition size (which means it could also be used to grow if you somehow had a filesystem a lot smaller, for example because you restored an smaller image from somewhere).
Both patches are created with statically linked e2fsprogs binaries and its own static copy of busybox shell interpreter. So they should work on all Android devices that use ext file system (probably all V2.3.1 Gingerbread and higher androids), and you should not lose any data because of this. But it is always good to make a backup.
We tested this on 2 phones, both ICS phones, and with both CWM and TWRP type recoveries, and are fairly certain it is safe to use. But to repeat, you should always take a backup of your phone.
Both patches can be found on my shared drive:
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
Procedure:
- Make backup of your phone
- Place files on SD card
- Boot into recovery
- Apply the shrink update
- If it tells you the filesystem is damaged apply the fix-fs update first
The patch only shrinks the filesystem, nothing is actually installed or removed on the phone. But if you use encryption, you could leave this patch on your SD card so that every time you wipe data, you can run the shrink patch again afterward to enable encryption again.
If you do use this, please report back in this thread, possibly mentioning your phone model and ROM you are using.
Quick encryption guide (and more)
I won't go deep into useless details as everything has already been described about Android phone protection somewhere on the internet. I will just give some meaningful links and tips by illustrating how I have protected my phone. Really nothing new or innovative, just a compilation of a few hints that I have put in practice to protect the numerous pieces of information that are on my phone.
Step 0: awareness
----------------------------
Why bother with phone security?
In short, I am clearly paranoid. Well, in fact, I don’t really feel at ease when I know all the information, both personal and professional I have on my phone. Over the month, my Androphone has become a real digital Swiss-knife and personal secretary. This includes:
Personal and professional contacts
Personal and professional agendas
Personal and professional digital exchanges (SMS and email)
Personal and professional photos
Banking account information
Trails where I run
Etc… etc…
Don't want someone looks at them. Not you?
Fist step: on-line protection
----------------------------------------
The first step in protecting your data consists in making hard to access indirectly the data that lay on your phone memory. This access consists in using the system when the phone is on, either via the GUI and the phone controls, or remotely (essentially by network connections, or phone basic functionalities like sms). So, basically, you need to lock efficiently your phone from preventing someone else to unlock the user interface that allows interactions with the system, and protect all communication channels.
To lock efficiently your phone, you must use a pin code of at least 4 digits (6 is better) or a pass-phrase. The latter is much less practical without improving online security that much. Above all, you must avoid those silly locking solutions like face recognition unlocking, or pattern lock. Those are toys for naive young boys. Not for those concerned seriously by security.
For protecting remote access to your phone, I would suggest:
1) Double check that USB debugging is disabled. This a major security hole.
2) Turn on data connections (bluetooth, wifi and 2/G/3G/4G) only when required (email checking, web-surfing session, data synchronising), and off rest of the time.
3) Avoid install cracked unofficial apks, or applications that asks for permission far beyond their obvious and principal utility
4) Install a software security app, if possible, open source and recognised by xda members. Once an adept of Droiwall, I have switched to Avast mobile security because of its extra features. But it is not opensource and it is a question of taste. But do this carefully, see that for instance before making a choice: http://download.cnet.com/8301-2007_4-57391170-12/dont-get-faked-by-android-antivirus-apps/ and http://www.av-test.org/fileadmin/pdf/avtest_2012-02_android_anti-malware_report_english.pdf.
But, you must be rooted (which is in itself a security hole if not mastered) and one must have a kernel with netfilter functionalities activated. This is the case with the stock kernel of the phone I use at the present time (Lenovo A789). But was not the case of 2 Samsung phones I used before. You have to either install a custom kernel adapted to your phone, or make your own if you have access to its sources (see tutorials as: http://forum.xda-developers.com/showpost.php?p=22941057&postcount=1)
5) Personally, I would feel more at ease if I could find an easy to use firewall solution that could close, and better, make stealth all the local ports of my phone, especially when I am not behind a wifi router. But I haven’t found one yet. Droidwall, nor Avast, addresses this functionality, whereas it would be fairly easy to implement it with the netfilter system layer underneath.
Second step: offline protection
-------------------------------------------
Here we are. Now your phone is protected when it is on. But, what if you switch it off, or remove its sdcard? The data lay on the internal memory, unprotected (at best obfuscated). Really easy to find a custom recovery for almost all phones, write a script to dump /data on a sdcard and then make whatever you want with the copy.
Don’t like that? The only solution to prevent /data from being read by someone else is to encrypt the /data partition. To do that, your phone or tablet internal storage partitions must be seen by your system as block devices. This is the case with eMMC but not with Yaffs. So beware, if you want encryption you need to buy a device that answers this requirement. This is not always true and almost never documented. Notes on the implementation of Android encryption are there: http://source.android.com/tech/encryption/android_crypto_implementation.html
Now, as me, if you are reading these lines, you are certainly looking for extra information about your Android device and probably extra functionalities.
Certainly, the most frequent way to install extra functionalities and custom ROMs to your phone is to use an update zip file. With stock recovery, this zip file needs to be signed, otherwise it is rejected. For maximum flexibility and ease of use, alternative boot recovery have been developed, of which CWRP is certainly the most famous.
Usually, for 99% of users and operations, CWRP operates great. Sometimes, as nothing is perfect, a bug may occur. This is the case for built in ICS encryption process. As Cybermaus indicates in the first post, to be able to perform this encryption the /data filesystem must be slightly smaller than the underlying partition. But CWRP, at least up to the version 5.5, formats all the corresponding partition leaving no place for Android to store the required information to be able to start the encryption process. This is clearly described in the following links: http://forum.xda-developers.com/showthread.php?t=1792101 and http://rootzwiki.com/topic/25652-fixing-galaxy-tab-2-encryption/
I have discovered that by using aLogcat to track down the origin of the failure. The interesting part revealed to be: E/Cryptfs ( 87): Orig filesystem overlaps crypto footer region. Cannot encrypt in place.
To circumvent this problem, you will find in Cybermaus first post, two CWM update zip files that will do the trick in a simple and secure way. After flashing your ROM and wiping data with CWM, apply them, go to system encryption as described here:http://support.google.com/android/bin/answer.py?hl=en&answer=1663755, and after waiting one or two minutes (not more), the system should restart automagically to encrypt your /data partition.
Third step: making your phone even more secure and practical at the same time
-------------------------------------------------------------------------------------------------------------------
Android built-in encryption is in fact more or less Linux LUKS (http://en.wikipedia.org/wiki/Linux_Unified_Key_Setup). Plus, it is open-source so that everyone with the required skills can make an audit of the code to see if no security hole is present in the Android implementation. The underlying mechanism is strong and secure, as long as you use a strong password. I mean by strong, at least 12 characters that includes at the same time lower-case letters, upper-case letters, numbers and symbols. And it must be something impossible to guess for others while easy to remember for yourself. You will find a lot of resources on the internet on how to create such a password. For instance: https://help.ubuntu.com/community/StrongPasswords .
The problem with Android, in its attempt to keep the system not too complicated to use, is that the GUI (I insist: only the GUI, not the system) does not distinguish between the PIN or passphrase that you use to lock your phone when it is on, and the password used to encrypt the data that lay physically on your phone storage. So the casual user is in front of a paradigm: either he chooses a strong password for its data, but this will rapidly become tedious to type at least 12 characters to unlock his device several times a day; or he decides to use a PIN code, which is more practical to unlock the phone, and consequently uses a really weak password to encrypt its data which contains only digits, and thus may be cracked in a breath by any PC.
Fortunately, this paradigm is addressed and solved by small tools like EncPassChanger or Cryptfs Password (both requiring that your phone be rooted, which is by the way, paradoxically, a security hole if not used with caution ). See: http://nelenkov.blogspot.fr/2012/08/changing-androids-disk-encryption.html for complete notes about that. So for me, the only way, both secure and practical, to secure your phone is by using a PIN code of at least 4 numbers (6 is better). Then use a handy tool like EncPassChanger to have a true complex password for decryption at boot time.
Fourth step: increase security, without sacrifying practicability
-----------------------------------------------------------------------------------------
As I am paranoid, but at the same time don’t want my phone to become a source of annoyances, the previous “basic” steps were not enough for me.
So I decided to improve security in two ways:
1) By following the following tip, which I find great and is itself self-explaining: http://forum.xda-developers.com/showpost.php?p=26730989&postcount=2
2) By encrypting the photos I take with my phone, because these are linked with my private life and I won’t like that somebody gain access to them.
3) By encrypting documents I scan with CamScanner, for home and work, which may be sensitive.
4) By automating the action that disables USB debugging in case I forget to put it off after using it .
For point 2 and 3, documents lay on your sd card uncrypted. Android built-in encryption does not deal with both internal and external sdcard (just to be clear, by sdcards I mean partitions mounted as /mnt/scard or /mnt/scard2). To encrypt them you have to use once again an external tool. As I am an opensource fanatic for all that deal with security, I would recommend to use LUKS Manager (https://play.google.com/store/apps/details?id=com.nemesis2.luksmanager&feature=search_result and http://forum.xda-developers.com/showthread.php?t=1141467) which is based on dm-crypt module (yes, the same that Android uses for its build-in encryption), or Cryptonite (https://play.google.com/store/apps/details?id=csh.cryptonite&feature=search_result) which is completely open-source and implements the rock-solid Linux encfs on Android.
The latter is my personal choice. I do not use Crytonite in itself, except for creating the initial .encfs6.xml file. For everyday use, I use directly the Android port of the binary file encfs that comes with Cryptonite, and embed it into shell scripts. Up to now, no flaw, no problem. The password to open my encfs encrypted volumes is stored in a text file located on the /data partition. It is thus encrypted by Android and made accessible on boot when you decrypt this partition. So nothing more to remember.
To make things usable and practical, I use Tasker to automate the following things:
- Mount encfs volumes on start-up, by reading directly the password in the file located on /data
- Umount encfs volumes when usb is plugged
- Copy photos on a regular basis from the unencrypted /mnt/sdcard/DCIM to the safe place I created with encfs, delete AND wipe the original ones
Fifth step: be coherent about security
-----------------------------------------------------
Some people, torn apart by the paradigm described in Third step, by negligence or by lack of knowledge, strongly secure one part of the system, but make other parts big security holes.
Concretely, I am thinking about two examples: mixing encryption with pattern lock (or, even worse, with face unlock), or mixing encryption with usb debugging. Face recognition is just a jock. It is not reliable and fails very often. Moreover it is really easy to crack, with a photo for example. One of my colleague even achieved to unlock my phone with its own face, just because we are morphologically close enough. Pattern lock is not much better. (See: http://forum.xda-developers.com/showpost.php?p=37649447&postcount=6 and https://www.google.fr/search?q=smudge+attack).
So always ponder over (two times rather than one) each action you take that may touch system security.
Thanks lolo
I'm trying to use this on my VZW Galaxy S3 16Gb and this is what I'm seeing in TWRP v2.2.0:
Mounting System
Extracting system fixes
Update script starting...
Update script started
Disk /dev/block/mmcblk0p15: 13.1GB, 13140754432
4 heads, 10 sectors/track, 401024 cylinders
Units = cylinders of 64 * 512 = 32768 bytes
ERROR: unlikely size of KB
aborting operation!
Update script ended
Unmounting system...
Update Complete
Click to expand...
Click to collapse
edit: The same thing happens with both scripts.
I need to enable device encryption because my employer requires it for email and other Google Apps for Business apps. Thank you for your help!
Anyone know why full disk encryption isn't available on some (if not most roms)? Is it something that needs to be added with intent aside in the building process, or dependent on how the stock rom was set-up to work with?
I was hoping this would help get encryption working on an EVO ics rom which has encryption available, but when you click "encrypt phone" it just hangs on an android screen and doesn't actually do anything.
i was really happy to find your solution to enable encryption on my HTC desire S (ICS, rooted), but unfotunately it doesn't work. the same thing happen to me as it happened to mushu13, only different numbers in lines 5 and 6. same result whichever script i choose. please help, i really need system encryption.
thanky you very much!
First thing you should know, I am not an Android Guru. And unfortunately, if your phone is not an A789, I won't be able to help you deep in technical details. Cybermaus is the most skilled of the two of us, technically speaking, and he may lack time to answer correctly every request he is regurlarly faced with.
Okay, I do not know your phones and don't own them. So, distant debugging is much harder in these conditions. But the first things you should check, before applying Cybermaus' patches, are :
1) if encryption works with stock rom
2) follow thoroughly all steps I described in "Second step: offline protection" of the second post of this thread :
- your phone or tablet internal storage partitions must be seen by your system as block devices. This is the case with eMMC but not with Yaffs. If you don't have this information from the manufacturer, install Terminal Emulator from the Play Store and type 'mount' in it. You should see lines beginning with /[email protected] and /[email protected] If this is not the case, I fear encryption won't be able to work on your device.
- use aLogcat to track down the origin of the failure (see resources on the internet to learn how to use it, and links I have put in the second post)
3) Be sure that required modules are built into the kernel you use, especially dm-crypt
4) Post your results and cross your fingers that either this is a problem I have already encountered (in this case I may help you further), or Cybermaus see your posts.
While this script did allow me to encrypt my phone, it also shrunk my /data partition to roughly 1.1 GB.
Any ideas on how to expand it back to a reasonable size? I supposedly have 4 GB of ROM, and I assume more than 1 GB ought to be available for data.
Sent from my HTC Sensation using xda app-developers app
Thank you for your nice guide.
Only one thing is missing: baseband security.
Attacks on the baseband system requires very skilled people. Such as government agencies. It is believed they use baseband attacks to break into almost every mobile device. And there is only little you can do. Some vendors like Cryptophone have mobile devices with a hardened Android system. All others have no way to protect their device against baseband attacks.
Is this patch and reasoning still valid for newer android releases.
I am running a custom kitkat rom and twrp on a note 3 and can't encrypt so im looking for a fix.
I have been looking around for fixes but different posts blame different things.
Sometimes its the fact its a custom recovery, sometimes its that root is on the device and then there is this reasoning
Is there a way to find out the cause and fix for kitkat?
Virus
Hi, i tried to download your files
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
But the file are exe files with viruses.
Any ideas?
u2funker said:
Hi, i tried to download your files
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
But the file are exe files with viruses.
Any ideas?
Click to expand...
Click to collapse
Maybe false alarm.
Lossyx said:
Maybe false alarm.
Click to expand...
Click to collapse
no, but if you search for these file, you will find some which work and which are without viruses. Check the link..it is not an zip file..it is an exe file
@cybermaus: just tried flashing the two *.zips on my Galaxy S 4 Mini running CM 12 (Android Lollipop) because my logcat tells me I'm getting the described cryptfs error. It seems my /data partition doesn't have that 1 MB of unused space needed for encryption. Now I would love to encrypt my phone using CM's integrated function without having to completely format the internal storage (because that's the other workaround I found: flash stock rom, wipe data (factory reset), flash Custom Recovery, flash CM again)
Do you have the time and device to update your script so it works with Android Lollipop as well? I see a lot of people come across this issue recently so there would be definetly use for such a nice script like yours!
Thanks for sharing this with us!
-Teutone
no available for download any mirror ?
Or write the script on the thread.
Thanks
Can you post the scripts? links are dead!
---------- Post added at 16:33 ---------- Previous post was at 16:32 ----------
cybermaus said:
If you ever used CWM, CWMT or other non factory recoveries to wipe your data, you probably noticed that you lost the ability to encrypt your phone. Or maybe you did not even realize this is why encryption does not work.
For the Android phone encryption to work, it needs the /data (usrdata) partition to have a little bit of unused space between the end of the filesystem and the end of the partition. And as soon as you use CWM to wipe, it actually reformats using all space, and encryption does not work anymore.
User lolo250612 brought this to my attention, and together we created a update.zip that shrinks the /data filesystem by 1MB
In fact, we created 2 patches: One to shrink, and one to first repair the filesystem. The first will refuse to shrink if the file system is not clean and healthy. They will automatically find the correct usrdata partition device and its size. The shrink will then resize to 1MB less then the partition size (which means it could also be used to grow if you somehow had a filesystem a lot smaller, for example because you restored an smaller image from somewhere).
Both patches are created with statically linked e2fsprogs binaries and its own static copy of busybox shell interpreter. So they should work on all Android devices that use ext file system (probably all V2.3.1 Gingerbread and higher androids), and you should not lose any data because of this. But it is always good to make a backup.
We tested this on 2 phones, both ICS phones, and with both CWM and TWRP type recoveries, and are fairly certain it is safe to use. But to repeat, you should always take a backup of your phone.
Both patches can be found on my shared drive:
ICS_usrdata_fix-fs.zip
ICS_usrdata_shrink.zip
Procedure:
- Make backup of your phone
- Place files on SD card
- Boot into recovery
- Apply the shrink update
- If it tells you the filesystem is damaged apply the fix-fs update first
The patch only shrinks the filesystem, nothing is actually installed or removed on the phone. But if you use encryption, you could leave this patch on your SD card so that every time you wipe data, you can run the shrink patch again afterward to enable encryption again.
If you do use this, please report back in this thread, possibly mentioning your phone model and ROM you are using.
Click to expand...
Click to collapse
links are dead. Can you post the scripts?
I have a kinda complex problem with this device.
Someone called me last night scared because she (kinda) deleted an entire messages thread from the phone ( SMS ) and it was an accident.
Now, she really need those messages because of a juridic case. Is there any way to recover old messages that were deleted from that thread ?
I tried to recover with EaseUS mobile Data Recovery, which (for desktop version) i used in the past for other laptops and Desktops to recover deleted files or formatted disks, but with phones, it requires me Root. I recently Rooted my HTC device and this process "wipe" whole data from your phone. As i know, Windows Kernel and Linux Kernel (the one used in Android OS too) have the same file management system, so the point is, this "Wipe" what actually does with MBR ? It hard format the disk, or it just modify MRB's File Table by deleting it's references pointers to files ? A hard format (switching every single bit to 0 from where MBR ends, all the way to the end of the partition) takes a lot of time, especially in these smartphones, because they have a lot slower storage speeds, and on my HTC device, this process took like 1-2 seconds so i hardly can imagine that actually it deletes the data, so with the MBR's mechanism in mind, i really take this in consideration to root the phone and use that software to recover that thread.
My main concern is that, i don't really know what file shall i restore. If these SMSes are stored into a database, when a thread is deleted, the phone can simply "drop records"... i mean, it can simply delete records from the tables, and tables being stored in a database as a single file, ell, and they are gone... for good in this case scenario....
The only way i think i will act if someone confirm that this is the way to recover it, i will restore every single file of the phone that was deleted before accident happen and i will look for SMS database to be modified a day after the thread was deleted. Is this the good way, i'm about to do a pointless action by rooting this device?
I have to mention, the thread is now created again, have 3 new mesages with that person, but there were like 1k+ in that thread before... so, how to recover them?
Thank you in advance for any information.