Database Users

MS Cloning Method to Quickly Restore Data After ROM Upgrade

I'd like to toss this out for discussion. This might already exist, it might not.
Microsoft had system builders use a Disk Differencing technique to extract what exactly had changed after installing applications and customizations.
First, we'd install the OS and create a base image.
Second, we'd install the apps & customizations we wanted for our customers.
Third, we'd create another image. An app would then compare these two images and determine what files were added/changed and what registry additions, deletions, alterations were performed.
I'm hoping we can do the same for ROM upgrades.
1. OLD ROM - Base image.
2. Install apps and customizations - Image "Apps". This will give us our customizations.
3. Install new ROM - NEW base image created. Compare to OLD ROM base image to see what has changed (that we need to look out for that our APPs image might adversly change).
4. Apply Apps image to restore all customizations & applications.
Ideas anyone?

The theory sounds good.
Maybe buzz can comment?

The theory is very feasable, ive used similar building Win PE rescue disks, you have a app that gives you the base bootable image made from a Win XP install CD. then you add home made "modules" ie, your installed apps or networking capabilities etc. then it makes an image for you to use,
that cant really be that difficult to do the same for Win CE,
Ive also seen a M$ webcast thing where the guy took 45 mins to build an entire image for a win ce device,
so yer, it is very feasible that such dev apps exist, after all, they were made and then sold to the hardware vendors before ever being a rom image.
check in msdn for win ce 5 stuff, its a very intresting read, :shock:

it is a good idea, but only for linear ROMs, where each file has always it's own and same place in the rom and such rom is not compressed.
hmm...
but you can do partial rom upgrades with wm5 roms...
...and it would be possible to make registry diff + DOC Storage partition backup, where you actually install programs (so called \Device) and userdata.
buzz

buzz_lightyear said:
it is a good idea, but only for linear ROMs, where each file has always it's own and same place in the rom and such rom is not compressed.
hmm...
but you can do partial rom upgrades with wm5 roms...
...and it would be possible to make registry diff + DOC Storage partition backup, where you actually install programs (so called \Device) and userdata.
buzz
Click to expand...
Click to collapse
Ok...Since I'm not a programmer and somewhat of a noob with respect to WinCE, I am NOT constrained by what might be a logical limitation to a programmer or someone in the "know". So, below is my thinking ... outside of the box (er, PDA) style.
But first, I need to grasp further Buzz's post above about the locations of things in ROM. Does this mean that there isn't, for lack of a better example, a FAT for the ROM? That every individual thing is expected to be at some specific, static location that can't change? If so, I see your point. But that tells the system that this location is static? Also, doesn't that make bug fixes difficult as the fix might exceed the designated space in ROM? Or do bug fixes always get re-applied after each reset? Patching in memory is how I'm looking at this. (Remember, I'm a non-programmer...deal with me, please.) LOL
Ok..out of the box time.
1. Can a shim be used to capture calls to a certain location in the ROM and redirect to another location (within the ROM or even the storage card. Linux builds anyone?). Like what HD disk managers did to get around LBA limitations or even boot sector viruses did.
2. Can an emulator be used to simulate/backup/build/test the ROMs on a PC before wiping the PDA? Don't laugh, but if not, could something be done with Virtual PC? Oh..Ok, laugh.
(Should I go hide somewhere?)

Flash storage usage??

I have searched the forum but could not find something of relevance.
It might sound stupid and maybe there is an easy explanation, but what I just don't understand is the following:
TyTN specs say 128meg ROM, 64meg RAM.
Looking at Start->Settings->Storage it tells me (translating from German):
File storage total: 56.22 megs
Program storage total: 48.80 megs
Then the usual stuff like 'used' and 'available'.
Of course there is the ExtendedROM which shows a total of 9.57 megs.
Where's the 128 megs gone or what am I missing here??? :roll:
Thanks for your help.
Cheers,
StonyA

the operating system basically. a perfectly intact copy is held in rom so that you can reinstall at any point using hard reset, this image is what you flash when you update your phone (same goes for your radio protocol).
hope this helps

herman3101 said:
the operating system basically. a perfectly intact copy is held in rom so that you can reinstall at any point using hard reset, this image is what you flash when you update your phone (same goes for your radio protocol).
hope this helps
Click to expand...
Click to collapse
Hello herman3101,
Thanks for your reply! Still a bit confused though. I understood the files stored in the "ExtendedROM" to be those who are used for reinstall.... ???
Even though, why doesn't it show Total ROM 128megs and xxx used???
I guess, I am still something missing :?
Cheers,
StonyA

The extended rom contains the customizations and patches that your carrier decides to put on, this is the way that different roms vary between carriers. The extended rom does NOT contain the operating system, this in a seperate hidden region of your rom (note that rom in this case has two variations of a meaning, one refers to the physical memory on your phone, the other refers to the image that is placed into that memory)
Think of that part of your memory as being an image of your windows xp cd rom, for normal day to day running its not needed as all the files it needs recide on your hard disk but when you come to reinstall you need to work of the cd. Same principle applies to that part of the rom. apologies if my analogy only makes sense to me, I have a habit of doing that.

Hey herman3101,
Many thanks for your explanation. Yes it does make a lot of sense what you are saying. Still don't understand why HTC does not openly show that, i.e. 128 megs rom, xxx megs used, xxx megs available.
Cheers & great weekend,
StonyA

Hidden ROM capacity???

While bussy with testing Extendir V2, I've noticed somthing strange...
Our HD2 has 3 FATFS volumes???
We have 1 IMGFS volume where our cooked stuff is,
1 FATFS volume as our root storage,
1 FATFS volume as our storage card
and 1 FATFS volume as our ....?
Could it be extra storage memory that isn't show? (maybe with the correct HSPL?)
Does our storage memory exists from 2 volumes of 256MB?
I realy don't know, but I lack the time to get my hands dirty on this...
You could check it yourself with for example FdcSoft's Task Manager.
ShaDrac

I'm trying right now
what i've to exactly search?

could it be this hidden 20mb as discussed HERE?

Fab985 said:
I'm trying right now
what i've to exactly search?
Click to expand...
Click to collapse
In FdcSoft's Task Manager, on the tab "Processes",
if you press and hold on filesys.exe and select details on the submenu,
you will see all the libraries loaded bij filesys.exe.
In the case of file system filter drivers an library will be loaded once per volume that it's hooked on (thats the number in the column "Proc Count").
As you can see for example: cachefilt is 3 times loaded while cachefilt is only loaded on FATFS volumes

2 screenshots here

samsamuel said:
could it be this hidden 20mb as discussed HERE?
Click to expand...
Click to collapse
Yes, I think you're right!
Next time I will try to search on more keyword combinations.
Still it would be nice to know for what it's used and when/how the volume was created.

ShaDrac said:
Next time I will try to search on more keyword combinations.
Click to expand...
Click to collapse
Heh, i couldn't find it either, i had to trawl through my past post history. Good job i commented in that thread !

ShaDrac said:
While bussy with testing Extendir V2, I've noticed somthing strange...
Our HD2 has 3 FATFS volumes???
We have 1 IMGFS volume where our cooked stuff is,
1 FATFS volume as our root storage,
1 FATFS volume as our storage card
and 1 FATFS volume as our ....?
Could it be extra storage memory that isn't show? (maybe with the correct HSPL?)
Does our storage memory exists from 2 volumes of 256MB?
I realy don't know, but I lack the time to get my hands dirty on this...
You could check it yourself with for example FdcSoft's Task Manager.
ShaDrac
Click to expand...
Click to collapse
I think is bluetooth storage
there is an option somewhere that you can set it and you will see it on the file manager

iosub said:
I think is bluetooth storage
there is an option somewhere that you can set it and you will see it on the file manager
Click to expand...
Click to collapse
It is used by Facebook_Engine package:
\PINET_ROM\FBOnlineAlbum\

Can we access data in this volume??????
djet said:
It is used by Facebook_Engine package:
\PINET_ROM\FBOnlineAlbum\
Click to expand...
Click to collapse
Hi would you please tell me how can i access these files as I need to extract few pictures from it ..... only coz they are here only..

[SOLVED] Is there a max number of modules to be cooked in EVK? [ROM Stucks]

hello chefs
yet another question from me
i use ervius kitchen to make blackstone roms
we dont have official 6.5 kernel bt we do have one from da_g wich works real nice
my problem is the kitchen is setup to have 650 modules
is there a way to update that number to say like 700
cause i have 649 modules
i just tried to update some packages and i came to 656 wich gave me a rom that hangs on the screen ''preparing youre device first use bla,bla,bla''
had this a couple of times and i thought upping the modules the rom can handle to 700 is easier then looking to refille some modules as htc is making more modules wich every new rom build:-o
thansk in adnavce for any answers

I dunno if your Blackstone is the same as the one in this post , but i would search better in future

af974 said:
I dunno if your Blackstone is the same as the one in this post , but i would search better in future
Click to expand...
Click to collapse
thansk for the answer but i think you misunderstood me
i already use the 6.5 kernel but its limited to 650 modules
my question is is there a way so it can handle more modules?

miniterror said:
thansk for the answer but i think you misunderstood me
i already use the 6.5 kernel but its limited to 650 modules
my question is is there a way so it can handle more modules?
Click to expand...
Click to collapse
I have 685 already in my ROM.

The actual module count doesn't matter; it's the memory allocated to Slot 0 that matters. If you have too much allocated for modules (maybe 10 MB or so), then you'll run into out of memory issues. You need to check your memory map in your build-log to see how many modules (if any) are allocated to slot 0. Your problem may be coming from somewhere else.
I don't think I'm the only person who's confused by this:
we dont have official 6.5 kernel but i think this doesnt really matter
Click to expand...
Click to collapse
followed by this:
i already use the 6.5 kernel but its limited to 650 modules
Click to expand...
Click to collapse
If you have a 6.5 nk, maybe you should edit the first post. It doesn't make much sense.

sorry my fault
gonna edit the first post
forgot to say we dont have a official 6.5 but we do have a 6.5 nk from da_g like posted above
as for the buildlog i only have one with 649 modules and from what i can see 0mb is located
i would ahve to check with the dll's recmodded again to see what happens then
for now i will attach my buildlog maybe you guys can see something strange besides a lot of overwriting
Memory Map...
SLOT 0: 0x02000000 - 0x018b0000 (END: 0x00060000, 0 MODULES)
0x02000000 - 0x01fc0000 - ROM 0
0x01f80000 - 0x018b0000 - ROM 1
SLOT 1: 0x04000000 - 0x02021000 (END: 0x02020000, 213 MODULES)
SLOT 60: 0x7a000000 - 0x78da0000 (END: 0x78020000, 57 MODULES)
SLOT 61: 0x7c000000 - 0x7a022000 (END: 0x7a020000, 228 MODULES)
RAM IMAGE: 0x80000000 - 0x803e6560
RAM: 0x803e7000 - 0x80475000 - Used for kernel modules
0x80475000 - 0x83000000 - 43 MB free
edit: and a buildlog with 656 modules wich hangs at the screen ''preparing youre device for first use bla,bla,bla''
the same packages refilled works but still i think i will short of modules soon with one left

You have a module that must be recomdded then.
I would suspect NtfConfig.dll in the Notification Enhancement package then.
That one is a known issue that hangs the ROM at that screen.
RecMod that file and then you should be good.

rgb-rgb said:
You have a module that must be recomdded then.
I would suspect NtfConfig.dll in the Notification Enhancement package then.
That one is a known issue that hangs the ROM at that screen.
RecMod that file and then you should be good.
Click to expand...
Click to collapse
that is refilled
the only thing in there not recfilled is NotificationRes.dll
and in my other notificationenhancement that one isnt refilled either:s
besides notification enhancment there is coming in the kitchen as new packages htcmessage, resourceproxy, text selection and usb to pc pop up
edit:also when i open up erviuskitchen and load a rom and press other
the section max numbers of modules is set to 650
i tried to put it to 675 but also the same result
edit2: lol i forgot to attach the buildlog:S
smoked a little to much i think
will make a new one as i just deleted my trashcan

mmmm it seems that your new PKGS has something wrong.. maybe a bad module or a bad entire PKG or regs try to undo the changes you have done since your roms works, then try to add one pkg by one
Mod edit: And yes the thread name has no sense.... iw ill change it

Only thing I could advise is:
- Go back to the situation it did work building ROMs and flashing it
- Update packages one by one, building and flashing making sure it boots
At the moment you have a non-bootable ROM you'l know what packages makes the ROM unbootable.
I know it takes a lot of time, but I think it's the only way to troubleshoot this issue.

i dont htink there is a max, it just increases the file size, and as long as ur not over the internal memory size, ur fine

The other thing you could do is just post the group of packages (and/or just the modules) that you include that result in a non-booter. Someone here might be able to point out the problem. Also, check your build-log for any errors or over-writes. Post that, too.

+ Que PPC said:
mmmm it seems that your new PKGS has something wrong.. maybe a bad module or a bad entire PKG or regs try to undo the changes you have done since your roms works, then try to add one pkg by one
Mod edit: And yes the thread name has no sense.... iw ill change it
Click to expand...
Click to collapse
sorry for the wrong name
english isnt my native language and try to type the best i can
as for th rom not booting it isnt treu
with the new packages i can get a booting rom
all i have to do is refille modules in those packages so i get under the 650 and the rom wil start
i also tryed with a few refilled to 651 and again it hangs at the screen where it says preparing youre device for first use
EpiphanyNL said:
Only thing I could advise is:
- Go back to the situation it did work building ROMs and flashing it
- Update packages one by one, building and flashing making sure it boots
At the moment you have a non-bootable ROM you'l know what packages makes the ROM unbootable.
I know it takes a lot of time, but I think it's the only way to troubleshoot this issue.
Click to expand...
Click to collapse
look above
as i wrote there the rom can boot if i put down the modules to files and go under 650
SiL3nTKiLL said:
i dont htink there is a max, it just increases the file size, and as long as ur not over the internal memory size, ur fine
Click to expand...
Click to collapse
Farmer Ted said:
The other thing you could do is just post the group of packages (and/or just the modules) that you include that result in a non-booter. Someone here might be able to point out the problem. Also, check your build-log for any errors or over-writes. Post that, too.
Click to expand...
Click to collapse
as for the size i think i have enough spare
i module isnt much bigger in size then the dll file itself or is it much bigger?
just checked mine and it says i have 298.89 mb total storage memory
40.70 mb in use of storage memory
and 258.19 mb free of storage memory
for program memory
total 191.05 mb
in use 125.45 mb
free 65.60 mb
as for a new buildlog i will make one when im back at my own computer and post the packages that wont work when they are moduled(they come this way by the dump program)
anyway thanks again for all the help trying to sort this out

as said here in this post i will attach the ext packages i want to add like they are default
the way they are posted here is the way i get them from the dumper and never had problems before with the dumped packages
this way i have 656 modules and the rom hangs at preparing youre device for first usebla,bla,bla
if i refille all modulles in those packages i get a working rom
with the modules refilled i have 649 modules
also attached the buildlog from the booting and the non booting rom
thansk again for looking

miniterror said:
as said here in this post i will attach the ext packages i want to add like they are default
the way they are posted here is the way i get them from the dumper and never had problems before with the dumped packages
this way i have 656 modules and the rom hangs at preparing youre device for first usebla,bla,bla
if i refille all modulles in those packages i get a working rom
with the modules refilled i have 649 modules
also attached the buildlog from the booting and the non booting rom
thansk again for looking
Click to expand...
Click to collapse
Your prob is here:
Code:
...
Processing "dump\NTFConfig.dll" as module
...
You did reversmode also NTFConfig.dll

Cloudyfa said:
Your prob is here:
Code:
...
Processing "dump\NTFConfig.dll" as module
...
You did reversmode also NTFConfig.dll
Click to expand...
Click to collapse
verry verry strange
now i had a booting rom with 655 modules:-o
only ntfconfig refilled and the rest in the packages have been modules
dont know why its works now as i tryed it with only that one refilled and the rom hang(i know that could be the bugger but with that one refilled it didnt work either so i thought it whas the modules not possible of going over 650)
maybe a bad rom build that attempt
thanks to all of you guys for helping me

That is a known issue in EVK with that particular module. Drove me about nuts until some kind souls on this forum helped.

jwzg said:
That is a known issue in EVK with that particular module. Drove me about nuts until some kind souls on this forum helped.
Click to expand...
Click to collapse
well i know that one could give trouble as i have read it many times on xda(thanks for the chefs posting this troublemaker) but when i tried the new packages with only that one refilled i had the same problem so i thought it whasnt caused by it
after that non bootable rom i refilled everything and then it booted
when i checked the buildlog i saw that 650 being low
so i tried another one with that one refilled again and the rest as modules and i whas coming over the 650:-o(and again no bootable rom:S)
think those made broke during flash or build(i have to say my cable looks reall crap with the ends a little broken open:-o

rgb-rgb said:
You have a module that must be recomdded then.
I would suspect NtfConfig.dll in the Notification Enhancement package then.
That one is a known issue that hangs the ROM at that screen.
RecMod that file and then you should be good.
Click to expand...
Click to collapse
miniterror said:
that is recmodded
the only file in the not recmodded is NotificationRes.dll
and in my other notificationenhancement that one isnt recmodded either:s
besides notification enhancment there is coming in the kitchen as new packages htcmessage, resourceproxy, text selection and usb to pc pop up
edit:also when i open up erviuskitchen and load a rom and press other
the section max numbers of modules is set to 650
i tried to put it to 675 but also the same result
edit2: lol i forgot to attach the buildlog:S
smoked a little to much i think
will make a new one as i just deleted my trashcan
Click to expand...
Click to collapse
I will re quote this message cause seems the first mistake, Glad you have sort it out, thanks to the guys helping Cloudyfa JW and many more this kind of details can be forgoten using Oskitchen cause it recmods automatically many modules to void errors.
Post at the same time... its weird, maybe you have some rewrites

+ Que PPC said:
I will re quote this message cause seems the first mistake, Glad you have sort it out, thanks to the guys helping Cloudyfa JW and many more this kind of details can be forgoten using Oskitchen cause it recmods automatically many modules to void errors.
Post at the same time... its weird, maybe you have some rewrites
Click to expand...
Click to collapse
seems like for this kind of trouble i will need to use google translate
major errors in the way the text is set up
sorry bout that

Help with increasing phone internal storage / Data.img size

I am currently running Superram 1.5 and am trying to increase the "internal" storage of the phone. I have skewered the forums and I understand this involves increasing the Data.img size beyond the initial 256mb compaction. Using TopoResize I have resized my data.img to 1.9 GB, however, i still had access to only 256 MB. I reinstalled Superram 1.5 and tried again with a fresh data image and also failed.
Can someone point me in the right direction, please?
Is there an XML i am to edit or something so the system recognizes the change?
App2SD just isn't cuttin it.
Much thanks to anyone who can help.

Well once the data img is used I don't think you can simply just enlarge it, it has never worked for me that way.. The only solution I can think of, is for you to get a new data image, a bigger one than the one you currently use, and restart your rom with that data img. You can find lots of Build threads with blank data images in them, eg. JDMS/MDJ. If you want to back up stuff, use titanium backup from the market.

data image resizing
Firstly, that should read "scoured" lol -not skewered. And "compaction" should read capacity. Sorry, was trying out swype with XDA App.
Now, more impotently. When you say "restart your rom," you mean to just substitute one image file for another in windows and restart superRAM, correct?
I will not have to reinstall SuperRAM 1.5?
Also, since superram runs off the RAM of the device, will using a image file from a different build cause a problem?
Finally, has anyone out there increased there Image file in size via TopoResize and had success? Was it before the initial launch of your android build or after?
Any help would be much appreciated. Thanks all

Kailkti said:
Well once the data img is used I don't think you can simply just enlarge it, it has never worked for me that way.. The only solution I can think of, is for you to get a new data image, a bigger one than the one you currently use, and restart your rom with that data img. You can find lots of Build threads with blank data images in them, eg. JDMS/MDJ. If you want to back up stuff, use titanium backup from the market.
Click to expand...
Click to collapse
I big to differ. It can be increased. I used to do that during my android SD days.
@OP, you haven't scoured "a lot" yet. Search for datai.img maker, it can resize and create data.img.
Sent from my HD2 Droid HD using XDA App

Issue resolved
Issue resolved.
In the interests of others who may encounter this problem, here is the solution.
Simply get titanium backup and back up all your apps and info, then find a 2GB image from another build and substituted the old image file (data.img) with the new image file (renaming in appropriately) and used titanium backup to restore mostly everything. Titanium backup does a very good job restoring, even Astrid tasks returned, yay.
I originally didn't what to attempt this because SuperRAM is a unique build in the way it runs but there seems to be no issue.
One thing did come up, however.
I attempted to use a 4GB image as a replacement at first and that did not work.
Something about it being too large.
I read someplace that android has a difficulty with image files larger then 2 GB, can someone confirm?

Resize
@Jose - I have tried using toporesize, It seems to increase the size of the data.img but only allow me to use the original 256MB. End result is ineffective.

thedarklion said:
@Jose - I have tried using toporesize, It seems to increase the size of the data.img but only allow me to use the original 256MB. End result is ineffective.
Click to expand...
Click to collapse
Sorry but I did not suggest toporesize but data.img maker.
Sent from my HD2 Droid HD using XDA App

jose makalolot said:
Sorry but I did not suggest toporesize but data.img maker.
Sent from my HD2 Droid HD using XDA App
Click to expand...
Click to collapse
Easy enough mistake to make, but toporesize is data.img maker. To OP (and for posterity's sake), I don't know if you tried it but if you tried making a new data.img using toporesize/data.img maker, you "should" have achieved about the same effect. Glad you got this solved though.

Hey Guys. How are you toady?
I come here with new problems! Probably this thread is solved but i got problem with Toporesize.
Anyway I solved why my Android crash all the time! Was the sd and now I installed few games but I will be glad to install Chainfire also but no way to go true because I do not have storage enough.
I try to use toposize to resize my data.img (inside i have many app and I did a backup) but I got this error, see pic.
If I make the fresh data.img no problem, but the Android do not turn on with "new" data.img, i looking just to resize mine without compromise it.
someone know how to do it?