Database Users

mkrom: make your own ROM for your device

Using 'mkrom', you can create your own ROM, including your own files, using a new boot splash image, changing cold-boot registry settings to match your own taste.
Click here

Thanks for your work

Peter Poelman said:
Using 'mkrom', you can create your own ROM, including your own files
Click to expand...
Click to collapse
Can this be used to add files to a WM2003 ROM ?? Or is it very version specific. Are there tools to split out the contents of a WM2003 ROM yet ?

working on it.
dumprom can extract files from 4.x roms with the '-4' commandline option.

i tried running dumprom with -4 option and nothing is extracted for some reason.
is this the right command? dumprom nk.nbf -4 files >nkinfo.txt ?
thanks
alex

XDA developer Itsme said:
working on it.
dumprom can extract files from 4.x roms with the '-4' commandline option.
Click to expand...
Click to collapse
OK, any doc's in progress - is the ROM image well documented somewhere on the web ? Just downloaded the SDK from MS ...

A Little help please? I think I'm close to getting mkrom to work for me.
Here's the output for my mkrom command (the result is the same if I use my Nb1 or Nbf)
$ ./mkrom.sh test.nb1
write xip block starting at 81800000, with 13 files
write xip block starting at 819c0000, with 198 files
./mkrom.sh: line 69: file: command not found
316+0 records in
316+0 records out
013c0000 added mainrom
0+1 records in
1+0 records out
013d0000 added xipchain
60+0 records in
60+0 records out
017c0000 added MISC
14+1 records in
15+0 records out
018b0000 added xda1
153600+0 records in
2+1 records out
018e5800 added bitmap
1+1 records in
2+0 records out
01980000 added operator rom
16+1 records in
17+0 records out
01dc0000 added xda2
0+0 records in
0+0 records out
01ec0000 added end
rommap: 80000000-80040000, 80040000-81f00000
80000000 - 80040000 -- bootloaer 0 files 1 modules
80040000 - 8015df08 -- kernel 5 files 5 modules
80180000 - 80376ef0 -- kernel 10 files 14 modules
80380000 - 8064306c -- kernel 20 files 36 modules
80670000 - 80be66a8 -- kernel 107 files 88 modules
80c00000 - 8102ce98 -- kernel 11 files 36 modules
81050000 - 813ef114 -- kernel 95 files 44 modules
81400000 - 814019a4 -- xip chain 10 xip entries
815f0000 - 8171bc7c -- kernel 56 files 19 modules
81800000 - 818e0c14 10 XDA_DEVELOPERS1 13 files 0 modules
81900000 - 81925800 -- bitmap : 424d4858 .. f9fff9ff
81940000 - 8198b6e5 -- operator rom 20 files
819c0000 - 81de4928 11 XDA_DEVELOPERS2 198 files 0 modules
e59d3134 - e5933004 -- kernel -509550577 files -44373180
0 modules
[email protected] /cygdrive/c/perl/bin
$
It loks like there;s one error at line 69? What is that?
The process actually produces an nb1 and nbf file. When I upload the nb1 to my sd card with xdatools then into the xda it all looks good until the cold reboot. Then it comes up with a black screen(not the default xda boot.img) with the 4.00.21 and right radio stack version in the bottom right corner. It never gets past that. I upload my backup 4.00.16 and all is back to normal.
I think I'm on the brink of getting the mkrom to work. Does anybody have any suggestions? Thanks - Jim

Well, I got it to work for a 3,17 ROM now I'm looking fopr advice on the 4.x ROms. Here's the link to my post. Thanks - Jim
http://forum.xda-developers.com/viewtopic.php?t=4450&highlight=mkrom+sh

it is advisable to use a more recent version of mkrom, which can be found at http://www.xs4all.nl/~itsme/projects/xda/romtools.html

Thanks for the pointer. I'm using the scripts from there now and having a few problems. The splitrom.pl is finding an overlap in it's checkforoverlap subroutine and exiting at that point.
I kow that could be any of a thousand things but is there any one or two things it's more likely to be? Thanks - Jim

you should be able to see from the output what memory ranges overlap.
most likely it is the operator rom, or the bootsplash image.
you have to put the correct parameters in the params file.

kalex said:
is this the right command? dumprom nk.nbf -4 files >nkinfo.txt ?
Click to expand...
Click to collapse
Not with the versions I am using. Use this:
dumprom nk.nbf -4 -d files >nkinfo.txt
LD

I just can not get the default.fdf right.
I used dumprom on 4.00.xx rom. tried 05 11 16 21.
1. download the latest dumprom
2. used -4 option
3. tried both on windows and linux
but the extracted default.fdf is only about 17k large which is not right.
any hint here?

well, itsme pointed out the default.fdf should be about 17k large.. However, still get into trouble.
downloaded the romtool, use 4.00.11 rom image. followed the instruction. I could cook up a good rom without putting any files in the "files" directory. xda works fine
however, when I put in about 3M files in "files" directory. all the rest not modified, XDA won't boot up with the cooked rom. froze at the screen shows the image and the rom version and radio stack version number.
I got from the kitchen that I can put 3906k files inside this rom. is that right?

pine said:
I got from the kitchen that I can put 3906k files inside this rom. is that right?
Click to expand...
Click to collapse
This depends on the settings for mkrom.
What settings do you use?
Stefan

what kind of setting? I assume the params file
I used
wincever=4
start1=81740000
size1=00040000
start2=81b00000
size2=003c0000
startbmp=81ec0000
startop=81b00000
otherwise, I don't remember there is any place for setting. I used 4.00.11 rom, which i suppose does not include the t-mobile stuff

Hi all (sorry for my english)... First post
I dont want to seem stupid but I want to know thomething...
I've got a SPV 1G and want to know if there is also a ROMCoocker (mkrom) for Smartphone (WinCE 2k2/2k3) devices ?
Thx for your participation
Sidarus

Ping? :mrgreen:

create or rename into my own extended rom version name.. hel
can any1 help me how to create own version name for my own cooked rom? tnx! id like to customize my version into my own numbers.. tnx!

hi i have tryed to get the rom to work but just need to first 3 steps to get me on the right path :? thanks guys

DIY your ROM

Hi,
I write some code that can modify the ROMs, it can save your time to add and delete files by hand.
RomMaster V2.0 Beta
Usage: RomMaster [options] imagefile
-d[m] <dfile> - delete file
replace file/module together with -a option
'm' delete module, deleting module isn't suggested
-a[c] <afile> - add file into the rom
'c' means use compress(need CECompressv4.dll)
-o <ofile> - output imagefile name
-v <0~9> - print info, 0 detail, 9 only show errors, default is 5
-w <5> - 5 is 2005, default is 2003&SE
-x - only save XIP(OS) data
-s <0x...> - Fix XIP start address(Hex)
-e <0x...> - Fix XIP end address(Hex)
In replace mode, 'c'&'m' is useless
It is now 2.0 Bata Release.
You can delete file/modules you don’t like from the ROM.
RomMaster –d “filename” –o “newROMname” “ROMname”
You can add files into the ROM.
RomMaster –a “newfilename” –o “newROMname” “ROMname”
You can replace file in the ROM
RomMaster –d “filename” –a “newfilename” –o “newROMname” “ROMname”
“newfile”’s size should be the same or small than the file you want to replace, new file will occupy the same space as the old one.
I test some ROM in my SP; include SDA, Dopod 575 & 585. I only tested one 2005 ROM. Replace module may don’t work, I am still working on it.
Before you burn the image generated by the tool, make sure you finish follow step:
1. RomMaster –o “TestROM” “SrcROM”
2. Do binary compare “SrcROM” with “TestROM”
a) If they are 100% same, I think you can safely use this tool.
b) If they are 99% same, you should be careful, make sure you only burn the OS part. Because some ROM are modified by someone before, there are maybe some useless data in the ROM, only burn the OS part won’t damage your SP.
c) Else, the “SrcROM” may contain some unknown structure or data, the “TestROM” may won’t work, don’t try burning it into you SP. If you want to modify it, tell me where I can find the ROM, if I am free, I can give some help.
3. I only tested one 2005 ROM, its structure isn’t very correct, and I think that ROM is extracted form emulation ROM. So if 2005 ROM isn’t 100% same, don’t try and be careful even they are 100% same.

This is great!!!
Going to try it!

ncruz,
I'll wait for your experience, cause if this is working we can all save space by directly burning upgraded cameras etc into the rom. will save me at least 1MB ram or storage.
The tools sounds great gmap.

There already exists MKROM tool - http://www.xs4all.nl/~itsme/projects/xda/romtools.html
it is 100% working with WM2003/2003SE devices. But it is rather inconvenient.
I'll test your "-w 5" option on a real device. Real WM5 device has one XIP kernel section with only few modules and about 1Mb free space. All other data is kept in IMGFS partition, I'm currently working on a tool that would work with it.
And one question. When you add new files to ROM, do you add them to a new XIP or extend the existing XIP? And when you delete modules, do you reuse the freed space after adding new ones?

mamaich said:
There already exists MKROM tool - http://www.xs4all.nl/~itsme/projects/xda/romtools.html
it is 100% working with WM2003/2003SE devices. But it is rather inconvenient.
I'll test your "-w 5" option on a real device. Real WM5 device has one XIP kernel section with only few modules and about 1Mb free space. All other data is kept in IMGFS partition, I'm currently working on a tool that would work with it.
And one question. When you add new files to ROM, do you add them to a new XIP or extend the existing XIP? And when you delete modules, do you reuse the freed space after adding new ones?
Click to expand...
Click to collapse
That's great if we can modify IMGFS partition I am waiting for it.
I know that tool and i don't know how it works. I made this tool only for interesting.
You can find XIP chain in 2003 ROM, by XIP chain, you can know the address and length of each XIP section. I will scan the hole XIP region before inserting the new file to reduce memory fragment. When a module is deleted, its space will be freed and reused when adding files. I freed about 6M space in my own ROM by deleteing the useless files, and add about 5M files into it, it works OK.
It seems 2005 don't have XIP chain information in the ROM, i only test one 2005 ROM, and i didn't find the XIP chain info. If your 2005 ROM don't have XIP chain info too, you should modify ROMHDR.physlast to a correct value by hand. Because if i can't find the XIP chian info, I use ROMHDR.physlast to decide the end address of XIP. Or, there are almost no space for you to add new file. My 2005 ROM physlast=0x8c253278, and only about 78732 bytes free before 0x8c253278.
I update the the tool V2.2 , fixed a bug when deeling with MDA(818) ROM.

gmap said:
That's great if we can modify IMGFS partition I am waiting for it.
Click to expand...
Click to collapse
I've PMed you a test version. I'll make it available to public later.
... If your 2005 ROM don't have XIP chain info too, you should modify ROMHDR.physlast to a correct value by hand.
Click to expand...
Click to collapse
My ROM has all needed info, I had to extract everything from rom image after 1C0000 address to a separate file and gave it to your tool. It is working perfectly. I've managed to delete and add a new file to ROM. I have not tested "-dm" option. It seems that all modules/files in XIP section of WM5 are uncompressed. I'm using BlueAngel's WM5 ROM. Later I'll try to replace boot.hv file with my own version.
Can you add a switch to your program "-s bytes" so that it woud skip the given number of bytes from file start, so it would be possible to work directly on NBA files with header?

gmap, can you tell me what file i can use this with? Is it for nbk or nba files? Thanks

I'm not able to edit the xip sextion ...
i've tried your tool on 1.60c.07CHS rom for xdaII :
RomMaster.exe -w 5 -x -o test.bin nk.nba
result :
[Info] It is a common ROM.
[Error] File is damaged, end address small than start address.
[Error] File is damaged, end address small than start address.
RomMaster.exe -w 5 -x -o test.bin imgfs_raw_data.bin (created with mamaich's tool)
result :
[Info] It is a common ROM.
[Error] Load nb00 failed.
RomMaster.exe -w 5 -x -o test.bin img.bin (created by nba part 1C0000 to end)
result :
[Info] It is a common ROM.
[Error] File is damaged, end address small than start address.
[Error] File is damaged, end address small than start address.
How to save the XIP section ?

TofClock said:
RomMaster.exe -w 5 -x -o test.bin imgfs_raw_data.bin (created with mamaich's tool)
Click to expand...
Click to collapse
This would not work. My tool works with IMGFS and you need to edit XIP

and ... how to edit XIP ?

mamaich said:
My ROM has all needed info, I had to extract everything from rom image after 1C0000 address to a separate file and gave it to your tool. It is working perfectly. I've managed to delete and add a new file to ROM. I have not tested "-dm" option. It seems that all modules/files in XIP section of WM5 are uncompressed. I'm using BlueAngel's WM5 ROM. Later I'll try to replace boot.hv file with my own version.
Can you add a switch to your program "-s bytes" so that it woud skip the given number of bytes from file start, so it would be possible to work directly on NBA files with header?
Click to expand...
Click to collapse
mamaich, or anyone else, what did you use to extract after 1C0000?
splitrom?
any help would be greatly appreciated.
Russ

Wow , just an hex-editor like winhex or edithexa

i tried that w/xvi32, and it didn't seem to work. I guess I'll try again

Thx gmap for this great tool. 8)
Recently I want to replace the TureFFS.dll at the XIP area of Wizard OS ROM.
ftp://xda:[email protected]/Uploads/HTC_Wizard/Roms/Qtek/Qtek_9100_1_6_7_ENG__OS_ONLY.zip
I type the following command and it says that it cannot replace the file.
Code:
D:\>RomMaster.exe -d TrueFFS.dll -a TrueFFS.dll -w 5 -o new.nba os.nba
[Info] It is a common ROM.
[Warning] o32_rom(0x8c268ef8)'s o32_data at 0x00000000 is zero.
[Warning] Found dif-referenced region [OLD] Address=0x8c1e1290 Length=0x00000800 ObjectType=0x00200000
[Warning] Found dif-referenced region [New] Address=0x8c1e1290 Length=0x00000800 ObjectType=0x00008000
[Warning] Found dif-referenced region [OLD] Address=0x8c1fe538 Length=0x00000600 ObjectType=0x00200000
[Warning] Found dif-referenced region [New] Address=0x8c1fe538 Length=0x00000600 ObjectType=0x00008000
[Warning] Found dif-referenced region [OLD] Address=0x8c211018 Length=0x00000a00 ObjectType=0x00200000
[Warning] Found dif-referenced region [New] Address=0x8c211018 Length=0x00000a00 ObjectType=0x00008000
[Warning] Memory Block(0x8c101000,0x8c1510ac) overlap with Block(0x8c10288c,0x8c1028b8).
[Error] You want to get an object whose size is 108, but it is 112. RomOffset=0x8c268e00
[Error] Could not replace 'TrueFFS.dll' with 'TrueFFS.dll'.
However, I can successfully replace the boot.hv file. Do you know why?

ahlok_hk said:
Thx gmap for this great tool. 8)
Recently I want to replace the TureFFS.dll at the XIP area of Wizard OS ROM.
ftp://xda:[email protected]/Uploads/HTC_Wizard/Roms/Qtek/Qtek_9100_1_6_7_ENG__OS_ONLY.zip
I type the following command and it says that it cannot replace the file.
Code:
D:\>RomMaster.exe -d TrueFFS.dll -a TrueFFS.dll -w 5 -o new.nba os.nba
[Info] It is a common ROM.
[Warning] o32_rom(0x8c268ef8)'s o32_data at 0x00000000 is zero.
[Warning] Found dif-referenced region [OLD] Address=0x8c1e1290 Length=0x00000800 ObjectType=0x00200000
[Warning] Found dif-referenced region [New] Address=0x8c1e1290 Length=0x00000800 ObjectType=0x00008000
[Warning] Found dif-referenced region [OLD] Address=0x8c1fe538 Length=0x00000600 ObjectType=0x00200000
[Warning] Found dif-referenced region [New] Address=0x8c1fe538 Length=0x00000600 ObjectType=0x00008000
[Warning] Found dif-referenced region [OLD] Address=0x8c211018 Length=0x00000a00 ObjectType=0x00200000
[Warning] Found dif-referenced region [New] Address=0x8c211018 Length=0x00000a00 ObjectType=0x00008000
[Warning] Memory Block(0x8c101000,0x8c1510ac) overlap with Block(0x8c10288c,0x8c1028b8).
[Error] You want to get an object whose size is 108, but it is 112. RomOffset=0x8c268e00
[Error] Could not replace 'TrueFFS.dll' with 'TrueFFS.dll'.
However, I can successfully replace the boot.hv file. Do you know why?
Click to expand...
Click to collapse
In replace mode, new file should not biger than old file. So, you should delete old 'TrueFFS.dll' first, then add the new one(You can delete some file next to 'TrueFFS.dll' to get free space or move it to a big free space, then modify the file size and related infomation of 'TrueFFS.dll' by hand to make it bigger). Replacing module isn't stable, I confused by some of the module data. If you want to replace a module, it may not work. I haven't find a way that can be used to calculate all of the data.

gmap said:
In replace mode, new file should not biger than old file. So, you should delete old 'TrueFFS.dll' first, then add the new one(You can delete some file next to 'TrueFFS.dll' to get free space or move it to a big free space, then modify the file size and related infomation of 'TrueFFS.dll' by hand to make it bigger). Replacing module isn't stable, I confused by some of the module data. If you want to replace a module, it may not work. I haven't find a way that can be used to calculate all of the data.
Click to expand...
Click to collapse
Thx for your explanation. Actually the new file is smaller than the one being replaced. And I just found that I can only delete those non-module files while all modules could not be deleted.
Thx again. Hope to see new version if you have time to find out how to calculate the data. :wink:

How can i extract a kernel file (kbbdrv.dll)?
thanks

dherrero said:
How can i extract a kernel file (kbbdrv.dll)?
thanks
Click to expand...
Click to collapse
dumprom tool
But if you'll extract this or any other XIP DLL and then readd it to the same, or any other ROM it would not work.

Hello,
I have a Apache Rom,
I would like to delete nk.exe are replace it.
iv tryed:
rommaster -w 5 -d nk.exe nk.nba
also
rommaster -w 5 -d nk.exe -a nk.exe nk.nba
keeps telling me i can delete that file.
just to let you know: if i use dumprom i get the boot partition files (containing nk.exe) and if i use imgfs i get all the os files (not containing nk.exe)
Any help would be great thanks

you cannot delete nk.exe, and you should not even need to do that.

Extracting files from a segment with SRPX signature, how?

I am trying to extend the bepe's kitchen in order to include support for Mio A701 and Mio A700 (Scoter platform). Some of you are already aware of it.
Our DOC architecture is quite simple:
- DOC's static RAM: G3/G4 Initial Program Loader
- DOC BDK0 Binary partition that keeps the Bootloader
- DOC BDK1 Binary partition that keeps the Microsoft Initial Program Loader (also called SPL over these forums, isn't it)
- DOC BDTL0 TrueFFS partition that keeps the WM5. This partition is exactly 50MB (0x3200000 bytes). It is a MSFLSH50 image containing a 0x400 bytes header followed by 4 subpartitions.
- DOC BDTL1 TrueFFS partition that keeps the user data in a FAT32 filesystem.
BDTL0 has 4 subpartitions:
- Part00 Starts at offset 0x400 inside the MSFLSH50 image. Unknown format, it has 'SRPX' signature at offset 0x40.
- Part01 Unknown format, it has 'SRPX' signature at offset 0x40.
- Part02 IMGFS segment.
- Part03 segment with an empty FAT16 filesystem used for padding the size of 50MB required for the BDTL0_MSFLSH50 partition.
I can extract everything but those files stored in Part00 and Part01. IMGFS can be easily extracted and built with the IMGFS_tools by Mamaich.
In HTC devices the kernel and critical drivers are stored in 2 XIP chains, but these files do not seem to be XIP chains since they are compressed or encrypted. Thew SRPX signature is not very common, Buzz Lightyear talked about it here:
buzz_lightyear said:
hi willem,
hmm... I know, it's a problem...
wm5 compression signature is 'SRPX' (as far as I remember coz i'm 1 month away from it). it's XPRS other way around. XPRS is some standard compression. I guess it is also included in cecompress.dll from CEPB5.
...just a thought... maybe a bit of help...
is it also used in smartphones with wm5?
thanx
buzz
Click to expand...
Click to collapse
After that no one else has talked about this kind of segments or SRPX signature.
If you want to take a look at the unknown segments/subpartitions of the MSFLSH50 WM5 image then you can download a dump of Part00 and Part01 from here.
I need to extract and insert files into this segments, can you help me with any related information about it please?
Thanks a lot,
Oki

Hi Oki,
where did yo dig that post about SRPX out, please )))
Anyway, i still have no info about that, but i'm wondering, what would you like to put inside...
Oki said:
Microsoft Initial Program Loader (also called SPL over these forums, isn't it)
Click to expand...
Click to collapse
))) it actually is SPL
buzz

It is nice receiving a quick answer here. I have already posted this in your site.
It seems that Microsoft calls the SPL as MS IPL. It does not matter, in the MiTAC world bootloader is known as UBoot and has a nice menu for selecting the part that you want to flash so we only need to create a customized MSFLSH50 image and that's all, the OS is upgraded.
I want to create a customized image for my device so I need to apply the certmod.dll patch described by mamaich. Any other solution?
The kernel file, some critical DLLs and boot.rgu among other important files are in those two segments, so in order to create a customized OS I will need to access these files and replace them.
Let me ask you where did you found the SRPX signature? Is there any other device with this image format?
Thanks,
Oki

Oki said:
It is nice receiving a quick answer here. I have already posted this in your site.
Click to expand...
Click to collapse
)))) maybe because i was on this site, when i've got notification...
But i first answered at buzzdev.net ))))) LOL
"Hi Oki,
so SRPX... )) i saw that very long time ago in some Himalaya WM5 ROM. i really can't remember, where exactly.
all i know is, that XPRS is a kind of compression, so i thought that time, that XPRS is actually SRPX other way around.
Then, as other things poped up, i somehow forgot about that totally ))
CU
buzz"

For Oki: SRPX signature found on ATOM LIFE
Hello Oki,
The XDA Atom Life has MSFLASH50 format as well as SRPX signature for the kernel part. I was wondering what is the start of the segment for the MSFLASH50...? I couldn't seem to get msflshtool.exe to work with this ROM. It keeps on saying not a MSFLASH50 format.
BTW, your Scoter Kitchen tools worked on XDA ATOM, we are trying to port the files from XDA ATOM LIFE into our ROM... Fortunately you have covered this format so we can extract its contents...
Jiggs

request for other srpx-tool
Hello, and sorry for digging in this old thread.
I have a XDA Comet aka Atom Life and the XIP is SRPX compressed like Jiggs described.
I'm trying to update the Kernel.
I use the SRPX tools from Scoter kitchen. With MSFLSHTOOL i get 2 XIP and 1 imgfs part.
I use SRPX2XIP for the second part and the XIP is 1728 KB.
If I change back with XIP2SRPX the new part is only 1442 KB.
So I write back this part to my ROM image and the image doesn't boot.
Is this an error from SPRX tools or did I miss something ?
I can't find an other tool for that job. Google gives only a hint to "sushi-repeat-containing protein" but i guess that's not the information i realy need.
May be someone can enlighten me.
Attached a link to Atom Life XIP (If someone is interested)
http://rapidshare.com/files/79622471/LifeXIP.rar.html

scorpio16v said:
Hello, and sorry for digging in this old thread.
I have a XDA Comet aka Atom Life and the XIP is SRPX compressed like Jiggs described.
I'm trying to update the Kernel.
I use the SRPX tools from Scoter kitchen. With MSFLSHTOOL i get 2 XIP and 1 imgfs part.
I use SRPX2XIP for the second part and the XIP is 1728 KB.
If I change back with XIP2SRPX the new part is only 1442 KB.
So I write back this part to my ROM image and the image doesn't boot.
Is this an error from SPRX tools or did I miss something ?
I can't find an other tool for that job. Google gives only a hint to "sushi-repeat-containing protein" but i guess that's not the information i realy need.
May be someone can enlighten me.
Attached a link to Atom Life XIP (If someone is interested)
http://rapidshare.com/files/79622471/LifeXIP.rar.html
Click to expand...
Click to collapse
Did you do a hex comparison between old and new XIP? you could try dumping and rebuilding first without modifications, and see the difference. vivi was able to sort this thing with his asus p525.

tjlabais said:
vivi was able to sort this thing with his asus p525.
Click to expand...
Click to collapse
Thank you for the hint.
After comparing the Comet-, the Atom Life- and the rebuilded file, I'll try to hexedit the beginning and fill the end of the rebuilded file to match the right filesize.
Will report later.
edit:
after simply cosmetical changes with a hexeditor the files are identical.

The ExtRom Research thread (Trinity/Hermes, maybe others)

While cooking my german GPS rom I noticed that there is very little information about the ExtROM nb format. Right now we can't extract it, we can't rebuild it, we can't resize the partition.
So I figured it was time to put some research into the matter. I made a package that contains the following:
- 04_ExtROM.nb (from RUU_Trinity_HTC_GER_1.23.407.2_103_6275_1.38.00.11_108)
- extrom_dump.raw (a dump of the extrom area after flashing it)
- Content (content of the extrom, copied from device after unhiding)
The idea is to analyse how files are written from NB to flash and how they are stored inside the NB. The format should be identical with Hermes and maybe other devices.
The goals of this research are:
1) understanding the Extrom NB format
2) making an extraction tool for getting files out of extrom.nb files
3) making a rebuild tool that allows us to make custom extrom.nb files
4) Resizing the Extrom partition
Please post your findings in this thread, you can also contact me on IRC (#xda-devs on irc.freenode.net
Here's the file:
http://rapidshare.com/files/24740192/ExtromResearch.rar.html

Here's what I found out so far (only worked on it for a few minutes):
R1:
The Extrom nb seems to hold 7 versions of each file, probably for 7 different languages. Search for PP_AKv33-DefaultPage_ and you'll find:
PP_AKv33-DefaultPage_FIN-040b.CAB
PP_AKv33-DefaultPage_WWE-0409.CAB
PP_AKv33-DefaultPage_WWE-0409.CAB
PP_AKv33-DefaultPage_RUS-0419.CAB
PP_AKv33-DefaultPage_FRA-040c.CAB
PP_AKv33-DefaultPage_GER-0407.CAB
PP_AKv33-DefaultPage_FIN-040b.CAB
Those occur in the 7 different config.txt files. Maybe those cabs are not all really in the NB, at the very least they must be very similar. Otherwise it wouldn't be so easy to compress. I found 7 occurences of several cab files in the NB by searching for the first few bytes of them.
R2:
The raw dump does not contain the config.txt files (at least I couldn't find them). Maybe config.txt gets stored elsewhere. I also didn't find the cabs in the dump so far, maybe a different format or a bad dump.
I had used
pdocread.exe -w -d EXT_FLA -p Part00 0 0xa00000 extrom.raw
R3: Only one of the 7 files in NB actually is actually in the ExtROM content.
My conclusion from R1 and R3:
The extrom.nb holds information for different languages or OS versions. Depending on some information only one of those actually gets flashed.

I'm finding it strange that the extracted ext_rom and the dumped ext_rom haven't the same structure.
I own a wizard (Qtek9100) and using Typho5 to extract it from the RUU and using podcread to dump it from the phone i always get a FAT16 image file. I can then use a program like Winimage to browse and edit it as i like.
Are you sure the extracted ext_rom is correct?

I believe I did everything correctly, yes.
Trinity and Hermes are different from Wizard but my dump also seems to be FAT16. If you have experience with it could you please see if the Wizard tools work on my dump?
R4: The filesystem used for ExtROM seems to be TFAT16 (Transaction-Safe FAT).
The NB files contains 90 TFAT16 occurences. I'll see if there are tools for viewing/editing TFAT16.

ZakMcRofl said:
I believe I did everything correctly, yes.
Click to expand...
Click to collapse
If you remove the first 0x1040 bytes from the nb file you get something looking like a FAT16 image, but still not working correctly. I wonder if the nbh decoder by itsme as a bug that produces a corrupted extension_rom?
ZakMcRofl said:
Trinity and Hermes are different from Wizard but my dump also seems to be FAT16. If you have experience with it could you please see if the Wizard tools work on my dump?
Click to expand...
Click to collapse
I did I downloaded your dump and used Winimage to check the raw file and it showed an empty confused FAT16 image
ZakMcRofl said:
R4: The filesystem used for ExtROM seems to be TFAT16 (Transaction-Safe FAT).
The NB files contains 90 TFAT16 occurences. I'll see if there are tools for viewing/editing TFAT16.
Click to expand...
Click to collapse
Winimage allows it. There's a tutorial by Faria on how to cook ext_roms and flash it back to wizards and winimage is advised
cheers

This post says otherwise (regarding TFAT16, not FAT16)
The simpliest method.
1. Take MS_.NBA (a decrypted version of MS_.NBF). Open it in any hex editor, and search for bytes "EB FE 90 4D 53 57 49 4E 34 2E 31 00" ("ыРMSWIN4.1", there would be "FAT16" string a bit lower). The string should be found near offset 0x70000. Extract everything starting from the place you've found and up to the end of file to a file named "extrom.img"
2. Open "extrom.img" in WinImage, edit it as you like, save the file
3. Open the hex aditor and place modified "extrom.img" to the same place in MS.NBA file where it was before extraction.
that's all. Convert NBF to NBA and flash your ROM.
The same method cannot be used on Universal. It has TFAT16 instead of FAT16, WinImage knows nothing about TFAT and destroys FAT table. But there is a simple workaround.
Click to expand...
Click to collapse
Source: http://forum.xda-developers.com/showpost.php?p=847312&postcount=10

Well, winimage works perfectly with wizard nb files (decrypted nbf files)

Yes, apparently Wizard uses FAT16 whereas Universal (and Trinity Extrom) use TFAT16. The former can be opened, the latter not.
I'm currently compiling a file list for further analysis.

ZakMcRofl said:
Yes, apparently Wizard uses FAT16 whereas Universal (and Trinity Extrom) use TFAT16. The former can be opened, the latter not.
I'm currently compiling a file list for further analysis.
Click to expand...
Click to collapse
Maybe that's why Wizard's extended roms get corrupted when users try to delete files in it. Perhaps WM5 or WM6 use TFAT16 upon rebooting and it
screws up the reading

Filelist
I took the content files and searched for their occurrences in extrom.nb.
Here is the filelist with hex positions:
Code:
0x0000A280 BT_Table.CAB
0x0001F3B0 Config.txt (FIN)
0x00020A08 HTC_WM5DST_signed.cab
0x000485D0 MP_CVSDcpl_20060920.cab
0x000685B0 PP_AKv30-DefaultPage_ALL.CAB
0x0007DCF8 PP_AKv33-DefaultPage_???.CAB
0x00093850 PP_ExtVersion.xml
0x000956C8 PP_FixITS2654_SMD.CAB
0x000A2800 BT_Table.CAB
0x000B7930 Config.txt (WWE)
0x000B8F88 HTC_WM5DST_signed.cab
0x000E0B50 MP_CVSDcpl_20060920.cab
0x00100B30 PP_AKv30-DefaultPage_ALL.CAB
0x0012B7B8 PP_ExtVersion.xml
0x0012D630 PP_FixITS2654_SMD.CAB
0x0013A768 BT_Table.CAB
0x0014F898 Config.txt (WWE)
0x00150EF0 HTC_WM5DST_signed.cab
0x00178AB8 MP_CVSDcpl_20060920.cab
0x00198A98 PP_AKv30-DefaultPage_ALL.CAB
0x001C3720 PP_ExtVersion.xml
0x001C5598 PP_FixITS2654_SMD.CAB
0x001D26D0 BT_Table.CAB
0x001E7800 Config.txt (RUS)
0x001E8E58 HTC_WM5DST_signed.cab
0x00210A20 MP_CVSDcpl_20060920.cab
0x00230A00 PP_AKv30-DefaultPage_ALL.CAB
0x00246148 PP_AKv33-DefaultPage_???.CAB
0x0025BCA0 PP_ExtVersion.xml
0x0025DB18 PP_FixITS2654_SMD.CAB
0x0026AC50 BT_Table.CAB
0x0027FD80 Config.txt (FRA)
0x002813D8 HTC_WM5DST_signed.cab
0x002A8FA0 MP_CVSDcpl_20060920.cab
0x002C8F80 PP_AKv30-DefaultPage_ALL.CAB
0x002DE6C8 PP_AKv33-DefaultPage_???.CAB
0x002F4220 PP_ExtVersion.xml
0x002F6098 PP_FixITS2654_SMD.CAB
0x003031D0 BT_Table.CAB
0x00318300 Config.txt (GER)
0x00319958 HTC_WM5DST_signed.cab
0x00341520 MP_CVSDcpl_20060920.cab
0x00361500 PP_AKv30-DefaultPage_ALL.CAB
0x00376C48 PP_AKv33-DefaultPage_GER-0407.CAB
0x0038C7A0 PP_ExtVersion.xml
0x0038E618 PP_FixITS2654_SMD.CAB
0x0050DE80 MP_CVSDcpl_20060920.cab
0x00527EE8 Config.txt (FIN)
0x005282F8 HTC_WM5DST_signed.cab
0x005C9298 PP_AKv30-DefaultPage_ALL.CAB
0x005DE9E0 PP_AKv33-DefaultPage_???CAB
0x005F4538 PP_ExtVersion.xml
0x005F63B0 PP_FixITS2654_SMD.CAB
R5: Files are stored sequentially for each language.
I haven't found where the offsets and how the offsets are stored, maybe relative to the beginning of a language section. I haven't found the absolute offsets anywhere yet.

[PORT] WP7 XPH compression for WM

XPH compression library ported from WP7 (Mango).
XPH is faster and more effective than XPR. It means, it is the best compression to date.
Compatible kitchen:
only the latest OSBuilder (>=15-Sept-2011 version).
Compatible devices:
only ARMv7 instruction set-compatible devices - all Snapdragon devices. OTHERS AREN'T SUPPORTED!!!
(older devices will not boot)
(no, no way to port to older devices)
Little comparison of libraries
In terms of size (from the worst to the best):
xpr -> xph -> lzx
In terms of speed (from the worst to the best):
lzx -> xpr -> xph
Click to expand...
Click to collapse
(C) Barin and UltraShot (asusmobile.ru team).
Special thanks to AndrewSh and HD2Owner for tests.

thanks alot buddy. great work.

Latest OsBuilder - means that one http://forum.xda-developers.com/showpost.php?p=17570707&postcount=480

Thanks a lot UltraShot !! It's a great news !! : ) : )

ultrashot said:
XPH compression library ported from WP7 (Mango).
XPH is faster and more effective than XPR. It means, it is the best compression to date.
only ARMv7 instruction set-compatible devices - all Snapdragon devices. OTHERS AREN'T SUPPORTED!!!
Click to expand...
Click to collapse
...sigh, nothing for Arm V4 oldtimers
What is the observed improvement of XPH over LZX in terms of size and speed?
LZX produces (from my observation, but I have several MB already UPX-ed) about 15% smaller builds than XPR. Need to check that without my optional packages to see the real ratio - other observations anyone?

little comparison of libraries
in terms of size (from the worst to the best):
xpr -> xph -> lzx
in terms of speed (from the worst to the best):
lzx -> xpr -> xph
I haven't tried to calculate percents.

I have installed OS Builder V1.3.163. I went into settings and selected XPH as build method. When I tried to build a rom I saw the following:
----------- CREATING IMGFS PARTITION FILE ----------
IMGFS data (template):
Compression - XPR
Boot sector size - 0x0800
Building IMGFS partition file (XPR)...Ok
Compression - XPR
Boot sector size - 0x0800
Empty sectors - 23 (46 Kb)
--------- OPERATION COMPLETED SUCCESSFULLY ---------
Is there any reason it is still using XPR over XPH?
EDIT: Found the .DLL folder in Personal Build\Dump\ROM\XIP\OEMXIPKernel replaced it with the new one for XPH. Reloaded osBuilder but it still says XPH not supported, but now getting the following:
----------- CREATING IMGFS PARTITION FILE ----------
IMGFS data (template):
Compression - XPR
Boot sector size - 0x0800
Building IMGFS partition file (XPH)...Ok
Compression - XPH
Boot sector size - 0x0800
Empty sectors - 48 (96 Kb)
--------- OPERATION COMPLETED SUCCESSFULLY ---------
EDIT2: Compiled the rom but now wont boot on my HD2 very odd. Going to bed now

GhostXSeries, read this. That's the way how to deal with new cecompr.dll. Do not replace, but create additional folder with it inside and define the path in settings.

Something obvious: you must rebuild your XIP too. IMGFS is not enough - this will be compressed with XPH when creating the ROM but the cecomp.dll on the device must support the unpack of that as well - otherwise the device won't boot.
Check the XIP part of the build options for that - this is where AndrewSh is pointing to with the previous post.

Ah that might be why my rom isn't booting then! It was unpacked using XPR I can see during the build process that XIP is compress using XPR and then IMGFS is compressed using XPH.
I will try that cheers!

Compression is tricky - and in a way nested as well:
the XIP as a whole may be compressed with SRPX on some devices that support it from the bootloader
parts of the XIP maybe compressed with entries in their DSM, but then the OS (nk.exe and related modules to access the compressed files - including the cecomp.dll) have to be loaded already - separate info in the history on the use of this part
the imgfs as a whole - what you are after.
Good luck

Do I need to use the Barin's OSbuilder essentially for XPH compression? can't we use bepe's platformrebuilder? I remember while I used to cook Diamond's ROMs then there was no problem while compressing with LZX with the help of bepe's platformrebuilder.

platformrebuilder doesn't build imgfs. That's ImgFsFromDump's work. If integrated ImgFsFromDump supports unknown (for it) compression, then it is ok.

ultrashot said:
platformrebuilder doesn't build imgfs. That's ImgFsFromDump's work. If integrated ImgFsFromDump supports unknown (for it) compression, then it is ok.
Click to expand...
Click to collapse
I agree that platformbuilder does not build IMGFS. But, XIP building and compression is done by it. Because, when I put the relevent cecompr.dll in XIP, the PB gives error while processing it.
Even in the Barin's OSB there is option to replace the cecompr.dll later on sometimes during the process. If I replace the cecompr.dll manually before launching the OSB and uncheck the replace option in XIP tab of OSB the there is error as well.
Since I am still a user of bepe's PB I was looking for a possibility of XPH compression.
Can it be possible?

XIP isn't being compressed by XPH. XIP is compressed using other methods. So, its pb's problem.
OSB is the only solution then.

I put the relevent cecompr.dll in XIP, the PB gives error while processing it
Click to expand...
Click to collapse
Platformrebuilder CANNOT process cecompr.dll (XPH), because PRB CANNOT work with relocations of type 5 and 7
Compatible kitchen:
only the latest OSBuilder (>=15-Sept-2011 version).
Click to expand...
Click to collapse
It was written in the fist post

c_shekhar said:
Do I need to use the Barin's OSbuilder essentially for XPH compression? can't we use bepe's platformrebuilder? I remember while I used to cook Diamond's ROMs then there was no problem while compressing with LZX with the help of bepe's platformrebuilder.
Click to expand...
Click to collapse
Don't be afraid to upgrade - it is worth it. You may not get all the OSB power in the first round (recook a shipped ROM as a start) but you will learn as you go. Mind to read the WWE manual which gives you a good walkthrough for most options.

-=Barin=- said:
Platformrebuilder CANNOT process cecompr.dll (XPH), because PRB CANNOT work with relocations of type 5 and 7
It was written in the fist post
Click to expand...
Click to collapse
Thanks a lot @Barin.
Now you has made me understand the limitation of PRB against the OSB.
tobbbie said:
Don't be afraid to upgrade - it is worth it. You may not get all the OSB power in the first round (recook a shipped ROM as a start) but you will learn as you go. Mind to read the WWE manual which gives you a good walkthrough for most options.
Click to expand...
Click to collapse
Thanks a lot @tobbbie.
Actually I am using my own kitchen which uses the bepe's PRB. and separate implantxip.exe by azz. If I can manage to create, using Barin's OSB, the XIP having XPH cecompr.dll then I can perhaps implant that using implantxip.exe and compress the IMGFS with XPH.
It is only my guess may be it works and may be it does not.
Whatsay?

c_shekhar said:
If I can manage to create, using Barin's OSB, the XIP having XPH cecompr.dll then I can perhaps implant that using implantxip.exe and compress the IMGFS with XPH.
It is only my guess may be it works and may be it does not.
Whatsay?
Click to expand...
Click to collapse
I am not really familiar with the details how the XIP is linked with imgfs, but the connection is quite close regarding end and start of adresses. So if you need OSB for creating the XIP with XPH compression contained and then you need OSB to compress the imgfs with XPH - why any need for cooking outside OSB anyway?!
I had tried in the beginning to get some pieces together like a XIP from OSB in my old kitchen but it failed (I guess due to the memory boundaries). As OSB could import the packages unchanged (I used an old style batch kitchen with BuildOS.exe and option.xml) - there was only very little to be done for a start. Mind that OSB will care for defined content in the *.dsm files which my old kitchen did not (just picking all in the package directory), so there is some work ahead to get that cleaned. You should however move over to the OSB thread to read and discuss there when it comes to OSB usage and migration from older kitchen.
To get a clean start, just dump a ready cooked ROM from your old kitchen with OSB (to get the right structure set up) - of course you have to keep all packages intact in the build. Then try to rebuild that including the re-creation of the XIP. Play with advanced options there if you like. After this is ok, then you can iteratively add your other packages and setup options for these packages.