Related
I'd like to toss this out for discussion. This might already exist, it might not.
Microsoft had system builders use a Disk Differencing technique to extract what exactly had changed after installing applications and customizations.
First, we'd install the OS and create a base image.
Second, we'd install the apps & customizations we wanted for our customers.
Third, we'd create another image. An app would then compare these two images and determine what files were added/changed and what registry additions, deletions, alterations were performed.
I'm hoping we can do the same for ROM upgrades.
1. OLD ROM - Base image.
2. Install apps and customizations - Image "Apps". This will give us our customizations.
3. Install new ROM - NEW base image created. Compare to OLD ROM base image to see what has changed (that we need to look out for that our APPs image might adversly change).
4. Apply Apps image to restore all customizations & applications.
Ideas anyone?
The theory sounds good.
Maybe buzz can comment?
The theory is very feasable, ive used similar building Win PE rescue disks, you have a app that gives you the base bootable image made from a Win XP install CD. then you add home made "modules" ie, your installed apps or networking capabilities etc. then it makes an image for you to use,
that cant really be that difficult to do the same for Win CE,
Ive also seen a M$ webcast thing where the guy took 45 mins to build an entire image for a win ce device,
so yer, it is very feasible that such dev apps exist, after all, they were made and then sold to the hardware vendors before ever being a rom image.
check in msdn for win ce 5 stuff, its a very intresting read, :shock:
it is a good idea, but only for linear ROMs, where each file has always it's own and same place in the rom and such rom is not compressed.
hmm...
but you can do partial rom upgrades with wm5 roms...
...and it would be possible to make registry diff + DOC Storage partition backup, where you actually install programs (so called \Device) and userdata.
buzz
buzz_lightyear said:
it is a good idea, but only for linear ROMs, where each file has always it's own and same place in the rom and such rom is not compressed.
hmm...
but you can do partial rom upgrades with wm5 roms...
...and it would be possible to make registry diff + DOC Storage partition backup, where you actually install programs (so called \Device) and userdata.
buzz
Click to expand...
Click to collapse
Ok...Since I'm not a programmer and somewhat of a noob with respect to WinCE, I am NOT constrained by what might be a logical limitation to a programmer or someone in the "know". So, below is my thinking ... outside of the box (er, PDA) style.
But first, I need to grasp further Buzz's post above about the locations of things in ROM. Does this mean that there isn't, for lack of a better example, a FAT for the ROM? That every individual thing is expected to be at some specific, static location that can't change? If so, I see your point. But that tells the system that this location is static? Also, doesn't that make bug fixes difficult as the fix might exceed the designated space in ROM? Or do bug fixes always get re-applied after each reset? Patching in memory is how I'm looking at this. (Remember, I'm a non-programmer...deal with me, please.) LOL
Ok..out of the box time.
1. Can a shim be used to capture calls to a certain location in the ROM and redirect to another location (within the ROM or even the storage card. Linux builds anyone?). Like what HD disk managers did to get around LBA limitations or even boot sector viruses did.
2. Can an emulator be used to simulate/backup/build/test the ROMs on a PC before wiping the PDA? Don't laugh, but if not, could something be done with Virtual PC? Oh..Ok, laugh.
(Should I go hide somewhere?)
Hi!
So what it's all about?
There are so many projects dedicated to porting linux to WM devices.
Mostly the only way to boot into linux and kick windows mobile out of RAM is the haret tool.
For thoose who never heard of it is a linux bootloader (it is even more), that is started from within windows mobile.
This is the result of the fact that the native bootcode of the device could not easily be replaced.
Apart from that, there are to many things to consider to rewrite own bootcode for a closed device.
So what if the native bootcode "thinks" it boots in windows mobile again, but it in fact jumps into linux kernel code?
Whaaat, you might think and i thought a lot of it in the last few weeks:
The idea is, to embed a linux kernel in a XIP container and boot it with the native bootloader
So that's crazy stuff and it's even more crazy, that some insane hackers already did some successful attempts.
The project aims to port Android to a device called Meizu M8.
Step into this thread and start reading:
http://www.meizume.com/modding-development/9014-project-port-android-meizu-m8-18.html
One of the developers already has started to push some GIT-repos of the toolset here:
http://gitorious.com/~banxian
Of course there's a lot to investigate and other things to consider:
1. The injected code must be in good shape (kernel needs excellent hardware support)
2. The device will need to be hard-SPLed (no security check)
3. The files must match the Flash layout of the bootcode
4. JTAG support for the device would be mandatory for testing
5. Lots of information about internal Flash structure is required
....
Please tell me what you think about it!
Cheers,
scholbert
That's an excellent idea!
Multiboot would seem to be a better (albeit more complicated) alternative to me though... In their current stage, Linux ports to WM devices tend to be unstable and rather limited in functionality. Replacing a fully functional WM with a semi-functional Android on one's primary device is not as tempting as having both at the same time.
I'd say the first step in this direction is bootstrapping custom XIP from RAM with a patched SPL. Flashing XIP nbX every time custom XIP is patched is slow and tedious. Once a working XIP with booting Linux kernel is available, it should be relatively easy to switch SPL back to normal booting from NAND. Also ULDR XIP can be used as a container for the custom XIP rather than the WM kernel XIP. This way one can boot WM by default and then reboot to ULDR/Linux if required (not quite multiboot, but something close).
The caveat here is that when HaRet is used to boot Linux kernel from under WM, some hardware init could have been carried out by WM kernel at boot time. When WM kernel/XIP was never ran, there's a good chance some of the hardware that works when booting Linux via HaRet won't work anymore Completely bypassing WM kernel initialization means more initialization may have to be done in Linux kernel/custom XIP.
The progress that Meizu people made is certainly impressive, but there's a long way from a 100-byte piece of code that fills the framebuffer to a fully working Linux kernel. I'm not being sceptical here, on contrary I'm pretty sure this is possible, but it will take a lot of time and dedication to make this happen. At any rate, good luck and I'm sure you'll find plenty of support here.
This project, is cool, i am waiting any progress, i want put the android in the MS20(Brazillian KS20 without wifi and 3G)
Hi again,
thanks for the feedback so far
@ stepw:
I really share your thoughts concerning long and dusty road of development.
So perhaps i was in kind of euphoria when i decided to post it yesterday.
Anyway, let's see what the future will bring us
Maybe we should start with some kind of ramloader and place it in XIP area.
BTW, could you be more specific about this ULDR XIP thing?
Sounds interesting and to be honest, never heard of it...
Please consider the thread as a starting point for an open discussion.
Anyway, i will need help, because i'm a horrible hacker.
So maybe i should have written IDEA not PRJ
Have a nice day!
scholbert
As per http://channel9.msdn.com/wiki/CEDeveloper/BSP/
ULDR and IPL
For BSPs that are for Windows Mobile products, the ULDR and IPL are required parts of the BSP.
“ULDR” stands for “Update Loader”, and is part of the Image Update system. This system allows deployed devices to be updated with new software after they ship. The Update Loader reads a configuration stored in persistent memory and downloads and installs new versions of operating system or OEM files.
“IPL” stands for “Initial Program Loader”. This piece of code is launched by the bootloader or executed directly at startup if a bootloader has been removed from a board. The sole job of this program is to choose whether to execute the ULDR software, or load and execute the operating system that is currently on the device. If a user has downloaded new versions of operating system or OEM files, the IPL will be configured to launch the ULDR. Otherwise, it will load and launch the OS.
===
Da_G's thread http://forum.xda-developers.com/showthread.php?t=520009 has plenty of information about ULDR, although it's more about keeping it rather than about using it for something else.
Replacing ULDR is a valid way to inject another item into NAND partition table. Unfortunately with WM all 4 primary partitions are used in the MBR by default, so taking ULDR out allows for reuse of one of the parition slots for other purposes. IPL already has a way to bootstrap either ULDR or OEM XIP (WM kernel), it should be possible to control boot partition selection from each of the OSes. Manipulating partition type and flags should make it possible to choose the default OS too.
ULDR partition is typically fairly small, but it can be expanded to store Linux ramfs image or even the filesystem. Alternatively, FATFS partition in NAND could be mounted at boot and filesystem image could be located there. Yet another location for it is IMGFS partition, but that calls for a file system driver (read-only at least) that I don't think exists for Linux. Anyway, if at least FATFS can be mounted, access to all user files accessible from WM should also be possible from Linux/Android once it's booted.
Sorry for deviating from the original topic
Hey stepw!
Sorry for deviating from the original topic
Click to expand...
Click to collapse
Hey we got an open discussion here...
Thanks for all this useful information so far!!
I knew about starting up WM platform and i also looked deeper into IPL and stuff. What i am missing a little is just these information about XIP and WM image in general.
So i think i'll first step through Da_G'S thread, it looks very promising. Great stuff!
Again leave your technical comments here, because i think it's the only way to get best solution
Best regards,
scholbert
Hi,
I've been working on this for a few days now and have a simple bootloader that loads and runs a kernel on my vogue. Unfortunately the kernel doesn't boot properly because it can't initialise all the hardware correctly but it definitely runs. Attached is my code and a script to insert it into an xip payload with a kernel.
Woooow.....
Hey dzo,
this is a real breakthrough or whatever you may call it.
Really great stuff!
Maybe this won't reach peoples interest right now,
but let me forecast, that someday this will give us the opportunity
to wipe out windows mobile completely if we like to
Anyway i think it would be nice to get some stuff pointed out more clearly.
So let me sum up:
1. Let's assume we got excellent kernel zImage to support the hardware of our device.
We will need to initialize even parts of hardware in this kernel,
that we did not even know about, while testing with haret.
2. We need some hard-SPL bootcode on our device, because we need to avoid security check of the image.
3. We got WM ROM for our device and we got some kitchen to extract it and work with it.
The starting point will be the file OS.nb.payload, because this is pure binary.
(the image like it is stored in NAND flash memory).
The image (OS.nb.payload) itself is organized in different parts and partitions.
At the moment we don't care much about it, because we leave it mainly untouched.
We need to find the entry point from the WM kernel, which is pure XIP code (XIP.bin).
4. We inject a tiny loader for elf binaries at this point, which is also compiled as XIP code (tinboot).
5. We step a bit further and place the kernel zImage (which is an elf file) at a certain offset.
At least this should be the address the tiny loader points at (e.g. offset + 0x8000).
6. We use some kitchen tools to reconstruct a flashable image.
7. We flash this image to our platform, using the same tools we use to flash a cooked ROM.
8. We boot into linux!
Please correct me if i forgot something or made a wrong assumption here.
Would be really nice to get some more hackers and ROM cookers over here to benefit the discussion.
Thanks again dzo!
Best regards,
scholbert
Hi again!
A little research at the forum gives some more details about OS.nb.payload:
http://forum.xda-developers.com/showthread.php?t=446506
This maybe all well known, but should help to point at it again for a better understanding.
So if we use mtty and type the info 8 command on HTC loaders this prints out these partitions (should be all the same on HTC devices):
Partition[0], type=0x20, start=..., total=... BOOT (ULDR)
Partition[1], type=0x23, start=..., total=... RAWFS (XIP)
Partition[2], type=0x25, start=..., total=... IMGFS (SYSTEM)
Partition[2], type=0x04, start=..., total=... FATFS
TBC
scholbert
dzo said:
Hi,
I've been working on this for a few days now and have a simple bootloader that loads and runs a kernel on my vogue. Unfortunately the kernel doesn't boot properly because it can't initialise all the hardware correctly but it definitely runs. Attached is my code and a script to insert it into an xip payload with a kernel.
Click to expand...
Click to collapse
That's some excellent progress! Did you build your kernel yourself too? Is it mission critical hardware that fails initalization or some minor stuff?
Well hello everyone.
I am an application development student from Belgium.
I am proficient in Java, VB.NET, some c/c++ and also PHP/JS/HTML/CSS
Anyways, my recent interest has been going out to creating custom roms for the galaxy S!
The problem now is though, I was able to find some guides and such, but most were incomplete, and for HTC(and other) devices.
I have no idea what is different between making a ROM for galaxy S or a ROM for an HTC device.
If anyone would like to link me to some good resources that can help me on my way, or would like to collaborate (and in the meanwhile teach me a bit) I would be more than happy! (I am open for collaboration as I am pretty proficient in Java and I'm always happy to code and learn more about coding!)
I think the question is really want you want to do!
Custom ROMs for Android are as simple as using recovery console to replace files on the system with your own versions.
ie. Replace the framework.apk which contains all theme images with your own modified one, which you can use to skin the device. Same for replacing other system apps.
Unfortunately, most of these apps are closed source, so modifying them is difficult. The stock android system has open source versions of most of them though, and you can modify those and build them, and then replace the ones on the device. I'm not entirely sure how to build them up though, but it probably isn't too hard.
Then there are custom kernels, which let you actually change the underlying linux system underneath android - Samsung has an open source version of the kernel available that you can compile, and then flash to your phone using Odin or other tools. Compiling the kernel is pretty tricky though.
Well i'm really talking about stuff like SamSet, MoDaCo, Doc's ROM and others.
I'm not saying I want to create a pro ROM in a day or anything, but i'd really love to learn more to get better (always looking for more stuff to put on the resume, I guess?!)
Thanks a lot,
Kenny
Bump for some help
(sorry for the double post but i'm really eager to learn)
I don't have time to give a detailed response right now. However you will want to read up on yaffs (file system) and odexing/deodexing ROMs.
Thank you I will definately do that,
I am following a guide now to start out creating custom ROMs for android, however it was written for Motorola Droid and I have no idea what you have to do different for a samsung galaxy s.
I am a good coder, even though I say so myself, with a lot of experience. I also have a lot of linux experience, you name it
Only the problem is the old, and much known "where do I get started?"
So if anyone can point me out in the right direction on that, tell me how/where you started out, I would appreciate it greatly!
Thanks!
Very sorry to bump again
I set up my environment in ubuntu today to get started (just gotta fix a few errors, didn't know android wouldn't compile on java 1.6, only on 1.5)
Anyways, still need some help here! thanks!
Again I am awefully sorry for the bump here but i'd really like some help.
Note that i'm not a newbie at coding whatsoever and i'm not unfamiliar with some hardcore java, c/c++ and other languages, I'm just having trouble getting started when it comes to modding an existing android release (there are tutorials on HTC and other devices but some methods do not apply to the galaxy S)
If you're trying to create a ROM from scratch then unfortunately you're out of luck at the moment. No one has really managed to do so yet. By "from scratch" I mean the Android Open Source Project (i.e. official Android git repository).
What you can do is grab an existing firmware. Extract the contents, de-odex the apks, modify them or add new apks, optionally you can also compile a custom SGS kernel using Samsung's kernel source. Then you can re-odex the ROM, pack it back up and distribute it.
There's no real "starting place" as such and you won't yet find any tutorials about how to create SGS ROMs. Just give it a go and see where you get.
Yes, I do mean creating a ROM from stock samsung firmware, of course.
So my next logical question would be now:
How do I go over extracting contents from the existing samsung firmware, de-odexing and so on?
Any links/guides/tips are greatly appreciated, as long as they are applicable to this cell phone and not totally crap, thanks!
Samsung uses rfs, I think you should know. There is a thread somewhere in this forum regarding extracting .rfs file. Understood you need Debian though.
For some readup, go search for a famous guy named JesusFreke. Tons of materials...
Hope that from here onwards, this can lead you on, and to find out more. Google is your friend too...
I'm afraid my biggest problems already occur in setting up the environment, so it's really the very beginning that gets me stuck, i'm sure coding would be a bikeride compared to setting up the environment for me (i'm no hero in linux,not at all, i'm afraid)
How the hell do I get from point A (being downloading the full source) to point B (starting to create a custom ROM?)
thanks
of course if anyone has the time, and wants to, you can PM me your msn or something, if anyone is up to it
Raykoid666 said:
I'm afraid my biggest problems already occur in setting up the environment, so it's really the very beginning that gets me stuck, i'm sure coding would be a bikeride compared to setting up the environment for me (i'm no hero in linux,not at all, i'm afraid)
How the hell do I get from point A (being downloading the full source) to point B (starting to create a custom ROM?)
thanks
of course if anyone has the time, and wants to, you can PM me your msn or something, if anyone is up to it
Click to expand...
Click to collapse
Agreed completely! Getting a full cross-compilation to Samsung's Android going is seriously difficult! There have been some promises for tutorials and guides for setting things up though, but none have appeared yet. Probably because it's hard!
Well thank you for agreeing with me on that.
I am really hoping for anyone that can help me out here, either on here or in private, and give some detailed instructions on how to get started.
Hell there is enough information on creating roms alread, deodexing, adding/removing apps etc etc etc... The real problem lies in setting everything up properly.
I think I have gotten to the point where I'm compiling the android source (without samsung specific files)
note i'm saying "i think i am compiling" because i'm hoping it won't error out on me again after an hour of compiling!
But once that is done... I have no idea where to go next, where to get the samsung specific files, or what to do with them whatsoever once i have them
thanks!
Android generic source won't actually help you much. Samsung has made a huge number of changes to it, and while it is all open source, backporting their changes to get your stock source running on the phone is a major undertaking. I believe cyanogenmod was actually working on this, but it seems without much success so far.
What the other rom makers you see are doing is taking a snapshot of a running rom, copying over all of the APKs and the files on a running device, and then tweaking those. ie. Opening up framework-res.apk and changing the image files inside.
The update.zip roms that are what most people are doing are really just using the Android recovery to replace files on the phone with their own versions. Since the underlying system is linux, and everything is a file, this actually works pretty well.
The kernel development effort is completely separate from the rest of the system though, and Samsung has provided the entire source tree that just has to be compiled. I'm a bit stumped myself on all the details of setting it up, but I have seen some threads on this forum about it, and it didn't seem too bad to set up. Apparently you shouldn't use the codesourcery toolchain though, or something. Not too clued up on it, sorry. Hopefully someone with the details can chime in, or give us a link to the old post on it (I'm failing to find it using the search function! )
Alright sounds good!
So these recovery files you are talking about (the contents of /system) how do i retrieve those from a base ROM such as JM7 or the likes after a clean flash?)
Raykoid666 said:
Alright sounds good!
So these recovery files you are talking about (the contents of /system) how do i retrieve those from a base ROM such as JM7 or the likes after a clean flash?)
Click to expand...
Click to collapse
Root your phone, zip them up onto the sdcard, and then copy them off.
There are probably better methods, but this works fine.
Also, with credits to Unhelpful, he has told me that the correct toolchain to be using is 'uClibc or eglibc toolchain with ct-ng'. I have no idea what to do with that yet, but it's a start.
RyanZA said:
Root your phone, zip them up onto the sdcard, and then copy them off.
There are probably better methods, but this works fine.
Also, with credits to Unhelpful, he has told me that the correct toolchain to be using is 'uClibc or eglibc toolchain with ct-ng'. I have no idea what to do with that yet, but it's a start.
Click to expand...
Click to collapse
You don't even need to root your phone to pull /system Ryan
adb pull /system <random folder>
is all you need ^^
Alright thank you, I managed to get all the /system files using abd pull /system
now next thing up would be de-odexing right, before I go editing stuff?
Now before I go de-odexing. Can anyone tell me what exactly de-odexing is and why I need it?
and what do I need to do to create an update.zip again?
thanks a lot all!
Tayutama said:
You don't even need to root your phone to pull /system Ryan
adb pull /system <random folder>
is all you need ^^
Click to expand...
Click to collapse
Haha cool, didn't know you could pull whole folders!
Raykoid666 said:
Alright thank you, I managed to get all the /system files using abd pull /system
now next thing up would be de-odexing right, before I go editing stuff?
Now before I go de-odexing. Can anyone tell me what exactly de-odexing is and why I need it?
and what do I need to do to create an update.zip again?
thanks a lot all!
Click to expand...
Click to collapse
The apk files in Android are odexed - that means they have a special index applied for dalvik that speeds them up somehow! I don't know how, I haven't checked how it actually works.
The important point is though:
odexed is faster, but you can't edit it because of signatures or something technical
deodexed means you can change it however you want
More useful info can be found from google - lots of it!
update.zip is easy - it's a zip file in a special format that can be understood by Android's recovery console. The easiest way to do it is to grab an existing update.zip and modify it as needed. Tayutama has millions of em around, so use his!
Then you need to sign it once you're finished with it so that the SGS will accept it. Samsung made a boo-boo and didn't disable the test keys for the 2.1 SGS firmwares. That makes it easy!
http://www.londatiga.net/it/how-to-sign-apk-zip-files/
Just use the test keys to sign it, and then copy it to /sdcard/update.zip and reboot into recovery and it'll work.
We have an imageinfo.bin, and S000, S001, S002, and S003. We need to combine them somehow.
So, I've found threads that will teach you how to create ROMs based on imageInfo.bin files, but does anyone know how to take the pieces of a DLL (or executable) and put them together as a standalone DLL? We are trying to do that in this thread: http://forum.xda-developers.com/showthread.php?t=844008&page=3
For an example of what we are trying to assemble, check here: http://forum.xda-developers.com/attachment.php?attachmentid=444604&d=1290160423
You need to get recmod.exe.
Awesome, thanks! You rock.
I'm running into an error message though, "Error! ProcessCase0: bit 5 is zero!" I think maybe the file format for windows phone 7 is different. Is there anywhere I can go to understand this problem, and maybe get the source code and fix it or something?
FWIW my suspicion is that recmode.exe doesn't properly handle signed binaries, since all the WP7 binaries are signed, but previous WM versions weren't all signed.
Ondraster probably knows the right way to re-file them.
OndraSter doesn't know the actual way .
Binaries in WP7 are a bit different, maybe those e32 and whatever (I haven't dug into this ever, don't understand it and I'm not sure it is worth learning, since WM is going off and WP7 won't be cookable or worth cooking) are different and recmod can't handle its new awesomeness .
Hey OndraSter,
Thanks for the reply. It seems it would be something useful in our efforts to run native code on Windows 7 to be able to recmod those things. Do you have the source code, or any tips on how to go about figuring it out? I can give you all the credit, all I want is to be able to assemble the files!
I have the same problem...
Would anyone be willing and able to create a dump of a clean Windows 10 for Phones system image and share it with me? I searched around in the installation for non-supported devices threads, but did not see a reference to anything.
An FFU image extracted from the updater would also work, thanks in advance!
Why is that?
I see.
But at least it's possible, even if not very probable.
Although, we should check if we can use WinRAR, since it's possible to edit the images without breaking the signiture.
Assuming we have a signed image.
Not possible. There are many threads trying to achieve what you're hoping to do with cabs and such, and it's not possible when the bootloaders are signed and damn near everything in the system requires a signed cert.
But how does that prevent us from opening and modifying them with WinRAR?
Even if we can't boot the new files, it's still a step.
So open it with WinRAR, if possible.
At least to get an idea of the structure of the OTA, that peice of information may help us form an idea as to at least part of the structure of the system.
It's better to have (theoretical) partial read access, than no access at all.
feherneoh said:
FFU is not OTA
Click to expand...
Click to collapse
Alright, but can you open it inside WinRAR?
Now we're talking, what we need now is someone to examine the partitions and their layouts.
Unfortunately , I'm not at that level but at least I can understand these things, so I would like to hear the results.
Is anyone exporting the partitions yet?
Unfortunately, I don't know the partition layout in Windows 10 Mobile, but perhaps they should all be exported and examined?
Not for myself, however I had thought that it may help others attempting to port Windows 10 Mobile in the future.
you can try to download MI4 rom
http://en.miui.com/thread-189556-1-1.html