Related
Hi - notice many having problems with country ID's and provider codes in XDA II images. The common complaints inexperienced people trying to use hex editors etc. I decided I would write a nice win32 GUI replacement for the er2003edit and the nbf2 tools to roll them into one easy to use application. I have also addressed the second prov/country in the ms.nbf so I think would be usefull to many less experienced users on the forum trying to flash new roms into there XDA II's
It so far will open and display the rom file as hex or ascii - and automatically works out the 'key' to decrypt the actual file (if its an nbf) etc - and all is funtional.
Have got it substantially finished - but am needing some feedback from other developers on a couple of technical issues.
Issue 1 is that I have noticed that the padding bytes in the older nk.nbf files are 00's - as opposed to FF's in newer nk's. Is this consistant across most images based on vintage etc (need this for auto calculate key).
Issue 2 is I have not yet worked out the checksum algo - which I am happy to experiment with - but I can assume seeing as both the apps I am trying to roll into one 'know' how to re-checksum the files etc - that someone here could explain to me how (if they see fit)
Not and issue - so Question 3 - are there any other functions that anyone here believes would be a good addition to this tool - speak up now (I am short of ideas here).
I will post app as soon as I egt the checksum working (which should be fairly soon if I can get the basic algo without too much work) as all the other bitsa and pieces are already functional.
cheers
^lave
Good Idea brother....
Waitng for it....
I have prgressed on application - have written the CRC algo now and working (better way to learn) - but am still lokking for some advice from the more knowledgable people on this board.
So far I can now re-checksum both nk and radio rom - but for some reason algo is failing on the ms_ roms - do I need to delve into structure a little further (ms being different to the rest as far as checksum goes) - or have I simply got a bug which only appears on ms roms (not too likely as working on multiple other roms).
Am able to decrypt the ms roms like the other but just not correctly CRC it - any clues anyone ????
cheers
^lave
See PM I sent you.
Sounds like a great project.
As I remember correctly, itsme also tried to calc CRC...not sure if he ever succeeded.
Personally, I can not help you.
@HappyGoat - yes thanks for the PM - that advice gave me some pointers and let me find a description of the algo which allowed me to write my existing work so far - I am perusing the source of xda2nbf to try to find answer for the ms_.nbf's as they definately seem to follow a slightly different format (ie I think either more header - or datablocks start at different point) with reguards the CRC therefore is different.
I did try to reply with thanks in PM but xda.developer was so slow at the time I gave up waiting for reply page after a few minutes. Thanks a lot for your input
cheers
^lave
OK - at last some progress - I finally worked out my bug crc'ing the ms_.nbf - was a silly assumtion that I had mada re the file sizes (ie radio 4MB + header) - (nk 32mb + header) - (ms 16mb + header - WRONG) - how about I check the filesizes - was missing 210 odd KB off the end in the crc - derrr.
Never mind - next question is an easy one and simply to prevent having to DL lots and lots of images - are all the MS_.nbf's a consistant size ??? - I have download 3 so far and they have been - but without download many many 40MB+ bundles - someone must know the answer to this one ????
cheers
^lave
PS what I am really looking for is a reliable method to identify one type of nbf from the next - trying to identify if filesize is the obvious answer.
Hi everyone - this is a totally useless version of app - from perspective has no write functionality yet and is very alpha. Idea is if anyone wans to have a play am interested in suscess or failure at opening different rom files. Is only design to handle XDAII images atm. Also interested in getting feedback on if the calculated checksum matches the one in the file. If anyone does get a chance to try out let me know.
cheers
^lave
More progress - a less useless version of the editor. Still treat with caution as is still beta and not a lot of feedback on compatibility wth a variety of NB10 roms yet.
Changelog:
several bugfixes
speed improvement in CRC
Write functionality
Plain to XOR and XOR to plain conversion
2nd details in ms_.nbf now working and editable
Allows changing the XOR key
If you save a file from the app with no extension or a non .nbf extension will save out plaintext rather than re-encrypting with the XOR key etc.
Again - if you download to play - consider giving some feedback on bug and features additions.
Cheers
^lave
good work!!!
Just an email to say good work as you don't seem to be getting much feedback for your program. The problem is that those who most need the program (noobs) are the least able to help, and those who can help don't need the program so are less inclined to help. There are some real whizzes lurking around here, some of them should be able to give you a bit more support.
Good Luck,
JJ
oh and the other thing is, if you want more coverage post this program as a news article on the main pageas it tends to stay up for a few weeks!
Thanks!!!!
Hi ^lave,
It's a very nice tool. Is this a alpha version, right? Will you are going to add more features, ie make change to the ROM content to prevent Authentication Check. etc.
I think you may need to change some label wording:
Provider ID ----> Country ID
Country ID ----> Language ID
Am I right?
Hello
Do you plan to adapt tools to 2020i (PH10C) Alpine ??
It will be greater to have tools to manage ROM on this PDA
Thank's for reply
Yes - is very early code still and I intend more features as I work things out. I only got my first PDA at Christmas just past (a bottom end iPaq - enough to catch the bug though)- and the O2 a few weeks ago so its still all new to me (yep I am a newbie).
Most of my more recent coding for past few years has been satellite related - so my terminology certainly has that bent at the moment and needs some serious work.
I certainly intend the app to be more universal as far as PDA models go - but of course that will rely somewhat on availability of hardware to test with (or alternatively people with other hardware to test for me).
Thanks for feedback and suggestions (I certainly need them at this stage of things).
cheers
^lave
Thank's
I can be tester for you on Qtek 2020i (PH10C) Alpine.
The extended ROM do work different than 9090, S100, 2020.
I have new registry entry and TRUEFFS_DOC that was for Extended ROM is now for Storage and EXTENDED ROM Profile is VDISK that use vdisk.dll.
hi,
I have a program and it has lots of settings saved in memory, when I exis the program I need to save settings in a config file.
My program has CString(s), CArray(s) and other basic int and DWORD varibles, so how can I save such complex settings?
Mohammad
as xml maybe
If you're using MFC you could always attach a CArchive to a file and serialize
Perhaps you need to either create a log file - for debugging purposes or save your settings to the registry. If its the former then, check this one http://www.codeproject.com/ce/GenericLogFunctionality.asp it should be helpful. If its the latter, then you just search codeproject and you'll find lot of tutorials on the registry. Also, you can use CEDB which is a flat table data base that can store many data type structures including files and images, use it if you need to.
It may seem like you're lost now, but you can take an overview about the cons and pros of each technique so as to choose the best one for you.
For my config file I use plain english bounded by my own tags. I use a character sequence as a seperator for lists. I convert numerical values into text that can be human readable. There are some benefits to being able to read and edit it manually if the need arizes. I also made a class purely for writing and reading from my config file. Now when I to use a config file I can just drop that class into my project.
XML sounds good and similar but it would require a greater knowledge of xml.
Registry is easier and faster to do/execute but for data that can potentially grow too much it is probably best to keep it out of registry.
Is CArray serializable? do we just simply use CArray::Serialize and it will dump all included objects into a file?
thanks
http://msdn2.microsoft.com/en-gb/default.aspx
this site may be helpfull while you wait for somebody to answer your question
If you can it would be easier to use CObArray and ensure that the array members are subclassed from CObject with the DECLARE_SERIAL / IMPLEMENT_SERIAL macros. You could also add a new << operator to the CArchive class and write the serialization routine but that would kinda defeat the object I guess.
I solved the issue using serialization, for CArray structures I used the following method:
File<<MyArray.GetSize();//store how many objects
for(all my objects in the CArray)
serialize them one by one
I did the same when loading and it works flawlessly.
Thanks all for your help
I've waited for a while to see how everything would develop, but RA2 just remains slow as hell.
I'm getting main memory write of ~1700 and read of ~6100.
Storage card gets stuck at ~650 write and ~ 2600 read.
It's drama. Did anyone find a way to improve this, or must we start PM'ing tnyynt?
All I did was reflash back to the R1 firmware. I didn't see anything in the R2 that was better. So unless I missed something, I just went back to the R1 with the tweaks and it seems to be a lot smoother and quicker than the R2. Just my .02
I think and maybe wrong here, but the r2a roms are using a different memory model, that caches a lot more.
I've found that letting wm control the cache works best.
16mb pagepool is also good
despite the benchmarks I definitely find r2a faster than 1, inclduing sdcard access.
have you tried spb benchmarks "real world" tests, using word etc?
fards said:
I think and maybe wrong here, but the r2a roms are using a different memory model, that caches a lot more.
I've found that letting wm control the cache works best.
16mb pagepool is also good
despite the benchmarks I definitely find r2a faster than 1, inclduing sdcard access.
have you tried spb benchmarks "real world" tests, using word etc?
Click to expand...
Click to collapse
So what exactly did you change?
I haven't tried a SPB benchmark. I know SKtools isn't the most reliable test, but I think that the numbers I gave you tell us enough; it's not good.
Angelusz said:
I've waited for a while to see how everything would develop, but RA2 just remains slow as hell.
I'm getting main memory write of ~1700 and read of ~6100.
Storage card gets stuck at ~650 write and ~ 2600 read.
It's drama. Did anyone find a way to improve this, or must we start PM'ing tnyynt?
Click to expand...
Click to collapse
Can you verify that the registry entries from the cab have been indeed written to the registry?
If yes, use a tweaking program to verify File System Cache is enabled and set according to SD TuneUp.
Write operations should not be cached by default, so that result I seem fair but the SD read result I find low. What was your result on the previous ROM?
I checked and I think I found the issue. The settings are not applied indeed.
Funny thing is, I can't change them. I've tried with SOTI Pocket controller, as well as with SKtools' regeditor. The registry keys cannot be modified! Will try in another way, will post results.
EDIT: After some more thorough inspection, I found something is wrong.
The registry does show the correct HEX keys. In decimal however, it shows completely the wrong thing!
For instance the DWORD CardDetectControlMask:
In tnyynt SDtuneup.cab
Hexidecimal: fffff3ff
Decimal: -3073
In my registry:
Hexidecimal: fffff3ff
Decimal: 4294964223
How can this be explained? I've never seen HEX differ from decimal before..
Edit2
Third inspection shows me that the registry doesn't like a "-". In your CAB, - values are used. I can now modify positive values (took the SD card out), but can't enter minus.
Angelusz said:
EDIT: After some more thorough inspection, I found something is wrong.
In tnyynt SDtuneup.cab
Hexidecimal: fffff3ff
Decimal: -3073
In my registry:
Hexidecimal: fffff3ff
Decimal: 4294964223
How can this be explained? I've never seen HEX differ from decimal before..
Click to expand...
Click to collapse
i can explain this one. (i will use h for hexadecimal numbers and d for decimal numbers and b for binary numbers)
normally FFFFF3FFh =4294964223d in 32bits
but in binary you can not enter "-" in a cpu registry only 0 and 1
and you can only do addition. So to do 5d-2d they must do 5d+(-2d).
but (-2d ) can not be express in ordinary binary.
the example is only on 4bits
5d=0101b
2d=0010b
3d=0011b
the result must be 3d=011b
0101b
+xxxxb
_____
0011b
we find that xxxxb=1110b=(-2d) but 1110b also = 14d !!!
it works because of a registry overflow. on 4 bits you can only count up to 15 normally 0101b+1110b = 10011b -> 5 bits but only 4 bits can be stored so the one on the left is not in memory. so the result of the operation is 0011b.
the shortest way to find the representation of (-2d) is
take the biggest number on the number of bits you use. remove the absolute value of the negative number you are are looking for and add 1.
(computer can not do binary subtraction but human can )
on 4 bits
1111b - 0010b +0001b =
1101b +0001b=1110b
or 15d-2d+1d =14d
now on 32bits.
3073d=00000C01h
so
-3073d=FFFFFFFFh - 00000C01h +00000001h
-3073d=FFFFFFFFh- 00000C00h
-3073d= FFFFF3FFh !!!!!
all that to say that the value in the registry are the same....LOL
all depend if you consider it or not has a negative number.
normally we use the first bit on the left to know if its a positive or negative value.
Alright. So it actually is the same. Thanks for your elaborate explanation, it's never a bad time to learn something.
So it's all in the way it's displayed. I gues WinCE cab manager does allow showing -xxxx in registry entries, where the winmo registry does not.
This does mean that the values should indeed be applied correctly. Tnyynt?
After updating my firmware I've notice, faster boot up times, and TF3D is a lot smoother. A lot. Before the update TF3D would crash my phone. It hasn't crashed yet. I wondering why you guys are experiencing a sluggish update.
yogibear2009 said:
After updating my firmware I've notice, faster boot up times, and TF3D is a lot smoother. A lot. Before the update TF3D would crash my phone. It hasn't crashed yet. I wondering why you guys are experiencing a sluggish update.
Click to expand...
Click to collapse
This thread is not about it being sluggish, I'm pointing at the raw numbers. File transfer btw, is much slower.
Angelusz said:
This thread is not about it being sluggish, I'm pointing at the raw numbers. File transfer btw, is much slower.
Click to expand...
Click to collapse
My apologies. I didn't mean to highjack the thread.
tnyynt said:
What was your result on the previous ROM?
Click to expand...
Click to collapse
i got around 3000 on R2A
on R1A i use to have 12000 on my 16g sd card.
so it's significantly lower on R2A
Angelusz said:
Alright. So it actually is the same. Thanks for your elaborate explanation, it's never a bad time to learn something.
So it's all in the way it's displayed. I gues WinCE cab manager does allow showing -xxxx in registry entries, where the winmo registry does not.
This does mean that the values should indeed be applied correctly. Tnyynt?
Click to expand...
Click to collapse
it depends if Tnyynt have used decimal or hexadecimal to enter the values. registry are always imported in hexadecimal
if Tnyynt has entered the initial value in hexadecimal so the values are good but if Tnyynt entered the initial value in decimal there is a problem, but this one can be easily solved. it will just mean that R1A and R2A may not use the same code for numbers so in this case we will just have to recode the value to have the good numbers.
but i logicaly for a mask you use hexadecimal...
Has somebody tried to contact se about this problem?
Tylwith said:
i got around 3000 on R2A
on R1A i use to have 12000 on my 16g sd card.
so it's significantly lower on R2A
Click to expand...
Click to collapse
yeah same here. This is why I was wondering if it was worth staying on the R1A since most programs are installed onto the mem card, the R1A firmware should be quicker at loading programs correct?
Ok, why are you people so keen about the benchmarks? Copy a file on your SD and compare the result. This is a lot more realistic test and there you can see if there is any real difference for your programs. Everything else is just for bragging, sorry.
Dandie said:
Ok, why are you people so keen about the benchmarks? Copy a file on your SD and compare the result. This is a lot more realistic test and there you can see if there is any real difference for your programs. Everything else is just for bragging, sorry.
Click to expand...
Click to collapse
it's not that simple because of the file system and how machine handles access to them. depending of the size of file you are copying the result can be very different. Some fs are better with tiny files some other with bigger files. there is also the access speed of the file that you can not really tell even if it has a big importance. it's a bit like internet speed . you can have a fast non responsive internet connection(will take long time to access a server but will download very fast for example). Or a responsive but very slow internet connection(remember of the days people were playing counterstrike on rtc modem because the rtc modem gave them a better ping than adsl or cable)
benchmarks try to copy different kind of file size and try to make an average score they also test multiple simultaneous access and thing like that which is important for multitasking. Futhermore it's not easy to try launch a copy a several tiny files by hand.
if your system is faster reading bigger file it doesn't mean that program that use smaller files will be faster.
Very true. tnyynt, what does all this tell you?
Sorry to be so late, time is not on my side nowadays.
I find it really strange that the TuneUp would work on any device sporting a Card but on the Xperia with the latest firmware. I think we can easily rule out registry and import incompatibilities since it's the same operating system.
Nothing clever comes to my mind but this:
try exporting the regustry entries from my CAB with WinCE CAB Manager and apply them as a registry patch on your device. See how that goes.
Hi,
I'm interested to know which debugging/logging tools are available.
I'd like to get more information about the processes at startup, and specially logs of the CPU usage by each application over a period of time.
I've been searching for some time and the only I've found are the old Htc Test applications, but can't get what I want.
Noted that HTC devices have a builtin debug tool (debuglog.dll). Anyone knows how to use it?
Also found the following the following post describing the HTCDiagDriver and the possibility to analyze the device using QUALCOMM eXtensible Diagnostic Monitor.
http://forum.xda-developers.com/showpost.php?p=12624471&postcount=2
Anyone uses it?
Global debuglog can be enabled via DebugTool.exe (available in Htc Test Applications). It depends on debuglog.dll, yeah. Read manual, it works quite well.
Then, if you want to get log for selected app, launch it via IDA.
Also we have CeLog available, I will post needed launchers soon. I can hardly call it useful as we have retail/ship SYS builds. The only useful purpose for us is page faults chart.
EDIT: CeLog attached.
Some of the builds come with the Perfman package. That s.o.b. will really slow down your device, though, and it creates a massive log file, which I could never find the tools to analyze. I think celog does it, though, which is pretty sweet.
The htc debugger works better. You just change one of the debug flags and reset, and the device starts writing the log file. It doesn't slow down the device nearly as much as perfman. I think celog may work on that log file, too. You can royally eff up your device with that tool, though, if you mess with the radio flags. It's pretty cool how it writes to flash memory. Too bad you can't change other things with it like the page pool size.
ultrashot said:
Also we have CeLog available, I will post needed launchers soon. I can hardly call it useful as we have retail/ship SYS builds. The only useful purpose for us is page faults chart.
EDIT: CeLog attached.
Click to expand...
Click to collapse
I have been playing with the tool on my Tornado and observed the following (usage related):
Use it while the device is disconnected from PC. The overhead of repllog.exe (connected to ActiveSync on XP PC) and syncing is just filling your log. For my installation (no further MS Mobile development tools on the PC) kerneltracker.exe does not connect to the device anyway.
Though obvious, the files CeLog*.exe have to run on the device, so copy them to a convenient place there.
The CeLogAttach.exe seems to start the kernel logging and it slows down the device (kind of obvious). There is no way to stop this logging. Something like CeLogDetach would be needed, if it exists, to restore the state before CeLogAttach.exe was run.
The CeLogFlush.exe will flush the existing log but also immediately start the logging again.
The CeLogStopFlush.exe does just what the name tells - it stops the flush to file of the (still ongoing) logging.
After transfering the celog.clg file (from \Release\ directory of the device) it can be opened in kerneltracker.exe. Then you see all the kernelactivities logged and aligned per process/thread on a zoom-able timeline (10ms - 10s) including the labels of the logged primitives. With event filtering you can sort out what you are not interested in. Here you may need advice on what to look after when you want to hunt down a certain device behaviour.
I have checked for page-faults, Virtual Memory related actions (Allocate, Copy, Free) and also Module actions (load, free) to get a clue if and how modules and paging (or better said: the use of the Page-Pool) is correlated. Nothing eye-striking coming up here, but it may just be for the unknowing observer like myself.
@ultrashot: I could not find anything I would call a "page faults chart" - where is that - or what is that?
Looking further: If I change certain device properties (like increase the pagepool or playing with OSB advanced options) I fear that the logged information here is just far too detailed for a useful compare. For that you would have to create identical conditions for the action under scrutiny - something that cannot be done with a disconnected device.
So I have to admit that all objective compare of such tuning and tweaking is far above my head and I just have to join the many that make more or less clever assumptions trusting on their model of actions in their heads. I hope that the better knowing heads continue to spread their wisdom without only telling RTFM or guide with LMGTFY (which can help if the results really point to right places).
tobbbie said:
@ultrashot: I could not find anything I would call a "page faults chart" - where is that - or what is that?
Click to expand...
Click to collapse
Use Event filter->Miscellaneous->Page fault.
I am not too advanced user of this tool. If we had builds with extra celog instrumentation, we could have take much more from this tool. However, there are some articles in the internets about celog, so anyone who wants to be get more info may just try to google it. I don't want
ultrashot said:
Use Event filter->Miscellaneous->Page fault.
Click to expand...
Click to collapse
That is what I did already when telling about the items I cared below. Page faults are however part of generic virtual memory management and they do also apply for any normal loaded executables. As you know I seek for traces of module related paging and the use of the page-pool.
So it will stay with the trial and error and side-by-side compare with two devices having different settings. Not a big thing doing that...
Some interesting articles on MSDN regarding the paging pool (aka "pagepool"):
Kernel Blog article explaining the fundamentals (highly recommended): http://blogs.msdn.com/b/ce_base/archive/2008/01/19/paging-and-the-windows-ce-paging-pool.aspx
Pagepool Variable explained and simple methods to measure impact: http://msdn.microsoft.com/en-us/library/aa451041.aspx
-> this is what kitchentools are patching in the kernel
Then some more backup on virtual memory - just to complete on that:
http://blogs.msdn.com/b/ce_base/archive/2006/10/30/what-is-virtual-memory.aspx
http://blogs.msdn.com/b/hegenderfer/archive/2007/08/31/slaying-the-virtual-memory-monster.aspx
http://blogs.msdn.com/b/hegenderfer...aying-the-virtual-memory-monster-part-ii.aspx
And to get back to the debug tools topic of this thread, linked form the first article an introduction to the Remote Kernel Tracker to explain what you can actually see there (and why you cannot see certain things as we have shipped ROM builds and not profiling builds to deal with): http://blogs.msdn.com/b/sloh/archive/2005/05/17/introduction-to-remote-kernel-tracker.aspx
Great insight if you want to get a glimpse of how Windows CE operates under the hood.
...reading a little deeper in the MSDN articles, Sue Loh mentions there when talking about the paging pool size determination:
The best tool I know is that readlog.exe will print you a page fault report if you turn on the “verbose” and “summary” options. If you get multiple faults on the same pages, your pool may be too small (you may also be unloading and re-loading the same module, ejecting its pages from memory, so look for module load events in the log too). If you don’t get many repeats, your pool may be bigger than you need.
Click to expand...
Click to collapse
To avoid dealing with a full setup of the Mobile Development toolsets, could any one (ultrashot - you have been so helpful - could you??) post that mentioned "readlog" tool? If there is something like "CeLogDetach.exe", please add it too.
BTW: you may notice that the paging pool is a central part of the Windows CE memory management when it comes to running executable code from "memory mapped files" (as Sue Loh calls them). In my understanding these are simply what we know as "modules".
A lot of tweaking strategies go around that when building ROMs with OSBuilder. There are several ways how to avoid or optimize the use of the paging pool for certain or all modules in OSB. I think these options deserve an own thread and I am not sure if the one OSB thread we have should be cluttered with discussing this.
don't have any of those.
Hello - I'm doing some security testing on a WP8 application. The isolated storage has a bunch of .DATA files. If I try and view these in notepad ++ it is gibberish. Any ideas on how to view the contents of these files? I'm assuming they are database files.
OK, first of all, you already asked this in another thread. DON'T DO THAT.
Without knowing what the app in question is - which I don't really expect you to tell me - I can't much help you. I assume this is a blackbox assessment, or you'd have checked the source code. If the app is managed code, you can try decompiling it (you can try this if it's native code too - IDA Pro supports ARM - but that's obviously more difficult).
The extension .DATA doesn't mean anything. There are only a few common database formats used on WP8, so it might be one of those... but then, it could also be encrypted so even if you figure out the format you'll need the key too. It could just be a flat text file that has been compressed and/or encrypted, for that matter. You could try checking the file for magic numbers, either manually using a hex editor or by checking the file using libmagic (on *nix systems, /bin/file is usually a front-end to libmagic and should be able to tell you if the file is, for example, a known compressed format).
GoodDayToDie said:
OK, first of all, you already asked this in another thread. DON'T DO THAT.
Without knowing what the app in question is - which I don't really expect you to tell me - I can't much help you. I assume this is a blackbox assessment, or you'd have checked the source code. If the app is managed code, you can try decompiling it (you can try this if it's native code too - IDA Pro supports ARM - but that's obviously more difficult).
The extension .DATA doesn't mean anything. There are only a few common database formats used on WP8, so it might be one of those... but then, it could also be encrypted so even if you figure out the format you'll need the key too. It could just be a flat text file that has been compressed and/or encrypted, for that matter. You could try checking the file for magic numbers, either manually using a hex editor or by checking the file using libmagic (on *nix systems, /bin/file is usually a front-end to libmagic and should be able to tell you if the file is, for example, a known compressed format).
Click to expand...
Click to collapse
Sorry about posting in multiple spots. I figured I shouldnt ask multiple questions (diff topics) in one thread so I thought I would do a stand alone. Won't happen again. The vendor has told me that these files are encrypted...but I am also trying to get some more info on what that actually means and what they are doing for key storage.
It's quite likely that the files are encrypted using the data protection API for WP8 apps (http://msdn.microsoft.com/en-us/lib...ity.cryptography.protecteddata(v=vs.105).aspx). From what I've read, that uses 3DES with a key derived from the user (probably meaning app) and system (probably meaning device-specific) identities. The key derivation and management is invisible to the user, though.