UPX - Compression app to free up space - JASJAR, XDA Exec, MDA Pro General

I found this thread on aximsite, a forum I still visit from when I had a Dell Axim and it looked interesting. I didn't find the app in a search here, so I thought I'd post it for anyone who might be interested.
UPX is apparently an app that lets you compress .exe and .dll files without really impacting perfomance in any noticeable way (per the postings in the thread.) I haven't tried it yet, but might give it a shot. It seems the app is not resident on the pocket PC, but needs have files moved to the pc, compressed, then moved back. It's a little bit of work, but it only needs to be done once, so after the set up, it sounds like you're good to go.
Anyway, just thought I'd pass it on as an FYI for anyone who might be interested.
Oh yeah, and it's free too.

Finally UPX is available for WinCE. You should have posted in "General" in the first place.
This program makes your programs even start faster (less to read from usually slow SD card, storage, ...).

For a non-technical person, I am a bit confused... Reading thru the thread, it seems that more ram are required to run the compressed apps.
What exactly do we gain if the apps are stored on a storage card where storage space is not really the issue? Need help.

Wow, thank God. UPX is old skool genius.
Can someone give it a go on the enormous Skype executable? I'm not near my phone for a while and can't check it.
V

vijay, I tested it on the big MobileNavigator.exe which is around 3,7 MB originally. UPX'd only 1,1 MB or so. And it runs! Even Voice Command (300KB down to 100KB). Since I have it installed in Storage it starts faster (Storage is soo slow).

Thank goodness. This is a God send, when reversing PPC binaries you can see that they're full of so much slack.
I had a look at UPX quite a while ago - I use it to compress all my PC VB and C++ stuff, but was disappointed to find it didn't work on PPC. I'm now.
Decompressing presumably incurrs some overhead, but it'll help us out when we've got limited storage.
V

vijay555 said:
Decompressing presumably incurrs some overhead, but it'll help us out when we've got limited storage.
Click to expand...
Click to collapse
I can't agree with that. You can read on the authors page, that
http://upx.sf.net/ said:
Your executables suffer no memory overhead or other drawbacks because of in-place decompression.
Click to expand...
Click to collapse
Although decompression takes some time it is still faster, because it is read into RAM a few times faster than it normally would. This saves more time than it takes.

Good point Chatty. My concern was CPU overhead on slower processors (OMAP), but as you say, since you're using RAM or faster storage, it'll probably compensate.
V

i tried it with opera and tomtom which are both on my main memory
results:
original opera startup 14 sec
upx opera startup 9 sec
original tomtom startup 15 sec
upx tomtom startup 14 sec
i also tried it with tcpmp which is on my SD card
original tcpmp startup 3 sec
upx tcpmp startup 2,5 sec
so it seems to speed things up a little
HOWEVER... in opera the the '-' buttons dont work anymore.. they do work when you use them through the touch screen though..
and the text ('Stop'/'Go to' and 'Menu') is aligned to the left of the screen instead of the normal at 1/3rd and 2/3rd of the screen.
--------------------------------------------
[edit] extra information
original opera 4,86mb
upx opera 1,63mb
original tomtom 2,62mb
upx tomtom 1,67mb
original tcpmp 1,21mb
upx tcpmp 1,21mb
not only a speed gain but it also saves room

OPERA
after a few tests i found out the opera.dll is responsible for the movement of the ´-´ buttons and the not functioning with the keyboard
the opera.exe, xmlparse.dll and zip.dll have nothing to do with it..
with the upx´ed above three and the standard opera.dll startup takes 12 sec so its still some faster then all original files
however the original opera.dll is 4,74mb and the upx opera.dll is 1,57mb, so it doesnt save much room :!:
im just glad ill receive a 4gb SD card today which will give me some extra breathing room compared to my 512mb

Re: OPERA
Bartjan said:
however the original opera.dll is 4,74mb and the upx opera.dll is 1,57mb, so it doesnt save much room :!:
Click to expand...
Click to collapse
1/3! What do/did you expect? I find this rather great!

i meant:
when you use the original opera.dll so the ´-´ buttons still work right, you dont save much room compared to the all original opera :roll:

I went through last night and UPX'd every DLL and EXE that I could, including WisBar (and Desktop), Resco, Opera, PocketInformant and other stuff.
PocketInformant won't start if one of the DLLs is UPX'd, but I don't remember which one. The rest of them and all the EXEs are fine.
Opera - as above. Screw the soft keys, 3MB is a lot of slack to be rid of.
Resco was interesting. The Today plugin messed up at one point, and the Explorer went completely scatty with fonts. Even after uncompressing all the Resco files again the Encrypt/Decrypt context menu items were blank but functioning.
WisBar (compressed) can now live in the 10MB Ext_ROM along with Opera and Resco.
All in all it is something to use carefully, but can save a LOT of space.

Hmm.. so far the responses from the trial seems good. I'll give it a try too later this weekend
--- UPDATE (12 May 06) ---
My results, using UPX -9 (e.g. max compression)
All programs are working so far, but performance are negligible (e.g. less than 2 seconds improvements)
from \Windows
PITools.dll 1.18M -> 504K (save 700K)
from \Program Files
Mobibook.exe : 1.61M -> 619K (save 1.0M)
Tom tom navigator : 1.59 -> 692 (save 900K)
WorldMate.exe : 747K -> 207K (540K)
So, that is a saving of 3M of space! Anyway, for those of you that are adventurous, there are various 500K size dll files in \Windows folder, which I doubt can save much (e.g. the effort/space ratio isn't that attractive).
Anyway, I've found the following files on \Windows that seems to be compressable.
ppt.exe (powerpoint)
gwes.exe (???) <-- DO NOT RUN. It will hang your unit.
gdiplus.dll (???)
msxml3.dll (???)
However, I'm not sure if it is a good idea to compress the ppt that is in ROM, and others are unknown programs (e.g. not sure how to test it after compression)
I've also looked around my SD card and found the Tennis.exe game that I have, that is 2.4M in size which were then compressed downed to 2.17M, which is a bit disappointing. I didn't test if it runs on my unit or not.

hanmin said:
Hmm.. so far the responses from the trial seems good. I'll give it a try too later this weekend
Click to expand...
Click to collapse
An pages anywhere with a list of what NOT to compress?

This is an excellent app! I have compressed all of my apps and have not had any problems.
The big ones for me are PlanMaker and TextMaker - both have compressed to a fraction of their original size. Infact I was running a little tight on space I hadn't even installed TextMaker. But now I have compressed both it isn't a problem.
The only strange one so far is ListPro which works fine but loses it's icon.
I haven't tried to compress everything possible, just the apps and may be the main dll.
I have now got about 5mb more storage space than when I started with TextMaker installed.
Excellent!

hanmin said:
Anyway, I've found the following files on \Windows that seems to be compressable.
ppt.exe (powerpoint)
gwes.exe (???) <-- DO NOT RUN. It will hang your unit.
gdiplus.dll (???)
msxml3.dll (???)
However, I'm not sure if it is a good idea to compress the ppt that is in ROM, and others are unknown programs (e.g. not sure how to test it after compression)
Click to expand...
Click to collapse
You're right that compressing something that is in ROM isn't a good idea, that takes up space!
If you have a file in ROM like \windows\PPT.EXE and it is 1MB in size, then compress it to 500KB, copy it back to \windows, you actually USE 500KB of Program Storage rather than saving it.
Your ROM is Read Only, you can't delete files from it or overwrite them.
What actually happens is that you create another file with the same name and the OS ignores the one in ROM.
I do this with ClearStorage.exe.
I created a .TXT file and in it put
Code:
Clear Storage is dangerous.
then renamed it to ClearStorage.exe. I put it in the \windows folder to stop anyone erasing my device by accident(!).
If I need to clear my device I can delete the file and the original ClearStorage.exe is still there.
Try it yourself.
There is no easy way to save space.
If you want to load your apps to ROM and delete the crap ones that come with the OS this will work, but you will have to decode the ROM, edit the hive files, re-encode the ROM and flash it.
This is no small task, but it can be done (I plan on doing it myself actually).

Wow - This is fantastic. In just a short amount of time I have saved several meg of space, and some programs (such as TomTom) run significantly faster.
Superb!
Cheers
Rowan

Amazing. My Big Storage now BIGGER

Some results (on a Wizard)
MS Communicator Mobile - Compressed the exe and one dll down to 33% (1.3mb). The app would not run with one dll (lclang-res96.dll) compressed.
GoodLink - Compressed GoodApp.exe (the biggest file) from 3.5mb to 1.2mb successfully, however by compressing the largest dll (GoodLinkRes_EN.dll) I lost the soft key labels (and probably some other stuff).
TomTom Navigator - 41%
MS VoiceCommand - Compressed from 1.6mb to .6mb
Those were my big files on my device. Using UPX gained me an extra 6mb storage memory - which was a 50% increase (I went from 12 to 18 ). Off to try it now on my Storage Card apps.
Also, .NET files are not currently supported.

Related

Size matters - how to make png's, exe's, tsk's and cabs smaller

Hello, this is one of the first posts, I don't know if you have any interest on this, but here it goes:
fact 1: EXE's and DLL's may be compressed
fact 2: PNG's may be compressed without any quality loss
based on that, I will explain what I do to make smaller CAB's and do not waste so many space when you have those installed.
tools used:
UPX (http://upx.sf.net)
pngout (http://advsys.net/ken/util/pngout.exe)
msceinf (http://www.codeppc.com/telechargements/msceinf/msceinf.htm)
cabwiz (from Microsoft)
let's put this together with a simple example with HTC Audio Manager CAB file:
size before: 1.135.294
size after: 327.204
(yes, 347% reduction, and all works well)
1. used msceinf to decompile the cab, and decompress it to a directory
2. used command "upx *.exe *.dll" on the decompressed directory
3. used command "for %i in (*.png) do pngout "%i" /kp" on the same directory
4. recreated the cab using "cabwiz Audio Manager.inf /compress"
that's it, until now I could regain a lot of space, of course that using exe compression makes programs a bit slower, but I believe there are more advantages than disavantages.
On Themes you can also achieve better compression using the same technique, decompiling the file, then using pngout on them and recreating the cab's back.
some results on a theme file:
Htc_New_Default.tsk original: 106.828
Htc_New_Default.tsk optimized: 44.835
I also saw some problems on some icon packages: sometimes the authors compile them with thumbs.db on it, resulting in complete waste of space.
Regards.
interesting. I never thought about it this way.
Optimized rom chefs, start your engines!!!!
In other news, dig the title.
It should be pointed out that one of the main negatives to upx'ing files is that they take up more memory. Example..a 100k program upx'd to 50k takes up 150k when ran and not all of the ram is released. This is typically why everyone doesn't just go crazy with compressing everything in site. If you need the space and are willing to sacrifice some memory then upx it..otherwise you'd be better off leaving it be. Also..if you are going to use upx or one of the other utils make sure you are using the most recent version to get best performance.
Yes, that's why I wrote that last sentence regarding EXE compression, some figures for CommManager that occupies 508KB as a process, it was tested by reading the in use memory when CommManager was running, and after I forced it's close (to check real memory usage, since process memory reads the same):
using UPX EXE: 33.77-32.89 = 0,88MB
original EXE: 33.67-32.89 = 0,78MB
so it's about 100KB difference for this EXE (since I don't usually run a lot of programs at the same time, I don't care about this), but for PNG's that are used more and more on pocket pc programs, it really makes a difference.
I've updated this with how to optimize Themes as well...
BullGates said:
I've updated this with how to optimize Themes as well...
Click to expand...
Click to collapse
You know there is a upx --ultra-brute switch that basically tries everything to get the smallest size?
Yes thanks. But as you pointed well, it's better to use EXE and DLL compression evaluating your needs. I've rebuilt many cab's concentrating on the graphics compression and I'm quite happy with the results. Smaller CAB's means smaller install times (not very noticed, but it's a fact) less need for storage - if you build your setups based on extended rom storage, this is quite handy. Also you have better performance since less bytes are involved on the loading (so better loading times).
Actually, I was looking to do exactly the same!
I had already found upx and was going to combine this with infocabxp and my own simple parser for the *.000/_setup.xml files, until I found the free msceinf tool - and about an hour later I find your post.
I just wish I found your post first, it would have saved me a lot of time
It is probably best to apply upx compression on exe's and dll's of standalone applications that you start manually or for short time use, and not all background processes as well, but I see there are already some thoughts on that subject
Thanks
How to do this
. used command "for %i in (*.png) do pngout "%i" /kp" on the same directory
Would u please explain more?
Regards
my 3 cents..
there's no reason to upx small files, lol...
40 kb of space may be equal of 40 kb mem less, if dll is persistent/resident(works all the time) - also not every dll/exe may be upxed - it may be broken after upx(i.e. isilo).
.net apps CANNOT be upxed for now.
DO NOT upx today plugins' dlls! they may lose stability/ you may have loading problems etc + mem loss, of course.
you can upx cpl's too.
always crunch at maximal pack setting!
do not pack exe icons, and relocs.
do not try to make xip modules from upxed files!
gfx resources like bmps may became smaller after pallette decrease(it is quite useful while reshacking - i.e. commgr may be smaller = faster launch times etc) regardless of saving method(24,32bit).
almost 100% of pngs, that are used in software/system etc usually loses its size after reedit = faster software loading/ working/less mem used(dialpad res's, home plug, etc) - usually palette decreasing works in same way, as with bmps - smaller file size(yeah, i know, that png is saved as 8bpp, but..sometimes palette may be 4 bit i.e.).

UPX

Ahoi, I want to compress as much as possible of the files in the kitchen with UPX compressor, but as I tried to do so, some of the programs gave bugs (some programs seem to use offsets, which are not corrected by UPX.)
So I would need a list of dll's, exe's and so on,in the file, for which things they are, so I can test the compression out. If it should work, I would post this things out, for everyone.
If someone want's to help me, help is welcome!
UPX can compress files for more then 50% (some up to 80%!!!!)
wfg. - Miha
will be here in 2-3 days!
Please!!!! I need help in this TASK!
there are far too much dll's and ocx's and exe's, to do it all by myself!!!!
Or I will just compress the files and upload them somewhere for testers... or so...
I think somebody (not sure who it was, maybe Tuatara, but search always work) made a research on this and found out, that it does not make much sense after all and brings more problems, than benefits (slowdowns, more memory used, compatibility problems, problems you mentioned). Only some selected files (biggest and rarely used) can be UPXed for good.
Works fine
actually I have an improvement at all! Better speed! more free space!
And - UPX can be decompressed, so I will re-upload the citchen completely upx'ed.
Compressed with 2.9 alpha (or so, not shure anymore....) and I'm going to test the new 3.01 version, and I am also going to install this one:
UPX4PPC
How to do you judge the improvements in speed? As with any compression technology, it will slow down your system. However, for PPC, there is a fine line in between that due to the slow reading speed of the storage card. However, if that were to be something on the device itself, chances are, probably not. Need to be careful on your benchmarking.
Personally, I would not do much on the system stuff, for which the problem may not be that visible (i.e. bug may not appear on a simple run). As jakubd mentioned above, you ought to just pick a few large files for your compression, which I dont think it is a lot. I played with UPX before and trying to look for big files to compressed.. and it seems that there isn't much around to play with. The only thing I remeber that i've found the last time is the powerpoint.
It works!!!
1) I have compressed almost all files and it didn't work. Than I uncompressed some other files again, in the \SYS\OS folder, and - it works!!!!
There is no speed impruvement for flashing the rom trough (it is still 32mb!!!), but there is much more space!
Even the coldstart runs a little bit faster now (only 3-5 sec. but more than nothing).
UPX works better than zip or rar, just read on the HP!
And - you have several MB's in small files, I have compressed them aswell! Youn know, a small cent makes the big dollar (or something in this way).
Folders Compressed_|_Original___|_Compressed
--------------------|-----------|--------------
OEM Folder________|__6.835 kB_|__4.409 kB
SYS______________|_79.927 kB_|_58.760 kB
____It isn't much, but - more than 20.000 kB? (No Word and Excell!!!)
And now I'm testing it, until now - no issues!
2) I have compressed some files in "Program Files" folder - it works aswell!
(CAUTION! MANY PROGRAMS START TO WORK WITH SMALL OR BIG BUGS!)
==> PhoneAlarm Loads FASTER!!! PhoneAlarm Skin editor (around 70kB uncompressed, less that 32kb compressed) loads much longer! So every programm (*.exe) and library (*.dll) should be tested!
Adobe Reader starts two times faster, or more!!! (I haven't used a timer or so....)
DO NOT compare UPX with zip-folders or similar!
Question: Any testers around here? Should I upload?
Changes: another Calc, GS-finder is fexplorer and not cecmd, cecmd included!
Skreen Rotate, Office - excluded (I use other proggies for that - IndiBar and SoftMaker Office 2006)
Because I puted some programs, which are not originally maid for Kitchens and so on, I would ask the proggers, before to re-post the Kitchen btw. to post my mod.
So if anyone wants to have it, or to test it aswell - go on!
PS: It could be, that I included some bugs into the Kitchen, which are not connected to UPX - I do not expire any problems until now! (testing for already 3 days!).

New reg-hack: MemHack. Pre-Alpha.

I fiddled around with my wm6'd hx4700 (thanks kuzuhara!) and a registry editor and found a way to make it regard SD as ram and xip from it.
This renders my device to achieve usage smooth enough i allow myself to reffer to the device as a mini-umpc^_^
For SD only atm, miniSD may or may not work depending on your devices driver. No MMC for the time being.
I sooooo hope this isn't old news
Here's to me not ****ing any of your devices up, it worked wonders for me.
P.S
IMPORTANT!
Delete the Program Files folder and reset again once you see the SD's been renamed to Program Files2.
On the next boot it'll mount as Program Files and everything will load from it.
Excuse me, my brains are stuck in google translator mode
4.11.2007
It seems i was rather in the wrong and am taking a whole new direction.
One which involves reading more carefully before i charge head into the guts of windows embedded.
Will take me a while until next post, and it might become hx4700 exclusive, but i promise you some cool stuff to come yet.
Anybody wanting to make their SD into Program Files to save space,
just start resco reg editor and change hklm\System\StorageManager\Profiles\SDMemory\Folder: Program Files.
(Originally is SD Card).
You can also try messing with it, making it into diffenent sys folders, just be careful
Mine is currently application cache (Volatile) and [email protected] Files.
(Running sweet with what i got to with the enablement of complete graphics accelaration, except for a minor hang up when returning from sleep).
Sooooooooooo...
ONLY DOWNLOAD AND APPLY THE CURRENT PATCH IF YOU KNOW WHAT YOUR'E DOING AND WANT TO GIVE BEATING ME TO IT A SHOT!
I'm hoping for a next release in a day, but not necessarily of this project but of an off-chute.
BTW, is it just me or is WM real *nix like?^^
anyone tried this ?
ranasrule said:
anyone tried this ?
Click to expand...
Click to collapse
Downloaded, but waiting to see what others think.
Want to know what it does to the SD card and device as the readme does say to move the devices contents of its program files folder to the sd card?????
downloaded it too. but i notice its his forst post.
Lol... it's my first post, sorry..
All it does is make the device call the card drive "Program Files" and set a swap file that's used as most device memory (don't worry, it's faster than most none-nand flash). i had to leave a small buffer in device memory though.
The memory file is unallocated, so just don't forget to leave 528MB free at all times.
Re-read the original post
It is and will be updated with periodic updates should they come.
I've been using it for 72 hours now.
It takes the odd reset to free up memory, but nothing worst than what we had before.
well if all thing got wrong.. anyway to revert it back?.. well ofcourse hard reset is one of it. uhmm.. lets put hardreset aside first..
well.. cant wait to test this.. i made a whole backup of my registry so if thing went wrong.. ill just restore the registry backup..
Good point doube_ofour.
Just export the StorageManager registry key before applying the changes, and if worst comes to worst restore your backed up original.
So, anybody else installed and can connect to tell the tale?
DuperMan said:
So, anybody else installed and can connect to tell the tale?
Click to expand...
Click to collapse
I made a cab with the regs and installed it But I'm still experimenting .. Nothing bad happened so far how do i know the device is faster ?!! how did you measured your device's speed ?
So let me get this straight -- it will use the microSD in my TyTN as RAM -- the icky sticky stuff I run out of because internet explorer, pocket player, and spb mobile shell grab it all for themselves?
Is there any way to specify how much of the SD card is used as RAM -- since I don't really need 1 GB of RAM on my device.
I saw your post about 558 MB free but didn't quite understand it. You also said you need the odd softreset to free memory? but why would you need to free up a few MB when you have access to hundreds?
But thanks for sharing with the community -- if this really does work out well for us, that's pretty freaking amazing.
@DuperMan
Does this hack simply make our "Storage Card" transform into "Program Files"?.. Thats it?.. cant see any speed improvement.. besides my memory usage increased by that means my device will perform slow..
My "Program Memory and Storage Memory" Stays the same as 47mb total on both memory..
Exactly Ragart!
Only it's set to use miniSD, i think you'd have to mess with it a little to make it work on miniSD... if response is good i'll make a mini version.
Remember - It's only been tested on wm6 hx4700!
double_ofour said:
@DuperMan
Does this hack simply make our "Storage Card" transform into "Program Files"?.. Thats it?.. cant see any speed improvement.. besides my memory usage increased by that means my device will perform slow..
My "Program Memory and Storage Memory" Stays the same as 47mb total on both memory..
Click to expand...
Click to collapse
in my case my program memory decreased by something around 10mb ! and I can't see any improvement in speed too ...
miniSD, hmm? Well that's the slightly-larger brother of my 1GB MicroSD. So I guess I'll just watch the thread and see others' reactions before I try anything (though I do have fullbackups)
Thanks for the quick response. So how much RAM does your hx4700 'have' now? and did you set it to 558 extra for a reason? Just curious questions from a person who doesn't understand the intricacies of reg editing
@doube_ofour:
Use it for a while, constant usage should be smoother. Did you free the space up by deleting the old Program Files as well?
Use it to surf for a while, multiple windows style, and you'll notice the difference.
Also, don't be mean to my hack Don't load it to spite, wait for each app to load and keep on starting apps without (majorly) worrying about a system crash.
@hma4:
The age old method of feel
It doesn't boost anything per say, but your device shouldn't be as compaction ridden as it was, so less mem degradation over time=more time between resets under heavy work loads. and i mean heavy. also, multi-tasking is MUCH more usable.
@ragart:
I set it to have a 512Mb paging area on the SD, plus 512kb in main for caching and another 16Mb for caching on SD, so that's a total of 528.5Mb overall mem, 16.5Mb cache and half a gig paging (good for them internet chaches etc..
i check on the registry that u have made in ur reg hack. seems like all the folders etc is the same even for a miniSD card ... so do u think it will work on miniSD too?

Batch upx compress (compress whole folders of exes)

Hi folks,
In the course of making my own ROM (something I admit to being new at) I found it rather tedious to individually compress exes with the upx utility and batches included with HyperCore kitchen. I've made a batch job that will look through all files in an input folder and compress all the exes (so you can just copy your entire SYS and OEM folder out of your kitchen, compress their contents and copy back). Perhaps something to do this already exists, but if so I couldn't find it. I've also included a more recent version of upx with this than was in HyperCore.
Hopefully this will save someone a little time.
To use:
1. Copy/Move folders containing exes (I suggest just copying the whole SYS and OEM folders) into the input folder.
2. Run compress.bat
3. Copy folders in output back to your kitchen.
While UPX is good for compressing PE files (which results in a smaller output file), one should point out, that it's a tradeoff. You get a smaller file, but a higher startup latency, and sometimes a tad larger memory footprint. So UPXing is not good for responsivenes of a device.
UPXing small .exe files may then be beside the point. You'll gain for eg. 5KB but you'll loose a second every time you load the file.
UPXing usually makes sence on executable files larger then 200-400 KB, and smaller then 15MB.
You can google for UPX performance for more information.
Just my 5 cents, and I think it's important for cookers.
NIXin said:
but a higher startup latency
Click to expand...
Click to collapse
mehehehe.
heh. it was not really good text, imo(maybe it is, on e-105..but even omap have VERY fast decrunch speed).
4 cents are ok, in general...
my one cent:
test EVERY upxed dll, EVERY upxed exe, some things will be broken after operation(isilo - i.e.).
very good idea is to pack office pack exes(much space for useful soft while cooking).
.net apps CANNOT be uxped for now.
imo, batch upxer is just NOT good idea...
Good input from both of you, particularly in regard to testing the executables. There is an option to force byte-identical decompression, perhaps it would be a good idea to enable this? I am aware that there is a trade-off in performance with compression, but reading the readme on upx one notes that decompression is very fast. Even assuming our ppcs are 10 times slower than the Pentium 133 decompression would still be 1MB/sec, and most of our exes are much smaller than that.
Just curious if this case where files don't work after compression is a common one (noting I'm not compressing dlls as they are unable to share data when compressed; this is noted in the upx documentation, where they comment it's probably only a good idea to compress things like plugin dlls). Perhaps someone with more experience can comment on this?
caeci11ius said:
Good input from both of you, particularly in regard to testing the executables. There is an option to force byte-identical decompression, perhaps it would be a good idea to enable this? I am aware that there is a trade-off in performance with compression, but reading the readme on upx one notes that decompression is very fast. Even assuming our ppcs are 10 times slower than the Pentium 133 decompression would still be 1MB/sec, and most of our exes are much smaller than that.
Just curious if this case where files don't work after compression is a common one (noting I'm not compressing dlls as they are unable to share data when compressed; this is noted in the upx documentation, where they comment it's probably only a good idea to compress things like plugin dlls). Perhaps someone with more experience can comment on this?
Click to expand...
Click to collapse
While compression is necessary to fit some things in it really ought to be a last resort. I recall the author stating that a 100k program compressed to 50k requires 150k of memory to expand and run and it doesn't free up all of the extra memory needed...so there has to be a balance or you are going to be using up all your memory to save storage....if the app HAS to be installed to phone storage then upx is something to consider but I wouldn't recommend globally upx'ing all your apps on your minisd for instance.
.net apps CANNOT be uxped for now.
Click to expand...
Click to collapse
Because .NET files don't have the standard PE header. It's a requirement for UPX (or any other executable file packer) (PL: pozatym witam )
Mr. famewolf just confirmed what I have said and even added some more. It's perfectly okay for PCs, where you don't notice the difference, but PPC's are just way to slow and the gain is too little (if any). Batch method is not the way to go, unless you wan't to "cripple" your PPC.

sygic cant calculate routes above 250 km

hello fellow chefs
i have a problem that whas brought to my attention by a user of my blackstone rom
he uses sygic as navigation and he wanted to calculate a route from netherlands to france and it cant finish calculating as it says it runs out of memory:s
when i tap the taskmaanger it mostly says arround 60% so i gues there is enough also when i look at storage in taskmanager it says
total 191.05 mb
in use 93 mb
free 98.05 mb (these valeus are after a soft reset with 49%)
there is more free then in use so i guess there is enough
also he pointed me to a program where you could test ram wich isnt working on my rom but is working on multiple other roms he has used:S
i also tried it myself by downloading sygic and i came to the same conclusion:S
also the program tells me the same when i want to do a ram check
when i start the program it says at least 21.25 mb free space is needed on the storage device
when i look at taskmanger at the storage section it tells me i have 255 mb free:S
you guys can downlaod the program here: http://rapidshare.com/files/334876190/pocketmechanic.2.99.283.CBD.zip
its a free program so no warezlink
when you instal the program and open it you have the option card benchmark
there you can choose over wich memory you want to perform it
when i say ram it tells me 15% of my ram is used and
302.25mb, 84.68% free
by my calculation 84% of 302 is way more then then the 21 mb the cecker talks about:-o
hope some one can help me solve this strange ram problem as i think the calculation fails of the strange ram thing
edit: gonna put the memorymap from my buildlog here maybe you can see something there
as it says by ram 43 free:-o still its more then the 21 the ram chacker talks about:s
Code:
Memory Map...
SLOT 0: 0x02000000 - 0x018b0000 (END: 0x00060000, 0 MODULES)
0x02000000 - 0x01fc0000 - ROM 0
0x01f80000 - 0x018b0000 - ROM 1
SLOT 1: 0x04000000 - 0x02020000 (END: 0x02020000, 211 MODULES)
SLOT 60: 0x7a000000 - 0x78e40000 (END: 0x78020000, 52 MODULES)
SLOT 61: 0x7c000000 - 0x7a022000 (END: 0x7a020000, 239 MODULES)
RAM IMAGE: 0x80000000 - 0x803e6560
RAM: 0x803e7000 - 0x80475000 - Used for kernel modules
0x80475000 - 0x83000000 - 43 MB free
bumping my 1 post question and got another one
can someone share the htc in call recoreder package including certificates?
thanks in advance
It's probably running out of virtual memory in the slot used by sygic.
Farmer Ted said:
It's probably running out of virtual memory in the slot used by sygic.
Click to expand...
Click to collapse
and can this be changed cause he says he only has the problem with the rom i make:S
How big do you have the datacache (and others in boot.rgu) set? Maybe that's a problem, if it's too big. You should run virtualmemory.exe while the app is running to see if the problem is virtual memory management.
every setting i have is found here on xda so i gues everybody should have the problem then as all of them are in hd2 general section
also the bootrgu i have changed one thing there wich is
[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\FLASHDRV\FATFS]
"DataCacheSize"=dword:00001000 ;4096 sectors(4096*2048=8MB)
normally this whas at 4 mb but found it in a pdf from da_g i believe that you could set it to 8 but on the default valeu and the 8 mb valeu does the same
so i dont thing its this setting thats interferring:-o
will try that program you mentioned
anything special i need to look at there
edit: any chance you can point in the direction where to download that program as i cant seem to find it only source code:S
edit2: never mind after some more googling i found you already posted it somewhere thanks
maybe bit of a noob question but where does the program puts the snapshot:-o
i made two but i cant seem to find them stored anywhere:-o
You need to take a screen-cap; it doesn't save the snapshots for you.
Farmer Ted said:
You need to take a screen-cap; it doesn't save the snapshots for you.
Click to expand...
Click to collapse
lol
i saw snapshot so i thought it whas to make a picture
here is a foto of the phone in idle after a day of use
i first did the snapshot as im not sure what it does
can you see something strange on the picture?
So, you don't have sygic running in that one? Slot 15 is pretty porky; what process is that? You need to scroll left and right to see the processes for each slot. You may need to run vol d-pad to do that on the HD, though. Slots 9 and 10 have low-lying dll's (or some other resource). I suspect that what happens is that sygic loads a resource that is even lower, and maybe it crunches into slot 2 (filesys.exe) or the process in 15. The other possibility is that sygic creates really large heaps when it's doing big calculations (>250 km), and then the memory grows up in that slot to the level where those lower lying dll's are in slots 9 and 10. Even though those dll's (if that's what they are, and not some type of static resource) may not load into the sygic slot, the memory manager still accounts for them in all of the VM slots, so they will limit how big the heaps can grow in any process.
If I were you, I'd run sygic and do a VM snapshot. Also, get devhealth.exe (search on it), run it, and see what those dll's are in slots 9 and 10. They may be the issue. Mainly, see if their shared or static (like a .plg file in tcpmp). Of course, I could be totallly wrong about this, lol.
Farmer Ted said:
So, you don't have sygic running in that one? Slot 15 is pretty porky; what process is that? You need to scroll left and right to see the processes for each slot. You may need to run vol d-pad to do that on the HD, though. Slots 9 and 10 have low-lying dll's (or some other resource). I suspect that what happens is that sygic loads a resource that is even lower, and maybe it crunches into slot 2 (filesys.exe) or the process in 15. The other possibility is that sygic creates really large heaps when it's doing big calculations (>250 km), and then the memory grows up in that slot to the level where those lower lying dll's are in slots 9 and 10. Even though those dll's (if that's what they are, and not some type of static resource) may not load into the sygic slot, the memory manager still accounts for them in all of the VM slots, so they will limit how big the heaps can grow in any process.
If I were you, I'd run sygic and do a VM snapshot. Also, get devhealth.exe (search on it), run it, and see what those dll's are in slots 9 and 10. They may be the issue. Mainly, see if their shared or static (like a .plg file in tcpmp). Of course, I could be totallly wrong about this, lol.
Click to expand...
Click to collapse
muchias gracias for youre help farmer ted
i opend up that virtual memory again and changed to different things
then i opend sygic and took a snapshot again
as you can see its going into slot 14 and every time i took a snapshot the bar went up showing more green(green means in use?)
i just looked up that slot without sygic and it tells me its the slot of tmail.exe
dont think tmail does much in that slot or its a big thing
also on the end i have all those empty slots cant it be reasignd to a new slot so it has a full bar(see the screenshot with sygic calculating a route)
will also try the program you mentioned when i get back from doing my things today as i have some appointments
edit: see you also asked what slot 15 is when i look it up it tells me its manilla.exe and im running sense2.5 from leo 3.04 so it could be heavy
Well, that's it, then. It sounds like sygic has a massive memory leak, if the heap size keeps growing. And this just confirms what I've known since the first day I got my Fuze: Manila sucks, lol.
I'm not sure why the slot 9 and 10 dll's are loaded so low. What processes are those? Maybe something in startup is causing it, and can be fixed.
Farmer Ted said:
Well, that's it, then. It sounds like sygic has a massive memory leak, if the heap size keeps growing. And this just confirms what I've known since the first day I got my Fuze: Manila sucks, lol.
I'm not sure why the slot 9 and 10 dll's are loaded so low. What processes are those? Maybe something in startup is causing it, and can be fixed.
Click to expand...
Click to collapse
thank god inever uused sygic then as it seems a stupid deep going problem
as for manilla sucks im not sure about it
the hd is my first htc device and i think the experience is nice(better then all samsungs i had
for the slots i will write them all down maybe thats easier then asking what is this and what is that(check code box below)
further when i look at windows/startup i have three things there
htc startup, pkg, and poutlook
the htcstartup also has a speaker infront of it with a x behind the speaker(hope you get what i mean)
Code:
slot1: rom dlls
slot2: filesys.exe
slot3: cprog.exe
slot4: backportrait.exe
slot5: device.exe
slot6: gwes.exe(this is a colorfull rom so i can imagine its a little high:p)
slot7: shell32.exe
slot8: virtualmemory.exe
slot9: fexplore.exe
slot10: services.exe
slot11: connmgr.exe
slot12: repllog.exe
slot13: empty
slot14: tmail.exe(sygic will be loaded in this one)
slot15: manilla.exe
slot16: jblenddaemon.exe(have read this can turned off from startup to save a little ram what do you think about that?)
slot17: commmanager.exe
slot18: empty
slot19: empty
slot20: sapsettings
the rest are all empty
I would say remove jblenddaemon.exe; it must be a java app? If you don't use java, there's no reason to have it running. You might sapsettings.exe from startup as well, if you're not using it at all. I removed it a long time ago, but I don't use bluetooth much, and when I do, it works fine without sapsettings. I think I read that it's for connecting to you car or something for hands free.
It's weird that you have the lower lying dll's (if I'm seeing what I think I'm seeing in slots 9 and 10). They definitely seem to be the wall that driver.exe is running into. I also don't know what's up with fexplore.exe. It's using a lot of VM. Is that the extended file explorer? Last time I used it, it would eat a lot of ram if you highlight pics and it showed the previews.
Farmer Ted said:
I would say remove jblenddaemon.exe; it must be a java app? If you don't use java, there's no reason to have it running. You might sapsettings.exe from startup as well, if you're not using it at all. I removed it a long time ago, but I don't use bluetooth much, and when I do, it works fine without sapsettings. I think I read that it's for connecting to you car or something for hands free.
It's weird that you have the lower lying dll's (if I'm seeing what I think I'm seeing in slots 9 and 10). They definitely seem to be the wall that driver.exe is running into. I also don't know what's up with fexplore.exe. It's using a lot of VM. Is that the extended file explorer? Last time I used it, it would eat a lot of ram if you highlight pics and it showed the previews.
Click to expand...
Click to collapse
thanks for the quick overview
the jblenddeamon is the java app
i personally never used it but there are some people using my rom who do use it:-o
guess it will be activated when they start up jblend
im gonna build a rom with jblend and the sapsettings taken out of the bootlauncher and gonna try it again to see what happens
also the file explorer is the normal fexplorer from the 21911 sys
it looks to me the verry same as every other file explorer i have seen on other roms so no preview of pictures you think i need a different fexlorer.exe?
and if so can you recommend one
I was just wondering if you were using the extended file explorer (2.06, I think; by Houming). It's pretty cool, but I personally use total commander and almost never use file explorer, so I don't use the extended on anymore, either. I was never able to make a package with it that worked, so I just did a cab install during customization. The cab is kind of weird, and I can't say I understand how it works. It actually loads up two file explorers (should have remembered that, and known yours wasn't the extended one). I attached the cab for 2.06-I packed this one, and it's compressed and doesn't have the uninstall crap loaded into it.
Sapsettings just doesn't seem to do anything. It doesn't use much memory, but you get quicker boot times when you take stuff like that out of the queue. If anyone really needs it to run, you can just give them a shortcut. You can always just remover the registry keys from your own device after a flash.
Farmer Ted said:
I was just wondering if you were using the extended file explorer (2.06, I think; by Houming). It's pretty cool, but I personally use total commander and almost never use file explorer, so I don't use the extended on anymore, either. I was never able to make a package with it that worked, so I just did a cab install during customization. The cab is kind of weird, and I can't say I understand how it works. It actually loads up two file explorers (should have remembered that, and known yours wasn't the extended one). I attached the cab for 2.06-I packed this one, and it's compressed and doesn't have the uninstall crap loaded into it.
Sapsettings just doesn't seem to do anything. It doesn't use much memory, but you get quicker boot times when you take stuff like that out of the queue. If anyone really needs it to run, you can just give them a shortcut. You can always just remover the registry keys from your own device after a flash.
Click to expand...
Click to collapse
just a quick update as im in a hurry
need to go away to arrange some things for my house in 15 min and i still need to shower:-o
i made a rom without those two thins and damn the rom has bacome faster
also the virtualmemory looks a lot emptier:-o
will post screenshot later today when im finished arranging things
also tried sygic and it managed to calculate alot further but still missing 200 km of the route i want it to be calculated:-o
do you know the differnece between poutlook and tmail.exe?
maybe i can take one of those two out
I'm not really sure what the difference is, to be honest. I don't run either at startup. I don't text much and almost never email, but regardless, you can still do it w/o any issues. I guess maybe outlook loads quicker with one or the other running. I run 11 processes at startup:
nk.exe
filesys.exe
device.exe
gwes.exe
services.exe
shell32.exe
connmgr.exe
cprog.exe
AEBPlus.exe (AE Button)
SpeedBmonitor.exe (Speed Booster; sort of helps, doesn't use much memory)
XHook.exe (XTask task manager)
I also have a shortcut that runs at startup to expand the file dialogue and another to activate resco keyboard, but those processes only run a few seconds each.
That's just 11 processes; the first 6 are mandatory, the next 2 are needed for most things (although you can kill them for maybe a little better performance gaming or something like that-you definitely don't need cprog on an airplane), and the latter 3 are just apps that I run/use all the time. I don't miss any of the other processes, and with fewer processes going, you get faster boot times. My fuze usually boots in 60-65 seconds, which is pretty nice.
well used the rom for a day now as i flashed it yesterday evening before i went to bed
the strangest thing is jblend has activated itself:-o
wonder what triggerd it to get activated as i havent opend java:S
i only used messenegr a little today but its windows its own messenger so i dont think its java related:S
you have any idea what could have opend the jblend?
also the order of what is where has changed but i dont think that does matter as i think all slots are equal of size or have they different sizes?
i will also post a screenshot of the new virtual memory but with jblend in it and sygic gets loaded in that one when i start it up:S
im totally starting to hate this program
you think the screenshots looks better now on how everything is managed?
also i read somewhere if i format my memorycard with bigger clusters it will become faster
what do you think about that?
could this speed up route calculating as the maps or on my sd card
also i will try to remove some more from the startup and hopefully this will give me better sygic calculating things
again thansk or helping em with this probblem as i think i cant solve this alone
Code:
slot1: rom dlls
slot2: filesys.exe
slot3: backportrait.exe
slot4: virtualmemory.exe
slot5: device.exe
slot6: gwes.exe
slot7: shell32
slot8: fexplore.exe
slot9: manila.exe
slot10: services.exe
slot11: connmgr.exe
slot12: cprog.exe
slot13: poutlook.exe
slot14: tmail.exe
slot15: commmanager.exe
slot16: jblendeamon.exe
the rest are all empty
I would check the jblend package and make sure there's not a reg key that's starting it up in the .reg file. The order of processes in the slots depends on the order that the processes are booted up. You can get some processes in an early slot just because there's some transient processes initiated during bootup that only run briefly, then close, which re-opens the slot for the next process in line. If you kill cprog, then start another app, it will go into cprog's slot, so then if you restart cprog, it will be in slot 13 or something like that.
Larger cluster sizes definitely speed up an sd card; I have a 16 g card, which normally had 32 kb clusters. I lowered it once to 4 kb clusters, which freed up maybe 100 mb of memory, but scanning the card became nearly impossible. Normally, if I run scandisk (I do it daily), it takes 5 min. But it would take 40 min or so with the smaller cluster size. Also, benchmarking showed that writing to the card was a lot slower (reading too, I think).
You still seem to have some low lying dll's in slots 9 and 10, which is weird.

Categories

Resources