Related
Look at this (from 1:44 on):
It's a quadrant benchmark run on a android port on the HD2. Graphics are really bad, but in the end it has approximately the same score as the benchmarking score of the Galaxy with the original firmware. I mean what is in a score? If I look at the beginning of the movie, the UI is very slow and not as responsive as the Galaxy
(BTW i got 55.7 FPS with the neocore benchmark on JM2)
This is not to say that I don't have deep respect for what the HD2-android development team is doing. Really amazing job. I just can't wait to get my HD2 back from repair.
appelflap said:
Look at this (from 1:44 on):
It's a quadrant benchmark run on a android port on the HD2. Graphics are really bad, but in the end it has approximately the same score as the benchmarking score of the Galaxy with the original firmware. I mean what is in a score? If I look at the beginning of the movie, the UI is very slow and not as responsive as the Galaxy
(BTW i got 55.7 FPS with the neocore benchmark on JM2)
This is not to say that I don't have deep respect for what the HD2-android development team is doing. Really amazing job. I just can't wait to get my HD2 back from repair.
Click to expand...
Click to collapse
Quadrant scores have been criticized for their non-descript breakdowns, at least on their free suite. Also, the fact that they chose the weighting of the scores, so should they chose 2D is equal to 3D weight, I don't know their formula (and for all I know, they give equal weighting to all or they give equal weighting to all test where the CPU has 12 tests and the 3D graphics has 4), but the fact that we, as users don't have access to their formula on their website is a bit unnerving.
Add to that the fact that many reviews and videos rely on it so heavily leaves users a bit misinformed. In reality, and thorough review should definitely run a custom test suite to give individual scores to:
CPU
Memory
I/O
2D graphics
3D graphics
That way users can compare what's important to them. The Galaxy S suffers from terrible I/O and the hacks that have given the fixes typically boost Galaxy scores to nearly double their rates, and it's majorly attributed to improving a bunk I/O score.
Totally agree. In addition, it would be really nice to know which benchmarked factors are responsible for which functions. For example it is really interesting to see how the hd2 performs before the user is running the tests. When the user is scrolling through the setting menu there is a very noticible lag. Given the fact that the total score is nearly the same as the scrore for the SGS, and thar the graphic score of the hd2 is bad in comparisson to the SGS, I would conclude that graphic performance is very important for the way the ui responds.
Sent from my GT-I9000 using XDA App
appelflap said:
Totally agree. In addition, it would be really nice to know which benchmarked factors are responsible for which functions. For example it is really interesting to see how the hd2 performs before the user is running the tests. When the user is scrolling through the setting menu there is a very noticible lag. Given the fact that the total score is nearly the same as the scrore for the SGS, and thar the graphic score of the hd2 is bad in comparisson to the SGS, I would conclude that graphic performance is very important for the way the ui responds.
Sent from my GT-I9000 using XDA App
Click to expand...
Click to collapse
From what I can tell, the HD2 got a decent score 'cos it was running Froyo. When we get bumped up to an official froyo build with JIT fully optimized, We should be top of the pile.
don't forget, android isn't working 100% on the HD2.
I personally think it's pointless comparing to a not complete port.
woops dbl post
alovell83 said:
Quadrant scores have been criticized for their non-descript breakdowns, at least on their free suite. Also, the fact that they chose the weighting of the scores, so should they chose 2D is equal to 3D weight, I don't know their formula (and for all I know, they give equal weighting to all or they give equal weighting to all test where the CPU has 12 tests and the 3D graphics has 4), but the fact that we, as users don't have access to their formula on their website is a bit unnerving.
Add to that the fact that many reviews and videos rely on it so heavily leaves users a bit misinformed. In reality, and thorough review should definitely run a custom test suite to give individual scores to:
CPU
Memory
I/O
2D graphics
3D graphics
That way users can compare what's important to them. The Galaxy S suffers from terrible I/O and the hacks that have given the fixes typically boost Galaxy scores to nearly double their rates, and it's majorly attributed to improving a bunk I/O score.
Click to expand...
Click to collapse
Even then though, it's possible to write a benchmark which wins constantly for any phone.
In regards to "terrible I/O", that might even be due to a bug in the FAT32 drivers. Yes you can benchmark it, but it wont mean much. The best way is to actually TEST the applications you need, rather than select a phone based on benchmarks. However, you are possibly best off looking at the component specs, because they ignore software bugs.
scrizz said:
don't forget, android isn't working 100% on the HD2.
I personally think it's pointless comparing to a not complete port.
Click to expand...
Click to collapse
But the topic is about "what's in a score". Maybe one can generally say that is pointless to compare devices this way. I think that such benchmark scores are only (a bit) relevant at the two poles of the benchmark score spectrum. Everything in between can be neglected due to the uninformed way sub-scores are evaluated.
You got 55.7 FPS on Neocore as the sgs has vertical sync enabled, the refresh rate on the sgs'es screen is 56 fps and thus you can only go up to 56 fps as the v-sync is on. This proves that the sgs is indeed a much more powerful device that is actually being held back. If you can disable the v-sync then you can get a higher fps score
appelflap said:
But the topic is about "what's in a score". Maybe one can generally say that is pointless to compare devices this way. I think that such benchmark scores are only (a bit) relevant at the two poles of the benchmark score spectrum. Everything in between can be neglected due to the uninformed way sub-scores are evaluated.
Click to expand...
Click to collapse
I just read in a post that the Galaxy S gets a 0 on the 2D score:
"JIT isn't fully enabled in the current froyo versions, and quadrant, frankly, is bull**** (for exmple, 2d acceleration gets the same weight in the final result as 3D. Due to the fact that the SGS doesn't have a dedicated 2D accelerator, quadrant doesn't try to use the cpu- it just gives a round zero in that part)"
I can't confirm this, but that definitely seems like a terrible set-up, seeing as how I'm pretty sure I have games run in 2D, so to say that it can't do it just seems wrong regardless of if the SGS has a dedicated 2D accelerator or not (so if you aren't testing the way it performs in real-world, why are you testing?)
http://forum.xda-developers.com/showthread.php?t=737787&page=3
Qazz~ said:
You got 55.7 FPS on Neocore as the sgs has vertical sync enabled, the refresh rate on the sgs'es screen is 56 fps and thus you can only go up to 56 fps as the v-sync is on. This proves that the sgs is indeed a much more powerful device that is actually being held back. If you can disable the v-sync then you can get a higher fps score
Click to expand...
Click to collapse
It isn't really being held back - the screen can't display more than 56 fps as you say, and it wouldn't really be visible even if it could. Disabling v-sync isn't really that important, we need a benchmark that can actually use the advanced features in the SGS GPU (Neocore just pushes a fairly small amount of polygons with no real extras.) Using current 3D benchmarks to benchmark the SGS is like using quake 1 to benchmark the brand new ATI/nVidia cards.
The benchmark is what is at fault here, not the device
RyanZA said:
It isn't really being held back - the screen can't display more than 56 fps as you say, and it wouldn't really be visible even if it could. Disabling v-sync isn't really that important, we need a benchmark that can actually use the advanced features in the SGS GPU (Neocore just pushes a fairly small amount of polygons with no real extras.) Using current 3D benchmarks to benchmark the SGS is like using quake 1 to benchmark the brand new ATI/nVidia cards.
The benchmark is what is at fault here, not the device
Click to expand...
Click to collapse
I don't want to speak for the other poster, and I agree with your premise, however, it isn't actually solving the issue at hand. Better FPS wouldn't be noticed, however, it would give a better score and, more importantly, indicate it's potential. So, getting 56FPS isn't doing the phone any justice within the score, which is what reviews are using, giving it an artificially low score, and putting it more in line with units that can't compete on higher end games. So, when a site like anand pushes 150FPS on a game, I know that means that their rig is entirely too powerful for the game in question, but it still means something when you compare it to the lower end graphics card that only gets 90...then when they run Crisis you see these results play out more with differences that we can notice with the eye.
I think the HD2 gets that score because, as I can see in the video, the CPU tests run faster compared to my SGS, probably because of Froyo, and I know, from the time I had the Diamond and the HD2, that the internal memory and RAM are very fast. Sadly SGS has a slow internal memory, atleast when used by the phone`s software, when copying from PC is faster than my class 6 microSD. Luckily, we have mimocan`s fix. Hope this will be fixed in future FW`s.
NexusHD2 with-FRG83D V1.7 with hastarin r8.5.1 On my HD2 got 1920 in quadrant,31.5 on neocore, and 37 on linmark.
The lag might be because you are using launcher pro, I use launcher pro and sometimes it makes the the lock lag on my phone but it doesn't happen when I use the default lock also if you have alot of Widgets on your screen it will cause lag also
appelflap said:
Look at this (from 1:44 on):
It's a quadrant benchmark run on a android port on the HD2. Graphics are really bad, but in the end it has approximately the same score as the benchmarking score of the Galaxy with the original firmware. I mean what is in a score? If I look at the beginning of the movie, the UI is very slow and not as responsive as the Galaxy
(BTW i got 55.7 FPS with the neocore benchmark on JM2)
This is not to say that I don't have deep respect for what the HD2-android development team is doing. Really amazing job. I just can't wait to get my HD2 back from repair.
Click to expand...
Click to collapse
same galaxy s scores 6000+ in quadrant with custem roms
The HD 2 is a better fit for quadrent then the sgs as quadrent was made for the snapdragon processor which the hd2 has and the sgs does not. Comparing apples to orenges in an apple juice contest doesn't really prove much. Use real life feel. If you care about the scores a rom can be made to get you over 3000 quad score but is laggy as hell. Don't believe me? Look at my sig
interesting... I was using quadrant to see how a stock xxjvo and gingerreal compared. Surely that would indicate a real speed difference and not just be some kind of "hack" ?
zelendel said:
The HD 2 is a better fit for quadrent then the sgs as quadrent was made for the snapdragon processor which the hd2 has and the sgs does not.
Click to expand...
Click to collapse
That's right.
HD2 uses two android OS :
- Cyanogenmod, that is faster than our samsung os..
- Nexus one's port to HD2, greatly optimized by google...
It's really fast. I upgraded my father's HD2 last month, replacing windows in the NAND with CM7. It really makes a big change, the phone is like brand new ^^
http://forum.xda-developers.com/showthread.php?t=1012556
Quadrant is pretty flawed. And I say that being someone who had a phone (before modifications) that was mid-range in Quadrant (Galaxy S), and having a phone that's right top of the heap (Galaxy S II)
One of my coworkers has a tmobile vibrant with some lag fix according to him.. he did a quadrant benchmark right in front of me and it was showing 2500 plus everytime.. Im very curious as to what is making his phone so fast. And can it be dont to ours. Hes not running a custom rom or overclocking. Im only getting 1030 with mine clocked at 1.2ghz. Any Ideas? I couldnt get into too much details with him yesterday and I dont know whens the next time ill see him..
If you were to look at a test break down you would see generally all the scores are identical or the epic a little ahead except in the read/write area. The scores from their read/write are just inflating their overall score. It's a issue with quadrant and how it handles its overall score. Basically it just makes the system easy to abuse/cheat. So I wouldn't worry much about the difference in your score and his.
Sent from my Samsung Epic
The reason other Galaxy S phones score high in quadrant is because of the lag fix they use. The lag fix mounts a different file system on the phone with DRAMATICALLY increases read-write times. That portion of the quadrant benchmark gets inflated beyond reason. Using this game technique, Cyanogen was able to score more than 3000 on a snapdragon phone.
All of the Galaxy S phones have the same processor. Also, quadrant is a terrible benchmark. It's the most over-quoted and abused benchmark for android phones
Ahh ok.. thats good to know.. so what would be a better benchmark to use? Linpack?
jok3sta said:
Ahh ok.. thats good to know.. so what would be a better benchmark to use? Linpack?
Click to expand...
Click to collapse
Linpack is good for measuring raw CPU processing power... but only on devices running the same version of android. Phones with 2.2 will score insanely high due to the JIT compiler. For example, a snapdragon phone with Froyo can score ~40 Mflops. A snapdragon phone with eclair scores around 7 Mflops. Does Froyo make the phone run 5-6X faster? Hell no. In some cases, the difference is almost unnoticeable to the human eye.
Here is a rundown of what I believe to be the pros and cons of various benchmarks:
Linpack
Pros:
- Good for measuring CPU processing power on the same version of Android
- Great tool for measuring the performance gain from overclocking
Cons
- Scores are boosted unreasonably by Froyo's JIT compiler on snapdragon phones
Quadrant
Pros:
- Great tool for measuring the performance gain from overclocking
- Decent tool for measuring 3D graphics performance (just pay attention to FPS, not the end result)
- Decent tool for measuring 2D graphics performance (again, look at FPS)
- The paid version ("Quadrant Pro" I believe) shows which parts of the benchmark contributed to the score. Easier to spot the inflated CPU or I/O inflation
Cons:
- I/O portion isn't valued as much as others, but can boost scores beyond reason via exploits, hacks, fixes, etc.
- CPU portion is inflated on phones running 2.2. A Nexus One is not faster than any Galaxy S, Droid X, Droid 2, etc.
Neocore
Pros:
- Good tool for measuring graphics processing power
Cons:
- Graphics are not intense enough to push the power of very fast GPU's. Some phones will hit their FPS limit
- Only measures graphics processing power.
Nenamark1
Pros:
- Great tool for measuring graphics processing power
- Effects are advanced enough to show the performance of faster GPUs in relation to phones with lesser GPUs.
Cons:
- Only measures graphics processing power.
Sweet thanks for all the info man..
Agreed, this is great info thanks. I think the quadrant score is the most quoted becuase it provides a very easy to read graph built in with it for instant comparing/gratification. I guess I am gonna start going by linpack and nenamark1.
hydralisk said:
Linpack is good for measuring raw CPU processing power... but only on devices running the same version of android. Phones with 2.2 will score insanely high due to the JIT compiler. For example, a snapdragon phone with Froyo can score ~40 Mflops. A snapdragon phone with eclair scores around 7 Mflops. Does Froyo make the phone run 5-6X faster? Hell no. In some cases, the difference is almost unnoticeable to the human eye.
Click to expand...
Click to collapse
Linpack is ok for when your using same CPU comparison, different CPU's can cause issues...The reason why snapdragon gets scores of 5-6x is for some reason the snapdragon utilizes the VFP rather then using raw processing power..aka snapdragon cheats on the Linpack.
In reality our I/O scores should be a lot higher then it is as even in the Epic some of samsung's crappy file system still exists. But not as high as the lagfixed Vibrant of course.
Quadrant Pro is probably best indicator out of them all(The non-pro version is pretty much useless unless your comparing the same phone)...the con of having 2.2 show is higher is expected as it is a measure of efficiency of JIT in comparison to the current. The OS always played a role in Benchmarks so it is expected.
it can be faked by using a different partition to test on. IIRC the data partition making the speeds much faster than they should be so be careful when accepting those high scores
rjmjr69 said:
it can be faked by using a different partition to test on. IIRC the data partition making the speeds much faster than they should be so be careful when accepting those high scores
Click to expand...
Click to collapse
It is not exactly faking it..as you are increasing performance..thing is you cannot see at what it performs well at unless you see the individual scores from the Pro version....
So I just bought my girl a G2, and I am waiting for the tab. I was very impressed with teh speed of the g2 and its perfomance in all games. I was really wondering since the galaxy s scored even below the nexus one with 2.2 on it in a benchmarking review. how do you think the tab will do?
Everything seems to be the same from the galaxy s to the tab... so does anyone expect the tablet to perform slower than say the g2? Or what about aleast better than the Galaxy s?
Also is the chipset in the tab the same as the one in the galaxy s? Or are their minor differences?
I can assure you that the galaxy tab will be plenty fast on 2.2 I have a samsung galaxy s (captivate) running 2.2 and I never have lag you cannot compare the galaxy s series though quadrant scores as that benchmark is biased to the snap dragon processor which the galaxy s does not run on.
Sent from my SAMSUNG-SGH-I897 using XDA App
laxwillsch said:
I can assure you that the galaxy tab will be plenty fast on 2.2 I have a samsung galaxy s (captivate) running 2.2 and I never have lag you cannot compare the galaxy s series though quadrant scores as that benchmark is biased to the snap dragon processor which the galaxy s does not run on.
Sent from my SAMSUNG-SGH-I897 using XDA App
Click to expand...
Click to collapse
Quadrant scores dont count for everything, and its hardly biased, snapdragon does better at the I/O tests more than anything. My Desire on a myTouch 4G port is quicker than stock Android roms according to quadrant.
Make sense of that
Sent from my HTC Glacier using XDA App
laxwillsch said:
I can assure you that the galaxy tab will be plenty fast on 2.2 I have a samsung galaxy s (captivate) running 2.2 and I never have lag you cannot compare the galaxy s series though quadrant scores as that benchmark is biased to the snap dragon processor which the galaxy s does not run on.
Sent from my SAMSUNG-SGH-I897 using XDA App
Click to expand...
Click to collapse
lol, No it's not.
Stock vs Stock The G2 beat the Galaxy S at every bench mark I've put on them
(5 total) except CPU benchmark and those scores were 812 for the G2 vs 799 on the Galaxy S ( lower the better )
The g2 has the latest snapdragon, it will be a little faster than the galaxy s in cpu benchmarks once overclocked. This is what happens when google coded froyo optimization's for only one cpu type, before that we kicked snapdragon ass
Sent from my GT-I9000 using XDA App
siberslug said:
lol, No it's not.
Stock vs Stock The G2 beat the Galaxy S at every bench mark I've put on them
(5 total) except CPU benchmark and those scores were 812 for the G2 vs 799 on the Galaxy S ( lower the better )
Click to expand...
Click to collapse
JIT.
RFS.
go read up on those before you go around correcting people on what's better than what. Benchmarks don't mean a thing. There's a reason why people like Cyanogen don't trust them.
It's kinda hard or weird (or pointless) to compare SGS and Tab CPU and GPU wise, since they both have THE EXACT SAME HARDWARE as long as those are concerned... They both run on the Samsung-built 1GHz Hummingbird chip (that includes the PowerVR GPU). So as long as only CPU/GPU are benchmarked, hardware-wise the SGS and Tab are identical.
Now, comparing those two on a general level makes more sense as they are bound to have small software differences. (At least my Tab does not have the 'SGS-lag' present at all and generally feels smoother to operate)
tl;dr: Basically Galaxy Tab = SGS with different screen, no real point comparing these two extensively
But they must have done a lot of tweaking on the tab that has yet to be ported to the S(if it ever does....).
Seriously, the Tab is smooth as silk, runs like you'd want and as we all know, essentially the same hardware, 2.2 and even RFS.
Also, benchmarks are worthless. My Galaxy S, with Docs JPO 7.5 ROM, ULF ext4 lagfix (no overkill here, not a score whore) gets around 1500 on quadthingy. My Tab, stock, gets around 900-1000. Yet the Tab is sooo much smoother, even with all my tweaks on the S. linpack is pretty much the same (as you'd expect) and I can't be bothered after that.
The only test that matters, usability, favours the Tab.
If I could get the stock Tab rom/fw/whatever onto my S, I'd be a happy camper.
That and a decent browser (dolphin doesn't do it for me either, just don't like it, gotta try the firefox alpha/beta/whatever)
I think the galaxy tab gpu is clocked at higher speed than the s,using neocore it gives same fbs as. Galaxy s although tab has higher resolution and also tab is announced with 1080p playback while galaxy s is anounced with 720 p correct if im wrong
Both equally good. If size does matter, big and small screen, your choice. For me, tab is best.
Sent from my GT-P1000 using XDA App
tab performs significantly worse as it has a bigger screen to power + more pixels.
But the tab 8.9 and sgs 2 (dual cores) will be a good match!
spekesel said:
But they must have done a lot of tweaking on the tab that has yet to be ported to the S(if it ever does....).
Seriously, the Tab is smooth as silk, runs like you'd want and as we all know, essentially the same hardware, 2.2 and even RFS.
Also, benchmarks are worthless. My Galaxy S, with Docs JPO 7.5 ROM, ULF ext4 lagfix (no overkill here, not a score whore) gets around 1500 on quadthingy. My Tab, stock, gets around 900-1000. Yet the Tab is sooo much smoother, even with all my tweaks on the S. linpack is pretty much the same (as you'd expect) and I can't be bothered after that.
The only test that matters, usability, favours the Tab.
If I could get the stock Tab rom/fw/whatever onto my S, I'd be a happy camper.
That and a decent browser (dolphin doesn't do it for me either, just don't like it, gotta try the firefox alpha/beta/whatever)
Click to expand...
Click to collapse
my tab isnt smooth at all!
I preffer sgs.
I just scored 2597 on a DK17 FroYo ROM (Quantum Rom v1.5 {WarpSpeed}) running at a stock 1.0 GHz. I believe this sets a record for a stock CPU on Android.
(At least it does according to smartphonebenchmarks.com) UPDATE - It's official. This is the 8th highest recorded Quadrant score and the very highest recorded stock clocked CPU score on that website.
This score was made possible by the EXT3 (EDIT - Correction, ramdisk) hack implemented in the most recent version of the ROM, released this morning.
EDIT - I'd like to note that the "lagfix" was incorporated into the ROM by the author with noobnl's assistance to prove a point; neither condone the serious use of Quadrant scores in it's current form. This hack is purely a Quadrant scores boost and does not provide real-world benefits. It is a demonstration that we should not rely upon inaccurate measurements to tell us what ROM or hardware is best.
An exploitable benchmark is no benchmark at all.
http://dl.dropbox.com/u/11825908/snap20101127_124836.png
http://dl.dropbox.com/u/11825908/snap20101127_131805.png
So just to clarify, does the ext3 hack do nothing other than improve benchmark scores? Is there any real-world improvement?
As common knowledge as the Quadrant thing is, I'm amazed people KEEP IMPLEMENTING this tweak and bragging about the results... unless there was an actual (less-drastic than Quadrant shows) improvement.
I agree that first we have to look at the big picture..while it may not help as much as the number makes it out to be, that doesn't necessary mean that it gives no benefit.
Next is this, why do people complain about Quadrant so much? you are using a FREE version..the FREE version only gives you a total number..Quadrant Pro on the other hand gives you the scores in each category..and is actually one of the best for measuring atm on android...
Electrofreak said:
I just scored 2597 on a DK17 FroYo ROM (Quantum Rom v1.5 {WarpSpeed}) running at a stock 1.0 GHz. I believe this sets a record for a stock CPU on Android.
(At least it does according to smartphonebenchmarks.com)
This score was made possible by the EXT3 hack implemented in the most recent version of the ROM, released this morning.
An exploitable benchmark is no benchmark at all.
http://dl.dropbox.com/u/11825908/snap20101127_124836.png
http://dl.dropbox.com/u/11825908/snap20101127_131805.png
Click to expand...
Click to collapse
it is true, we mounted benchmark data folder as a ramdisk/tmpfs, theres no ext3 filesystem in the rom because just for the lulz and educate the group that a lag fix won't do ****.. just to cheat a benchmark.. we were pissed at roms user thinking making rfs to ext3 will make performance better...
gTen said:
I agree that first we have to look at the big picture..while it may not help as much as the number makes it out to be, that doesn't necessary mean that it gives no benefit.
Next is this, why do people complain about Quadrant so much? you are using a FREE version..the FREE version only gives you a total number..Quadrant Pro on the other hand gives you the scores in each category..and is actually one of the best for measuring atm on android...
Click to expand...
Click to collapse
It has nothing to do with cost. It has to do with the fact that Quadrant is the most commonly used benchmark and it is deeply flawed.
Yes, the Pro version does break down the scores. Most people don't pay for Quadrant. Secondly, the I/O scores are weighted very heavily... the score nearly tripled because an I/O test completed a little quicker. There's a significant problem with that.
I've been impressed by the Epic's speed, but your scores aren't truly valid because you are "cheating"
I assume you're using the new Quantum ROM (which I also have installed) that implements a lagfix in which one moves the app data and dalvik cache away from Samsung’s embarrasingly slow internal Storage. Doing this significantly boosts I/O scores in Quadrant, thereby cheating the system. In real-life use, your system is barely performing any faster than before the exploit/lagfix
Read here for more info to see how a dev exploited this to create a 3300 Quad score (can't post links so remove the spaces):
http : // androidspin .com /2010/08/23/my-quadrant-is-bigger-than-your-quadrant/
This exploit has even been confirmed by the almighty Cyanogen himself
So is there any real benifits to this Epic 4G "lag fix"? I've seen that the "lag fix" within the other galaxy S phones actually does give real world results...
Eazail70x7 said:
So is there any real benifits to this Epic 4G "lag fix"? I've seen that the "lag fix" within the other galaxy S phones actually does give real world results...
Click to expand...
Click to collapse
Sure, it provides a speed benefit. The only thing being addressed in this thread is that, just because your quadrant scores double, it doesn't mean your device is twice as fast in actual use
Look for Dameon's post a couple of days ago in the quantum rom thread (on tapatalk..and linking sux). He talks about this exploit, but sadly (and understandably), most of the rom migrants want something to quantify expected results.. Even if bs
sent from my RAZR
Its not fair to say quadrant cheats.. rather it just may or may not be measuring performance of an operation that is ever really used on your system. In fact your system does use some fsyncs like quadrant and those will be faster like quadrant. Your system could start using fewer fsyncs though and quadrant would never know. Also your device certainly doesn't use only fsyncs..but those are very related to the lag. So basically, quadrant says the lag fix works if I'm not missing something. That's OK...but you don't need a crazy number to say that.
Bottom line... the only thing that tells how your phone performs...is how your phone performs and that's not just some little disclaimer. Hardware speed depends DRAMATICALLY on how it is used.
Sent from my SHW-M110S using XDA App
biff6789 said:
Sure, it provides a speed benefit. The only thing being addressed in this thread is that, just because your quadrant scores double, it doesn't mean your device is twice as fast in actual use
Click to expand...
Click to collapse
Actually the tweak, as noobnl states, quite specifically provides NO performance improvements to the phone. He would know because he made the hack. See the quote below.
noobnl said:
it is true, we mounted benchmark data folder as a ramdisk/tmpfs, theres no ext3 filesystem in the rom because just for the lulz and educate the group that a lag fix won't do ****.. just to cheat a benchmark.. we were pissed at roms user thinking making rfs to ext3 will make performance better...
Click to expand...
Click to collapse
Moving on...
biff6789 said:
I've been impressed by the Epic's speed, but your scores aren't truly valid because you are "cheating"
I assume you're using the new Quantum ROM (which I also have installed) that implements a lagfix in which one moves the app data and dalvik cache away from Samsung’s embarrasingly slow internal Storage. Doing this significantly boosts I/O scores in Quadrant, thereby cheating the system. In real-life use, your system is barely performing any faster than before the exploit/lagfix
Read here for more info to see how a dev exploited this to create a 3300 Quad score (can't post links so remove the spaces):
http : // androidspin .com /2010/08/23/my-quadrant-is-bigger-than-your-quadrant/
This exploit has even been confirmed by the almighty Cyanogen himself
Click to expand...
Click to collapse
This was the whole point of my post... I think you might have missed that. I would also recommend this article as well: http://briefmobile.com/cyanogen-demonstrates-quadrants-flaws
Electrofreak said:
It has nothing to do with cost. It has to do with the fact that Quadrant is the most commonly used benchmark and it is deeply flawed.
Yes, the Pro version does break down the scores. Most people don't pay for Quadrant. Secondly, the I/O scores are weighted very heavily... the score nearly tripled because an I/O test completed a little quicker. There's a significant problem with that.
Click to expand...
Click to collapse
Well realistically speaking even without the I/O exploit the none pro version never had any meaning to it..why?
Well it measures:
CPU
Memory
I/O
2d
3d
Now lets look at it from perspective..lets say we have a score of 3000 and 99% of it comes from 3d..the phone would still be laggy because even though it may have a good gpu it won't have the other things to back it up.
Quadrant weighs everything evenly...it exists for the sole purpose of getting people to buy the pro version..
So its not Quadrant that is flawed but the people who actually think a large number means everything is automatically super fast...
So in most cases yes, the I/O increase would be almost useless as nothing would utilize it to the point where you would need to utilize such high I/O..unless maybe you plan to run a database server on your phone? lol
But like any benchmark we have to not only look at the number but understand what the number means..similar to how on a linpack gets cheated on with use of a VFP since it measures with floating points...
Heck I can write a program called gTen Score which does an md5 sum of every file on the device and the closer the device is to mine the higher score it would get..would that make the device with the highest score the best? lol..benchmarks are benchmarks...we have to understand them and look at the details to how the numbers got there..In my opinion if its not Quadrant Pro people are wasting their time with the benchmark...
gTen said:
Well realistically speaking even without the I/O exploit the none pro version never had any meaning to it..why?
Well it measures:
CPU
Memory
I/O
2d
3d
Now lets look at it from perspective..lets say we have a score of 3000 and 99% of it comes from 3d..the phone would still be laggy because even though it may have a good gpu it won't have the other things to back it up.
Quadrant weighs everything evenly...it exists for the sole purpose of getting people to buy the pro version..
So its not Quadrant that is flawed but the people who actually think a large number means everything is automatically super fast...
So in most cases yes, the I/O increase would be almost useless as nothing would utilize it to the point where you would need to utilize such high I/O..unless maybe you plan to run a database server on your phone? lol
But like any benchmark we have to not only look at the number but understand what the number means..similar to how on a linpack gets cheated on with use of a VFP since it measures with floating points...
Heck I can write a program called gTen Score which does an md5 sum of every file on the device and the closer the device is to mine the higher score it would get..would that make the device with the highest score the best? lol..benchmarks are benchmarks...we have to understand them and look at the details to how the numbers got there..In my opinion if its not Quadrant Pro people are wasting their time with the benchmark...
Click to expand...
Click to collapse
You make some very good points, but Quadrant IS flawed in that it should be able to test performance in a manner that is more consistent with real-world performance. A benchmark tool that provides poor results for a different configuration of hardware or software (in this case, Samsung's RFS file system) when performance is in on par or even superior to a higher-scoring configuration is a benchmark tool with serious flaws.
Electrofreak said:
You make some very good points, but Quadrant IS flawed in that it should be able to test performance in a manner that is more consistent with real-world performance. A benchmark tool that provides poor results for a different configuration of hardware or software (in this case, Samsung's RFS file system) when performance is in on par or even superior to a higher-scoring configuration is a benchmark tool with serious flaws.
Click to expand...
Click to collapse
The flaw of quadrant is grouping up all of those categories into 1...usually as you said things are weighted into a benchmark and each benchmark measures things like 3d performance, i/o performance an etc would each be a separate benchmark..Quadrant will never do that because their goal is for you to buy the pro version...it was never intended to be used for anything other then a proof of concept for selling the pro version...
Since people are not gonna buy Quadrant Pro..the best way is to get them to use a different benchmark...I hear some people touting GLBenchmark is it any better?
gTen said:
The flaw of quadrant is grouping up all of those categories into 1...usually as you said things are weighted into a benchmark and each benchmark measures things like 3d performance, i/o performance an etc would each be a separate benchmark..Quadrant will never do that because their goal is for you to buy the pro version...it was never intended to be used for anything other then a proof of concept for selling the pro version...
Since people are not gonna buy Quadrant Pro..the best way is to get them to use a different benchmark...I hear some people touting GLBenchmark is it any better?
Click to expand...
Click to collapse
I've been following GLBenchmark and it seems pretty accurate as far as I can tell. A buddy of mine outed the HTC Glacier (Mytouch 4G) when he found that a developer had erroneously left scores posted on GLBenchmark.com. We could tell it was a next-gen Snapdragon by the scores, and that it wasn't running an Adreno 200. Became big news soon thereafter.
Electrofreak said:
I've been following GLBenchmark and it seems pretty accurate as far as I can tell. A buddy of mine outed the HTC Glacier (Mytouch 4G) when he found that a developer had erroneously left scores posted on GLBenchmark.com. We could tell it was a next-gen Snapdragon by the scores... became big news soon thereafter.
Click to expand...
Click to collapse
Well then the best bet is to contact rom developers and ask if they could include GLBenchmark instead of quadrant and explain the benefits of having a specified benchmark over a general unweighed one..
and yeah I saw th HTC Glacier thing..
Electrofreak said:
This was the whole point of my post... I think you might have missed that. I would also recommend this article as well: http://briefmobile.com/cyanogen-demonstrates-quadrants-flaws
Click to expand...
Click to collapse
You're right, I originally DID miss your point. My bad... I thought you were just another fanboi raving about the awesomeness of Quadrant. I should have read more carefully
But in essence, you and I were saying the same thing all along: Quadrant is bunk and can easily be manipulated by a tmpfs tweak
As for my other point about lagfixes offering real-world performance boosts in other phones, I was saying "yes" in a general way as often times they do, not "yes" specifically regarding Dameon's tweak
biff6789 said:
You're right, I originally DID miss your point. My bad... I thought you were just another fanboi raving about the awesomeness of Quadrant. I should have read more carefully
But in essence, you and I were saying the same thing all along: Quadrant is bunk and can easily be manipulated by a tmpfs tweak
As for my other point about lagfixes offering real-world performance boosts in other phones, I was saying "yes" in a general way as often times they do, not "yes" specifically regarding Dameon's tweak
Click to expand...
Click to collapse
Yeah, and from what I understand (forgive me, I am still a relative newcomer here) the other Galaxy S phones mount their file directory a little differently than the Epic does, negating the need for a real lag fix. However, the Captivate lag-fix, for example, does indeed provide actual performance improvements. Perhaps this misunderstanding is what is causing people to believe that the Epic needs to have some sort of lag fix too.
Electrofreak said:
Yeah, and from what I understand (forgive me, I am still a relative newcomer here) the other Galaxy S phones mount their file directory a little differently than the Epic does, negating the need for a real lag fix. However, the Captivate lag-fix, for example, does indeed provide actual performance improvements. Perhaps this misunderstanding is what is causing people to believe that the Epic needs to have some sort of lag fix too.
Click to expand...
Click to collapse
You're exactly right. Dameon himself stated that the other Galaxy phones need a lagfix while the Epic does not (read the last line of the following post):
http://forum.xda-developers.com/showpost.php?p=9440522&postcount=1174
quadrant has been known to very inaccurate. I forget who it was but someone tweaking their evo was able to get like 6000 on quadrant awhile ago. After that it has been tricked many times into giving big numbers with no actual improvements over the actual phone itself.
both running 2.2 (epic on dk28)
epic gets 991-1000 and the evo just scored 1241... wtf?!
the evo isn't EXT4 is it?
the epic is currently RFS... I'm having problems going to EXT4
The Quadrant was made specifically for the Snapdragon process; which the Evo uses if i'm not mistaken.
Besides; they don't prove real world performance. I've gotten my hands on numerous evo's and all seemed to be 'laggy' they arn't nearly as responsive as my Epic.
Quadrant scores do not accurately represent Snapdragon vs the Galaxy S Hummingbird (or any other Cortex-A8 like those in Verizon's Droid lineup). Certain functions such as the Virtual Floating Point extension in Snapdragon allow for artificially inflated Linpack scores which do not represent real-world performance. Additionally, the RFS file system that Samsung uses on the Galaxy S phones is not well-understood by Quadrant, resulting in drastically lower scores.
For example, when we switch to EXT4 from RFS, our FroYo Quadrant scores can jump from 1100 to the 1600 range (which kicks the pants off that EVO). The actual performance increase however is hardly perceptible.
In short, Quadrant sucks, and Linpack is susceptible as well. For ****s and giggles, a little while back I made a minor modification to my Epic and produced a Quadrant score of 2597, which is currently listed as the 10th highest score on SmartphoneBenchmarks.com, and is the highest stock clock Quadrant score recorded. I accomplished this score utilizing a simple hack devised by a few developers here on XDA that fools the Quadrant application by utilizing a ramdrive for the I/O test. I posted about it on their forums and was advised by an administrator that they are aware of the problems with Quadrant and are releasing their own benchmark tool.
Android runs on top of a virtual machine so really what you're testing is virtual machine performance, and currently that virtual machine is tweaked for Snapdragon. With the Nexus S now a Google flagship phone, we'll likely see the VM better optimized for Hummingbird in the near future, and in fact, Gingerbread had a few more JIT enhancements as well.
Hummingbird is a slightly better performing chip MHz for MHz than the first-generation Snapdragons, if by a small amount. However it is significantly more power-efficient. Second-gen Snapdragons do draw even in terms of efficiency and performance, but both are going to be blown away by the Cortex-A9 Tegra phones we'll be seeing this spring.
EDIT - So I just noticed that SmartphoneBenchmarks.com has released their own benchmark tool recently, Smartbench 2010. I just ran it on my phone and scored 1178 on the Productivity Index (CPU) and 2610 on the Games Index (GPU). The highest-scoring competitor, the HTC G2, scores 1045 and 1396 respectfully. DRockstar on IRC ran the benchmark on his Epic that has RFS, and scored 1133 and 2521. So, this benchmark tool actually performs fine on RFS. Amazing! Grab it off the Android Market!
Electrofreak said:
Quadrant scores do not accurately represent Snapdragon vs the Galaxy S Hummingbird (or any other Cortex-A8 like those in Verizon's Droid lineup). Certain functions such as the Virtual Floating Point extension in Snapdragon allow for artificially inflated Linpack scores which do not represent real-world performance. Additionally, the RFS file system that Samsung uses on the Galaxy S phones is not well-understood by Quadrant, resulting in drastically lower scores.
For example, when we switch to EXT4 from RFS, our FroYo Quadrant scores can jump from 1100 to the 1600 range (which kicks the pants off that EVO). The actual performance increase however is hardly perceptible.
In short, Quadrant sucks, and Linpack is susceptible as well. For ****s and giggles, a little while back I made a minor modification to my Epic and produced a Quadrant score of 2597, which is currently listed as the 10th highest score on SmartphoneBenchmarks.com, and is the highest stock clock Quadrant score recorded. I accomplished this score utilizing a simple hack devised by a few developers here on XDA that fools the Quadrant application by utilizing a ramdrive for the I/O test. I posted about it on their forums and was advised by an administrator that they are aware of the problems with Quadrant and are releasing their own benchmark tool.
Android runs on top of a virtual machine so really what you're testing is virtual machine performance, and currently that virtual machine is tweaked for Snapdragon. With the Nexus S now a Google flagship phone, we'll likely see the VM better optimized for Hummingbird in the near future, and in fact, Gingerbread had a few more JIT enhancements as well.
Hummingbird is a slightly better performing chip MHz for MHz than the first-generation Snapdragons, if by a small amount. However it is significantly more power-efficient. Second-gen Snapdragons do draw even in terms of efficiency and performance, but both are going to be blown away by the Cortex-A9 Tegra phones we'll be seeing this spring.
EDIT - So I just noticed that SmartphoneBenchmarks.com has released their own benchmark tool recently, Smartbench 2010. I just ran it on my phone and scored 1178 on the Productivity Index (CPU) and 2610 on the Games Index (GPU). The highest-scoring competitor, the HTC G2, scores 1045 and 1396 respectfully. DRockstar on IRC ran the benchmark on his Epic that has RFS, and scored 1133 and 2521. So, this benchmark tool actually performs fine on RFS. Amazing! Grab it off the Android Market!
Click to expand...
Click to collapse
Electrofreak,
I just wanted to thank you (I pushed the button, too!) for this post. I'm trying to decide between three phones for my Sprint upgrade next month. My three candidates are the Epic, Evo, & new Evo Shift.
I was not aware of everything you stated, so it helped me look at the Epic in a different light.
Again, thanks.
tps70 said:
Electrofreak,
I just wanted to thank you (I pushed the button, too!) for this post. I'm trying to decide between three phones for my Sprint upgrade next month. My three candidates are the Epic, Evo, & new Evo Shift.
I was not aware of everything you stated, so it helped me look at the Epic in a different light.
Again, thanks.
Click to expand...
Click to collapse
No probs, and if you're interested in still more info, you're welcome to read an article I wrote comparing the hardware in multiple smartphones back in April (though the focus was on the EVO 4G and the Samsung Galaxy S I9000). The article is starting to get a little outdated, (neither the EVO nor the Galaxy S line had been released at that point yet) and it also doesn't cover some other details I've unearthed since then (my blog in my signature is where you'll find that) but most of it is still relevant.
http://alienbabeltech.com/main/?p=17125
Edit - I think I have an addiction to parenthesis (which I'm ashamed to admit)
Electrofreak said:
Quadrant scores do not accurately represent Snapdragon vs the Galaxy S Hummingbird (or any other Cortex-A8 like those in Verizon's Droid lineup). Certain functions such as the Virtual Floating Point extension in Snapdragon allow for artificially inflated Linpack scores which do not represent real-world performance. Additionally, the RFS file system that Samsung uses on the Galaxy S phones is not well-understood by Quadrant, resulting in drastically lower scores.
For example, when we switch to EXT4 from RFS, our FroYo Quadrant scores can jump from 1100 to the 1600 range (which kicks the pants off that EVO). The actual performance increase however is hardly perceptible.
In short, Quadrant sucks, and Linpack is susceptible as well. For ****s and giggles, a little while back I made a minor modification to my Epic and produced a Quadrant score of 2597, which is currently listed as the 10th highest score on SmartphoneBenchmarks.com, and is the highest stock clock Quadrant score recorded. I accomplished this score utilizing a simple hack devised by a few developers here on XDA that fools the Quadrant application by utilizing a ramdrive for the I/O test. I posted about it on their forums and was advised by an administrator that they are aware of the problems with Quadrant and are releasing their own benchmark tool.
Android runs on top of a virtual machine so really what you're testing is virtual machine performance, and currently that virtual machine is tweaked for Snapdragon. With the Nexus S now a Google flagship phone, we'll likely see the VM better optimized for Hummingbird in the near future, and in fact, Gingerbread had a few more JIT enhancements as well.
Hummingbird is a slightly better performing chip MHz for MHz than the first-generation Snapdragons, if by a small amount. However it is significantly more power-efficient. Second-gen Snapdragons do draw even in terms of efficiency and performance, but both are going to be blown away by the Cortex-A9 Tegra phones we'll be seeing this spring.
EDIT - So I just noticed that SmartphoneBenchmarks.com has released their own benchmark tool recently, Smartbench 2010. I just ran it on my phone and scored 1178 on the Productivity Index (CPU) and 2610 on the Games Index (GPU). The highest-scoring competitor, the HTC G2, scores 1045 and 1396 respectfully. DRockstar on IRC ran the benchmark on his Epic that has RFS, and scored 1133 and 2521. So, this benchmark tool actually performs fine on RFS. Amazing! Grab it off the Android Market!
Click to expand...
Click to collapse
That was great!
I scored 1147/2704 but i'm rooted/rommed.
the evo scored 700/910
razorseal said:
That was great!
I scored 1147/2704 but i'm rooted/rommed.
the evo scored 700/910
Click to expand...
Click to collapse
That EVO on 2.2? I would have expected it to score around 1000 at least. I wonder how it would score on EXT4 running CM6...
Smartphone benchmarks is a great benchmark.
Sent from my SPH-D700 using XDA App
Electrofreak said:
That EVO on 2.2? I would have expected it to score around 1000 at least. I wonder how it would score on EXT4 running CM6...
Click to expand...
Click to collapse
yup, it's a stock evo running whatever sprint updated for it
Scored 1257/2751 CM6 EXT4
Sent from my CM6 EXT4 Epic
1255p 2945g,Im running my ROM,how could I be faster then CM6? maybe not the best benchmark.
Sent from my SPH-D700 using XDA App
My Epic w/ Cmod's latest gave me 1115 and 2600
My wife has an Evo with CMods latest as well but only got 1089 and 1050, why so low on the second one?
My Epic scored 606\1901 in smartbench 2010. Weird... much lower productivity score than other people, but really high gaming score.
My Epic is stock.
EDIT: I ran it a few more times and watched it carefully.
603/1808
618/1954
633/1941
Seems I/O is pretty slow...
I'm just wondering why it matters? It's not like Android has a robust collection of high performance games.
razorseal said:
both running 2.2 (epic on dk28)
epic gets 991-1000 and the evo just scored 1241... wtf?!
the evo isn't EXT4 is it?
the epic is currently RFS... I'm having problems going to EXT4
Click to expand...
Click to collapse
Probably because she keeps her Evo in her bra and you keep your Epic in your pocket. It's a fact (check quadrant scores) that smartphones prefer boobs to guys hips 9 out of 10 days of the week. So obviously her Evo is happier and therefor performs better than yours. Do your Epic a favor and give him some booby time and watch those Quadrant scores rise!
+1,agreed and its been proven time and time again...
jirafabo said:
Probably because she keeps her Evo in her bra and you keep your Epic in your pocket. It's a fact (check quadrant scores) that smartphones prefer boobs to guys hips 9 out of 10 days of the week. So obviously her Evo is happier and therefor performs better than yours. Do your Epic a favor and give him some booby time and watch those Quadrant scores rise!
Click to expand...
Click to collapse