Related
Look at this (from 1:44 on):
It's a quadrant benchmark run on a android port on the HD2. Graphics are really bad, but in the end it has approximately the same score as the benchmarking score of the Galaxy with the original firmware. I mean what is in a score? If I look at the beginning of the movie, the UI is very slow and not as responsive as the Galaxy
(BTW i got 55.7 FPS with the neocore benchmark on JM2)
This is not to say that I don't have deep respect for what the HD2-android development team is doing. Really amazing job. I just can't wait to get my HD2 back from repair.
appelflap said:
Look at this (from 1:44 on):
It's a quadrant benchmark run on a android port on the HD2. Graphics are really bad, but in the end it has approximately the same score as the benchmarking score of the Galaxy with the original firmware. I mean what is in a score? If I look at the beginning of the movie, the UI is very slow and not as responsive as the Galaxy
(BTW i got 55.7 FPS with the neocore benchmark on JM2)
This is not to say that I don't have deep respect for what the HD2-android development team is doing. Really amazing job. I just can't wait to get my HD2 back from repair.
Click to expand...
Click to collapse
Quadrant scores have been criticized for their non-descript breakdowns, at least on their free suite. Also, the fact that they chose the weighting of the scores, so should they chose 2D is equal to 3D weight, I don't know their formula (and for all I know, they give equal weighting to all or they give equal weighting to all test where the CPU has 12 tests and the 3D graphics has 4), but the fact that we, as users don't have access to their formula on their website is a bit unnerving.
Add to that the fact that many reviews and videos rely on it so heavily leaves users a bit misinformed. In reality, and thorough review should definitely run a custom test suite to give individual scores to:
CPU
Memory
I/O
2D graphics
3D graphics
That way users can compare what's important to them. The Galaxy S suffers from terrible I/O and the hacks that have given the fixes typically boost Galaxy scores to nearly double their rates, and it's majorly attributed to improving a bunk I/O score.
Totally agree. In addition, it would be really nice to know which benchmarked factors are responsible for which functions. For example it is really interesting to see how the hd2 performs before the user is running the tests. When the user is scrolling through the setting menu there is a very noticible lag. Given the fact that the total score is nearly the same as the scrore for the SGS, and thar the graphic score of the hd2 is bad in comparisson to the SGS, I would conclude that graphic performance is very important for the way the ui responds.
Sent from my GT-I9000 using XDA App
appelflap said:
Totally agree. In addition, it would be really nice to know which benchmarked factors are responsible for which functions. For example it is really interesting to see how the hd2 performs before the user is running the tests. When the user is scrolling through the setting menu there is a very noticible lag. Given the fact that the total score is nearly the same as the scrore for the SGS, and thar the graphic score of the hd2 is bad in comparisson to the SGS, I would conclude that graphic performance is very important for the way the ui responds.
Sent from my GT-I9000 using XDA App
Click to expand...
Click to collapse
From what I can tell, the HD2 got a decent score 'cos it was running Froyo. When we get bumped up to an official froyo build with JIT fully optimized, We should be top of the pile.
don't forget, android isn't working 100% on the HD2.
I personally think it's pointless comparing to a not complete port.
woops dbl post
alovell83 said:
Quadrant scores have been criticized for their non-descript breakdowns, at least on their free suite. Also, the fact that they chose the weighting of the scores, so should they chose 2D is equal to 3D weight, I don't know their formula (and for all I know, they give equal weighting to all or they give equal weighting to all test where the CPU has 12 tests and the 3D graphics has 4), but the fact that we, as users don't have access to their formula on their website is a bit unnerving.
Add to that the fact that many reviews and videos rely on it so heavily leaves users a bit misinformed. In reality, and thorough review should definitely run a custom test suite to give individual scores to:
CPU
Memory
I/O
2D graphics
3D graphics
That way users can compare what's important to them. The Galaxy S suffers from terrible I/O and the hacks that have given the fixes typically boost Galaxy scores to nearly double their rates, and it's majorly attributed to improving a bunk I/O score.
Click to expand...
Click to collapse
Even then though, it's possible to write a benchmark which wins constantly for any phone.
In regards to "terrible I/O", that might even be due to a bug in the FAT32 drivers. Yes you can benchmark it, but it wont mean much. The best way is to actually TEST the applications you need, rather than select a phone based on benchmarks. However, you are possibly best off looking at the component specs, because they ignore software bugs.
scrizz said:
don't forget, android isn't working 100% on the HD2.
I personally think it's pointless comparing to a not complete port.
Click to expand...
Click to collapse
But the topic is about "what's in a score". Maybe one can generally say that is pointless to compare devices this way. I think that such benchmark scores are only (a bit) relevant at the two poles of the benchmark score spectrum. Everything in between can be neglected due to the uninformed way sub-scores are evaluated.
You got 55.7 FPS on Neocore as the sgs has vertical sync enabled, the refresh rate on the sgs'es screen is 56 fps and thus you can only go up to 56 fps as the v-sync is on. This proves that the sgs is indeed a much more powerful device that is actually being held back. If you can disable the v-sync then you can get a higher fps score
appelflap said:
But the topic is about "what's in a score". Maybe one can generally say that is pointless to compare devices this way. I think that such benchmark scores are only (a bit) relevant at the two poles of the benchmark score spectrum. Everything in between can be neglected due to the uninformed way sub-scores are evaluated.
Click to expand...
Click to collapse
I just read in a post that the Galaxy S gets a 0 on the 2D score:
"JIT isn't fully enabled in the current froyo versions, and quadrant, frankly, is bull**** (for exmple, 2d acceleration gets the same weight in the final result as 3D. Due to the fact that the SGS doesn't have a dedicated 2D accelerator, quadrant doesn't try to use the cpu- it just gives a round zero in that part)"
I can't confirm this, but that definitely seems like a terrible set-up, seeing as how I'm pretty sure I have games run in 2D, so to say that it can't do it just seems wrong regardless of if the SGS has a dedicated 2D accelerator or not (so if you aren't testing the way it performs in real-world, why are you testing?)
http://forum.xda-developers.com/showthread.php?t=737787&page=3
Qazz~ said:
You got 55.7 FPS on Neocore as the sgs has vertical sync enabled, the refresh rate on the sgs'es screen is 56 fps and thus you can only go up to 56 fps as the v-sync is on. This proves that the sgs is indeed a much more powerful device that is actually being held back. If you can disable the v-sync then you can get a higher fps score
Click to expand...
Click to collapse
It isn't really being held back - the screen can't display more than 56 fps as you say, and it wouldn't really be visible even if it could. Disabling v-sync isn't really that important, we need a benchmark that can actually use the advanced features in the SGS GPU (Neocore just pushes a fairly small amount of polygons with no real extras.) Using current 3D benchmarks to benchmark the SGS is like using quake 1 to benchmark the brand new ATI/nVidia cards.
The benchmark is what is at fault here, not the device
RyanZA said:
It isn't really being held back - the screen can't display more than 56 fps as you say, and it wouldn't really be visible even if it could. Disabling v-sync isn't really that important, we need a benchmark that can actually use the advanced features in the SGS GPU (Neocore just pushes a fairly small amount of polygons with no real extras.) Using current 3D benchmarks to benchmark the SGS is like using quake 1 to benchmark the brand new ATI/nVidia cards.
The benchmark is what is at fault here, not the device
Click to expand...
Click to collapse
I don't want to speak for the other poster, and I agree with your premise, however, it isn't actually solving the issue at hand. Better FPS wouldn't be noticed, however, it would give a better score and, more importantly, indicate it's potential. So, getting 56FPS isn't doing the phone any justice within the score, which is what reviews are using, giving it an artificially low score, and putting it more in line with units that can't compete on higher end games. So, when a site like anand pushes 150FPS on a game, I know that means that their rig is entirely too powerful for the game in question, but it still means something when you compare it to the lower end graphics card that only gets 90...then when they run Crisis you see these results play out more with differences that we can notice with the eye.
I think the HD2 gets that score because, as I can see in the video, the CPU tests run faster compared to my SGS, probably because of Froyo, and I know, from the time I had the Diamond and the HD2, that the internal memory and RAM are very fast. Sadly SGS has a slow internal memory, atleast when used by the phone`s software, when copying from PC is faster than my class 6 microSD. Luckily, we have mimocan`s fix. Hope this will be fixed in future FW`s.
NexusHD2 with-FRG83D V1.7 with hastarin r8.5.1 On my HD2 got 1920 in quadrant,31.5 on neocore, and 37 on linmark.
The lag might be because you are using launcher pro, I use launcher pro and sometimes it makes the the lock lag on my phone but it doesn't happen when I use the default lock also if you have alot of Widgets on your screen it will cause lag also
appelflap said:
Look at this (from 1:44 on):
It's a quadrant benchmark run on a android port on the HD2. Graphics are really bad, but in the end it has approximately the same score as the benchmarking score of the Galaxy with the original firmware. I mean what is in a score? If I look at the beginning of the movie, the UI is very slow and not as responsive as the Galaxy
(BTW i got 55.7 FPS with the neocore benchmark on JM2)
This is not to say that I don't have deep respect for what the HD2-android development team is doing. Really amazing job. I just can't wait to get my HD2 back from repair.
Click to expand...
Click to collapse
same galaxy s scores 6000+ in quadrant with custem roms
The HD 2 is a better fit for quadrent then the sgs as quadrent was made for the snapdragon processor which the hd2 has and the sgs does not. Comparing apples to orenges in an apple juice contest doesn't really prove much. Use real life feel. If you care about the scores a rom can be made to get you over 3000 quad score but is laggy as hell. Don't believe me? Look at my sig
interesting... I was using quadrant to see how a stock xxjvo and gingerreal compared. Surely that would indicate a real speed difference and not just be some kind of "hack" ?
zelendel said:
The HD 2 is a better fit for quadrent then the sgs as quadrent was made for the snapdragon processor which the hd2 has and the sgs does not.
Click to expand...
Click to collapse
That's right.
HD2 uses two android OS :
- Cyanogenmod, that is faster than our samsung os..
- Nexus one's port to HD2, greatly optimized by google...
It's really fast. I upgraded my father's HD2 last month, replacing windows in the NAND with CM7. It really makes a big change, the phone is like brand new ^^
http://forum.xda-developers.com/showthread.php?t=1012556
Quadrant is pretty flawed. And I say that being someone who had a phone (before modifications) that was mid-range in Quadrant (Galaxy S), and having a phone that's right top of the heap (Galaxy S II)
Thought i would drop in and let you know the results of my testing, as I have a lot of downtime.
I used antutu benchmark as it would give me values for each category.
Moto xoom - 1.5 oc kernel, stock 3.0
Fascinate - voodoo eb16 kernel, superclean 2.7
Processor - as you all have heard the new dual core tegra 2 is fast, through same benchmark tests I have found that it twice as fast as the hummingbird w/ no over clock, the tegra was oc to 1.2ghz
Xoom 1034 and 766
Fascinate 596 and 142
Graphics - this is where some eyebrows will be raised! Everyone says the tegra is supposed to be the best at 3d but I found quite to opposite. There are a lot of variables out there too though, multi threadding and is 3.0 as mature as 2.2 and last the screen resolution is higher on the xoom.
Xoom 2d 148 3d 205
Fascinate 2d 158 3d 364
These results can be seen by the naked eye as things just seem slower on the xoom in 3d.
I did not include linpack as its obvious the xoom will out perform the hummingbird no questions. I am just thrown off by the gpu scores, anything out for the tegra the fascinae should be able to run, extra help will come from the faster processor but i am not seeing an incredible difference that was advertised. After these findings I am more excited for the dual core snapdragon. Things my improve as honeycomb matures and they iron out the details, from first use right now it seems like its a beta 2.2 that we had on the fascinates, its just not smooth. Over all if you want a good tablet w/ verizon 3g/4g the xoom is a good chipoice, those wanting a finished product, I would hold out a few mnths. Thunderbolt vs fascinae coming soon, as soon as the phone hits the streets. I will compare stock TB to modded fascinate, for those considering the upgrade.
Final on the xoom, it is a good tablet, maybe expensive
forgot to include wifi, another big dissapoitment, xoom being 4g upgradeable, I thought would kill it, it did not.
Netgear wireless n router, 12mbps line
Xoom - 2mbps
Fascinate - 6.5mbps
Hopefully an update will fix that
Well, the fact that most benchmark applications, besides Smartbenchmark, are not optimized correctly for Tegra2 puts these benchmarks to shame. Nenamark caps at 30fps, Quadrant caps at 60 or 30fps, and 2D benchmarks cap at similar fps as well. Real world speed is faster, as well as games optimized for the Tegra2. Though, movie compatibility with the Fascinate is better because of the extra codecs courtesy of Samsung. Not sure whats wrong with your Wifi..
Hitting fps caps is not in the cards right now quadrant hits 6fps 2d and about 20 3d, maybe when its optimized it will reflect capability better but now it just looks like its close. Everyday useability, my only compplain is the keyboard is sometimes laggy in the browser, most likely will be patched in an update.
I have both. And its too early to do a fair comparison. Apps haven't been optimized for the xoom.
I'm just bummed out that i can't use my fascinate to tether the xoom. I'm stuck paying for verizon monthly bill until the fascinate offers wifi infrastructure not ad-hoc. or switch to a phone that actually offers it.
You can get 3g mobile hotspot free google it
Sent from my SCH-I500 using XDA App
I think partially the difference comes from the higher resolution on the Xoom. But we do have a great GPU indeed.
I know about the differences in benchmarks and how they arent set up for dual core, but I just ran smartbench 2011 and my gaming score is off by 1000 points on a stock xoom, I am rooted and running stock kernel. I am not sure why, maybe something is wrong with it.
My quadrant scores are lower than my dx but my linpack score is 64mflops! Don't know why our quadrant scores are so low but I'm having the same problem.
Sent from my Xoom using XDA App
Off from what? A phone? Synthetic benchmarks say almost nothing about real world performance, and they will always be different with devices at different resolutions.
A 1280x800 tablet will always score unusually low on a graphics benchmark that scales to resolution compared to a phone.
Usmc7356 said:
My quadrant scores are lower than my dx but my linpack score is 64mflops! Don't know why our quadrant scores are so low but I'm having the same problem.
Sent from my Xoom using XDA App
Click to expand...
Click to collapse
Quadrant also places a lot of weight on the filesystem, and my Xoom always hangs quite a while during database writes.
Also I think that getting ~6 FPS on their first 2D animation test can't help.
The Xoom is really zippy, take the benchmarks with many, many grains of salt.
I was just talking about smart bench, everything else is working fine, but the smartbench 2011 shows a galaxy s as being more powerful than my xoom, and the half the speed of a stock xoom. I am just wondering if other people were showing that they are below what a stock xoom should be too.
you have to make sure that these benchmarks are compatible with dual core processors. otherwise the results are moot.
I know that, but it is shows below the average xoom, thats the problem I am seeing, average xoom gets like 2k I get 1k
I was having the same issue, I believe it is because of spare parts for gaming full screen. I factory restored my xoom and scored higher than average. The benchmark ran on a much smaller screen when I ran it on a fresh xoom.
joepfalzgraf said:
I was having the same issue, I believe it is because of spare parts for gaming full screen. I factory restored my xoom and scored higher than average. The benchmark ran on a much smaller screen when I ran it on a fresh xoom.
Click to expand...
Click to collapse
Confirmed that is it, thanks for this I guess I was overlooking it and thinking my xoom wasnt up to par.
After updating to 3.1 I ran a few quadrant tests and instead of the 2000-2100 scores i normally get i am not getting 1500-1600 ... Usually updates boost performance not lower it
so i guess that begs the question: is your TF's performance lower?
Dark lord me said:
After updating to 3.1 I ran a few quadrant tests and instead of the 2000-2100 scores i normally get i am not getting 1500-1600 ... Usually updates boost performance not lower it
Click to expand...
Click to collapse
I noticed this too... BUT... the system seems a lot faster and more responsive, so i guess scores arent everything.
For sure score isn't everything, even more with quadrant.
Sent from my GT-P1000 using XDA App
quadrant came out and has not been updated since the nexus 1 got 2.2. so its kind of flawed and old. current best benchmark is either Vellamo or AnTuTu
Vellamo is a web browser benchmark IIRC, where as Quadrant is a CPU/GPU benchmark. I dont know about the other one you mentioned.
15xx is pretty damn low, I'm getting around 35xx with Quadrant at 1.5 GHz. Check your clockspeed in setcpu to make sure nothing is out of wack.
mrevankyle said:
quadrant came out and has not been updated since the nexus 1 got 2.2. so its kind of flawed and old. current best benchmark is either Vellamo or AnTuTu
Click to expand...
Click to collapse
Or CF Bench
Sent from my Nexus S using XDA Premium App
Or actually using the tablet. If it seems faster when you use it, its better. Benchmarks are pretty useless, especially since they can be skewed or manipulated
quadrant is a horrible benchmark. there are hacks and tweaks to get you stupid high scores.
Wierd i get2 2600
Tortel1210 said:
Or actually using the tablet. If it seems faster when you use it, its better. Benchmarks are pretty useless, especially since they can be skewed or manipulated
Click to expand...
Click to collapse
That's dumb. I clearly spent 400 dollars so I can get my electronics to tell me that I am cool. If my number is lower, then I am not cool.
sassafras
My quadrant is 1.7 not rooted or anything. I must say this tab runs extremely fast and I have no problems with it minus apps crashing once in a blue moon. If quadrant ment something my vibrant has 2.2k and it still doesn't run as smooth as my tab
Sent from my SGH-T959 using XDA App
Dark lord me said:
After updating to 3.1 I ran a few quadrant tests and instead of the 2000-2100 scores i normally get i am not getting 1500-1600 ... Usually updates boost performance not lower it
Click to expand...
Click to collapse
I too used to be quadrant this or that using it as a gauge...then after I owned a few android devices...I came to the conclusion...its a piece of ****... First its inaccurate...my EVO. 3d is way faster then my color nookut yet I get better scores with the nook...same with the tf...second...it uses testing methods that can be cheated by some settings...hardware stuff..3rd...if you run it 3 times...you will usually get 3 different darn scores that range widely. To me using is the best test...not benchmarks..however if you need to use this as a guage...do it...but be warned...for real life...it don't mean anything
sassafras_ said:
That's dumb. I clearly spent 400 dollars so I can get my electronics to tell me that I am cool. If my number is lower, then I am not cool.
sassafras
Click to expand...
Click to collapse
It's not dumb when the software is deeply, deeply flawed....quadrant that is.
life64x said:
It's not dumb when the software is deeply, deeply flawed....quadrant that is.
Click to expand...
Click to collapse
I think your sarcasm meter is broken.
...
Quadrant is broken because it doesn't weight different aspects of the benchmark equally. The Nexus One has a terrible GPU but a fast CPU, so it gets decent scores. The BN Nook Color has a mediocre CPU and a decent GPU so it scores better than the N1 even though the N1 is clearly the superior device.
Changing the file system to something journaled can bump your Quadrant score a few hundred points, which is dumb.
The ideal benchmark would somehow score in a way that represented the overall user experience. Unfortunately, no such benchmark exists for Android. Until then, it's just these pieces of crap that only exist so teenagers can show off their e-peen on the internet.
sassafras
sassafras_ said:
I think your sarcasm meter is broken.
...
Quadrant is broken because it doesn't weight different aspects of the benchmark equally. The Nexus One has a terrible GPU but a fast CPU, so it gets decent scores. The BN Nook Color has a mediocre CPU and a decent GPU so it scores better than the N1 even though the N1 is clearly the superior device.
Changing the file system to something journaled can bump your Quadrant score a few hundred points, which is dumb.
The ideal benchmark would somehow score in a way that represented the overall user experience. Unfortunately, no such benchmark exists for Android. Until then, it's just these pieces of crap that only exist so teenagers can show off their e-peen on the internet.
sassafras
Click to expand...
Click to collapse
I was saying that!!! I figured from your first reply...if you spent 400 it should be off the chain for the score. Quadrant is deeply, deeply flawed. If I mis-read your reply then it is my fault but I was not using sarcasm or being flippant but just stating what we both said.
life64x said:
I was saying that!!! I figured from your first reply...if you spent 400 it should be off the chain for the score. Quadrant is deeply, deeply flawed. If I mis-read your reply then it is my fault but I was not using sarcasm or being flippant but just stating what we both said.
Click to expand...
Click to collapse
he wasn't accusing you if being sarcastic, he was being sarcastic.
Sent from my Transformer TF101 using Tapatalk
Thanks, my bad. I am a optimist and thought my pessimist came out... With only a couple hours sleep my mind plays tricks on me. Oh well, go back to watching dune...I would have used my gom jabber( watch dune to know what I mean).
Only thing worst than benchmark nerds are benchmark nerds who are stupid enough to still be using quadrant software that's over a year old and is not optimized for dualcore or honeycomb.
Update: NOW the tests show 3720 average, so not sure what is going on....
I have done a bunch of tests and the average is 2650: Range as low as 2500 and 2760 was the highest. I thought people were getting about 1000 higher than that?
Been using the newest version of Quadrant and restarted the A510 as well and see no apparent signs of slowness and emulators & media play great. Something is not adding up here. Without doubt the 510 is faster than my Thrive, but that is not saying a whole lot (Tegra 2 is nearing two years old).
rushless said:
Update: NOW the tests show 3720 average, so not sure what is going on....
I have done a bunch of tests and the average is 2650: Range as low as 2500 and 2760 was the highest. I thought people were getting about 1000 higher than that?
Been using the newest version of Quadrant and restarted the A510 as well and see no apparent signs of slowness and emulators & media play great. Something is not adding up here. Without doubt the 510 is faster than my Thrive, but that is not saying a whole lot (Tegra 2 is nearing two years old).
Click to expand...
Click to collapse
I tweak the kernel of the A500 on ICS and the quadrant score is 2500-2600
But don't look to hard on the quadrant benchmark , it's probably your I/O score that is low.