I just scored 2597 on a DK17 FroYo ROM (Quantum Rom v1.5 {WarpSpeed}) running at a stock 1.0 GHz. I believe this sets a record for a stock CPU on Android.
(At least it does according to smartphonebenchmarks.com) UPDATE - It's official. This is the 8th highest recorded Quadrant score and the very highest recorded stock clocked CPU score on that website.
This score was made possible by the EXT3 (EDIT - Correction, ramdisk) hack implemented in the most recent version of the ROM, released this morning.
EDIT - I'd like to note that the "lagfix" was incorporated into the ROM by the author with noobnl's assistance to prove a point; neither condone the serious use of Quadrant scores in it's current form. This hack is purely a Quadrant scores boost and does not provide real-world benefits. It is a demonstration that we should not rely upon inaccurate measurements to tell us what ROM or hardware is best.
An exploitable benchmark is no benchmark at all.
http://dl.dropbox.com/u/11825908/snap20101127_124836.png
http://dl.dropbox.com/u/11825908/snap20101127_131805.png
So just to clarify, does the ext3 hack do nothing other than improve benchmark scores? Is there any real-world improvement?
As common knowledge as the Quadrant thing is, I'm amazed people KEEP IMPLEMENTING this tweak and bragging about the results... unless there was an actual (less-drastic than Quadrant shows) improvement.
I agree that first we have to look at the big picture..while it may not help as much as the number makes it out to be, that doesn't necessary mean that it gives no benefit.
Next is this, why do people complain about Quadrant so much? you are using a FREE version..the FREE version only gives you a total number..Quadrant Pro on the other hand gives you the scores in each category..and is actually one of the best for measuring atm on android...
Electrofreak said:
I just scored 2597 on a DK17 FroYo ROM (Quantum Rom v1.5 {WarpSpeed}) running at a stock 1.0 GHz. I believe this sets a record for a stock CPU on Android.
(At least it does according to smartphonebenchmarks.com)
This score was made possible by the EXT3 hack implemented in the most recent version of the ROM, released this morning.
An exploitable benchmark is no benchmark at all.
http://dl.dropbox.com/u/11825908/snap20101127_124836.png
http://dl.dropbox.com/u/11825908/snap20101127_131805.png
Click to expand...
Click to collapse
it is true, we mounted benchmark data folder as a ramdisk/tmpfs, theres no ext3 filesystem in the rom because just for the lulz and educate the group that a lag fix won't do ****.. just to cheat a benchmark.. we were pissed at roms user thinking making rfs to ext3 will make performance better...
gTen said:
I agree that first we have to look at the big picture..while it may not help as much as the number makes it out to be, that doesn't necessary mean that it gives no benefit.
Next is this, why do people complain about Quadrant so much? you are using a FREE version..the FREE version only gives you a total number..Quadrant Pro on the other hand gives you the scores in each category..and is actually one of the best for measuring atm on android...
Click to expand...
Click to collapse
It has nothing to do with cost. It has to do with the fact that Quadrant is the most commonly used benchmark and it is deeply flawed.
Yes, the Pro version does break down the scores. Most people don't pay for Quadrant. Secondly, the I/O scores are weighted very heavily... the score nearly tripled because an I/O test completed a little quicker. There's a significant problem with that.
I've been impressed by the Epic's speed, but your scores aren't truly valid because you are "cheating"
I assume you're using the new Quantum ROM (which I also have installed) that implements a lagfix in which one moves the app data and dalvik cache away from Samsung’s embarrasingly slow internal Storage. Doing this significantly boosts I/O scores in Quadrant, thereby cheating the system. In real-life use, your system is barely performing any faster than before the exploit/lagfix
Read here for more info to see how a dev exploited this to create a 3300 Quad score (can't post links so remove the spaces):
http : // androidspin .com /2010/08/23/my-quadrant-is-bigger-than-your-quadrant/
This exploit has even been confirmed by the almighty Cyanogen himself
So is there any real benifits to this Epic 4G "lag fix"? I've seen that the "lag fix" within the other galaxy S phones actually does give real world results...
Eazail70x7 said:
So is there any real benifits to this Epic 4G "lag fix"? I've seen that the "lag fix" within the other galaxy S phones actually does give real world results...
Click to expand...
Click to collapse
Sure, it provides a speed benefit. The only thing being addressed in this thread is that, just because your quadrant scores double, it doesn't mean your device is twice as fast in actual use
Look for Dameon's post a couple of days ago in the quantum rom thread (on tapatalk..and linking sux). He talks about this exploit, but sadly (and understandably), most of the rom migrants want something to quantify expected results.. Even if bs
sent from my RAZR
Its not fair to say quadrant cheats.. rather it just may or may not be measuring performance of an operation that is ever really used on your system. In fact your system does use some fsyncs like quadrant and those will be faster like quadrant. Your system could start using fewer fsyncs though and quadrant would never know. Also your device certainly doesn't use only fsyncs..but those are very related to the lag. So basically, quadrant says the lag fix works if I'm not missing something. That's OK...but you don't need a crazy number to say that.
Bottom line... the only thing that tells how your phone performs...is how your phone performs and that's not just some little disclaimer. Hardware speed depends DRAMATICALLY on how it is used.
Sent from my SHW-M110S using XDA App
biff6789 said:
Sure, it provides a speed benefit. The only thing being addressed in this thread is that, just because your quadrant scores double, it doesn't mean your device is twice as fast in actual use
Click to expand...
Click to collapse
Actually the tweak, as noobnl states, quite specifically provides NO performance improvements to the phone. He would know because he made the hack. See the quote below.
noobnl said:
it is true, we mounted benchmark data folder as a ramdisk/tmpfs, theres no ext3 filesystem in the rom because just for the lulz and educate the group that a lag fix won't do ****.. just to cheat a benchmark.. we were pissed at roms user thinking making rfs to ext3 will make performance better...
Click to expand...
Click to collapse
Moving on...
biff6789 said:
I've been impressed by the Epic's speed, but your scores aren't truly valid because you are "cheating"
I assume you're using the new Quantum ROM (which I also have installed) that implements a lagfix in which one moves the app data and dalvik cache away from Samsung’s embarrasingly slow internal Storage. Doing this significantly boosts I/O scores in Quadrant, thereby cheating the system. In real-life use, your system is barely performing any faster than before the exploit/lagfix
Read here for more info to see how a dev exploited this to create a 3300 Quad score (can't post links so remove the spaces):
http : // androidspin .com /2010/08/23/my-quadrant-is-bigger-than-your-quadrant/
This exploit has even been confirmed by the almighty Cyanogen himself
Click to expand...
Click to collapse
This was the whole point of my post... I think you might have missed that. I would also recommend this article as well: http://briefmobile.com/cyanogen-demonstrates-quadrants-flaws
Electrofreak said:
It has nothing to do with cost. It has to do with the fact that Quadrant is the most commonly used benchmark and it is deeply flawed.
Yes, the Pro version does break down the scores. Most people don't pay for Quadrant. Secondly, the I/O scores are weighted very heavily... the score nearly tripled because an I/O test completed a little quicker. There's a significant problem with that.
Click to expand...
Click to collapse
Well realistically speaking even without the I/O exploit the none pro version never had any meaning to it..why?
Well it measures:
CPU
Memory
I/O
2d
3d
Now lets look at it from perspective..lets say we have a score of 3000 and 99% of it comes from 3d..the phone would still be laggy because even though it may have a good gpu it won't have the other things to back it up.
Quadrant weighs everything evenly...it exists for the sole purpose of getting people to buy the pro version..
So its not Quadrant that is flawed but the people who actually think a large number means everything is automatically super fast...
So in most cases yes, the I/O increase would be almost useless as nothing would utilize it to the point where you would need to utilize such high I/O..unless maybe you plan to run a database server on your phone? lol
But like any benchmark we have to not only look at the number but understand what the number means..similar to how on a linpack gets cheated on with use of a VFP since it measures with floating points...
Heck I can write a program called gTen Score which does an md5 sum of every file on the device and the closer the device is to mine the higher score it would get..would that make the device with the highest score the best? lol..benchmarks are benchmarks...we have to understand them and look at the details to how the numbers got there..In my opinion if its not Quadrant Pro people are wasting their time with the benchmark...
gTen said:
Well realistically speaking even without the I/O exploit the none pro version never had any meaning to it..why?
Well it measures:
CPU
Memory
I/O
2d
3d
Now lets look at it from perspective..lets say we have a score of 3000 and 99% of it comes from 3d..the phone would still be laggy because even though it may have a good gpu it won't have the other things to back it up.
Quadrant weighs everything evenly...it exists for the sole purpose of getting people to buy the pro version..
So its not Quadrant that is flawed but the people who actually think a large number means everything is automatically super fast...
So in most cases yes, the I/O increase would be almost useless as nothing would utilize it to the point where you would need to utilize such high I/O..unless maybe you plan to run a database server on your phone? lol
But like any benchmark we have to not only look at the number but understand what the number means..similar to how on a linpack gets cheated on with use of a VFP since it measures with floating points...
Heck I can write a program called gTen Score which does an md5 sum of every file on the device and the closer the device is to mine the higher score it would get..would that make the device with the highest score the best? lol..benchmarks are benchmarks...we have to understand them and look at the details to how the numbers got there..In my opinion if its not Quadrant Pro people are wasting their time with the benchmark...
Click to expand...
Click to collapse
You make some very good points, but Quadrant IS flawed in that it should be able to test performance in a manner that is more consistent with real-world performance. A benchmark tool that provides poor results for a different configuration of hardware or software (in this case, Samsung's RFS file system) when performance is in on par or even superior to a higher-scoring configuration is a benchmark tool with serious flaws.
Electrofreak said:
You make some very good points, but Quadrant IS flawed in that it should be able to test performance in a manner that is more consistent with real-world performance. A benchmark tool that provides poor results for a different configuration of hardware or software (in this case, Samsung's RFS file system) when performance is in on par or even superior to a higher-scoring configuration is a benchmark tool with serious flaws.
Click to expand...
Click to collapse
The flaw of quadrant is grouping up all of those categories into 1...usually as you said things are weighted into a benchmark and each benchmark measures things like 3d performance, i/o performance an etc would each be a separate benchmark..Quadrant will never do that because their goal is for you to buy the pro version...it was never intended to be used for anything other then a proof of concept for selling the pro version...
Since people are not gonna buy Quadrant Pro..the best way is to get them to use a different benchmark...I hear some people touting GLBenchmark is it any better?
gTen said:
The flaw of quadrant is grouping up all of those categories into 1...usually as you said things are weighted into a benchmark and each benchmark measures things like 3d performance, i/o performance an etc would each be a separate benchmark..Quadrant will never do that because their goal is for you to buy the pro version...it was never intended to be used for anything other then a proof of concept for selling the pro version...
Since people are not gonna buy Quadrant Pro..the best way is to get them to use a different benchmark...I hear some people touting GLBenchmark is it any better?
Click to expand...
Click to collapse
I've been following GLBenchmark and it seems pretty accurate as far as I can tell. A buddy of mine outed the HTC Glacier (Mytouch 4G) when he found that a developer had erroneously left scores posted on GLBenchmark.com. We could tell it was a next-gen Snapdragon by the scores, and that it wasn't running an Adreno 200. Became big news soon thereafter.
Electrofreak said:
I've been following GLBenchmark and it seems pretty accurate as far as I can tell. A buddy of mine outed the HTC Glacier (Mytouch 4G) when he found that a developer had erroneously left scores posted on GLBenchmark.com. We could tell it was a next-gen Snapdragon by the scores... became big news soon thereafter.
Click to expand...
Click to collapse
Well then the best bet is to contact rom developers and ask if they could include GLBenchmark instead of quadrant and explain the benefits of having a specified benchmark over a general unweighed one..
and yeah I saw th HTC Glacier thing..
Electrofreak said:
This was the whole point of my post... I think you might have missed that. I would also recommend this article as well: http://briefmobile.com/cyanogen-demonstrates-quadrants-flaws
Click to expand...
Click to collapse
You're right, I originally DID miss your point. My bad... I thought you were just another fanboi raving about the awesomeness of Quadrant. I should have read more carefully
But in essence, you and I were saying the same thing all along: Quadrant is bunk and can easily be manipulated by a tmpfs tweak
As for my other point about lagfixes offering real-world performance boosts in other phones, I was saying "yes" in a general way as often times they do, not "yes" specifically regarding Dameon's tweak
biff6789 said:
You're right, I originally DID miss your point. My bad... I thought you were just another fanboi raving about the awesomeness of Quadrant. I should have read more carefully
But in essence, you and I were saying the same thing all along: Quadrant is bunk and can easily be manipulated by a tmpfs tweak
As for my other point about lagfixes offering real-world performance boosts in other phones, I was saying "yes" in a general way as often times they do, not "yes" specifically regarding Dameon's tweak
Click to expand...
Click to collapse
Yeah, and from what I understand (forgive me, I am still a relative newcomer here) the other Galaxy S phones mount their file directory a little differently than the Epic does, negating the need for a real lag fix. However, the Captivate lag-fix, for example, does indeed provide actual performance improvements. Perhaps this misunderstanding is what is causing people to believe that the Epic needs to have some sort of lag fix too.
Electrofreak said:
Yeah, and from what I understand (forgive me, I am still a relative newcomer here) the other Galaxy S phones mount their file directory a little differently than the Epic does, negating the need for a real lag fix. However, the Captivate lag-fix, for example, does indeed provide actual performance improvements. Perhaps this misunderstanding is what is causing people to believe that the Epic needs to have some sort of lag fix too.
Click to expand...
Click to collapse
You're exactly right. Dameon himself stated that the other Galaxy phones need a lagfix while the Epic does not (read the last line of the following post):
http://forum.xda-developers.com/showpost.php?p=9440522&postcount=1174
quadrant has been known to very inaccurate. I forget who it was but someone tweaking their evo was able to get like 6000 on quadrant awhile ago. After that it has been tricked many times into giving big numbers with no actual improvements over the actual phone itself.
The purpose of this post is to inform people of all the major benchmarks, pros/cons or each and what each of them do.
Quadrant Standard/Advanced - This benchmark is made for phones using the Snapdragon proccessor such as the g2/droids/evo etc. Please do not rely on this benchmark for the Epic as the Epic has a Hummingbird.
SmartBench - This benchmark was made to show fair scores for the actual speed of the phone. The developer of this app works very hard to work out "cheats" and bugs. This benchmark seperates the speed in games from the speed in doing other, non-game, tasks.
Linpack - This benchmark, like Quadrant, is mainly for Snapdragon. It shows floating point speed that does not really determine real life performance. Phones like the Evo and G2 get scores way higher than the Epic even though the proccessor on the Epic is faster.
0xBench - This benchmark is a combonation of different benchmarks. The purpose of this benchmark is to find out how fast your phone is for many different things.
Total Benchmark - This benchmark is a FULL benchmark. It not only measures speed but it measures many other things such as accelerometer, display, multitouch, etc.
An3dBench - This benchmark does a wide variety of tests to determine 3d speed
SQLight - This benchmark measures the time it takes to complete a series of non-gaming/non-media tasks.
SetCpu - If you didn't know, SetCpu has 3 different benchmarks included in it. These benchmarks show proccessor speed improvements from your overclock/kernal or rom change.
Nenamark - A GPU benchmark
Neocore - Another GPU benchmark. Made by Qualcomm to showcase their GPU. Since our GPU isnt Qualcomm, you cant fully trust it, but it seems pretty legit.
Did I miss any? Please send me a PM telling me. I know there are alot more but I think I got all of the major ones.
It would be great if this could be stickied.
Reserved for later use
A few others I've got on my phone:
Benchmark Pi: I don't recommend, since all it really does is test one function of the CPU, hardly a comprehensive benchmark.
GLBenchmark: Pretty decent benchmark, all things considered. Lots of options.
NenaMark1: Primarily a GPU benchmark, provides FPS (frames per second) score.
Neocore: Same as the above, mostly tests GPU performance. Created by Qualcomm to showcase their Adreno GPU, so take the FPS scores with a grain of salt.
Another I've heard of, BaseMark. AFAIK its currently only available to software developers. Looking to get my hands on it... looks to be pretty full-featured.
I still recommend SmartBench2010 over anything else right now, mostly because it seems to be accurate and the developer has shown himself to be very dedicated.
Also note AnTuTu. It's comprehensive, and gives a very good breakdown of what goes into its scores.
Added Nenamark and Neocore
Sent from my SPH-D700 using XDA Premium App
GLbenchmarks
Its one of the best. I personally only really care about benchmarks that Anandtech uses since they give the most detailed and comprehensive reviews and testing of new phones and devices. I would check out there site to make sure you got what they use. I know for a fact that GLbenchmarks is one of them. Its a graphical benchmark that tests the GPU. There are also a benchmark to test browser performance, java and i think one other one that they use.
You can get all these on the market?
No, some of these you have to get outside of the market. GLbenchmarks for example is downloaded from their website.
guys what is the best benchmark software?
I'm thinking SmartBench 2011. Test multi threads
Don't use Quadrant. It's only good for making your E-weenis look much bigger than it really is.
actually quadrant global is good doesn't test I/O
its not popular as it should be but that's becasue it doesn't show retarded inaccurate I/O scores caused by lagfix ext4 script like standard quadrant does
I was doing some benchmarks on my Galaxy Note and found that I could only achieve a maximum of about 6300 on Antutu, this is even when overclocking to 1600Mhz.
I have tried nearly all of the roms/kernels available and they never get higher than 6300.
Anyone know what I could try to get something higher, especially considering some results on Antutu are higher than 8000.
Thanks
Thread closed
Please refer to this benchmark thread to continue the discussion, we don't need two threads on the subject.
A few months, kernel guru Andreilux discovered the Exynos variant of the Galaxy S4 was detecting the presence of certain benchmarks and raising thermal limits in order to gain an edge on those benchmarks. Ars Technica found the same behavior in the Note 3. It turns out Samsung isn't the only one playing this game, virtually all OEM's do.
In typical Anandtech like thesis fashion, they have broken down the state of android benchmark cheating in this very well written article (at least to me it was).
http://anandtech.com/show/7384/state-of-cheating-in-android-benchmarks
Benchmarks have been irrelevant since gingerbread... But I think what Samsung and HTC do should be the real way to test a phones performance.. there are so many things that affect a phone's performance so testing it's max potential should be the point of a benchmark.. I don't see anything wrong with what they are doing.
Sent from my HTC One using Tapatalk 4
IMO except Exynos version of galaxy s4 none of them is cheating. What is cheating anyway? As samsung does on 9500, users only can use 533mhz in benchmarks. That is cheating. But what about devices just switching governors and thermal limits? Like NOTE 3 or some other phones, you can definately get the SAME level of scores if you throw those phones into a freezer and switch their governors to performance. This is not cheating.
LoL. BTW, I've been reading through the comments and find that none of the replies stating NVIDIA Shield's cheating behavior are replied or commented. NVIDIA Shield COULD BE the WORST of all. It does some trick to dramatically increase the 3D performance, which is not frequency/governor related. As tested, the fan-equipted device can only reach 75% of 3D scores in Antutu X than in original Antutu, and its gpu's frequency is always 672mhz. Apparantly the "better" results come not from governor switching nor from raising thermal limits, but from other tricks to fake the scores. Also NVIDIA Shield does no better in many games than adreno 320 does. Why benchmarks much higher? LoL
Screenshots here
http://tieba.baidu.com/p/2625483808