Ubuntu QA:
BlogBrainstormPackage status
Log in
Ubuntu QA
The Ubuntu community has contributed 21549 ideas, 132424 comments, 2606791 votes
Idea sandbox Idea sandbox
Popular ideas Popular ideas
Ideas in development Ideas in development
Implemented ideas Implemented ideas
Idea #5407: Promote development of modern benchmarking program

Written by Auzy the 22 Mar 08 at 14:26. Category: Gaming. Related project: Nothing/Others. Status: New
Rationale
We have dozens of ancient benchmarks, which mainly test I/O operations, or basic tasks. Or your typical Quake 3 ones.

We really should somehow promote development (maybe as a bounty) for a new benchmark (or benchmark framework) that can push modern hardware (similar to 3Dmark). Whilst it may not seem important, many computer guys use 3Dmark normally as a way to test their overclocking, and compare to other people.

If we design it in a modular fashion, as a framework, where every test is a plugin, the community will help making good plugins. Surely a framework would not take much effort to code.

After we have a proper framework, you could expect members of the community to code:
- I/O tests
- Shader tests
- OpenGL tests
- Even directX tests on windows (we have the advantage though of being able to plugin to winelib too to benchmark wine).

Its a project with a little time needed at the beginning, but it will take on a life of its own. And, since we are in total control, it will give us the ability to accurately test our performance with other OS's, to work out what we can improve (or where we pwn them).

Current benchmarks like 3Dmark are coded in Directx, so are unlikely to be ported to other platforms, and we have the advantage of having a modular system.

In the future, we could use it to test for default installed programs. ie, test automatically if the system is good enough for compwiz. and enable if it runs it well, allow users to only list games for install they can run well, etc.

----
From #16560 merge:
It would tell user how much ubuntu has improved (or not), which drivers are better, also which hardware (if he changed anything)
It should contain test for:
CPU
Graphics card
Disk(s)
etc
- It should store information on xml files, including hardware info (in case user changes something)
- It should store information about most important packages - like kernel versions, graphical driver etc
- This tool should also create graphs with combination with all other info from xml's, like:
- Compare system performance with recent kernels
- Compare GPU performance before and after enabling different driver
- Compare overall performance with next releases
- Compare disk performance with different hardware - file systems
Tags: (none)

453
votes
up equal down
Solution #1: Auto-generated solution of idea #5407
Written by Auzy the 22 Mar 08 at 14:26.
Ubuntu Brainstorm was updated in January 2009. Since the idea #5407 was submitted before this update, its rationale and solution are not separated. Please vote accordingly, and if you have the necessary rights, please separate the rationale from the solution. Thanks!

Propose your solution

Attachments
No attachments.


Duplicates


Comments
Auzy wrote on the 22 Mar 08 at 14:27
And happy easter all :)

neon wrote on the 22 Mar 08 at 17:54
+1000. xD And happy easter to you too. ^^

Vadim P. wrote on the 23 Mar 08 at 04:05
I believe the Phoronix Test Suite will take care of this, when it's finished.

http://www.phoronix-test-suite.com/

Lutz_Ifer wrote on the 26 Mar 08 at 00:01
absolutely +1

andrewfenn wrote on the 29 Mar 08 at 19:43
I would be willing to help begin coding if you start by providing some high poly artwork as the 3dMark test suite has. If you start by scripting out a scene and then have at least one done to 3dMark quality then I'll start coding.

andrew fenn at gmail dot com, with no spaces.

Auzy wrote on the 1 Apr 08 at 13:40
Well, i can guarentee a lot of polygons.. Cant guarentee that they are anything more then a teapot or sphere though :P


Never really been into art, always been more a coder. But we could always start by plugging into free games

mahjongg wrote on the 8 Apr 08 at 15:25
In my humble opinion the most important thing this code should do is to show a user that his open GL hardware is running, and the drivers are working.

At the moment it's very hard to find out if your freshly installed Ubuntu has all the ingredients to support a 3D game, or even compiz fusion, what this code should primarily do is to do a basic test of the functionality of 3D hardware and software, and give pointers when something is failing, for example the hint to install "restricted drivers", when needed.

Testing the performance of the 3D hardware is also nice, but just providing the basic knowledge that everything is working, and assisting the user to get it working, is more important, and should be the primary focus.

Vadim P. wrote on the 17 Jul 08 at 12:23
This is already done: http://phoronix-test-suite.com/

notyetroot wrote on the 10 Aug 08 at 15:32
+0. This would be nice, but we have more pressing issues, like the sub-10% market share to deal with first.

notyetroot wrote on the 12 Aug 08 at 19:00
Nvm, +1.

Eldmannen wrote on the 12 Aug 08 at 23:58
There has been suspicions about cheating in closed-source benchmark software. Example, developers optimized for a vendor, etc.
Having a open-source cross-platform benchmark software would be great.

Vadim P. wrote on the 13 Aug 08 at 02:10
There is one already available: http://phoronix-test-suite.com/

zoubidoo wrote on the 3 Sep 08 at 12:21
GLMark is a 3D benchmark comparing the different features / extensions of OpenGL. Made for the Linux operating system, but is portable to any other OS that SDL supports.

https://sourceforge.net/projects/glmark

stoobers wrote on the 3 Sep 08 at 15:57
We don't need more benchmark tools. What we need are STANDARDIZED tools, and a central database so we can see how specific computers compare while running the same benchmarks.

To compare performance, I do a google search for "bogomips". Some folks post this value. Its nice, but what i really want to know is:

How will mini-itx motherbord QFRP perform while adding integers? floating point multiplication? How fast will a computer access a simple database with 1 million rows of non-indexed data vs. indexed data? Which is faster, the QFRP or the XFRP etc.

A small handful of test stats will suffice. Maybe 4 or 5 tests that everyone runs. Very much like phorenix, but rolled into ubuntu, and (an optional) part of the install / update process.

Phoronix comes close, but it emphasizes allowing people to run their own test suites, which doesn't help in a comparison. We need a standardized, centralized repository.


Post your comment