I'vs posted some related info of potential interest on benchmarks here.
(Edit: corrected URL)

An unoptimized benchmark is incapable of correctly measuring the potential of a CPU. IF you are going to measure a cpu, you need to measure its entire potential to be fair, not its castrated potential. This is at the BOINC level, as it it not in their control whether a project optimizes their code. What use is taking a measurement you know isn't fully accurate?

What use is taking a measurement you know is wrong? What credits are given for is not potential but work done. The new system is on the right track, not optimal but much better then the old system. Thinking loud: Perhaps we need a per-project-benchmark which per host uses those (and only those!) accelerators the science application will use? But no, I don't want a different amount of credits per hour given from the projects. Credits should not be a cause to choose a project.

Norbert

---

On an unrelated note, I got a bunch of failed WUs too; I don't think I changed anything to do that...anyone else seeing that?
[EDIT]: Just received some new WUs that appear to be running fine.

FloridaBear: David Kim posted a message [url=https://boinc.bakerlab.org/rosetta/forum_thread.php?id=1891#25458]here[url] about the errors you were getting.

---

By straight, I would have meant unoptimized. 3 am posts can be bad for word choices.

Precisely my point. An unoptimized benchmark is incapable of correctly measuring the potential of a CPU. IF you are going to measure a cpu, you need to measure its entire potential to be fair, not its castrated potential. This is at the BOINC level, as it it not in their control whether a project optimizes their code. What use is taking a measurement you know isn't fully accurate?

Ah but an unoptimised benchmark does properly measure the potential of a CPU that using nothing but standard X86 instructions (or PowerPC for those of a non-PC persuasion). Rosetta doesn't use anything but X86 instructions, therefore the standard BOINC benchmark properly measures what a processor is capable of on Rosetta. The only way to increase the throughput of the processor is by using extended instruction sets like SSE, which is something Rosetta doesn't use. Is it really fair to reward processors for capabilities they have but which aren't used? If that was the way it worked then Prescott core Pentium 4s would deserve more credit than Newcastle core Athlon 64s, merely because the Prescott has SSE3 and the Newcastle doesn't. But I'm sure you realise the Newcastle is far far faster than the Prescott.

Enough work can be completed to exceed the std benchmark,

Only with optimisations from extended instruction sets like SSE and MMX - which are only really used in optimised SETI science applications.

thus making it a false measure Or at best, incomplete).

But as I've explained, the standard BOINC benchmark is spot on for Rosetta and all other non-optimised BOINC projects. SETI and Einstein are the only projects where there have been science applications with extended instruction set optimisations.

On the other hand, the opt bench can describe the work done but exceeds the proc's throughput limit, also making it false from what you say. So which is "right"? Both and neither it would seem. One is incapable of correctly measuring full potential, the other can measure it, but uses a "naive" method to do so.

The standard BOINC benchmark correctly measures the full potential of a processor that isn't using optimisations that come from extended instruction sets like SSE and MMX. And that fairly covers all BOINC projects and their standard science applications. The only extraordinary cases are the optimised science applications that were found for SETI and Einstien - both of which now have new science applications anyway.

Remember, the optimised BOINC clients reason for existing was to optimise the benchmarks to claim extra credit to account for the faster processing of work units on SETI when optimised science applications were used. Up to a point I agreed with this. That point was passed when one of those optimised BOINC clients, Crunch3r's 5.5.0, threw the laws of everything out the window.

One uses the lowest common denominator to score "fairly",

Yes, but that LCD in this case is standard X86 performance. As this is what is used by all BOINC projects, this is fair. If optimised science applications were freely available for all BOINC projects then it wouldn't be particularly fair.

the other attempts to score the true potential, but does so in a poor manner.

The other, basically just Crunch3r's 5.5.0 (it's the only one I know of that gives such high benches), attempts to get three or more times the credit. It can't really be said to be trying to measure true potential - the other clients already do that with relative increases in benchmark scores for the addition of things like SSE optimisations. 5.5.0 tried to get relatively more credit, because Crunch3r's optimised SETI apps were that much more efficient. That was fair and understandable on SETI, but not anywhere else.

At the end of the day, Trux had a more elegant solution for fair rewarding of SETI credit with optimised science applications. 5.5.0 worked only for SETI, but broke the BOINC ecosystem by throwing out the laws of maths and physics and returning benchmark results from out of this world.

If you had 2 theoretical machines, identical except for SSE2 enabled on one, and disabled on the other, the std benchmark would score them the exact same. If they ran an opt project, the SSE2 enabled machine would by far outproduce the disabled machine, yet would get the same score. With an opt benchmark, the difference would be shown, and the true potential would be scored.

Yup. This is the case, and to begin with, optimised BOINC clients stuck to this. But 5.5.0 went way beyond the curve and gave results that were physically impossible.

As a general purpose benchmark, it should encompass all possibilities. Yes, that would "inflate" the general scores, but in the computer world inflation is inevitable.

Anyway, I think I'm repeating myself or talking in circles, been typing this in spurts through many distractions and hours. Caring for an alzheimer's patient is not conducive to typing a long post in one sitting.

The major point is that one needs to back up from staring intently at the microdetails and look at the general picture. That being the std client cannot properly describe the potential power of modern procs. An opt client is the only way to measure the full potential. (not saying crunch3r's is the best, just a step, or attempt, in the right direction) If the work is possible, then A benchmark is also possible.

Well the thing is the standard BOINC client does fully measure the potential power of modern processors doing just standard X86 stuff. It doesn't take into account things like SSE2, because the projects themselves aren't using it in general. If they were using it then that would be different because it should be taken account of in that case. An optimised BOINC client does take it into account, but is only necessary when the projects themselvs are issuing science applications that use these extra features. When that happens, the BOINC benchmark needs to be overhauled to measure these extra capabilities. But they do not exist in normal usage.

Being as this is no longer relevant to rosy, I'll stop here. The last words are yours, tho, you've basically stated the same facts, with a different end opinion. That or your distaste was simply how crunch obtained the results to describe what was happening.

The only way this all seems to relate to the new credit system would be in that the work based units are scored in a manner that mirrors the original benchmark, which is known to be flawed. Many cannot stomach that.

Also, with the averaging, it would seem that even if Rosy were to get optimizations, more work would not actually be scored, but would instead be reaveraged back down to the "dumbed down" levels of the std bench.

The new system seems to have been a ton of effort to create a new measuring stick, that was wasted by shortening it back to the old one.

I agree with you that the new credit system is flawed in that it determines the credit value for structures based on the benchmark results. But this is averaged out over the whole project, so at least the playing field is level. It's still a bit inaccurate, but it does stop people from having the same unfair advantage that was present before the new credit system.

Therefore, I can only conclude that despite its flaws, it is a step in the right direction and that it is progress.

---

In my mind, the MAIN thing that is flawed with the standard BOINC benchmark is simply that it is so trivial to modify your measured benchmarks into something they are not.

---

Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/

does the new credit system show flops? the old one showed every 100,000 credits was 1 teraflop.

does the new credit system show flops? the old one showed every 100,000 credits was 1 teraflop.

Unless the science application has flop counting in it, there is no way to even guess flops from credits.

---

Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)

If you wanted an 'in the ballpark' estimate. .

Take the flops count for a computer using the standard client (since rosetta doesn't take advantage of sse, sse2, etc) and divide it by the number of credits per day that computer gets running rosetta 24x7. Then multiply that by the number of credits returned in the past 24 hours from the R@H homepage.

There are several sources for error in this calculation, but it's probably closer than the current "divide by 100,000" one.

---

I did my suggested calculation and came up with 21.6 teraflops.

---

Evening All

I have been chasing stats for a bit on the Cross Project Calibration issue... I came over here to see if there was something that I could add to an intelligent discussion... At this point after reading all this, I can only add.

I Survived the Credit Wars @Seti... I know that the hundreds of machines that I have looked at various Projects do not match nor can they match what the Cobblestone is based on... Then depending on how the application is written it favors the front end or the back end... Or Intel VS AMD... Or older machines do better on some projects than others...

So with that said I will offer several machines to look at... Currently they have quite a few Results to calculate from.

Selay AMD XP 2000 currently running BOINC 5.4.6
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=284093

Plygoria AMD XP 2000 currently running BOINC 5.3.12tx36
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=282737

Deltaflyer AMD XP 2800 currently running BOINC 5.4.9
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=282729

Parada Intel PIII 550 currently running BOINC 5.3.12tx36
https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=284685

Please Note that while this machine does not meet the "specs" which equals 100 credits/day
Measured floating point speed 1000.00 million ops/sec
Measured integer speed 1000.00 million ops/sec

It does measure
Measured floating point speed 508.53 million ops/sec
Measured integer speed 1416.55 million ops/sec

IF, you do the quick math does it recieve 100 credits/day?

Tony has given you numbers, I also have numbers and am working with Tony and others... I could give the numbers, however it better that you go look. Then form your own opinions about what you see... I will not debate BOINC Optimized Core, BOINC Optimized Application... "We" are here to do the Science and someone has to test as broadly as they can and report. Then it can be a fair as it is possbile to be Fair!

You are welcome to click on my name and see what machines I have and currently running here...

Otherwise, Rosetta (and other projects) is not my primary interest... I still believe that the Science is my primary interest...

Regards
Pappa

Edit
Mod.DE in the Credit system explained threa you point to the old thread that you took offline...
https://boinc.bakerlab.org/rosetta/forum_thread.php?id=2197
Thought you should know..
Tip of the Hat

---

I just looked at the results of my systems at WU's from 23 aug and later.
To me it looks like the credits are granted in random order, that the cpu-time spend on a WU doesn't matter anymore.
I'll give it another day or two, but if it still not make any sense to me, I switch.

You have a target CPU time from 1 hour. That will always result in greater variations, since for bigger proteins you are just doing one model per WU. Model runtimes vary and so will your credit for such short WU. If you want smoother credit granting switch to 4 hours or more and give it a week or two instead of a day or two. ;-)

What Tralala said. Take a look at my systems, all of which run WU's for 12 hours. It's eerie how close the claimed and granted numbers are except for the two linux systems which are now getting fair credit.

---

MovieMan:

"Had you read my posts you wouldn't have made that statement."

There actually have been one (it's been said a number of times) or more people posting here claiming that the reason they use(d) Crunch3r's 5.5.0 client was that it gave them the credits they felt they deserved for having advanced features sets on their cpus.

Obviously, not everyone used the same justification as you do. I know that on another board I was taking part in, that they voted on using an optimized client to stay competitive with the teams around them, so some of teams you didn't mention - felt the same way you do.

---

XS Vietnam Soldier posted:

In no way I wanted to imply that the opt clients delivered falsified results. I stated so on many occasions and will do so in future. You explained very nicely how those impressive benchmarks can be achieved by utilizing all the power modern processors offer. However as you said, Rosetta can't use all those features yet so in fact the benchmark offers more of a potential speed which could be gained from such a computer if the app could utilize all the features. Whether credit should be based on the real work done or the potential a host offers is another question and I'm undecided about that. However I was and am against a system in which every host just gets what he claims. That led to laughable claims of poor hosts and as I remember the most absurd were dealt with but every day new absurd appeared and the more subtle cheats were never caught. It was just a hole in the credit system, which required constant action from the project staff and prevented them from the important tasks.

Actually, Crunch3r's 5.5.0 client did return benchmarks that were higher than the physical abilities of a processor even if perfect efficiency was achieved.

This is my 2.3 GHz Athlon XP. It's a Barton core running at 11.5 * 200 MHz. Now look at my integer benchmarks, as given by Crunch3r's 5.5.0 client - "Measured integer speed 11476.56 million ops/sec". Thing is, an Athlon XP can only issue 3 integer instructions per clock cycle, meaning that my processor is only capable of 3 * 2,300,000,000 integer instructions per second. That means that with absolute perfect efficiency, which you don't get ever, my processor can issue 6,900 million ops/sec. Where did the extra 4,576,56 million ops/sec come from?!

This is my 2.14 GHz Duron. It runs Trux's 5.2.13 or something like that. Note it's integer benchmark - "Measured integer speed 6364.27 million ops/sec". But that is physically possible because 3 * 2,140,000,000 = 6,420 million ops/sec.

I am sick and tired of people saying that Crunch3r's 5.5.0 client rewards them based on what they can contribute. No, you can't contribute something that is physically beyond the capabilities of your processor.

Had you read my posts you wouldn't have made that statement.
I'll repeat it here for you:
The one and only reason that we on XS( and I assume Free Dc and the DPC also) used crunch3r's optimised 5.5.0 files is that they corrected the disparity on the credit level between Intel and AMD machines. This was a flaw within the BOINC manager app. That is the only reason.
Did we realise that they gave more points? Of course we did but with all the top teams using the same files it was a non issue. Parity between machines within the Rosetta project was restored.
We didn't care about Boinc cross project parity and still don't.
We cared about each machine within that one project getting a fair credit value based upon the work it was doing and those files accomplished that goal.
You can beleive otherwise if you wish, I really don't care any more but ask yourself this: IF XS was only interested in the pursuit of points why would we, on the same day that we passed Free DC to go into #1 place, email the esteemed Dr. Baker and request that app 3,500,000 points be removed from our teams score as we'd just that same day been informed that 2 individuals on our team had without permission installed Rosetta on a large number of machines.
Think on that just a bit before you try and judge me or my team mates.
We were also the ones that brought to the attention of Baker Labs the ability to manipulate the XML files and play with the benchmarks. Jose from our team, Bok from Free DC and I beleive Kristof from the DPC coordinated in searching for those manipulating those files.

I told myself that I would not post over here again but I will not quietly sit back and see people make statements that have no basis in fact and disparage my teams good name.
[Your team is not mentioned in the post you quoted. - T.M.]
We brought between 2000-2500 top end machines to Rosetta for eight months that ran 24/7 and JUST for Rosetta.
Points are a non issue in the big picture, the work that was done by those machines and the devotion to this project by the people who donated them is all that matters.
Beleive me or don't, thats up to you, but ask yourself what in god's name do I have to gain by lying about it?
I will never meet 99% of you, I don't interact with any of you on a day to day basis. The answer is I have nothing to gain by lying to you and I am not doing so.
The bottom line is this: A few individuals with the agenda of changing the credit system on Rosetta to meet with what they thought was correct

[Moving to a work based credit system was a solution you mentioned prior to CASP. There's more than a few individuals that share the credit and/or blame for the new credit system. - T.M.]

I orginally first posted here because I knew that a lie told often enough tends to be accepted as the truth over time.
[edit out] [<- this is the main reason for this edit. Please be as respectful as you would want the party or parties mentioned to reciprocate. - T.M.]
I won't support any group or individual that allows the people that support them to be called cheats on their own forum.
[Normally, such issues are brought to the attention of the moderation staff at the link at the bottom of this post, and a moderator will deal with the issue. 4 of us are new here, and were not present at the time. - T.M.]
Movieman

Mr. Moderator:
I named no individual so I don't understand why the editing of my post.
You do make me think that the answer to this is not to post here, but where it will be seen by people who support this project and perhaps it will make them question their wisdom in doing so.
There is more to any project than the science.
There is the effective management of that project and the character of lack of in regards to the people that manage said project.
It is not the people that attacked the team that bothered me,they are nothing in my eyes. They gave nothing to the project in any measurable terms.
It was the total lack of response to stop it from the project managers that brought out my anger.
As to your comment that I highlighted above, I find it amazingly coincidental that the so called new credit system awards points at a level to make them at least seem to give the pro Boinc parity people what they were yelling for.
I've also read enough to see that the current credit structure is flawed beyond beleif.
Feel free to delete this if you feel compelled to quell anything that isn't pro your feelings. My next comments on this will be where a national audience can see it with no thought Nazi's to cut out what is printed.
Movieman

---

Yes, Movieman, a DC project is more than just the science. It's also the people that contribute that give it a sense of community. Be it on their own team forums, or on the project's main forum. But endless fighting doesn't make for an enjoyable atmosphere for most contributors.
We need to treat each other with more respect to keep such fighting from breaking out. And that's where the thought police, that you were demanding, come in. Keeping the peace, and pushing for more respect. Therefore, I thank you for your attempt at conveying your comment in a more respectful manner.

This is a thread about discussing the New Credit System. Let's get back to something approximating the core topic. <smile>

---

Rosetta Moderator: Mod.Tymbrimi
ROSETTA@home FAQ
Moderator Contact

Yes, Movieman, a DC project is more than just the science. It's also the people that contribute that give it a sense of community. Be it on their own team forums, or on the project's main forum. But endless fighting doesn't make for an enjoyable atmosphere for most contributors.
We need to treat each other with more respect to keep such fighting from breaking out. And that's where the thought police, that you were demanding, come in. Keeping the peace, and pushing for more respect. Therefore, I thank you for your attempt at conveying your comment in a more respectful manner.

This is a thread about discussing the New Credit System. Let's get back to something approximating the core topic. <smile>

I find it funny that you use the word "respectful" when the project developers showed my team none by allowing this to continue for weeks at a time.
Where was "respect" then when every day my team was being called a cheat?
I'll tell you the answer, there wasn't any because the people who run this project did not and do not respect the efforts of the people who devout their time, money and machinery to them.In their eyes we are tools to be used and nothing more. Not people who pushed the limits for them but tools. There is no community in this project there is only big business that could care less about any one of us. If you think differently or need to be persuaded, just watch Dr. Baker's "needy children" video and you will immediately rethink your support of him and his project.
You want to discuss the new credit system, ponder this:
As long as a fixed value is used for a decoy and said decoy takes a different time to generate on the same machine between different work units you will never have a "work based " credit system that is realistic. To make a credit system like the one in place now you would need to run each and every work unit on a machine that you used as a standard, note the times that each decoy took to generate, then apply a time factor that could be used across the entire range of WU's..Totally impractical as then the lab is running each work unit, getting the results and there is no need of the thousands of crunchers that do this project..
That is presented "respectfully"..
Movieman

---

I find it funny that you use the word "respectful" when the project developers showed my team none by allowing this to continue for weeks at a time.
Where was "respect" then when every day my team was being called a cheat?
Movieman

Movieman, please don't try to rewrite history. Here is my version of history:
All cheating accusations against your team were deleted and it was forbidden to call anybody using 5.5.0 a cheater. It is true that it took a few days to remove such posts because at this time there was only one mod active. Later Dr. Baker even apologized on your board that it took so long and that you felt offended. Many felt this preferential treatment problematic since togehter with his apologies he offered you on your board not to backdate, which was until then under discussion in the Rosetta boards. After he realized that his involvement could make things only worse he confined himself to posting in his journal.

Although cheating accusations were not allowed against your team, backdating was discarded and Dr. Baker personally apologized on your board for these accusations and posted in his science journal that he is grateful for any contribution (whether 5.5.0 or standard client) you kept demanding yet anohter statement, in which he should take sides and denounce those as wrong who disagree with your viewpoint. It is clear that such a thing could not happen. The project team had already shown a deep respect to you and listened to your proposals attentively but you kept asking for choosing you or what you like to call the "vocal minority".

The bad side is, that in the end you have damaged your tremendous effort for Rosetta over the past months with a inapt behaviour when the new credit system was invented. The good side is, this did not damage Rosetta as much as one could have feared.

regards
tralala

Tony has given you numbers, I also have numbers and am working with Tony and others...

Nuff said. Which project is next then ?

I find it unbelievable what Bakerlabs has allowed to happen to this project, especially after what has gone on before elsewhere.

---

Tony has given you numbers, I also have numbers and am working with Tony and others...

Nuff said. Which project is next then ?

I find it unbelievable what Bakerlabs has allowed to happen to this project, especially after what has gone on before elsewhere.

@Whl. I use exclusively Stock Boinc Core clients. Pappa admittedly uses that of Trux on some of his puters, and I think he has others in his massive farm. Together this data will show "a picture" of what's happening "cross project". Perhaps you or someone else who uses 5.5.0 or other third party Boinc Core Client would like to join in to show a "bigger picture"?? We could set you up with a base form to use, which would help in collating the data into a "bigger picture". I've shared my data with him and others, but he's still collecting data so I don't have his yet. Wanna help? The list of projects I'm doing can be found in any of my "cross project comparisons". I'm not sure exactly which projects Al (aka Pappa) is doing.

tony

I find it funny that you use the word "respectful" when the project developers showed my team none by allowing this to continue for weeks at a time.
Where was "respect" then when every day my team was being called a cheat?
Movieman

Movieman, please don't try to rewrite history. Here is my version of history:
All cheating accusations against your team were deleted and it was forbidden to call anybody using 5.5.0 a cheater. It is true that it took a few days to remove such posts because at this time there was only one mod active. Later Dr. Baker even apologized on your board that it took so long and that you felt offended. Many felt this preferential treatment problematic since togehter with his apologies he offered you on your board not to backdate, which was until then under discussion in the Rosetta boards. After he realized that his involvement could make things only worse he confined himself to posting in his journal.

Although cheating accusations were not allowed against your team, backdating was discarded and Dr. Baker personally apologized on your board for these accusations and posted in his science journal that he is grateful for any contribution (whether 5.5.0 or standard client) you kept demanding yet anohter statement, in which he should take sides and denounce those as wrong who disagree with your viewpoint. It is clear that such a thing could not happen. The project team had already shown a deep respect to you and listened to your proposals attentively but you kept asking for choosing you or what you like to call the "vocal minority".

The bad side is, that in the end you have damaged your tremendous effort for Rosetta over the past months with a inapt behaviour when the new credit system was invented. The good side is, this did not damage Rosetta as much as one could have feared.

regards
tralala

That is your recollection of what went on, it is not mine.
Mine is I was informed by Jose that there was a small group over here trying to change the rosetta structure credit system to what they wanted in the name of cross project parity within Boinc.That is when I got involved.
We asked Dr. Baker to come to this section over and over and make a statement to put a stop to their name calling. He did not.
ONLY after the XS team started to shut down did he open his mouth and he still didn't come to this section and make a statement.What he said was fluff at XS. He was online a grand total of 3 minutes. There was no communication with him. He came to make a statement and left. The owner of XS even PM'd him but he didn't bother to reply.
Re the highlighted section above: We damaged nothing. The work that was done is done. It was Dr. Baker himself by his lack of action that damaged any further effort that would have been done by us on his project.
This had nothing to do with the new credit system, inept as it is.
This had to do with respect my friend, his lack of it for the people that did this work for him and his lack of understanding of the importance if that word to others.
Don't try and change the issue to points when it had nothing to do with points.
Movieman

---

Someone correct me if I'm wrong here, but "cross-project parity" is really a reference to the BOINC credit system. Which is supposed to be measuring the crunching effort in TeraFLOPS. Since TeraFLOPS is a standardized computing term for work throughput, and has nothing to do with BOINC, I don't understand why having a secondary goal of having your credits issued in line with TeraFLOPS of work done is a point to debate. What am I missing about the meaning of a credit and a cobblestone?

---

Add this signature to your EMail:
Running Microsoft's "System Idle Process" will never help cure cancer, AIDS nor Alzheimer's. But running Rosetta@home just might!
https://boinc.bakerlab.org/rosetta/