Discussion of the new credit systen (3)

Message boards : Number crunching : Discussion of the new credit systen (3)

To post messages, you must log in.

AuthorMessage
Mod.Zilla
Volunteer moderator

Send message
Joined: 5 Sep 06
Posts: 423
Credit: 6
RAC: 0
Message 61306 - Posted: 21 May 2009, 17:59:46 UTC
Last modified: 8 Dec 2012, 18:30:59 UTC

continued from Discussion of the new credit systen (2)
Rosetta Informational Moderator: Mod.Zilla
ID: 61306 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Steve Hawker*

Send message
Joined: 26 Oct 12
Posts: 2
Credit: 143,860
RAC: 0
Message 74128 - Posted: 30 Oct 2012, 5:19:51 UTC

I guess my comment applies to every BOINC project where cr/hr has been lowered to get rid of cheaters, but I just happen to be here so here goes...

On almost every project where the credit earning system has been altered there is a statement somewhere on the message forums justifying the move on the basis of getting rid of the cheaters. Usually there is some elitist clap-trap about "only doing it for the science".

What a bunch of baloney.

Who cares if some enterprising folks manage to wangle a way to get granted more credits? The science still got done, didn't it?

"Oh no!" say the admins, "we must purge ourselves from the unholy cheats and return to pure science".

Meanwhile, folks like me who enjoy the science but also enjoy the fun of gaining credits get screwed. So instead of the cheaters cheating themselves, the admins have decided to cheat real users instead.

Yeah, I only crunched a few Rosetta WUs but I was somewhat disappointed to receive a paltry number of credits. Yeah, go on, criticize me for my decision, I don't care. Just remember that I'm a part of the rank of file crunching community and you caught me up in the Admins-Cheaters war.

So back I go to NumberFields or some other project and look what happens. Rosetta loses another cruncher but hey! Rosetta is purged of those awful cheaters who dirty our science. That's the important thing, right?

Nice work admins, well done.
ID: 74128 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,134,655
RAC: 4,716
Message 74130 - Posted: 30 Oct 2012, 10:55:25 UTC - in response to Message 74128.  

I guess my comment applies to every BOINC project where cr/hr has been lowered to get rid of cheaters, but I just happen to be here so here goes...

On almost every project where the credit earning system has been altered there is a statement somewhere on the message forums justifying the move on the basis of getting rid of the cheaters. Usually there is some elitist clap-trap about "only doing it for the science".

What a bunch of baloney.

Who cares if some enterprising folks manage to wangle a way to get granted more credits? The science still got done, didn't it?

"Oh no!" say the admins, "we must purge ourselves from the unholy cheats and return to pure science".

Meanwhile, folks like me who enjoy the science but also enjoy the fun of gaining credits get screwed. So instead of the cheaters cheating themselves, the admins have decided to cheat real users instead.

Yeah, I only crunched a few Rosetta WUs but I was somewhat disappointed to receive a paltry number of credits. Yeah, go on, criticize me for my decision, I don't care. Just remember that I'm a part of the rank of file crunching community and you caught me up in the Admins-Cheaters war.

So back I go to NumberFields or some other project and look what happens. Rosetta loses another cruncher but hey! Rosetta is purged of those awful cheaters who dirty our science. That's the important thing, right?

Nice work admins, well done.


I cannot defend the number of credits awarded but can give some further insight into why they might keep things the way they are...1)credits can attract people to a project, give people a bazillion credits for 1 second of work and nearly EVERYONE will beat a path to your door. The problem for the project then is ALL those people want work, meaning a high workload on the infrastructure of the project meaning it will likely fail. So unless you can afford full time techs and have TONS of money to through at the project, which most don't, that is not a good idea. 2)giving low credits, compared to other projects, prevents #1 but IF you provide a reasonable project people that do come will stay as long as they get SOME credits. In the end the idea of most projects is to find that happy medium between too many users and not enough users, credits can help a project find that balance. Rosetta seems to be happy where they are, gaining some new users and losing older users, but mostly maintaining an even number of users over time. Believe it or not MOST people come to a project and NEVER crunch for ANY other project in their entire crunching life! They apparently come for a reason and as long as they get some kind of hope that it is helping get to their goal, they stay and are reasonably happy. Also MOST crunchers NEVER use the message boards, which is why the new versions of Boinc have the Notices tab in the Boinc Manager.
ID: 74130 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
googloo
Avatar

Send message
Joined: 15 Sep 06
Posts: 133
Credit: 22,677,186
RAC: 4,532
Message 74567 - Posted: 25 Nov 2012, 20:47:11 UTC

I'm not complaining (too much) about how much credit I get, but what I don't understand is why every (yes, every) task gets awarded lower credit than claimed. Can someone please explain that?
ID: 74567 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1831
Credit: 119,523,428
RAC: 9,566
Message 74568 - Posted: 25 Nov 2012, 22:35:28 UTC - in response to Message 74567.  

I'm not complaining (too much) about how much credit I get, but what I don't understand is why every (yes, every) task gets awarded lower credit than claimed. Can someone please explain that?

It's because the claimed credit is based solely on the benchmark which is overly-simplistic as it's a simple (50% whetstone/50% dhrystone I think) benchmark so it doesn't take things like cache size or memory speed into account. The granted credit is calculated more fairly - the idea is the same amount of credit for the same work done (so faster computers get relatively more credit in line with their increased compute power). It's based on the average claimed credit from every task of that type submitted to date, multiplied by the number of models you submit. So basically it averages the claimed credit from all submitted tasks so far and grants you that.

For example, assume for task type abc1:

  • 20 users have each returned one task
  • each of those tasks contains 6 decoys (each task that your computer submits can contain many models/decoys), so a total of 120 decoys submitted.
  • the average claimed credit for each of those decoys is 10 credits.


Then you submit a task containing 8 decoys and claim 26 credits for each. That makes a total of 128 decoys submitted with an average claimed credit of 11 for each: (120*10 + 8*26) / 128 = 11. Therefore you'll claim 26 credits per decoy and receive 11.

That way, increasing the benchmark score has little effect on the credit awarded - in reality the 120 decoys submitted would probably be into the thousands very quickly after a new task is released, further diluting the effect if one computer is claiming very high credits for the same work.

In your case, the benchmark is artificially high because of hyperthreading on the i7 - it's still just about the best consumer CPU out there though, so the granted credit per cpu second is still very high and it gets lots of CPU seconds because it's running 8 threads at once.

HTH
Danny


ID: 74568 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mad_Max

Send message
Joined: 31 Dec 09
Posts: 209
Credit: 25,844,167
RAC: 12,218
Message 74659 - Posted: 4 Dec 2012, 11:46:37 UTC
Last modified: 4 Dec 2012, 11:52:09 UTC

Yes one of the main cause it is i7 hyperthreading cores. HT cores does very optimistic estimate in syntetic benchmarks but not so good in real Rosetta@Home calculations. So granted credit almost always lower to claimed on HT CPUs.
On CPUs with real cores only (like Intel i5 or Core 2 Duo/Quad or AMD Phenoms) claimed and granted credits usual near par (excluding some "problem" WUs)
For example stats from my comp (AMD Phenom II X6 = 6 real cores, no HT)
Runtime   Claimed   Granted
10,372.47   66.14   70.28
10,569.98   67.45   76.99
10,202.48   65.05   71.25
10,687.62   68.15   70.96
10,170.27   64.85   69.95
10,273.89   65.51   71.25
10,427.88   66.49   70.5
10,790.77   68.80   71.25
10,791.20   68.81   79.74
10,846.67   69.16   78.22
10,702.70   68.24   75
10,553.97   67.29   74.52
10,582.39   67.47   69.51
16,969.39   108.2   69.24
10,231.17   65.24   64.15
10,631.16   67.79   74.18
10,139.06   64.65   72.72
10,759.66   68.65   72.74
10,403.58   66.33   60.45

Totals = 206106 CPU seconds, 1314 claimed credits, 1363 Granted credits

P.S.
But i7 still faster with HT on (just not as fast, as suggested by the benchmark. So if RAM is enough for running 8/12 R@H threads, no need to turn HT off).
ID: 74659 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mad_Max

Send message
Joined: 31 Dec 09
Posts: 209
Credit: 25,844,167
RAC: 12,218
Message 74675 - Posted: 7 Dec 2012, 0:00:26 UTC

By the way, someone can explain why the standard BOINC mechanism claims relative low credits? (And accordingly Rosetta@Home too, because the current credit system in R@H is based on estimates from BOINC clients)

Look for example above (my computer) or pick up any other.
Totals = 206106 CPU seconds, 1314 claimed credits, 1363 Granted credits
1314/206106 = 0,006375360251521 Cr per CPU second or 550 Cr(Cobblestones) per day (24hr of pure computing time, excluding BOINC/OS overhead and other load)

But by definition in BOINC system 1 Cooblestone = 1/200 day work on reference computer with speed of 1 GFLOPS. (http://boinc.berkeley.edu/wiki/Computation_credit)
And example computer speed (estimate by buildin BOINC test) is ~ 3.4 GFLOPS(https://boinc.bakerlab.org/rosetta/show_host_detail.php?hostid=1252064)
So it should claim 3.4*200=680 Cr / day, but in real it claim only 550 Cr/day
What wrong?

Or all BOINC clients (including latest versions - for example i use 7.0.28 and 7.0.31) still use old cobblestone formula? And official BOINC manual is completely wrong!?

Lets see
Old BOINC Cooblestone formula was:
claimed credit = ([whetstone]+[dhrystone]) * wu_cpu_time_in_sec / 1728000
Where whetstone and dhrystone are BOINC benchmark results (in millions operations per second) for float point and integer computation respectively.
For my computer benchmarks are ~ 3400 FP and 7850 integer
So it should claim (3400+7850)*86400/1728000 = 562.5 Cr/day
Hmm, it very close to actual average values i take from 20 Wus = 550 Cr/day
(A small ~2% deviation can be explained by the fact that BOINC periodically repeats benchmark test and the results from time to time may be slightly different).

So what wrong? Actual claimed credit calculation in BOINC clients (may be in some versions only)? Or BOINC documentation?
Or I have something wrong in my calculations? Then please do similar calculations for other computer and post here for comparison...
ID: 74675 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile microchip
Avatar

Send message
Joined: 10 Nov 10
Posts: 10
Credit: 2,237,769
RAC: 1,994
Message 74676 - Posted: 7 Dec 2012, 9:45:14 UTC - in response to Message 74567.  
Last modified: 7 Dec 2012, 9:46:08 UTC

I'm not complaining (too much) about how much credit I get, but what I don't understand is why every (yes, every) task gets awarded lower credit than claimed. Can someone please explain that?


I'm also wondering about that.

Team Belgium
ID: 74676 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1831
Credit: 119,523,428
RAC: 9,566
Message 74678 - Posted: 7 Dec 2012, 11:54:55 UTC

Might it be that it's due to using the whetstone * dhrystone benchmark, but that more modern computers' dhrystone benchmark results haven't improved as quickly as their whetstone benchmark scores?

Basically, if the FPU has improved more than the integer units, in comparison to the original benchmark machine, then your whetstone score might be 3.4x higher, but your dhrystone score might not be 3.4x the original machines' dhrystone score, so your claimed credit isn't either...
ID: 74678 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1831
Credit: 119,523,428
RAC: 9,566
Message 74680 - Posted: 7 Dec 2012, 18:29:39 UTC - in response to Message 74679.  

Like stated a few messages before, it could also be due to the hyperthreading in modern Intel CPU's. And maybe the sharing of FPU units on AMD processors, the latest phenom processors do that if I remember correctly, could cause contention for resources. There could also be contention for other resources (memory) on a PC.


It certainly does partially explain why granted credit is comparatively lower on those CPUs with inter-core resource sharing, but Hyperthreading/shared FPU doesn't explain why the maths doesn't add up in Mad_Max's post - no HT and no shared FPU (Phenom II x6)...
ID: 74680 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 74682 - Posted: 7 Dec 2012, 21:39:49 UTC

It is always important to keep in mind that the results of the benchmarks vary each time they are run. This is part of why Rosetta's credit system only uses the numbers indirectly and in aggregate.

If the machine is busy with something when benchmarks are done, you'll get rather different results. This sort of thing led to disproportionately high credit claims coming from machines where the user made no attempt to spoof the credit system and was just running the default install.
Rosetta Moderator: Mod.Sense
ID: 74682 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mad_Max

Send message
Joined: 31 Dec 09
Posts: 209
Credit: 25,844,167
RAC: 12,218
Message 74687 - Posted: 8 Dec 2012, 14:27:52 UTC - in response to Message 74679.  

Like stated a few messages before, it could also be due to the hyperthreading in modern Intel CPU's. And maybe the sharing of FPU units on AMD processors, the latest phenom processors do that if I remember correctly, could cause contention for resources. There could also be contention for other resources (memory) on a PC.

Wouldn't it be better if the BOINC manager ran the benchmark on all available (logical) CPU's instead of the one CPU it uses now? Or does it do that already?
That way contention for resources would be taken into account too, sort of, when running the benchmark.


Phemom CPUs do not use hyperthreading and do not share any computing blocks(AMD start using shared blocks only on next generation - AMD FX Bulldozer). Phemom has only logical=physicall, independent cores (only L3 cache and memory/system bus shared)

All latest BOINC versions (at least 7.x) do benchmark on all available cores.
Some of old clients do benchmark only on 1 core (but i do not remember at which version this change was made)
ID: 74687 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mad_Max

Send message
Joined: 31 Dec 09
Posts: 209
Credit: 25,844,167
RAC: 12,218
Message 74688 - Posted: 8 Dec 2012, 14:55:47 UTC
Last modified: 8 Dec 2012, 15:15:41 UTC

But we wander off the point. The question is not about specific credits obtained on a particular CPU. But the fact that the BOINC documentation and what real BOINC programs do are fundamentally different.
Since time from my original post I checked few more other computer(just few random comps from R@H stat). And found what all clients - both new (7.x) and older (6.12.x) - they all use the old cobblestone/credit formula:
(float point speed + integer speed) / 2
While the BOINC documentation described another formula:
float point speed х 2
By first formula computer with a speed of 1 GFLOPS (and 1 integer GIOPS) will be rated at 100 cooblestones(Cr) / day
2nd formula for the same computer will be rated at 200 Cr / day.

This is a serious contradiction that generates a lot of confusion as most oriented to the 2nd formula (as it is described in the documentation). For example, many statistics sites (including the largest boincstats.com) for determination of the rate of projects (and the BOINC platform in general) use the formula speed(in GFLOPS)=RAC/200. Many Wiki articles comparing distributed computing projects among themselves and to supercomputers also use this formula.
Where as in reality BOINC software use a different formula for scoring.

For example Rosetta@home use credit / 100 for own FLOPS estimate (110 TFLOPS now). And Einstein@Home too (~900 TFLOPS).
But for example POEM@Home use credit/200
While at boincstats R@H and E@H projects rated as ~55 TFLOPS and ~400 TFLOPS as boincstats author use formula from BOINC documentation.
ID: 74688 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Discussion of the new credit systen (3)



©2024 University of Washington
https://www.bakerlab.org