Message boards : Graphics cards (GPUs) : Asymmetric GPU installations
Author | Message |
---|---|
Get your votes in early ... Asymmetric GPUs from the same vendor is currently not supported. I suggested that this is a bad idea: You are going to have to eventually for the simple reason that people, particularly for the wider rigs are not going to be able to upgrade 3-4 GPUs necessarily at the same time... Understand clearly what this means, if you have a 3 GPU system their expectation is that all three will be same make, model, and internals... In that GPU cards in the same model line don't always stay the same from one year to the next this is an impossible standard to meet ... unless you have way more money than I and can afford to buy your GPU sets all on the same day ... I pointed out that this asymmetry was inevitable last year, that GPU installations are more than likely going to be asymmetric than not for many people ... but UCB in cloud land is going to expect that if you want to run GTX295 cards or HD5870s you are going to buy them in batch lots ... I don't know about you ... but I don't always have $1,500-$2,000 to spend all in one swell foop ... Of course, you can just let the least capable, that is all those GPUs that don't match the top one, sit idle ... won't that be fun ... Anyway, please make your objections known on the Alpha list or you are going to be stuck ... forewarned ... | |
ID: 12942 | Rating: 0 | rate: / Reply Quote | |
Anyway, please make your objections known on the Alpha list or you are going to be stuck ... forewarned ... I can't speak for an ATI config, but with nVidia cards it's not quite as simple as just enabling this functionality as anyone who ever tried to fold with a 200 series card and a previous gen card (in the same mb) will tell you. Some of the issues have been resolved in the latest series drivers, but I still don't think it is a case of enable by default. It's one thing having 2 cards with the same GPU but one has more memory on board, quite another when the shader count differs. This functionality should not be enabled (for Linux BOINC and nVidia GPU's at least) until the current stable driver release is >= 190! 185.xx and CUDA 2.2, I guarantee problems with non asymmetrical hardware on the same mb. At best, the fastest card will run as slow as the slowest card, at worse both will error out when you start a CUDA kernel on them. By all means vote, but know what you're voting for! ;) Sorry Paul, it always seems like I'm trying to rain on your parade ...... ____________ Crunching on Linux: Fedora 11 x86_64 / nVidia 185.18.36 driver / CUDA 2.2 | |
ID: 12943 | Rating: 0 | rate: / Reply Quote | |
Jack, | |
ID: 12945 | Rating: 0 | rate: / Reply Quote | |
Paul, | |
ID: 12949 | Rating: 0 | rate: / Reply Quote | |
Paul, And I am not asking for you to violate the NDA ... But when Rom indicates that the plan and the code they are putting in place is intended to support one ATI card and one Nvidia card and or some other combinations (not sure if they would support one ATI card with a pair of GTX260s or not) ... but if that is not supporting mismatches I don't know what is ... | |
ID: 12952 | Rating: 0 | rate: / Reply Quote | |
But when Rom indicates that the plan and the code they are putting in place is intended to support one ATI card and one Nvidia card and or some other combinations (not sure if they would support one ATI card with a pair of GTX260s or not) ... but if that is not supporting mismatches I don't know what is ... Where is this discussion taking place, the dev list? ____________ Crunching on Linux: Fedora 11 x86_64 / nVidia 185.18.36 driver / CUDA 2.2 | |
ID: 12953 | Rating: 0 | rate: / Reply Quote | |
But when Rom indicates that the plan and the code they are putting in place is intended to support one ATI card and one Nvidia card and or some other combinations (not sure if they would support one ATI card with a pair of GTX260s or not) ... but if that is not supporting mismatches I don't know what is ... boinc_alpha, thread "Memory, oh the memories" | |
ID: 12954 | Rating: 0 | rate: / Reply Quote | |
boinc_alpha, thread "Memory, oh the memories" Thanks. I had a quick squint at the dev list archive but I couldn't find anything relevant. ;) ____________ Crunching on Linux: Fedora 11 x86_64 / nVidia 185.18.36 driver / CUDA 2.2 | |
ID: 12956 | Rating: 0 | rate: / Reply Quote | |
Get your votes in early ... I usually agree with you, but not sure on this issue. You typically would not put two dissimilar CPUs in a dual-socket mobo and I think SLI wants to see two virtually identical GPUs. "They" have enough problems fixing BOINC for simple systems. | |
ID: 12959 | Rating: 0 | rate: / Reply Quote | |
I usually agree with you, but not sure on this issue. Well, Rom's explanation of the intention it is to support Nvida / ATI mixes and I cannot see how you can get more complex than that or more dissimilar in GPU architecture ... | |
ID: 12966 | Rating: 0 | rate: / Reply Quote | |
I saw that on the test msgs. | |
ID: 12967 | Rating: 0 | rate: / Reply Quote | |
I saw that on the test msgs. Well, there is this failure of imagination ... Knowing the history of GPUs over time especially when multiple GPUs came out was that people upgrade a little bit at a time. In the VERY early days GPU cards more often than not as an upgrade replaced a GPU on the MB that was just enough to draw dots on the screen. | |
ID: 12972 | Rating: 0 | rate: / Reply Quote | |
Hi Paul, | |
ID: 12975 | Rating: 0 | rate: / Reply Quote | |
It would be a nice to have, but as the other guys have said I think they need to get things working properly to start with, before we try and handle different cards from the same vendor. A dreamer ... we have a dreamer on aisle 5! :) My personal opinion is they should treat gpu tasks the same as cpu tasks and swap out as needed, honour TSI and so on. We haven't got to that point yet. Sadly, the history is that operational experience does not color design or implementation much ... and sometimes when it does it hatches new major issues like Strict FIFO breaks Resource Share with MW because of its restricted queue rule. My point is that they are saying, in effect, that they will be supporting the more difficult case first (getting the two different cards working correctly has been, historically, a far more difficult proposition - ATI and Nvidia have for years assumed that you want only their product and so make little allowance to run them together - that said, some have managed to get the cards to work together - I have not for what that is worth). But that is my point, if they think BOINC can support GPUs that are so different that they use different drivers, architectures, and programming models; then why is supporting cards from the same MFGR with minor differences so out of the question ... That is the joke, they are saying that the ATI / Nvidia mode is supported but not two cards from the same MFGR with a difference in memory size ... it is looney toons ... | |
ID: 12978 | Rating: 0 | rate: / Reply Quote | |
That is the joke, they are saying that the ATI / Nvidia mode is supported but not two cards from the same MFGR with a difference in memory size ... it is looney toons ... Paul, you do realize that the cuda_compare function already has a test that allows 30% memory size difference between otherwise identical cards? if (loose) { if (c1.prop.totalGlobalMem > 1.4*c2.prop.totalGlobalMem) return 1; if (c1.prop.totalGlobalMem < .7* c2.prop.totalGlobalMem) return -1; return 0; } If you don't like the (0.7/1.4) multipliers, change them and recompile! ;) Alternatively, why not just set the 'use_all_gpus' config option? ____________ Crunching on Linux: Fedora 11 x86_64 / nVidia 185.18.36 driver / CUDA 2.2 | |
ID: 12979 | Rating: 0 | rate: / Reply Quote | |
That is the joke, they are saying that the ATI / Nvidia mode is supported but not two cards from the same MFGR with a difference in memory size ... it is looney toons ... BOINC already automatically handles a 30% memory size difference if using the same type of card. Same card with more extreme memory differences are already supported with the <use_all_gpus> flag. Where you run into issues are where the cards are more different (eg shaders/chip). Lets explore this one shall we. To support same brand cards of different types we'd (roughly) need BOINC to: 1. Maintain array with card details (mem/shaders/compute capability etc) 2. Need to tweak scheduler to split work to the appropiate card 3. Tweak scheduler to maintain a benchmark/estimated speed so we can tell in relative terms how fast/slow it is (needed for work fetch/est run times) 4. Server side changes (for work requests, display of details, etc) 5. Some ability to exclude the card from boinc using it (optional) As you can see its more work than simply cloning the existing code for Nvidia cards to their ATI equivilivent, which is basically what has been done to get ATI cards added. Also consider that ATI has around a 50% market share (or whatever percentage it really is) to Nvidia in the graphics card arena. To include the other 50% makes sense as you are effectively doubling your capability as you now have both brands of gpu supported instead of just one. Adding support for mixed cards doesn't double the capability. Thats not to say it won't happen. I think it will given time. ____________ BOINC blog | |
ID: 12980 | Rating: 0 | rate: / Reply Quote | |
I am aware of the "Use All" flag and use it now. | |
ID: 12984 | Rating: 0 | rate: / Reply Quote | |
Tackling the within family problem first is also more logical because it is far more likely to be the normal situation. User buys card ... user buys faster card and links it to have better gaming ... user thinks that he can use this with BOINC ... user thinks UCB are idiots because thy did not think of this ... If I was looking at it from a cost/benefit point of view I think they made the right choice. I have a product (BOINC) that supports one brand of gpu (Nvidia). If I want to improve my reach I add support for the other brand (ATI) for a relatively small cost because I am basically cloning the code. It is also less risky because I already have code that while not great does the job. I have also reduced my dependancy on one brand remaining in business. If on the other hand I could sit down and tweak the code so it behaves better what is my cost/benefit? Next to nothing. I haven't improved my reach a great deal because I still don't support the other brand. Sure my product is better behaved and I can support more of the same brand, but there is less in that than getting the other brand supported. ____________ BOINC blog | |
ID: 12989 | Rating: 0 | rate: / Reply Quote | |
If on the other hand I could sit down and tweak the code so it behaves better what is my cost/benefit? Next to nothing. I haven't improved my reach a great deal because I still don't support the other brand. Sure my product is better behaved and I can support more of the same brand, but there is less in that than getting the other brand supported. Well, I can cite some cost benefit thoughts too ... It is far better to do it right the first time ... I am not arguing for the plan or how they implemented it to do one then the other ... though we made several suggestions about that too to make it smoother ... all ignored ... I am suggesting that the concept that BOINC can support one or more cards from one MFGR and one or more cards from another MFGR as long as each set is identical is no more difficult than supporting two cards from the same MFGR that may have differences. | |
ID: 12992 | Rating: 0 | rate: / Reply Quote | |
Well I just found this thread... I have been running a machine with 1 9800gtx+ and 1 8800 gt for a couple of months now and as of last week I can only get 1 wu at a time, for the 9800. The 8800 no longer runs. Bummer. Since I can't run it anymore I will retire it. I just can't afford to buy $1800 of cards at a time. My car is worth less than that...... | |
ID: 13000 | Rating: 0 | rate: / Reply Quote | |
Well I just found this thread... I have been running a machine with 1 9800gtx+ and 1 8800 gt for a couple of months now and as of last week I can only get 1 wu at a time, for the 9800. The 8800 no longer runs. Bummer. Since I can't run it anymore I will retire it. I just can't afford to buy $1800 of cards at a time. My car is worth less than that...... Had this been planned for you could still get work for the 9800 from here and for the 8800 from Collatz and the system would assign the tasks appropriately ... and that is something you still may want to try ... | |
ID: 13002 | Rating: 0 | rate: / Reply Quote | |
Searching through MW forums, about new ATI / AMD GPU 5870\5850, was found quite interesting message posted by Verstapp. | |
ID: 13029 | Rating: 0 | rate: / Reply Quote | |
Which is not the first time that ATI or Nvidia did something like that ... yet, Rom says that the UCB expectation is that mixed ATI and Nvidia installations are going to be easier to support ... | |
ID: 13037 | Rating: 0 | rate: / Reply Quote | |
An idea I think UCB should consider: Support a list of GPUs, each with a database section describing the capabilities of that list item. Allow including clusters of GPUs that meet their expectations for matched types within that cluster. Include a mark on each one saying whether it is free for use, already in use, or possibly the user has marked it as not to be used now. | |
ID: 13086 | Rating: 0 | rate: / Reply Quote | |
Message boards : Graphics cards (GPUs) : Asymmetric GPU installations