Author |
Message |
Erich56Send message
Joined: 1 Jan 15 Posts: 1132 Credit: 10,256,332,676 RAC: 29,209,412 Level
Scientific publications
|
I remember when way back there were these "short runs" ACEMD tasks, some people, including myself, made some settings in a config_app.xml with the effect that 2 such tasks were processed simultaneously with the same GPU.
Since the current Python tasks are not really that GPU-intensive, it might be interesting to also have 2 of them run simultaneously.
Is anyone here doing this already? Successfully? If yes, what would the entry in the app_config.xml look like?
Or are there reasons why this would not be a good idea? |
|
|
|
I ran 3x concurrently on my RTX 3060. was doing each task in about 13.5hrs (effectively 4.5hrs per task) for ones that run full-length. but now that I've added a second GPU, and 3 more tasks, the CPU can't keep up anymore. will update to a 16/32 part soon to see if it helps.
these tasks take about 3.6GB VRAM per task. so to run 2x you'll need 8GB VRAM.
also, each task will use about 7GB of system ram. so if you're running two, you need at minimum 16GB of system RAM.
doing multiples can help increase GPU utilization a bit, but you need to worry about overwhelming the CPU with so many concurrent processes (32x threads per task). and make sure you have enough GPU and system RAM to handle it. looks like your system has enough system ram at 32GB, but your 980Ti with 6GB VRAM might not be enough.
I did try running 2x on a GTX 1060 6GB, and the tasks just barely fit, but that was on linux and I'm not sure if it will work as well on Windows. you could try, just be sure to compare runtimes of the max length tasks to see if switching to 2x is worth it. if it's taking more than 2x as long to process tasks at 2x, then it's not worth it and slower than 1x.
____________
|
|
|
Erich56Send message
Joined: 1 Jan 15 Posts: 1132 Credit: 10,256,332,676 RAC: 29,209,412 Level
Scientific publications
|
thanks, Ian&Steve C., for the thorough explanation.
Well, concerning my system with the GTX980ti (6144MB VRAM) inside, I could try. Currently, GPU-Z shows a VRAM usage of some 2.888MB, and this value seems pretty much stable.
What I am more concerned about is the CPU usage. Currently, the Windows task manager shows a total of about 67% CPU usage for one Python task.
So, crunching 2 Pythons would most probably slow down the crunching process quite a bit (the CPU is 6-core/6-HT). I guess that total crunching time for 2 Pythons concurrently would be clearly more than double the time of 1 Python. But, again, one needs to try and see.
The situation would be much better on my other machine with 2 RTX3070 inside (and a 10-core / 10-HT CPU). However, there the problem is that I am running a 32GB Ramdisk (with additional 32GB system RAM). I wanted to try a Python on that system, but I could not even download any. BOINC told me that I need some 7GB more disk space (probably it took into account the 2 GPUs).
So I figured that the only way to circumvent this situation would be to install a second BOINC on the SSD - so far I didn't take my time to do this, according to the instructions I have seen this is not quite so simple; at least nothing which one could to within 5 or 10 minutes.
A new situation will come up when probably next week I will receive a second hand HP workstation with 2 Xeon processors inside (8 cores + 8-HT ea.) and a Quadro P5000 (16GB). However, I am aware that this GPU is not exactly the ideal or fast card for crunching; but I hope that it will still work satisfactorily.
But, to come to an end: I still don't know how I could get a GPU to process more than 1 task. Again I guess it needs specific entries in an app_config.xml. So please let me know :-)
|
|
|
gemini8 Send message
Joined: 3 Jul 16 Posts: 31 Credit: 2,212,787,676 RAC: 3,785,098 Level
Scientific publications
|
You need something like this:
<app_config>
<app>
<name>PythonGPU</name>
<max_concurrent>1</max_concurrent>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>2.0</cpu_usage>
</gpu_versions>
</app>
</app_config>
Put it into a file you call app_config.xml, put that into the GPUGrid project folder and re-read config files from the Boinc client, or just quit Boinc and start it again.
As I'm not at home atm I'm not able to have a look at the proper name for the Python GPU tasks. It's something along PythonGPU, but I'm not certain it's this exact spelling.
Boinc will tell you if it's not right, so you can change it afterwards.
I use the configuration with two CPU-threads per work-unit on one of my Ryzens which does other things on the rest of the CPU. Not sure if it would work as well with 0.45 or something like that, so the PythonGPU task would just take whatever CPU-cycles it can get from the sixteen threads the machine has altogether.
I run a MilkyWay task beside two GPUGrid Python tasks as the latter won't really satisfy the 1080.
____________
- - - - - - - - - -
Greetings, Jens |
|
|
|
You need something like this:
<app_config>
<app>
<name>PythonGPU</name>
<max_concurrent>1</max_concurrent>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>2.0</cpu_usage>
</gpu_versions>
</app>
</app_config>
Put it into a file you call app_config.xml, put that into the GPUGrid project folder and re-read config files from the Boinc client, or just quit Boinc and start it again.
As I'm not at home atm I'm not able to have a look at the proper name for the Python GPU tasks. It's something along PythonGPU, but I'm not certain it's this exact spelling.
Boinc will tell you if it's not right, so you can change it afterwards.
I use the configuration with two CPU-threads per work-unit on one of my Ryzens which does other things on the rest of the CPU. Not sure if it would work as well with 0.45 or something like that, so the PythonGPU task would just take whatever CPU-cycles it can get from the sixteen threads the machine has altogether.
I run a MilkyWay task beside two GPUGrid Python tasks as the latter won't really satisfy the 1080.
How do I run an Einstein@Home / Milkyway GPU task alongside one GPUGrid Python task? GPUGrid's app_config seems to decide only how many simultaneous GPUGrid tasks run.
With GPUGrid's Python tasks, I only get intermittent GPU usage, with constant 40ish% CPU usage - with a Core i7-12700K and RTX 3080 FE. GPU-Z screenshot attached.
I don't think I have enough VRAM to comfortably run 2 GPUGrid Python tasks - with a single task I get up to 4947MB VRAM usage, and the GPU only has 10240MB total. But Milkyway / Einstein tasks only use about 1GB, so I could run one GPUGrid task alongside one Einstein task; just don't know how to...
____________
|
|
|
Keith Myers Send message
Joined: 13 Dec 17 Posts: 1341 Credit: 7,678,756,915 RAC: 13,455,284 Level
Scientific publications
|
Set all your projects to 0.5 gpu_usage and then restrict GPUGrid to only one max_concurrent PythonGPU task.
That will only allow one Python task to occupy the gpu along with one of your Einstein or Milkyway tasks at the same time. |
|
|
gemini8 Send message
Joined: 3 Jul 16 Posts: 31 Credit: 2,212,787,676 RAC: 3,785,098 Level
Scientific publications
|
You'll need an app_config.xml in your respective Milkyway or Einstein directories.
<app_config>
<app>
<name>Milkyway</name>
<max_concurrent>1</max_concurrent>
<gpu_versions>
<gpu_usage>0.5</gpu_usage>
<cpu_usage>.04</cpu_usage>
</gpu_versions>
</app>
</app_config>
For Einstein there's a lot of application names.
Use this app_config.xml with 'Milkyway' in the Einstein directory as well, and Einstein will tell you it's wrong and its actual appliction names.
Einstein uses 1.0 for cpu_usage if on Nvidia or AMD GPUs.
____________
- - - - - - - - - -
Greetings, Jens |
|
|
AsghanSend message
Joined: 30 Oct 19 Posts: 6 Credit: 405,900 RAC: 0 Level
Scientific publications
|
The biggest lie for the new python tasks is "0.9 CPU".
My current task, and the one before, were/is using 20 out of my 24 cores on my 5900X... |
|
|
VesterSend message
Joined: 22 Nov 16 Posts: 3 Credit: 455,000 RAC: 0 Level
Scientific publications
|
I errored out on three work units (27347453, 27347587, and 27347506) while trying to run more than one GPU task on my single GPU. I don't know if they failed because of my app_config.xml file or not, but I did not error out on a single GPU task, 27346636. I was also running Rosetta@home concurrently but have suspended R@h while running GPUGrid.
This is my computer. |
|
|
Keith Myers Send message
Joined: 13 Dec 17 Posts: 1341 Credit: 7,678,756,915 RAC: 13,455,284 Level
Scientific publications
|
The tasks failed because you haven't properly set a large enough paging file for your host in Windows.
This is your error:
[WinError 1455] The paging file is too small for this operation to complete.
There is nothing wrong with your gpu which is more than adequate to run 2X tasks concurrent.
But each task needs about 35GB of virtual memory in the paging file.
See the main thread about the Python tasks.
https://www.gpugrid.net/forum_thread.php?id=5233
If you increase your paging file size, you should have no issues successfully completing the work units. |
|
|
VesterSend message
Joined: 22 Nov 16 Posts: 3 Credit: 455,000 RAC: 0 Level
Scientific publications
|
If you increase your paging file size, you should have no issues successfully completing the work units.
Thank you, Keith, for your patience and help. I should have looked at the stderr output. My NMVe C: drive is only 250GB, so I put a fixed 228000MB paging file in another SSD. (Now I need more RAM to run three or more GPU tasks simultaneously.) |
|
|
Erich56Send message
Joined: 1 Jan 15 Posts: 1132 Credit: 10,256,332,676 RAC: 29,209,412 Level
Scientific publications
|
Now I need more RAM to run three or more GPU tasks simultaneously.
how powerful is your CPU ? |
|
|
VesterSend message
Joined: 22 Nov 16 Posts: 3 Credit: 455,000 RAC: 0 Level
Scientific publications
|
how powerful is your CPU ?
Intel i9-10850K @4.9GHz, ten cores and hyperthreaded. It could be a limitation because it is utilized at 100% when running two tasks on the Nvidia Titan X (single GPU). |
|
|
Erich56Send message
Joined: 1 Jan 15 Posts: 1132 Credit: 10,256,332,676 RAC: 29,209,412 Level
Scientific publications
|
how powerful is your CPU ?
Intel i9-10850K @4.9GHz, ten cores and hyperthreaded. It could be a limitation because it is utilized at 100% when running two tasks on the Nvidia Titan X (single GPU).
well, the CPU is definitely powerful enough to run 2 Pythons concurrently.
And basically more than 2 Pythons cannot be downloaded anyway, unless one "tunes" the "coproc-Info.xml" in the BOINC folder. |
|
|