I built a box and put 1 EVGA GeForceGTX 780 in it. It worked fine with GPUGRID for 2 months. I just put 2 more in there; one EVGA and one PNY (yes completely compatible and working in the system.) Now that they are in, the first one is working exactly as it was at 90-92%. The PNY is working also at 90%. The BOINC Manager shows 2 tasks being run now. The third card, which is showing as properly installed, works with a monitor as the main monitor, and also shows up with NVidiaInspector (like the other 2) is not working any tasks and is sitting at 0%. This third one will, on occasion, go up to 9% or somewhere lower than that, as I switch windows, but I cannot get it to do a task with GPUGRID. I don't see any settings for the project or the BOINC Manager itself, so I need so advice or help on how to get this third graphics card working on its own task like the other 2 are. TY

Hi, possibly modifying - cc_config.xml - to use all GPUs BOINC system:

<cc_config>
<options>
<report_results_immediately>1</report_results_immediately>
<use_all_gpus>1</ use_all_gpus>
</options>
</cc_config>

With this configuration task forces report without waiting and force BOINC to use all GPUs present.

I hope will be useful. Greetings.

OK, when I do a search for that file, I can't find it. I found reference to it in stdoutgpudetect.txt where it keeps repeating the message:

20-Nov-2014 05:10:51 [---] cc_config.xml not found - using defaults

Should I make a new file with that name and the text you gave, should I reinstall, or is there a way to force the program to create a new copy of it?

I went ahead and made that file and started BOINC again. It looks like it has been accepted so I will wait out a full cycle of tasks to see if the other GPU kicks in.

I went ahead and made that file and started BOINC again. It looks like it has been accepted so I will wait out a full cycle of tasks to see if the other GPU kicks in.

It looks like this thread needs a link on how to setup cc_config.xml.
Here:
http://boinc.berkeley.edu/wiki/Client_configuration

Carlesa, thank you again.

Jacob, thank you very much for that page (and linked pages from that page)!

Sorry it took a while. After a short while of working, that PC stopped working. After putting in the second and third GPU and adding another 4Tb HDD I exceeded the PSU and killed it. Now that I have a new PSU in it I have modified the cc_config.xml a little more. It is still only running the 2 GPUs and the third is doing nothing without help.

By "help" I mean that I have cheated the instructions that BOINC is running under and manually added a slot "2" to the Slots folder of BOINC in the ProgramData folder. Then I copy the oldest of the 2 slots contents (minus the lockfile) to the new slot folder. After that, I open cmd, cd to the slot 2 folder, then do the command:

I actually do see a few corrupted (errored out) tasks in my online task logs, so it looks like cheating does have its consequences. I need to find a solution that actually downloads and works 3 tasks as one per GPU the way the program is supposed to and not 'rigged to blow'.

In addition to any help that can be offered on these forums, is anyone willing to help people on here actually directly check my installation, files, settings, etc via something like TeamViewer? Direct help can save a lot of frustration for me trying to make it work and for those helping by just troubleshooting and doing the fix instead of the back and forth. TYYTYTYTYTYVM in advance for any help or suggestions that are given.

I am using nvidiaInspector's Overclocking options to do nothing but up the fan speed, but what would make you think they are too hot? Does heated GPUs cause the BOINC Manager to only load 2 slots and use 2 devices when 3 are noticed by the OS, nvidiaInspector, and can manually be loaded via command line? I wouldn't think heat is the reason it only loads the top 2 (device 0 and device 1) even if there are 3 slots with units in them, which rarely happens unless I "Suspend" one unit of work and it loads a new third one to run on the one I turned off. But if I do "Resume" any '3rd' unit after 2 are already running, it will sit in "Waiting" mode until one of the other 2 finishes and then it will turn back on and run.

I even tried changing the third "init_data.xml" while the manager was off to

I mean I honestly bought 2 extra $400 GPUs to run THIS project and it is frustrating that one refuses to be used. I don't even game!

Which BTW, I have switched positions of the GPU cards and no matter which configuration, the top 2 are used by GPUGRID and the bottom one sits idle. So device 0 and device 1 run the project and device 2 will not.

I mean I honestly bought 2 extra $400 GPUs to run THIS project and it is frustrating that one refuses to be used. I don't even game!

Which BTW, I have switched positions of the GPU cards and no matter which configuration, the top 2 are used by GPUGRID and the bottom one sits idle. So device 0 and device 1 run the project and device 2 will not.

The log does see all three GPUs listed one 3 different lines and numbers them 0, 1, and 2.
I did change <use_all_gpus> to a value of 1.
I have 1 CPU allocated at the value of 1%. The rest of my CPU cores and usage I have allocated to distributed.net and have for years. Does the amount of cores or the % of cores from the CPU affect the project's usage of GPUs to where it would only allow 2 GPUs to run tasks?

As far as the specs, I am running an Intel Core i7 4960X CPU @ 3.60GHz OCd to 4124.9 MHz (33.0 x 125.0 MHz), 64GB Quad Channel DDR3 RAM @ 833.4 MHz, on an ASUSTeK X79-DELUXE Rev 1.xx, with American Megatrends Inc. BIOS 0701 - 01/07/2014 ROM size 8192 KB.

Hope this helps you help me. (Again, I have TeamViewer running if any kind soul would like to pop in and take a look, I would be happy to allow that.)

The log does see all three GPUs listed one 3 different lines and numbers them 0, 1, and 2.
I did change <use_all_gpus> to a value of 1.
I have 1 CPU allocated at the value of 1%. The rest of my CPU cores and usage I have allocated to distributed.net and have for years. Does the amount of cores or the % of cores from the CPU affect the project's usage of GPUs to where it would only allow 2 GPUs to run tasks?

As far as the specs, I am running an Intel Core i7 4960X CPU @ 3.60GHz OCd to 4124.9 MHz (33.0 x 125.0 MHz), 64GB Quad Channel DDR3 RAM @ 833.4 MHz, on an ASUSTeK X79-DELUXE Rev 1.xx, with American Megatrends Inc. BIOS 0701 - 01/07/2014 ROM size 8192 KB.

Hope this helps you help me. (Again, I have TeamViewer running if any kind soul would like to pop in and take a look, I would be happy to allow that.)

OK, quick update.

After killing the CPU project, still no love for the GPUGRID project getting more units than 2 to work on at one time.

I never had any graphics cards and only ever used on-board graphics before building this rig. First time I ever had the money to make rather than buy mass produced cheap rigs.

I have not tried using a dummy plug, but thanks for asking. Thinking about that question, I noticed that the one that won't get a task is the one that I have the monitor actually plugged in to. That may or may not make a difference since I rebooted with the monitor unplugged from the PC completely and it still won't load a third task.

I have been watching the Event Log as I try to Update for new tasks and start and stop the BOINC Manager and I noticed that even when I Update for a new task, it reads

There is a lot of confusion in this thread.

I run 3 GPUs, and have no work fetch issues with them while running BOINC 7.4.36.

Manipulating slots directories, or running executables directly, or editing projects/slots .xml files, is wrong wrong wrong. Don't do it.

I see you have BOINC showing 3 GPUs in the Event Log when BOINC starts up. That's good. So, it is finding them. Is the concern that you have downloaded tasks that won't run? Or is the concern that it won't even download 3 tasks?

Assuming it is a work fetch concern... Okay, edit your cc_config.xml and turn on <work_fetch_debug>, then restart BOINC, then show us what a work fetch iteration looks like.

Also, please note that, at the moment, GPUGrid is on fumes, and may not be able to provide GPU work for every request. If you are attached to other projects that have GPU apps, you should be able to get GPU work from them.

The best way for us to help you diagnose a work fetch behavior, is to turn on work_fetch_debug and give us some output to look at. I'm an expert at looking at work fetch output. If you'd like to try to translate it yourself, feel free to have a look at this post:
http://www.bitcoinutopia.net/bitcoinutopia/forum_thread.php?id=691&postid=7369

Once you give us some debug output, we can try to help you further.

PS: If you are still up for doing a TeamViewer session, I would be willing to connect and take a peek. I'm an excellent troubleshooter, usually. Send me a Private Message with details.

Regards,
Jacob

{{{Warning, long answer on its way.}}}

Thank you very much for your reply Jacob. I think I caused some of the confusion by being ignorant of what information to provide, some because of my ignorance of how BOINC works compared to other distributed projects I have worked on in the past, and some because of my fondness and willingness to troubleshoot and tinker to fix things in order to understand them rather than understand them in order to troubleshoot and tinker. Some of the confusion was also caused by replies to my issues with good answers that I just didn't understand or did not apply.

I think I understand your information better than much of the answers and help that I have gotten so far. Deciphering (not writing) code and reading the manual are two of my strong points because of my previous job working with the coders and doing customer support for a project I learned first for the company after our company acquired a different company that relied more on code than on Windows front end programs. I was pretty much in charge of helping rewrite the manual while simultaneously going through the manual with the product in hand making sure what the manual said is what the product did. Then I was tasked with teaching much of the rest of the staff on the product line that the company eventually made its main line for years. After that, I did customer service and troubleshooting along with working back and forth between the coders and the CS dept on bugs, new issues, upgrades, and old versions. So reading your information about logs and reading the linked 'man' pages on scheduling, I know much of why I confused people in my requests and why they were confused with my answers.

So I turned the BOINC client. Then I turned on the <work_fetch_debug> and thought that while I was at it, I would turn on the <sched_op_debug> to see the output of both. Here is the result:

So, it is finding them. Is the concern that you have downloaded tasks that won't run? Or is the concern that it won't even download 3 tasks?

Ok, nice answer :) I see you have a technical background, that's good. I'll give you answers that are hopefully at the right "level" of making sense for you. PS: Because of the GPUGrid work shortage, I won't be able to conclusively tell you what the problem is. But I am willing to troubleshoot this as long as it takes to help you solve it.

Here goes.

BOINC is primarily meant to set it and forget it. Configure your settings, attach to some projects, and let it do it's thing. The Advanced view has tabs for Projects and Tasks, and you should familiarize yourself with the buttons there. Especially the Tasks grid. When a GPU task is running, the task's status will include something like "Running (0.667 CPUs + 1 NVIDIA GPU (device 0))", telling you how much CPU it is budgeting as "used" for the GPU task, and which GPU it is running on.

I'd also recommend attaching to more projects than just GPUGrid, and setting up your "Use at most X% of the processors" computing preference to be equal to the number of threads you'd like to let BOINC manage.

Looking at your log posting, I see...

Starting BOINC client version 7.4.27
... If you'd like to upgrade to the latest release candidates (I do recommend 7.4.36), feel free to bookmark:
http://boinc.berkeley.edu/download_all.php

12/21/2014 8:35:17 PM | | CUDA: NVIDIA GPU 0: GeForce GTX 780 (driver version 344.75, CUDA version 6.5, compute capability 3.5, 3072MB, 2665MB available, 4878 GFLOPS peak)
12/21/2014 8:35:17 PM | | CUDA: NVIDIA GPU 1: GeForce GTX 780 (driver version 344.75, CUDA version 6.5, compute capability 3.5, 3072MB, 2809MB available, 4698 GFLOPS peak)
12/21/2014 8:35:17 PM | | CUDA: NVIDIA GPU 2: GeForce GTX 780 (driver version 344.75, CUDA version 6.5, compute capability 3.5, 3072MB, 2809MB available, 4576 GFLOPS peak)
12/21/2014 8:35:17 PM | | OpenCL: NVIDIA GPU 0: GeForce GTX 780 (driver version 344.75, device version OpenCL 1.1 CUDA, 3072MB, 2665MB available, 4878 GFLOPS peak)
12/21/2014 8:35:17 PM | | OpenCL: NVIDIA GPU 1: GeForce GTX 780 (driver version 344.75, device version OpenCL 1.1 CUDA, 3072MB, 2809MB available, 4698 GFLOPS peak)
12/21/2014 8:35:17 PM | | OpenCL: NVIDIA GPU 2: GeForce GTX 780 (driver version 344.75, device version OpenCL 1.1 CUDA, 3072MB, 2809MB available, 4576 GFLOPS peak)
... perfectly seeing your 3 GPUs (note: driver 347.09 beta is available now, and appears to work fine in BOINC)

Damn nice machine! 12-threads, 64GB RAM, 3 GTX 780 GPUs -- #Jealous

12/21/2014 8:35:18 PM | | [work_fetch] ------- start work fetch state -------
12/21/2014 8:35:18 PM | | [work_fetch] target work buffer: 180.00 + 432000.00 sec
... I see your buffer settings are maintain at least 0 days (BOINC enforces a 3-minute low-water-mark to allow some time to ask projects for work, which is why you see 180.00 there)
... and allow an additional 432000secs = 5 days cache

12/21/2014 8:35:18 PM | | [work_fetch] --- project states ---
12/21/2014 8:35:18 PM | GPUGRID | [work_fetch] REC 355175.655 prio -1.000 can request work
... "can request work" is good. BOINC would say "can't request work" and give a reason, if a reason applied. Some reasons are: Project set to No New Tasks, Project set to Suspended, or one of the Project's tasks is suspended (that's right, it will not request more work from a Project, if you have a suspended task for that project).

12/21/2014 8:35:18 PM | | [work_fetch] --- state for CPU ---
12/21/2014 8:35:18 PM | | [work_fetch] shortfall 5186160.00 nidle 12.00 saturated 0.00 busy 0.00
... 12 CPUs, times 432180 high-water-mark-per-resource, equals 5186160.00 instance-seconds of shortfall. nidle (number idle) shows 12 idle CPUs.

12/21/2014 8:35:18 PM | | [work_fetch] --- state for NVIDIA GPU ---
12/21/2014 8:35:18 PM | | [work_fetch] shortfall 1296540.00 nidle 3.00 saturated 0.00 busy 0.00
... 3 GPUs, times 432180 high-water-mark-per-resource, equals 1296540.00 instance-seconds of shortfall. nidle (number idle) shows 3 idle NVIDIA GPUs.

12/21/2014 8:35:18 PM | | [work_fetch] ------- end work fetch state -------
... time for work fetch to decide if it should ask a project for work

12/21/2014 8:35:18 PM | GPUGRID | [sched_op] Starting scheduler request
12/21/2014 8:35:18 PM | GPUGRID | [work_fetch] request: CPU (1.00 sec, 12.00 inst) NVIDIA GPU (1.00 sec, 3.00 inst)
12/21/2014 8:35:18 PM | GPUGRID | Sending scheduler request: To fetch work.
12/21/2014 8:35:18 PM | GPUGRID | Requesting new tasks for CPU and NVIDIA GPU
... it has decided to ask GPUGRID for 12 CPU instances, and 3 NVIDIA GPU instances. I'm a bit curious why the sec values are only 1.00. I would have expected the full shortfalls on each. But this is okay for now.

12/21/2014 8:35:18 PM | GPUGRID | [sched_op] CPU work request: 1.00 seconds; 12.00 devices
12/21/2014 8:35:18 PM | GPUGRID | [sched_op] NVIDIA GPU work request: 1.00 seconds; 3.00 devices
... The "scheduler" (a BOINC Project server-side-process) received the request successfully.

12/21/2014 8:35:20 PM | GPUGRID | Scheduler request completed: got 0 new tasks
12/21/2014 8:35:20 PM | GPUGRID | [sched_op] Server version 613
12/21/2014 8:35:20 PM | GPUGRID | No tasks sent
12/21/2014 8:35:20 PM | GPUGRID | No tasks are available for Short runs (2-3 hours on fastest card)
12/21/2014 8:35:20 PM | GPUGRID | No tasks are available for ACEMD beta version
12/21/2014 8:35:20 PM | GPUGRID | No tasks are available for Long runs (8-12 hours on fastest card)
12/21/2014 8:35:20 PM | GPUGRID | No tasks are available for the applications you have selected.
12/21/2014 8:35:20 PM | GPUGRID | Project requested delay of 31 seconds
... and has replied that it has no tasks for either of the resources, per your selections. NOTE: GPUGrid actually DOES have CPU tasks now, with their "Test application for CPU MD" multi-threaded application. You could edit your web preferences, to turn on that app (might also have to check the "Run test applications?" checkbox too), if you'd like to run CPU tasks from GPUGrid. I'm sure they'd appreciate your CPU support.

12/21/2014 8:35:20 PM | GPUGRID | [work_fetch] backing off CPU 580 sec
12/21/2014 8:35:20 PM | GPUGRID | [work_fetch] backing off NVIDIA GPU 312 sec
12/21/2014 8:35:20 PM | GPUGRID | [sched_op] Deferring communication for 00:00:31
12/21/2014 8:35:20 PM | GPUGRID | [sched_op] Reason: requested by project
12/21/2014 8:35:20 PM | | [work_fetch] Request work fetch: RPC complete
... Because you needed work for CPU and didn't get any from this Project, BOINC enforces a semi-random "resource backoff timer" for that Project. 580 secs, in this case.
... Because you needed work for NVIDIA GPU and didn't get any, it also enforced a backoff of 312 secs for that resource.
... And the server said "don't come back here for at least 31 seconds please" :)

Sorry, but it is a bad time to be troubleshooting this with GPUGrid, since they are "on fumes" in terms of having GPU tasks available. See the Server Status "unsent" column here: http://www.gpugrid.net/server_status.php

Once you get 3 GPUGrid tasks in the Task grid, I'd like to see a screenshot of the behavior you saw. Are you sure you saw 3 GPUGrid GPU tasks listed in the Tasks grid, and BOINC was only running 2 of them? I don't know how to troubleshoot further without seeing the issue.

One other thing to note is that, I believe GPUGrid has a server-side rule that only allows "2 GPU tasks per GPU" to be on a client. For you, that means you should only ever see up-to-6 GPUGrid GPU tasks in your Task grid, regardless of your buffer settings.

Once you get 3-or-more tasks, host a screenshot and let us take a look. :) Then we might have to turn on more of those awesome debug flags to get geeky!

Oh, you asked about whether tasks get sent out multiple times. BOINC Projects set this up on a per-application basis; they can decide how many "instances" of a work unit are initially replicated, and can also set how many results must be in agreement before considering it completed, and how many error results should trigger abortion of the work unit. If you look at some tasks (here are mine: http://www.gpugrid.net/results.php?hostid=153764)... if you click the "Work Unit", you'll see that unit's values for "minimum quorum" (# results that must agree), "initial replication" (# tasks initially created/sent), and "max # of error/total/success tasks". GPUGrid does not do verification on their GPU apps, but most other project choose to verify their results with at least 2 successful returns.

I too used to run Distributed.net back in the day, but I go for the science research now. You might consider attaching BOINC to more projects, like World Community Grid, Citizen Science Grid, etc.

Merry Christmas,
Jacob

PS: Here are my list of running projects (I give WCG 4x the Resource Share as my others, and I have a couple 0-Resource-Share projects, which are "Backup Projects" to BOINC, meaning it will only get work from those if there is no other work from other projects)... and here's a what my Tasks grid looks like. I use an app_config.xml to tell BOINC to "consider 0.667 CPU budgeted per GPUGrid Task", and then I run with "Use at most 100% CPUs." That way, if BOINC runs 3 GPUGrid tasks, it automatically budgets ("frees up") 0.667*3=2.001 CPUs. Fun.

http://1drv.ms/13tRIpR
http://1drv.ms/1zTv9I5

I agree with skgiven -- Your GPUs are getting too hot. For a GTX 780, althought it is rated to support up-to-100*C, that is it's "absolute maximum operating temperature" (TjMax). It actually starts thermal downclocking at 80*C, since GPU Boost v2.0 GPUs use an 80*C threshold. And 80*C is generally about the "maximum comfort zone" for a GPU that you want to take care of.

I'd investigate your cooling, and then consider using a program like Precision X or Afterburner to enforce a custom fan curve that keeps it below 80*C.

Well, I have good news. If the current gold version of the BOINC client does have a bug to not use all 3 GPUs, the .36 version does not. I updated the driver and the BOINC and now it is crunching n 3 GPUs with 3 tasks!

Funny that this whole conversation could have been solved (but I would not have learned as much) if someone just gave me the link to 7.4.36 and said, "Here, try this." lol

I will check on that heat thing.

Thank you thank you thank you so much for all your help. (I will not close the door on this until I see this repeated, but I will operate as if it is.)

Both your GPUs 0 & 1 are hitting 95-96C I wouldn't take too long to try to solve the temp problem.

OK done, using Afterburner.

This seems like a new topic, but it also seems like something those already responding here can answer for me. Sorry if this is answered all over the forums and for my own lack of investigation. I could not find anything with a few searches of the terms I was looking for.

I don't quite understand how the GPU works compared to a CPU with shaders and all that goes with them. With that in mind, I found that my laptop has a mobile version of an NVidia card and was able to run GPUGrid on it. After it started running, I noticed that it is running at 78 degrees, which is fine, but it is running at 97% GPU load also. This struck me as odd since the GPU loads on the 780s in the big rig run at loads in the mid to high 70s. Since my last post, I found a new cooling solution for that rig (it being super cold out now) with a window open and a room fan aimed at it. And since that is keeping the temps in check without limiting with Afterburner and knowing overclocking a GPU will corrupt GPUGrid results, I want to know how to push the GPUs to use closer to 100% load. I do have it running cooler and can increase the cooling to make it run even cooler yet, if need be.

1) The same GPU load is reported by Afterburner, OpenHardwareMonitor, and NVidiaInspector, but is that reported load correct according to actual load or is it just an semi-accurate approximate?
2) Is the reported load of 7x% actually close to 100% load but with things I don't understand (like shaders, VPU, MCU, Bus speed, Memory speed, "Boost", etc.), it is showing me a number that looks much lower?
3) Is the GPU load a task specific item? I only ask this because I do see some tasks in the 60s and some in the 80s.
4) If it really is running in the 70s and 80s, how can I get it to run the full GPU load?

If you would like to answer simple answers, I can take that or if you can give or point me to definitive detailed answers, I would appreciate those.

Thanks again,
Mike
____________
1 Corinthians 9:16 "For though I preach the gospel, I have nothing to glory of: for necessity is laid upon me; yea, woe is unto me, if I preach not the gospel!"
Ephesians 6:18-20, please ;-)
http://tbc-pa.org

In order for me to get better GPU utilization added a file in C:\ProgramData\BOINC\projects\www.gpugrid.net called app_config.xml with. I have 3 GPU's in the machine. This lets you run two WU's per GPU with each getting half. My GPU utilization runs around 98% on each GPU.


<?xml version="1.0"?>

-<app_config>


-<app>

<name>acemdlong</name>

<max_concurrent>6</max_concurrent>


-<gpu_versions>

<gpu_usage>.5</gpu_usage>

<cpu_usage>1</cpu_usage>

</gpu_versions>

</app>

</app_config>

I want to know how to push the GPUs to use closer to 100% load. I do have it running cooler and can increase the cooling to make it run even cooler yet, if need be.

Thanks again,
Mike

I found a way to get over 90% of the GPUs working. Since Dnetc is not a BOINC project, I set the Dnetc GPU priority to 2 and it took up the slack from GPUGrid. Unfortunately there is no happy medium and it is slowing down the GPUGrid project, but my BOINC stats went from 1,500,000 to 850,000 a day round abouts, but my Dnetc stats went from 7,000 a day to over 50,000. I wish I could get some happy place where the DNetc only took the idle that GPUGrid doesn't touch, but any other priority either takes all the GPU or almost none of it. Cancers and other diseases are my moral priority, but getting the GPU as close to 100% is why I spent the money on the cards making it the financial priority for the moment. And the extra stats on the other side where I've spent almost 10 years doesn't hurt either. Maybe if I can think about it one day, I can figure out a way to make BOINC and Dnetc play nicer with each other.
____________
1 Corinthians 9:16 "For though I preach the gospel, I have nothing to glory of: for necessity is laid upon me; yea, woe is unto me, if I preach not the gospel!"
Ephesians 6:18-20, please ;-)
http://tbc-pa.org

OK, so I see that the distributed.net GPU client can set its GPU runtime priority. I can't find any information out there on how to set the GPU priority in Windows like you can with the CPU. Does BOINC or the GPUGrid have any settings that can change the GPU runtime priority of it? Since I find that I need to run the DNetc at level 2, running BOINC GPUGrid at level 3 would then allow for BOINC to use as much of the GPU as it can force, then the DNetc will take the remaining percentage. If BOINC/GPUGrid does not have any way of changing or forcing this, does anyone know how to change it per task at the Windows OS level.

When I want to change the CPU runtime, I use Process Explorer. It is like Task Manager on steroids. It allows manipulation at program level, OS level, and CPU level and gives usage graphs and statistics. Now this also comes with "System Information" that gives graphs of your usages. Drilling into them, you can view your GPU engines. But even this program that can change all things program and CPU/OS level, it seems to not be able to manipulate the GPU in the same way (yet maybe.) Anyone? Anything? Even speculation if there is no clear answer.
____________
1 Corinthians 9:16 "For though I preach the gospel, I have nothing to glory of: for necessity is laid upon me; yea, woe is unto me, if I preach not the gospel!"
Ephesians 6:18-20, please ;-)
http://tbc-pa.org