Tim pointed out on IRC that RGS that are built too close to each other will not only work less efficient but will completely stop generation or even decrease BP. I think what we should be heading for is an approximation of this (in 3D space, ofc):

EDIT:
After going through the code and confirming that negative generation rates can't happen we found out that the decrease in BP might be related to human bots repairing buildings. Still the proposed approximation might be a good solution. :D
EDIT2:
On request a more formal description of my idea and the algorithm (which is just a rough approximation of it):
The ideal idea
I preclude that RGS have an area of effect (AOE) described by a sphere in 3D space. If a is an AOE of a RGS, |a| is the volume of a. The total mine rate should be proportional to the union of all AOE volumes, so that if n RGS don't intersect they would mine at a rate of n * base_rate but if they are in the very same spot they would mine at a total rate of 1 * base_rate. Now the idea is that if some of the RGS intersect, the total mine rate should be n * base_rate * (|A_union| / |A_max|), where A_union is the union of all a1,...,an and A_max is the union of all a1,...,an if they were rearranged to not intersect. This is what I try to illustrate with the image above: if the red, green and blue spheres are the AOE of three RGS, the total mine rate would be proportional to the part of the image that is not black. The red RGS would mine at full speed in the red area, at half speed in the yellow and purple areas and at a third of the normal rate in the white area. This asures that building another RGS in the range of a number of existing (friendly) RGS will never be worse than not building it.
The approximation
The exact implementation would be very computationally intense, so here's a rough approximation that should be easy to implement and light on performace. It is not accurate and may lead to a negative gain if a high number of RGS is built too close to each other (3 RGS in the same spot would mine at a total of 3/4 of the mine rate of a single RGS, 4 at 4/8, 5 at 5/16, etc). Maybe the algortihm can be improved to yield more accurate results when a high number of closeby RGS is involved.
For each pair of RGS r1,r2 whose distance is smaller than two times the RGS radius
{
Let r1.rate and r2.rate be the respective current mine rates of r1 and r2.
Calculate the (approximate) quota q in ]0,1[ of the intersection area of the two AOEs.
Set r1.rate to r1.rate * (1.0 * (1.0 - q) + 0.5 * q).
Set r2.rate to r2.rate * (1.0 * (1.0 - q) + 0.5 * q).
}
EDIT3:
I calculated the exact value of q which is a third grade polynom and doesn't need to be approximated ( q(r,d) = ((d/r)3 - 12*(d/r) + 16) / 16, r is the RGS radius and d is the distance of the two RGS). The patch is upstream and will be online on the development server soon.