Two 24-tooth gear wheels, (left in the diagram) are designed to mesh if their axles are three holes apart. The teeth are only there to prevent any slipping when one wheel drives the other. If we had a material that does not allow slipping, we could replace the gears with two smooth discs (the pale yellow circles) and those discs would touch in a point exactly halfway between axle holes.
The yellow discs would roll against each other, they would have identical diameters and those diameters would be exactly 24mm because that is 3 times the distance between axle holes.
The same reasoning applied to 8-tooth gears gives smooth discs of exactly 8mm diameter.
Because the discs touch on the vertical tangential lines that in the diagram lie exactly between axle holes, a disc representing a 24-tooth wheel will also roll on one representing an 8-tooth gear wheel, as indeed a 24-tooth gear meshes with an 8-tooth one.
Both 8-tooth and 24-tooth wheels also engage with 40-tooth gears, and therefore the disc replacing a 40-tooth gear has a diameter of exactly 40mm.
Definition
The rolling diameter of a gear is the diameter of an idealised disc that allows it to roll on the equivalent idealised disc of another gear that is normally meshing with it.
Observations
It turns out that the rolling diameter in mm of any Lego gear is the same as its number of teeth.
Gears with a number of teeth that is an odd multiple of 8 (8, 24, 40) will mesh when placed on axles that are an odd number of holes apart.
However, gears with even multiples (e.g. 16) will need half a space when meshing with odd multiple ones.
Rack and Pinion
Given that the circumference of a rolling disc is πd where d is the diameter and d is the same as the number of teeth, a tooth takes up πd/d=πmm along the circumference.
Lego gear teeth are 3.14159…mm long.
For a wheel of infinite diameter, the width of a tooth should still be 3.14159…mm, and the teeth then form a rack. But a Lego rack has 10 teeth over a span of 32mm (four studs of 8mm) and their length is therefore 32/10=3.2mm. The difference is 3.20 - 3.14159 or less than 0.06 in 3.2, less than 2%.
It is possible that the design of the gears was conditioned by having to find a whole number of teeth for a rack of 32mm.
But note that in any case the actual dimensions of gear and rack are not important as things can be scaled; only the ratio of the rolling circumference of the (smallest) gear (8mm×π) to the length of the rack brick (32mm) is important, and this ratio is 0.785398… or conversely 1.27324…. If the gear has n teeth then the rack has to have m=n×1.27324… teeth. That is not acceptable: if we "peel off" the teeth from the wheel and roll them out along the rack, there will not be a whole number of teeth along the length of the rack brick.
But it would not be so bad if the number came close enough to a whole number: then we use that whole number as the number of teeth on the rack, and given that there is a little play anyway, the wheel will roll over it well enough.
Let's make a graph of the percent difference between m and its nearest integer, as a function of n. We note that m is a transcendental number, π being involved, and that the integer nearest to m can be less than or greater than m. After a bit of spreadsheet work we get:
percent difference between m and its nearest integer [SVG]
Obviously, for a wheel with only one tooth the situation would be rather pathetic, but we also expect that as the number of teeth becomes very large it matters less and less and the error between the teeth of the wheel, rolled out over the length of the rack, and the rack with a whole number of teeth becomes insignificant.
What we want is a small value of n, which has a good m. N should have some properties such as being even, possibly a multiple of 3 as well. Looking at the graph we see that the first value where the difference is really small is 11, but that is not a good number for use in toy gear wheels. Anything else is too large. The next smaller number is 7, also a prime number and not good for toys. Eight has a difference of 1.86…%, not bad. It is not a multiple of 3, but it is even and a multiple of 4. That is the choice made by Lego, though perhaps as a result of a very different reasoning.
Note that 12 has the same percent difference, and although it is a multiple of 3, it is quite a lot larger than 8: the teeth would be much smaller, perhaps too thin (see discussion).
The rolling disc of a rack has infinite diameter, and may be represented by a line that lies somewhat above the bottom of the rack's teeth and should touch the rolling disc of the wheel that rolls over the rack.
I could not find a simple construction in which a gear meshes with a rack, whereby the gear's axle is in a Technics bearing and the rack at some distance that is a multiple of plate thickness (3.2mm) except for the gear of 8 teeth:
Rack and Pinion
Since the rolling diameter of the pinion is 8mm, its radius is 4mm, and as the axle hole is 5.8mm above the bottom of the Technics bearing, the "rolling circle" of the rack touches the rolling circle of the pinion if it is 5.8-4.0=1.8mm above the bottom of the Technics bearing and 3.2+1.8=5mm above the bottom of the rack brick. That gives us the rolling radius of the infinite-diameter gear:∞+5mm. ☺
Caveat
Two gears may satisfactorily mesh when placed in axle holes that are in other positions than the ones found on a straight Technics bearing beam. The quality of the meshing depends on the inner and outer diameters of the gears, i.e. the diameter as measured over the tops of the teeth and the diameter as measured at the bottom of the teeth. These depend on the particular gear and are not directly related to the rolling diameter. That is the reason for the gear meshing calculator.
Known rolling diameters
(This does not include the conical wheels of differentials.)
To my knowledge there are gear wheels of 8, 12, 16, 20, 24, 28, 36, 40, 56 and 168 teeth.
Of these the one with 28 teeth is found on one side of a differential box; the 56 one is the outer ring of a turntable, and the 168 one is the inner ring of a large wheel that comes with the Hailfire Droid (Technics part x784, apparently only in the Star Wars series set 4481 of 2003). There is also a 24-tooth inner ring in a turntable.
Inside the Hailfire wheel
Using the rolling diameters it is easy to prove that the following combination of gears will fit inside the large "Hailfire" ring:
Note that all but the two 16-tooth wheels have axles that fit in the bearings of a normal beam; the 16-tooth wheels have their axles exactly in between holes. The beam outlined in red does not exist (it is much longer than the longest existing one).
