Tue, 20 Apr 2004

distcc for gcj?

Anthony writes about approaches to make distcc work well with gcj. This is tricky because distcc works by preprocessing your C or C++ source locally, then shipping the result to another machine for compilation.

There seem to be a few plausible approaches to doing this. One approach is to modify gcj itself. gcj could read your source, analyze it, find all the dependencies, and then package up a jar file that you could send remotely. At the remote site you could invoke gcj pointing at just this jar file. The drawback of this approach is that it requires invasive and complicated changes to gcj. You might think there is also a performance problem here, but in practice parsing and analyzing java source is pretty cheap, code generation is what takes the most time.

Another approach Anthony puts forward is the idea of using an LD_PRELOAD library to catch all open calls and have the remote compiler ask your machine for the files in question.

Actually I think this is kind of cool, though it does beg the question of why you don't just set up NFS, since after all you're serving files. This puts all the nasty potential security problems into a known framework.

Anyway, I think that his approach will work ok. It would be interesting to see how well it performs. It looks like his approach caches the files it opens, meaning that for a large compilation the number of files shipped over the network will dwindle. Another benefit of this approach is that the debugging information will always turn out right, since gcc will always request things by their real names. Neat.

One remaining approach is to just write a wrapper for gcj that ships everything mentioned on the class path to the remote server. If the class path includes a directory, just ship all the java files. You could use rsync for this to avoid shipping files more than once.

posted at: 16:50 | path: /software | permanent link to this entry