Ventrilo is a VOIP server that let's groups of people talk to each other in real time. The free, non-commercial version allows up to 8 clients. This is plenty for me at the moment, but if you need more, you can look at TeamSpeak wich allows up to 32 (IIRC, you may need to double check the limits).
Ventrilo has instructions for installing on linux, but they are far from "complete." I hit quite a few blockers and it took just over 2 hours to get it setup and working correctly. The following instructions worked for me on Ubuntu 9.10, but I imagine they work fine for older/newer versions as well.