At the worst, hot swapping could damage the ADB controller chip on the logic board. The same would hold true for connecting video or SCSI cables while the computer is turned on. Unless the cable technology is specifically designed for hot swapping, such as USB or FireWire, you should not hot plug cables.
Restarting the computer will usually resolve performance issues with an ADB device that has accidentally been hot swapped.
Note: Such damage constitutes customer abuse and may not be covered under warranty.