To isolate the issue, try the following troubleshooting steps:
Confirm Connectivity
Check to see if the keyboard and mouse cables are securely connected.
Bypass Other USB Devices
If the keyboard or mouse is connected to the USB port of a device other than the iMac itself, try bypassing that device by connecting the keyboard or mouse directly to the iMac.
Try Alternate Ports
Confirm that a particular USB port isn't at fault by trying the keyboard or mouse in the alternate USB ports: both devices can be connected to either of the two USB ports under the iMac's side I/O door. The mouse can additionally be connected to either of the keyboard's two USB ports, though if the keyboard is unresponsive, don't trust either of its ports for testing the mouse--use the ports under the I/O door instead.
It is not necessary to power off or restart the computer before or after switching ports.
Rule Out Software
Make sure the issue is not caused by a software conflict or damaged software, which will usually affect both keyboard and mouse, and often intermittently. Note, however, that if the keyboard power button cannot be used to power on the iMac, the issue is not in software.
Software issues can be ruled out by attempting such basic troubleshooting steps as starting up with extensions off or starting the computer from the system CD, though note that neither of these steps may be possible if there is a hardware issue with either device or the USB port.