Microsoft Internet Explorer's applet security settings are intended for the Microsoft Java virtual machine, not Mac OS Runtime for Java (MRJ).
To change the settings for MRJ, use Apple Applet Runner which is in the Apple Applet Runner folder, in the Mac OS Runtime for Java folder, in the Apple Extras folder. Apple Applet Runner has a user interface to change the properties file which controls the global security restrictions used in MRJ. To get to the properties dialog, select "Properties" from the "File" menu in Apple Applet Runner.