If Mac OS X was not configured to use AppleTalk via the Assistant, then turning on AppleTalk via the "Computer Settings"->"Network" menu won't work.
To make your Mac OS X Server an AppleShare file server via AppleTalk, use the Setup Assistant. In the Setup Assistant, select Apple File Services, then click on the right arrow to go to the "Review Settings" panel and click on the "Go Ahead" button to apply the change.