Since attachments are part of the message itself, if the message is stored on your computer's hard disk, then the it is also on the hard disk. If you delete the message, all of its attachments are also deleted.
Other email programs may use the downloads folder you set in the Web tab of the Internet pane in System Preferences. Mail does not use the downloads folder for attachments. However, you can save files there by pressing Control and clicking the enclosure.
For information on how to save an attachment in Mail, see document 107150 "Mac OS X 10.2: How to Save Attachments in Mail".