As you might already know, on VMS, the Enqueue Quota (ENQLM) limits the number of locks a process (and its subprocesses) can own. VAX/Rdb uses RMS, which in turn uses the Lock Management Facility, to synchronize shared file access. One lock is used for every shared file, local buffer, global buffer section, and outstanding record lock. Users doing large amounts of file sharing should set the ENQLM to a large value. Typically, this number is recommended at 200. The DEFAULT account ENQLM is set at 100, so all accounts begin with a limit of 100 unless explicitly changed.
It is understandable why you need to increase ENQLM when selecting a large number of records from a database. Commands like DESCRIBE DATABASE do not require many locks and, therefore, worked with the low ENQLM. We don't know why Rdb SQL$ utility worked except for the speculation that IDAL might add the extra locking.
DAL support engineering told us that although they do not specify account quota requirements, in the documentation, they recommend changing user accounts quota to the VAX/Rdb requirements. The following are from VAX Rdb/VMS Installation Manual:
BYTLM (Byte Limit): at least 18,000 bytes
FILLM (Open File Quota): around 50 is an acceptable value
ENQLM (Enqueue Limit): at least 600 locks
The account should also have TMPMBX privilege.