MacTCP does NOT support 0's as broadcast address bits, so your systems need to be reconfigured to support 1's as broadcast bits. Changing your Sun and MacTCP values will fix the issue:
Hosts:
Sun: 128.1.1.1
subnet: 255.255.0.0
broadcast: 128.0.255.255
Macintosh: 128.1.1.4
subnet: 255.255.0.0
broadcast: 128.0.255.255
MacTCP-based application error messages often leave out the most common error possibility -- an invalid address. MacX, for example, gave the error message to check installation of the MacTCPTool, when in fact it was the IP addressing scheme itself that was wrong.
MacTCP DOES accept "INVALID" TCP/IP addresses without complaining, and will then fail to attempt any connections with those addresses. You can configure "invalid" addresses, like 128.0.0.0 or 0.0.0.255, without any complaints, but there will not be any support for application services on top of them.
1) Here are the valid IP address ranges for:
Valid Class A IP addresses: 1.0.0.1 through 127.255.255.254. In a class A IP address, the first octet contains the network number. The second, third, or fourth octet must contain a non-zero value. Examples of valid addresses include: 17.5.0.0; 45.0.0.200; and 50.0.120.0.
Valid Class B IP addresses: 128.0.0.1 through 191.255.255.254. In a class B IP address, the first and second octets contain the network number. The third or fourth octet must contain a non-zero value. Examples of valid addresses include: 135.0.17.0; 160.0.0.210; and180.22.120.0.
Valid Class C IP addresses: 192.0.0.1 through 223.255.255.254. In a class C IP address, the first, second, and third octets contain the network number. The fourth octet must contain a non-zero value. Examples of valid addresses include: 221.5.0.1; 198.0.0.200; and 202.0.120.23.
Also internet addresses in the range 128.0.rrr.rrr, and 192.0.0.rrr are
reserved. MacTCP doesn't allow use of this address, and issues a "bac
ip or LAP configuration error".
2) MacTCP, under the Macintosh OS, uses 255 for broadcasts, not zero. In other words, a broadcast on a class network would use an address like this: 17.255.255.255, not 17.0.0.0. A/UX, or MacTCP running under A/UX, can be configured to use zero for broadcasts (for example 17.0.0.0).
You cannot use MacTCP, under the Macintosh OS, on a network that does not use 255 for broadcasts.
3) RFC 1166 is the document with the information you need.