Though this isn't a Masquerading issue but many users do this so it should be mentioned.
SMTP: The issue is that you are probably using your Linux box as an SMTP relay server and get the following error:
"error from mail server: we do not relay"
Sendmail: Enable specific relaying for your internal MASQed machines by editing the /etc/sendmail.cw file and add the hostname and domain name of your internal MASQed machine. You should also check to see that the /etc/hosts file has the IP address and Fully Qualified Domain Name (FQDN) configured in it. Once this is done, you need to restart Sendmail for it to re-read its configuration files. This is covered in TrinityOS - Section 25
POP-3: Some users configure their internal MASQ'ed computer's POP-3 clients to connect to some external SMTP server. While this is fine, many SMTP servers out there will try to IDENT your connection on port 113. Most likely your problem stems around your default Masquerade policy being set to DENY. This is BAD. Set it to REJECT and re-run your rc.firewall ruleset.