Well you never said what OS you were using in the first place, not sure if that reg entry will work on xp. I tend to not think so.
Again it seems clear that your host is not listening on 110 on ipv6.. So that would explain why something that is trying to use IPv6 would not work

If you don't want change your OS to prefer ipv4 over ipv6, then either enable or get you host to enable pop on ipv6 or another option would be to just create a host file entry that points to an IPv4 address and use that in your outlook setup. Only issue with that would be if the actual IP of the host changes, you would have to update your host file.
As to nmap not working on your pfsense, when did I say it would? But it would if you installed it or just run it from another machine on your network that uses ipv6, etc. But there is a package for nmap for pfsense so you could install it if you want. I just normally run it from one of my linux boxes or even window - it can run on quite a few OSes.
If you PM me your host name I would be happy to scan it for you to see what ports its listening on its ipv6 address.. But just because you enable IPv6 on a box does not mean all its services you might be running will bind to that IPv6 address. Most services require you to enable it to do that, some services do not even support IPv6 even though the OS might, etc.
from your telnet command there looks like your server is running dovecat for pop3, so for it to listen on both ipv4 and ipv6 you would have to use
http://wiki.dovecot.org/MainConfig "*" listens in all IPv4 interfaces. [::]" listens in all IPv6 interfaces. Use "*,[::]" for listening both IPv4 and IPv6.