Apache & IPv6

[bottswana]

Hey guys, having a bit of a nightmare here
The server Ip seems to be completely accessable. I can ping it from both sides of the tunnel, however, apache dosent like it.
THere are no errors when it starts up, and it works fine over IPv4, however, when you try it on IPv6, the page never loads. It establishes a connection, and then just hangs on "waiting" for an indefinate period of time.
Apache's error logs report nothing, and the access log reports a code 200 for every connection (All OK)
Can't understand why it never loads anything?
DNS works fine, the Quad-A records are resolving as normal, so it cant be anything wrong with the IPv6 configuration?
Any Apache geniuses know whats going on here?

[cholzhauer]

Need more information...what's your domain?

Is Apache actually listening on IPv6?  what's "netstat -a" show?

[bottswana]

Yes, apache is binded and is listening on the port.
Shutting down the apache service stops the "waiting for" issue and it fails on connecting, which is logical.

v4: 81.138.181.108
v6: [2001:470:95bb::2]

Netstat output:

Code Select Expand


root@marge:~# netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 *:29000                 *:*                     LISTEN
tcp        0      0 *:mysql                 *:*                     LISTEN
tcp        0      0 *:submission            *:*                     LISTEN
tcp        0      0 *:imap2                 *:*                     LISTEN
tcp        0      0 host81-138-181-1:webmin *:*                     LISTEN
tcp        0      0 host81-138-181-108.:www *:*                     LISTEN
tcp        0      0 192.168.10.2:domain     *:*                     LISTEN
tcp        0      0 host81-138-181-1:domain *:*                     LISTEN
tcp        0      0 localhost:domain        *:*                     LISTEN
tcp        0      0 *:ssh                   *:*                     LISTEN
tcp        0      0 *:19000                 *:*                     LISTEN
tcp        0      0 localhost:11000         *:*                     LISTEN
tcp        0      0 *:smtp                  *:*                     LISTEN
tcp        0      0 localhost:953           *:*                     LISTEN
tcp        0      0 host81-138-181-10:https *:*                     LISTEN
tcp        0      0 host81-138-181-10:20000 *:*                     LISTEN
tcp        0      0 *:imaps                 *:*                     LISTEN
tcp        0      1 host81-138-181-10:51104 clamav.swishmail.co:www SYN_SENT
tcp        0     52 host81-138-181-108.:ssh cpc1-lanc6-2-0-cu:49506 ESTABLISHED
tcp6       0      0 marge.bottswanam:webmin [::]:*                  LISTEN
tcp6       0      0 marge.bottswanamedi:www [::]:*                  LISTEN
tcp6       0      0 [::]:ftp                [::]:*                  LISTEN
tcp6       0      0 [::]:domain             [::]:*                  LISTEN
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN
tcp6       0      0 localhost:953           [::]:*                  LISTEN
tcp6       0      0 marge.bottswaname:https [::]:*                  LISTEN
tcp6       0      0 marge.bottswaname:20000 [::]:*                  LISTEN
udp        0      0 *:27500                 *:*
udp        0      0 192.168.10.2:domain     *:*
udp        0      0 host81-138-181-1:domain *:*
udp        0      0 localhost:domain        *:*
udp6       0      0 [::]:domain             [::]:*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     4841     private/bsmtp
unix  2      [ ACC ]     STREAM     LISTENING     4845     private/scalemail-backend
unix  2      [ ACC ]     STREAM     LISTENING     4849     private/mailman
unix  2      [ ACC ]     STREAM     LISTENING     4749     private/tlsmgr


netstat -tulpn | grep :80

Code Select Expand


root@marge:~# netstat -tulpn | grep :80
tcp        0      0 81.138.181.108:80       0.0.0.0:*               LISTEN      1316/apache2
tcp6       0      0 2001:470:95bb::2:80     :::*                    LISTEN      1316/apache2


[cconn]

ip6tables is blocking your HTTP port?

[bottswana]

If it was a firewall issue, then surely it would not connect in the first place?
Plus an IPv6 port scan shows port 80 as open.

[jrocha]

What does your virtualhost config look like? What does your ip6tables config look like? What happens when you telnet to port 80 using the ipv6 address?

[bottswana]

Telnetting in results in a black screen in which you can do nothing (at least, I don't know what to do with it).
It didnt display any errors though.

ip6tables: (This is probably where the problem is. I hate iptables!)

Code Select Expand



# Generated by ip6tables-save v1.4.4 on Tue Feb  1 22:57:59 2011
*filter
:INPUT DROP [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [27:2807]
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 19000:19010 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 29000 -j ACCEPT
-A INPUT -p tcp -m tcp -m multiport --dports 20,21 -j ACCEPT
-A INPUT -p tcp -m tcp -m multiport --dports 10000,20000 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 5000:5100 -j ACCEPT
-A INPUT -p tcp -m tcp -m multiport --dports 993,25,3306 -j ACCEPT
-A INPUT -s ::1/128 -j ACCEPT
-A INPUT -d ::1/128 -j ACCEPT
-A INPUT -p ipv6-icmp -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
COMMIT
# Completed on Tue Feb  1 22:57:59 2011
# Generated by ip6tables-save v1.4.4 on Tue Feb  1 22:57:59 2011
*mangle
:PREROUTING ACCEPT [19:2299]
:INPUT ACCEPT [19:2299]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [27:2807]
:POSTROUTING ACCEPT [27:2807]
COMMIT
# Completed on Tue Feb  1 22:57:59 2011



Virtual Servers:

Code Select Expand



NameVirtualHost 81.138.181.108:80
NameVirtualHost [2001:470:95bb::2]:80

<VirtualHost [2001:470:95bb::2]:80>
ServerAdmin webmaster@localhost

DocumentRoot /var/www
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>

ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined

</VirtualHost>

<VirtualHost 81.138.181.108:80>
ServerAdmin webmaster@localhost

DocumentRoot /var/www
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>

ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined

</VirtualHost>


Edit: Added new Ip6tables config to reflect icmp > icmpv6

[broquea]

Can't seem to ping6 or reach that host, not even from the tunnel-server itself:

Code Select Expand

~$ telnet 2001:470:95bb::2 80
Trying 2001:470:95bb::2...
telnet: Unable to connect to remote host: No route to host

--- 2001:470:95bb::2 ping statistics ---
4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3017ms

--- 2001:470:1f08:13b3::2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms


I've confirmed the static route is in place on our side.

[bottswana]

Very, very strange. Just used an online IPv6 ping utility and it reports the same thing. Can't be my router because I can ping other hosts in my /48 prefix, and I can ping it from within the prefix.
I will check my routing

Edit: No, that was me blocking all ICMP requests with Ip6tables. Fixed now :/

Ping6 to google from troublesome server:

Code Select Expand


root@marge:~# ping6 2a00:1450:8006::93
PING 2a00:1450:8006::93(2a00:1450:8006::93) 56 data bytes
64 bytes from 2a00:1450:8006::93: icmp_seq=1 ttl=56 time=36.4 ms
64 bytes from 2a00:1450:8006::93: icmp_seq=2 ttl=56 time=23.1 ms
64 bytes from 2a00:1450:8006::93: icmp_seq=3 ttl=56 time=24.5 ms
64 bytes from 2a00:1450:8006::93: icmp_seq=4 ttl=56 time=25.1 ms
^C
--- 2a00:1450:8006::93 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3006ms
rtt min/avg/max/mdev = 23.141/27.294/36.421/5.317 ms


So, after spending a while tweaking ip6tables, apache is still displaying a blank white page. Running out of ideas here :(

[schvin]

do your apache access logs show the v6 clients? just curious if by any chance something else is intercepting the traffic.

i can open a socket to it, but it doesn't appear to respond to http requests, just closes the questions. if it shows up in the apache logs then at least it is something within apache (though your apache config looks sane to me).

alternately you could open a v6 connection from somewhere else (i.e. printf "GET / HTTP/1.0\r\nHost: bottswanamedia.info\r\n\r\n" | telnet 2001:470:95bb::2 80), or, even just the telnet portion and let it hang, will give you some more time, and see if you see a netstat connection from the client.

[bottswana]

So, I switched logs mode from errors to 'debug' and loaded up the site again. This is the only line that is output:

Code Select Expand


Wed Feb 02 17:43:42 2011] [debug] mod_deflate.c(615): [client 2001:470:95bb::4] Zlib: Compressed 177 to 128 : URL /index.html


access.log outputs:

Code Select Expand


2001:470:95bb::4 - - [02/Feb/2011:17:43:42 +0000] "GET / HTTP/1.1" 200 485 "-" (Loads of browser info)


Still not getting any joy on page loads though.

using netstat when trying to connect does show a connection:

Code Select Expand


tcp6       0    485 2001:470:95bb::2:80     2001:470:95bb::4:1028   ESTABLISHED 14042/apache2


[schvin]

Ah, the gzipping definitely makes it hard to debug w/ telnet. Can you try disabling the mod_deflate as a test. Any other modules of relevance that it might be hanging on (ssl, etc.)?

[bottswana]

I disabled that deflate module. Theres quite a few modules loaded as its set  up as a shared hosting environment with Virtualmin.

Heres the module output from virtualmin:

Code Select Expand


actions
alias
auth_basic
auth_digest
authn_file
authz_default
authz_groupfile
authz_host
authz_user
autoindex
cgi
dav
dav_fs
dav_svn
dir
env
fcgid
mime
negotiation
php5
proxy
proxy_balancer
proxy_connect
proxy_html
proxy_http
reqtimeout
rewrite
ruby
setenvif
ssl
status
suexec


[schvin]

got it. connection refused currently, but guessing you are working on the system :)

it's not expecting a proxy backend or anything for that vhost, right?

[bottswana]

I tried disabling every module I could but still got the same result.
That included disabling all the proxy modules.

(Servers back as it was now)