Troubleshooting suggestions for slow tunnel?

[mcsteve]

Hey folks, I have my tunnel up and running, terminated on a Cisco 2621XM connected to the internet via PPPoE DSL.

I get 7Mbps incoming on that DSL link, and about 864Kbps outgoing. When using IPv6 via the tunnel, however, I'm getting speeds that vary between 10-100Kbps. I understand that HE's tunnels are not throttled, and their endpoints are probably not too overloaded, so this seems excessively slow.

Connectivity seems to work fine, aside from the speed, passing all tests at test-ipv6.com and getting ping times between 100 and 100ms to most IPv6 sites.

Does anyone have any pointers to where I should begin troubleshooting?

[cholzhauer]

There are some IPv6 speed test sites...have you tried those to see what speeds you're actually getting?

[mcsteve]

Yes, I've tried a couple IPv6 speedtests. Sometimes they won't even load, but when I can get the one at test-ipv6.com to load it reports my IPv4 speed at about 6.5Mbps and my IPv6 speed at about 10Kbps. The noc.maine.edu speedtest reports around 100Kbps for IPv6.

I should also mention that I am physically in Minnesota, and using the Chicago tserv.

[mcsteve]

Well, now I've tried playing around with my MTU settings a bit, since I am on a PPPoE DSL connection, and it hasn't made a difference:



Anyone else have any ideas? Is anyone else experiencing speeds like this?

[mcsteve]

I got ambitious tonight and decided to delete my current tunnel and create a new one on tserv13.ash1, which the stats say is much more lightly used than tserv9.chi1. I thought it might be an improvement since the ping time is lower, despite being physically further away. The speed is still about the same, though:


It's definitely not just the speedtest server either; if I attempt to visit dns.he.net with IPv6 the page never completely loads, taking about 10 minutes just to start rendering.

Has anyone got any ideas? Should I email HE about this? I hate to complain about throughput on a service that's being provided for free...

[cholzhauer]

Well, let's think about this.

You've tried two different tunnel servers, so the odds are both tunnel servers aren't the problem (and especially because I haven't seen any other posts describing the same problems) That means the problem lies somewhere on your end.  The only things I can think of are: Firewall/router, ISP, and computer.

Have you tired the speedtest from another computer?  Let's see the output of ipconfig /all (or ifconfig) on the computer you're using, and a copy of your routing tables

[mcsteve]

I've just tried the speedtest from another machine on my network, with the same results. I would say this leaves us with the router and the ISP as potential problems. Here is the ifconfig and routing information from my PC:

Code Select Expand

eth0      Link encap:Ethernet  HWaddr 70:5a:b6:87:81:7f 
          inet addr:10.10.0.5  Bcast:10.10.255.255  Mask:255.255.0.0
          inet6 addr: 2001:470:e03e::face/64 Scope:Global
          inet6 addr: fe80::725a:b6ff:fe87:817f/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1280  Metric:1
          RX packets:5099344 errors:0 dropped:3279 overruns:0 frame:0
          TX packets:7174360 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1194053110 (1.1 GiB)  TX bytes:5852272150 (5.4 GiB)
          Interrupt:43

Code Select Expand

Kernel IPv6 routing table
Destination                    Next Hop                   Flag Met Ref Use If
::1/128                        ::                         Un   0   1    12 lo
2001:470:e03e::face/128        ::                         Un   0   1  7157 lo
2001:470:e03e::/64             ::                         U    256 0     3 eth0
fe80::725a:b6ff:fe87:817f/128  ::                         Un   0   1   328 lo
fe80::/64                      ::                         U    256 0     0 eth0
ff00::/8                       ::                         U    256 0     0 eth0
::/0                           2001:470:e03e::1           UG   1024 0     0 eth0
::/0                           ::                         !n   -1  1    66 lo

And here is the routing table and tunnel interface configuration from my router, where the tunnel is terminated:

Code Select Expand

IPv6 Routing Table - 7 entries
Codes: C - Connected, L - Local, S - Static, R - RIP, B - BGP
       U - Per-user Static route
       I1 - ISIS L1, I2 - ISIS L2, IA - ISIS interarea, IS - ISIS summary
       O - OSPF intra, OI - OSPF inter, OE1 - OSPF ext 1, OE2 - OSPF ext 2
       ON1 - OSPF NSSA ext 1, ON2 - OSPF NSSA ext 2
S   ::/0 [1/0]
     via 2001:470:7:EE::1
C   2001:470:7:EE::/64 [0/0]
     via ::, Tunnel0
L   2001:470:7:EE::2/128 [0/0]
     via ::, Tunnel0
C   2001:470:E03E::/64 [0/0]
     via ::, FastEthernet0/0
L   2001:470:E03E::1/128 [0/0]
     via ::, FastEthernet0/0
L   FE80::/10 [0/0]
     via ::, Null0
L   FF00::/8 [0/0]
     via ::, Null0

Code Select Expand

Tunnel0 is up, line protocol is up
  Hardware is Tunnel
  Description: Hurricane Electric IPv6 Tunnel
  MTU 1514 bytes, BW 9 Kbit/sec, DLY 500000 usec,
     reliability 255/255, txload 1/255, rxload 221/255
  Encapsulation TUNNEL, loopback not set
  Keepalive not set
  Tunnel source 67.6.67.171 (Dialer1), destination 216.66.22.2
  Tunnel protocol/transport IPv6/IP
  Tunnel TTL 255
  Fast tunneling enabled
  Tunnel transmit bandwidth 8000 (kbps)
  Tunnel receive bandwidth 8000 (kbps)
  Last input 00:00:35, output 00:00:35, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/0 (size/max)
  5 minute input rate 53000 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     20402 packets input, 25649738 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     8882 packets output, 983186 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

Code Select Expand

Tunnel0 is up, line protocol is up
  IPv6 is enabled, link-local address is FE80::4306:43AB
  Description: Hurricane Electric IPv6 Tunnel
  Global unicast address(es):
    2001:470:7:EE::2, subnet is 2001:470:7:EE::/64
  Joined group address(es):
    FF02::1
    FF02::2
    FF02::1:FF00:2
    FF02::1:FF06:43AB
  MTU is 1280 bytes
  ICMP error messages limited to one every 100 milliseconds
  ICMP redirects are enabled
  Input features: Common pak subblock feature ACL
  Output features: Firewall Inspection
  Inbound access list in6
  Outbound Inspection Rule in6
  ND DAD is enabled, number of DAD attempts: 1
  ND reachable time is 30000 milliseconds
  Hosts use stateless autoconfig for addresses.

[mcsteve]

Oh, and here's the entire router config, in case it contains any clues:

Code Select Expand


!
! Last configuration change at 00:27:25 CDT Sun Jul 24 2011 by admin
! NVRAM config last updated at 00:27:28 CDT Sun Jul 24 2011 by admin
!
version 12.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service sequence-numbers
!
hostname core-router
!
boot-start-marker
boot-end-marker
!
security authentication failure rate 3 log
security passwords min-length 6
logging buffered 4096 debugging
enable secret 5 --SNIP--
enable password 7 --SNIP--
!
aaa new-model
!
!
aaa authentication login local_authen local
aaa authorization exec local_author local
!
aaa session-id common
clock timezone CST -6
clock summer-time CDT recurring
no network-clock-participate slot 1
no network-clock-participate wic 0
no ip source-route
ip cef
!
!
!
!
no ip bootp server
ip domain name houseofhacks.net
ip name-server 10.10.240.10
ip inspect name CCP_LOW cuseeme
ip inspect name CCP_LOW dns
ip inspect name CCP_LOW ftp
ip inspect name CCP_LOW h323
ip inspect name CCP_LOW sip
ip inspect name CCP_LOW https
ip inspect name CCP_LOW icmp
ip inspect name CCP_LOW imap
ip inspect name CCP_LOW pop3
ip inspect name CCP_LOW netshow
ip inspect name CCP_LOW rcmd
ip inspect name CCP_LOW realaudio
ip inspect name CCP_LOW rtsp
ip inspect name CCP_LOW esmtp
ip inspect name CCP_LOW sqlnet
ip inspect name CCP_LOW streamworks
ip inspect name CCP_LOW tftp
ip inspect name CCP_LOW tcp router-traffic
ip inspect name CCP_LOW udp
ip inspect name CCP_LOW vdolive
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
ip ddns update method tunnelbroker
HTTP
  add https://ipv4.tunnelbroker.net/ipv4_end.php?ip=AUTO&pass=--SNIP--
interval maximum 1 0 0 0
interval minimum 1 0 0 0
!
ip ddns update method he-ddns
HTTP
  add http://houseofhacks.net:--SNIP--@dyn.dns.he.net/nic/update?hostname=houseofhacks.net
interval maximum 1 0 0 0
interval minimum 1 0 0 0
!
vpdn enable
!
!
ipv6 unicast-routing
ipv6 inspect name in6 tcp
ipv6 inspect name in6 udp
!
!
!
!
!
!
!
!
!
!
!
!
!
crypto pki trustpoint tunnelbroker
enrollment terminal pem
revocation-check none
!
!
crypto pki certificate chain tunnelbroker
certificate ca 00F17A2250E699D461 nvram:ipv6henet#D3D4CA.cer
username admin privilege 15 secret 5 --SNIP--
!
!
ip tcp synwait-time 10
ip ssh time-out 60
ip ssh authentication-retries 2
ip ssh version 2
!
!
!
!
!
interface Tunnel0
description Hurricane Electric IPv6 Tunnel
no ip address
no ip redirects
no ip proxy-arp
ip route-cache flow
ipv6 address 2001:470:7:EE::2/64
ipv6 enable
ipv6 traffic-filter in6 in
ipv6 mtu 1280
ipv6 inspect in6 out
tunnel source Dialer1
tunnel destination 216.66.22.2
tunnel mode ipv6ip
!
interface FastEthernet0/0
description $ETH-LAN$$FW_INSIDE$
ip address 10.10.254.1 255.255.0.0
ip access-group 100 in
no ip redirects
no ip proxy-arp
ip nat inside
ip virtual-reassembly
ip route-cache flow
ip tcp adjust-mss 1452
no ip mroute-cache
speed auto
full-duplex
ipv6 address 2001:470:E03E::1/64
ipv6 enable
ipv6 mtu 1280
no mop enabled
!
interface FastEthernet0/1
no ip address
no ip redirects
no ip proxy-arp
ip route-cache flow
shutdown
duplex auto
speed auto
!
interface Ethernet1/0
description WAN
ip address 192.168.0.254 255.255.255.0
no ip redirects
no ip proxy-arp
ip route-cache flow
half-duplex
pppoe enable group global
pppoe-client dial-pool-number 1
!
interface Ethernet1/1
no ip address
no ip redirects
no ip proxy-arp
ip route-cache flow
shutdown
half-duplex
!
interface Ethernet1/2
no ip address
no ip redirects
no ip proxy-arp
ip route-cache flow
shutdown
half-duplex
!
interface Ethernet1/3
no ip address
no ip redirects
no ip proxy-arp
ip route-cache flow
shutdown
half-duplex
!
interface Dialer1
description $ETH-WAN$$FW_OUTSIDE$
bandwidth 893
bandwidth receive 7167
ip ddns update tunnelbroker
ip ddns update he-ddns
ip address negotiated
ip access-group 101 in
no ip redirects
no ip proxy-arp
ip mtu 1492
ip nbar protocol-discovery
ip nat outside
ip inspect CCP_LOW out
ip virtual-reassembly
encapsulation ppp
ip route-cache flow
no ip mroute-cache
dialer pool 1
dialer-group 1
ppp authentication chap pap callin
ppp pap sent-username --SNIP--
ppp ipcp route default
!
ip forward-protocol nd
!
!
ip http server
ip http access-class 90
no ip http secure-server
ip nat inside source list 1 interface Dialer1 overload
ip nat inside source static tcp 10.10.240.10 22 interface Dialer1 22
ip nat inside source static udp 10.10.0.1 6881 interface Dialer1 6881
ip nat inside source static tcp 10.10.0.1 6881 interface Dialer1 6881
!
ip access-list extended bittorrent
permit tcp any any eq 6881
permit udp any any eq 6881
!
access-list 1 permit 10.10.0.0 0.0.255.255
access-list 90 permit 10.10.0.0 0.0.255.255
access-list 100 remark auto generated by CCP firewall configuration
access-list 100 remark CCP_ACL Category=1
access-list 100 remark Auto generated by CCP for NTP (123) 10.10.240.10
access-list 100 permit udp host 10.10.240.10 eq ntp host 10.10.254.1 eq ntp
access-list 100 deny   ip host 255.255.255.255 any
access-list 100 deny   ip 127.0.0.0 0.255.255.255 any
access-list 100 permit ip any any
access-list 101 remark auto generated by CCP firewall configuration
access-list 101 remark CCP_ACL Category=1
access-list 101 permit 41 any any
access-list 101 permit tcp any any eq 6881
access-list 101 permit udp any any eq 6881
access-list 101 permit tcp any any eq 22
access-list 101 deny   ip 10.10.0.0 0.0.255.255 any
access-list 101 permit icmp any any packet-too-big
access-list 101 permit icmp any any echo
access-list 101 permit icmp any any echo-reply
access-list 101 permit icmp any any time-exceeded
access-list 101 permit icmp any any unreachable
access-list 101 deny   ip 10.0.0.0 0.255.255.255 any
access-list 101 deny   ip 172.16.0.0 0.15.255.255 any
access-list 101 deny   ip 192.168.0.0 0.0.255.255 any
access-list 101 deny   ip 127.0.0.0 0.255.255.255 any
access-list 101 deny   ip host 255.255.255.255 any
access-list 101 deny   ip host 0.0.0.0 any
access-list 101 deny   ip any any
dialer-list 1 protocol ip permit
snmp-server community public RO
snmp-server location Basement Rack
snmp-server host 10.10.240.10 public
ipv6 route ::/0 2001:470:7:EE::1
!
!
!
ipv6 access-list in6
permit icmp any any
permit tcp any host 2001:470:E03E::2 eq domain
permit udp any host 2001:470:E03E::2 eq domain
permit tcp any host 2001:470:E03E::2 eq smtp
permit tcp any host 2001:470:E03E::2 eq www
permit tcp any host 2001:470:E03E:0:E965:C3BE:45AD:8583 eq 6881
permit udp any host 2001:470:E03E:0:E965:C3BE:45AD:8583 eq 6881
!
control-plane
!
!
!
!
mgcp behavior g729-variants static-pt
!
!
!
!
!
banner login core-router.houseofhacks.net - unauthorized access prohibited.

!
line con 0
logging synchronous
login authentication local_authen
transport output telnet
line aux 0
login authentication local_authen
transport output telnet
line vty 0 4
access-class 90 in
password 7 --SNIP--
authorization exec local_author
login authentication local_authen
transport input telnet ssh
line vty 5 181
access-class 90 in
password 7 --SNIP--
authorization exec local_author
login authentication local_authen
transport input telnet ssh
!
scheduler allocate 4000 1000
ntp clock-period 17180239
ntp server 10.10.240.10
!
end


[cholzhauer]

Is your MTU correct?  I have no idea what it should be, I just know it's been the cause of some speed issues in the past

[mcsteve]

It looks like I've got the MTU set to 1280, currently. I think I got that from another post somewhere on the forum here. I've also tried leaving it unspecified, but it doesn't seem to make a difference. I've got ICMP allowed through the firewall, and PMTU discovery seems to be working.

[mcsteve]

I think I figured this out now. I found another thread elsewhere wherein someone had performance issues with IPv6 CBAC on a Cisco router. So I've disabled my router's CBAC configuration, and speeds are definitely improved. Actually the new speedtest results are pretty odd, but I'll chalk it up to the test server being located overseas:



Now I just need to figure out how to fix the CBAC performance issue, or create a usable firewall configuration without CBAC.

[bjoxaa]

Do you have "ipv6 inspect" configured ?
If so, did you tried to disable it ?

For me, performance was really improved after disabling "ipv6 inspect" on my Router (Cisco 877, 12.4(24)T6).

Nb : For me, standard ipv6 access-lists have no influence on traffic speed.