Before proceeding I want to make it absolutely clear that these are my personal views only and nothing to do with my employer or their views.
Traffic exchange charges
You can have a look at NIXI tariffs here. While the Currently peered parties have to shell out ₹1/GB for every excess of downstream data that they consume through the exchange as compared to the upstream data that they end up pushing, unless you push 5 times more upload as compared to download. That is then paid to the parties pushing out the data. Assuming charge of ₹/GB and ratio of 5:1 of download to upload, the yearly bill for a 1 Gbps fully utilised circuit(assuming avg. down 1 Gbps, average upload 200 Mbps) will cost ₹322 cr. ($47438217), which is exponentially higher the cost of transit in India. Doesn’t make any sense for an eyeball ISP to connect! They often try to compensate by announcing lesser prefixes to balance traffic.
This has resulted in parties announcing partial routes, while receiving everything. Also since the rates are determined on RX/TX on the L2 switch, the smaller providers can just do a UDP flood to balance traffic.
Keeping flat port charges and allowing parties to exchange traffic for free is a much better solution. I don’t know if this is an arangement to keep the larger transit providers like Tata, Airtel happy.
The requirement of an ISP license
A lot of content providers who want to peer with eyeball networks in India like Google, Microsoft, Amazon etc. simply can’t because of this. Getting an ISP license is not an option for them since getting one is a bureaucratic mess, compliance with DOT regulations and the % of AGR fee charged an even bigger problem. They don’t need one for their operations.
An exchange without content players is much worse a proposition.
Broken L1 connectivity and down members
We can check how many of the connected networks at NIXI are working from the NIXI looking glass.
- Routers which are pinging: 88
- Routers which are down: 36
- Total routers: 124
- Routers up percentage: 70.96%
It was good to note that NIXI no longer announces the subnet used for peering globally and keeps it local.
There is also an issue of choking of links. These are IPs of connected AS9498(Bharti Airtel Ltd) and AS45528(Tikona Digital Networks Pvt Ltd) routers graphed as an eg. from Delhi(Noida) location. AS55644(Idea Cellular Limited) and AS9829(Bharat Sanchar Nigam Ltd) routers on the same subnet graphed absolutely fine, while some others were far worse than this.
Ping to 218.100.48.34(Tikona) router from Noida
Ping to 218.100.48.20(Tikona) router from Noida
From Tikona’s graph you can make out that the issue is of choking of downstream traffic during peak hours. For Airtel it was more uniform. The reason why nobody wants to upgrade capacities is because:
- Larger ISPs like Airtel/Tata – Better connectivity at exchanges affects their IP transit business. They have a large number of end customers directly connected and make a lot of large chunk of revenue through selling connectivity to them.
- Eyeball ISPs – Higher port charges, higher costs due to traffic exchange charges(requestor pays logic). It is cheaper to remove announcements and try to balance upload and download than to upgrade the link. For many ISPs it is very small part of their capacity. It’s running because nobody has cared to remove it.
I have tried to get these issues fixed in my personal capacity but mostly I’m asked to send them traffic via transit or they reduce prefixes announced at NIXI.
Datacenter choice and newer locations
The datacenters where NIXI is present are:
- Mumbai – Cyquator Technologies Pvt. Ltd.
- Delhi(Noida) – Netmagic IT Services Private Limited
- Chennai – Sify Technologies Limited
- Bangalore – Sify Technologies Limited
- Hyderabad – CtrlS Datacenters Ltd
- Kolkata – STPI
- Ahmedabad – (n) Code Solutions -A Division of GNFC Limited
- Guwahati – Assam Electronics Development Corporation Ltd
As you can see, the choice of datacenter in a lot of cases is not stellar, where a lot of networks are already present and don’t need to build out separately just for connecting to NIXI.
In 2008, four more locations were added to NIXI mostly from non-internet hub cities. The older locations have done fairly well in terms of traffic, but the newer locations have suffered, largely due to no large player or any content player being present. There are mostly local eyeball ISPs present there. Most of the traffic between them is probably peer to peer or gaming traffic. At the current traffic levels, it doesn’t make sense for anyone to connect there.
Traffic rate through newer locations is pretty bad
As it stands, NIXI is working on expanding to Lucknow and Mohali.
Miscellaneous
- Lack of bilateral peering option
There is no option for participating networks to bilaterally peer with each other. Providing just the L2 switching infrastructure can solve the problem of costly interconnects for a lot of networks hence allowing to connect to more parties.
- Route leaks (largely fixed)
As NIXI is an exchange only, nobody should ideally be announcing routes learnt through AS24029(NIXI is an IXP in India) through their transits into the global routing table. However due to incorrect filter application, the routes can end up on the global BGP routing table.
route-views>sh ip bgp regexp _24029_
...
Network Next Hop Metric LocPrf Weight Path
* 43.224.159.0/24 140.192.8.16 0 54728 6939 15412 55644 24029 4755 132556 i
* 207.172.6.20 86 0 6079 15412 55644 24029 4755 132556 i
* 132.198.255.253 0 1351 10578 11164 15412 55644 24029 4755 132556 i
* 162.250.137.254 0 4901 6079 15412 55644 24029 4755 132556 i
* 207.172.6.1 0 0 6079 15412 55644 24029 4755 132556 i
* 209.124.176.223 0 101 101 11164 15412 55644 24029 4755 132556 i
* 129.250.1.66 17635 0 2914 15412 55644 24029 4755 132556 i
* 64.71.137.241 0 6939 15412 55644 24029 4755 132556 i
* 103.247.3.45 0 58511 6939 15412 55644 24029 4755 132556 i
* 91.218.184.60 0 0 49788 12552 6939 15412 55644 24029 4755 132556 i
* 116.212.177.0/24 140.192.8.16 0 54728 6939 9583 24029 4755 55839 i
* 64.71.137.241 0 6939 9583 24029 4755 55839 i
* 132.198.255.253 0 1351 6939 9583 24029 4755 55839 i
* 202.62.116.0 140.192.8.16 0 54728 6939 9583 24029 4755 9941 i
* 129.250.1.66 18628 0 2914 9583 24029 4755 9941 i
* 132.198.255.253 0 1351 6939 9583 24029 4755 9941 i
* 64.71.137.241 0 6939 9583 24029 4755 9941 i
* 202.122.134.0 140.192.8.16 0 54728 6939 9583 24029 4755 4755 4755 38615 i
* 129.250.1.66 18628 0 2914 9583 24029 4755 4755 4755 38615 ?
* 64.71.137.241 0 6939 9583 24029 4755 4755 4755 38615 i
* 132.198.255.253 0 1351 6939 9583 24029 4755 4755 4755 38615 i
* 114.31.199.1 0 4826 9583 24029 4755 4755 4755 38615 i
* 223.31.122.0 194.85.40.15 0 0 3267 43531 9583 24029 18101 18101 18101 132215 ?
* 195.208.112.161 0 3277 3267 43531 9583 24029 18101 18101 18101 132215 ?
AS9583(Sify Ltd) and AS55644(Idea Cellular Limited) are still leaking routes learnt from peers at NIXI onto the global routing table. This was an issue a major issue with many of the transit players connected to NIXI. The filters would be applied on the customer ASN and not from where the route was learnt. This is now fixed to a large extent.
- Partial sharing of routes
Due to requestor pays logic and unwillingness of networks to carry traffic over their backbones, they end up sharing partial routes at NIXI, while accepting all routes. This has resulted in issues for root servers connected there and don’t have transit, ending up blackholing traffic. K-root server ended up removing their root servers from NIXI largely because of this.
The silver lining is that there is decent competition that has now come up due to these issues with NIXI in Extreme IX, Mumbai IX and Bharat IX.
Complaining is always easy, but able to solve problems is a real challenge. When someone do that extending support and working with them to make a change is absolutely awesome !
My Suggestion Join Mumbai IX 🙂 You wont have reason to complaint.