How to start an Internet Service Provider in India

So I was planning to start an ISP in India but a few things happened and I gave on the idea. I did however do some research which I can share with you and hopefully you’ll find that interesting and helpful. 🙂

An Internet Service Provider or an ISP is basically an organisation or a company which provides users with internet access. I will be exploring most of the needed things needed to start one, the regulations and a few other things. I know that a lot of people would have the idea of starting one and hopefully I can help you out. While this post is primarily geared towards the Indian ISPs, this should help people trying to start one in other countries as well.

Firstly I highly recommend you to view this YouTube video. Skip to 54:30.

The last mile

For a residential/commercial provider, this should be your major focus point. Most of the time you’ll be scrambling to splice up fibers or putting in an Ubiquiti device, not managing a bunch of devices at the NOC. You will need to be a hardcore last mile guy or have someone in your company/business to help you out on that front. I didn’t go forward with my idea exactly for that reason. Having a good last mile will make or break your company, not trying to have cheaper bandwidth. There are two options you can look at for the last mile.

Wireless

Many providers like to start out on wireless or aspire to be a fully wireless provider. The problem? Telecom Regulatory Authority of India(TRAI) only allows for usage between 5.825 Ghz and 5.875 Ghz for WISPs. 50 Mhz is very cramped up for an ISP to operate in, especially when it will share that spectrum with many of it’s competitors. You can always use other bands illegally, but so do your competitors and overall there is too much noise here. It can be very problematic and there can be numerous issues which can occur. Site to site P2P links for commercial usage can workout fine but in P2MP(Point to Multipoint) scenarios, sharing so little spectrum among so many users is very problematic. PPPoE can also be an issue on wireless and /30 VLANs can be problematic if you don’t want to NAT. A huge wireless bridge is an even bigger problem. I will be talking about this in later sections. Wireless is very nice for corporate/business customers or linking far away sites together for P2P links. Otherwise it’s a massive pain.

Does it make sense:

• In India to every customer? Absolutely not. Laying fiber is cheaper usually.
• Other countries where last mile laying of fiber is expensive and spectrum is more liberal for WISPs and less choked? Yes. Also explore Millimeter wave radios if you fall in this category.

Also do remember that for wireless Ubiquiti is usually better than Mikrotik, albeit a little more expensive.

Fiber

This would probably be better. The problem is that last mile access and right of way is badly screwed up in India. It would have been great if the govt. had run conducts(essentially pipes) along electricity poles or underground installations and leased them out to private players. Find out about the cost of laying fiber legally on poles in your area. In Goa this is super cheap at about ₹1000/km/month. Other places like NCR it is prohibitively expensive. You can always do it illegally and usually nobody would care, but you of course run the risk of future action being taken. Also fiber gets cut. A lot. So make sure you have a nice team set up who can take care of the fiber and lay new fiber and get a splicer, unless you can lease it from someone for a good price. For fiber you again have two options:

Active ethernet

This basically is running one core of fiber either to each apartment/customer or using active devices like switches to essentially split fiber to be used by more customers. A lot of people like to use media converters and regular cheap unmanaged switches downstream. Most of media converters are crap and this tends to cause a lot of issues. Active ethernet is very hard to do in a country like India where the cost of fiber a lot compared to the cost of laying it. Also you need uninterrupted power at all locations, which can be a huge headache to manage and is not at all scalable. Thankfully there is a better solution.

Passive Optical Network

Passive Optical Network or PON allows you to essentially split a single strand of fiber to upto 128 customers, although 32 or 64 is usually preferred. The PLC splitters are extremely cheap and do not need power. You will need an ONU or an Optical Network Unit on the customers end, which can be had for around ₹1500-2000(for GEPON) and OLT, which costs around ₹70000-80000 for 4 ports expandable for GEPON. As for technologies you can use GEPON(Gigabit Ethernet Passive Optical Network) or GPON(Gigabit Passive Optical Network). GEPON is pure ethernet, while GPON encapsulates ethernet packets. GPON will give you 2.5 Gbps downstream per fiber, while GEPON is restricted to 1 Gbps. There is 10-EPON/10-GPON too but as of writing this article, they are prohibitively expensive. 10-EPON is backwards compatible with GEPON, which is gaining more traction compared to the 10-GPON so keep that in mind. With GEPON you can also do a hybrid by putting in active devices or POE powered ONUs so keep that in mind. GPON supports direct TV content, without relying on IPTV. Both use Wavelength Division Multiplexing (WDM) to have uplink and downlink over one fiber strand and Time-division multiplexing (TDM) to divide upstream data. Generally GEPON devices are cheaper as compared to GPON.

Spiliting and power levels is something you will have to be aware of. Get a nice power meter to check for power levels. OTDR may be also needed. You can get one later if it is too expensive for you at this stage. Explaining how power levels work may be beyond the scope of this article. Hopefully the above slide may help you out. AFAIK PON equipment is mostly SC/PC and you will need single mode fiber. RX/TX of either ONU or OLT added onto the opposite of the other will give you the total power. Eg. for upstream if TX of ONU is 3 dBm and RX sensitivity of OLT is -27 dBm, then you have 30 dBm of power to use max. People usually prefer to do 1:32 split for <= 5 kms and 1:64 for <= 20 kms.

Not doing last mile at all

This is what most providers end up doing since the last mile is a major pain. Instead you can just sell your bandwidth and billing services to smaller ISPs or LCOs(Local Cable Operators) and they will do the last mile for you and take a cut. However to get to that stage you will need to have a network of your own since without major investment or cashflow, this is just difficult to do. Running an ISP needs money and cash to flow in since this is a business where economies of scale really matter. You can use a NLD(National Long Distance) link from a teleco operator or simply a wireless P2P link to deliver the circuit to a smaller provider if the distance is large. For a smaller distance you can use fiber. A healthy mix of both is nice, since the smaller providers can always buy the circuit directly from a teleco if they grow large enough.

The middle mile

This is a place where wireless really can be helpful. Ubiquiti sells Airfiber radios which are very nice for these scenarios. Yes you should really try to build out fiber but over long distance this can simply be not possible. Millimeter wave radios are nice but they are illegal in India and not easily obtainable.

You can also use NLD links for this. These are basically point to point links usually sold by a telecom operator like Airtel, Tata, Reliance Communications, Vodafone. Unless you have serious distance between which you want to transport data, do not have a look at this. Powergrid also sells these in India for a good price. The pricing? ₹200-300, depending on capacity you want and how you negotiate. Always negotiate. Everything in this business depends on your negotiating skills to get the best price possible. Everyone is ready to budge. You however will not be needing any kind of NLDs in the beginning, unless you are selling bandwidth to someone else.

Network Operations Center

The Network Operations Center or the NOC will be the central point in your network. It is where the bandwidth will be coming in and you will be doing the billing or shaping traffic. I will be using this topic to talk about these things in a fairly detailed way. Overtime you will need smaller Point of Presences(POPs) away from your main NOC. You can use teleco buildings to keep your equipment to keep your equipment in. You can pull fiber from there.

Bandwidth and buying IP space

You will need to get bandwidth from the nearest teleco site from where it is delivered to your NOC. I would recommend using two fiber installations with different paths or fiber plus wireless for redundancy for this. I will talk about regulations and from where to buy the bandwidth from in later sections.

About the choice of routing protocols at the core, you can either run BGP or use static routing. For anything serious, BGP is a must. You can then announce your own IP space and can have multiple uplink providers and peer with content providers. Peering essentially means you exchange traffic for free. About 40-50% of your bandwidth would be to Google. Wouldn’t it be amazing that you can get all that for free and get better latency for your users? Well you can by peering with Google. You will need to get a NLD link to one of their POPs and peer. My friend has written an excellent article on the content providers to peer with here. Do note that you will need some capacity before the other party can agree. For Google that is 100-150 Mbps. Also do note that the datacenter provider that any company you want to peer is present that may charge you for keeping your router/switch there and for the interconnect. GGC(Google Global Cache) may be better in the beginning for caching content. Redundancy for NLD link may be problematic too. You can also have a look at NIXI for peering.

All I’m saying is that running BGP gives you so much more flexibility and self reliance than just depending on someone else for IPs and bandwidth. You can say run Airtel, Tata links for IP transit(essentially internet connectivity) and peer for regional routes so essentially you have to pay for less for bandwidth and get better routing and redundancy.

For IP space you will have to get this from your upstream if you are doing static routing. However it is simply a better idea to get this from the local internet registry – APNIC for India. The costs however are prohibitively expensive. Luckily we have IRINN for India. The costs are a very low ₹25000 initial application fee and about ₹45000/year for a /22 IPv4 block, with free IPv6 blocks. That’s 1024 public IPv4 addresses. So ₹3.66 per month. That’s cheap! They are still giving out /22 IPv4 block for new applicants and /32 IPv6 block if you try hard enough. You will need an ISP license and a Pvt. Ltd. comapny to get IPs from IRINN. For dynamic BGP routing, you can announce them yourself or for static routing your upstream will have to announce them for you. If you need to run NAT for more IPv4 addresses, do make sure you keep logs since that’s mandated in your ISP license. More on that later.

Contention Ratio (or over subscription ratio) is the number of users sharing the same data capacity. Many people try to think too much about contention. Don’t. What you should be looking at is the peak bandwidth usage. Whatever you do, do NOT oversell your line. Less bandwidth is a good problem to have. It means you can start getting more and buying more is cheaper. While overselling may get you more customers, they will likely not stay. The relationship is very clear:

Less capacity>Buy more bandwidth>Cheaper prices with more capacity and better contention>More profits.

Also as you grow, keep one DDOS protected circuit from a transit provider. They usually cost about 30% more but can come in handy.

Billing software

You will need to have some kind of billing/subscriber management/IP address allocation/payment collection management service. Most of the solutions use RADIUS to interact with the router. You can ofcourse build your own with FreeRADIUS as a base, but that is too cumbersome when there are ready made solutions available for not a lot of price. Synnefo, Log2Space and SSRMS from Height8 are some of the more reputable and larger ones. Synnefo and SSRMS also provide a cloud hosted solutions. Otherwise you would have to host it yourself on a server. They also are RADIUS based, unlike the other meaning they can directly interact with your router. Pricing is usually ₹10-15/customer/month depending on how you negotiate and what kind of features/services you may want. I do have Synnefo’s PDF with me. You can have a look at it here. Ask for a demo before making your choice.

Router/switch choice

For router Mikrotik is an excellent choice for a small-medium sized ISPs. You can run BGP, PPPoE(for subscriber management), VLANs etc. all on the same box. Do remember to use PPPoE and VLANs for keeping broadcast domains different since putting everything in a bridge is a recipe for issues as you grow. Get something from the Mikrotik’s CCR line. You can also run your own x86 hardware or a VM to run Router OS.

For the switch, any managed switch would do. However Cisco’s SG300 switches are super nice and reliable. Also keep spare equipment and cheaper 10/100/1000 unmanaged switches in hand just in case you may need them some day. Doesn’t hurt to have them.

For NIC cards in servers always use Intel whenever possible. They work very reliably and are not very expensive. I will be playing around with some cheap cheap Chelsio and Mellanox ConnectX2 cards for 10G. I’ll keep this updated with the results for that. But generally you shouldn’t be needing 10G at this stage.

Traffic monitoring and DNS

I am a big fan of using open source software whenever possible. The software is higher quality, there are no costs other than managing it and overall the experience is much better. You can either run a DNS recursor to send queries to the root server or run it in forwarding mode to say Google. Whatever you do do, do not use third party non-upstream DNS servers. Most DNS servers do not have EDNS support and that can break CDN support. You can run Unbound or PowerDNS for the DNS server. You can use Pi Blocker to block off ads for customers who want it, for convenience and saving money off your bandwidth bill.

You will need some kind of traffic analysis to get an idea of what and how much the traffic in your network is. There is again an open source solution available. You will simply have to port mirror from your managed switch to the appliance that you build or a VM. Ntop is a nice solution. I believe their ntopng can work out nicely for you, which also has nDPI for deep packet inspection, which can tell you what quantity is of traffic is flowing where.

LibreNMS is something you can use for monitoring all of your devices. Uptime Robot is nice for uptime status pages.

Web/torrent caching servers

Web caches are pretty much obsolete these days. There is not much unencrypted traffic which can be cached by a HTTP transparant proxy. Do not fall in the trap of torrent caches since most of them heavily rely on PBR or Policy Based Routing. Without a router that can do routing decisions on ASIC, this can lead to CPU spikes on your router and overall is a terrible solution. If you do want to proceed, I’ll be listing out the vendors for such caches in later sections. Instead peering is simply a better solution.

For caching Steam downloads, Steamcache is an excellent solution that you can look at. It is completely free and doesn’t rely on PBR. For torrents, rolling out your own solution with BEP22 may be a good idea. Not only does this not rely on PBR, it can cache encrypted torrents as well. Integrating all of this with your billing software could be a challenge.

Also do not do much traffic shaping. This can generally lead to poorer performance and you are essentially trying to solve a non-technical problem with a technical solution. The solution is simply to buy more bandwidth.

Power

Redundant power at all sites will be very important. You can either get an inverter with sufficient capacity, or get an Online UPS, which will isolate and protect your equipment as well. Make sure you have enough capacity to last atleast 10 hours in case of a power outage. I have seen people run huge infra without inverters at places with reliable power so YMMV.

Regulations

It’s no secret that over regulation is hurting India and this industry is no exception. Your first priority should be getting a license. This makes your operation more legitimate and means that you can get cheap IPs and cheaper bandwidth since you can deal with the more legitimate players and teleco companies directly, instead of doing shady dealings. Only reselling someone elses pre made plans is allowed if you do not have an ISP license. Smaller upstream players can budge though, however you are less likely to get a good deal.

Getting your ISP license

The initial cost for getting a license is described as below.

Service	Min. Equity	Min. Networth	Entry Fee	PBG	FBG	Pro. fee
ISP “A”	Nil	Nil	0.3	2	0.1	0.005
ISP “B”	Nil	Nil	0.02	0.1	0.01	0.0015
ISP “C”	Nil	Nil	0.002	0.005	0.001	0.001

All prices are in ₹ crores. ISP A license is a PAN India license. ISP B is a Telecom circle/Metro Area license, which usually means for the entire state. ISP C is a SSA(Secondary Switching Area) license. For getting the license you will either have to file the application yourself(talk to the Department of Telecom(DOT) guys) or find a middleman who can help you out for a small cut. You may have to pay 8-12% of your AGR as fees every year, although that is up in court. The later is usually preferred since the process can be tedious.

This will be a huge chunk of your work starting up. After this you can go apply for IP resources from IRINN and get cheaper bandwidth.

NAT logs and lim devices

DOT needs you to keep NAT logs so make arrangement for this. Further there is a requirement for lim devices, but this is often not enforced. From what I know, they will require you to run a machine or a VM with their monitoring software and port mirror to that. So keep this in mind.

Importing

Importing things is a huge hassle in India. Buy things locally as far as possible. Importing in personal capacity upto ₹50000 shouldn’t be a problem, however for companies you need a license. DHL, Fedex and some other cargo carriers can do customs clearance on behalf of you so do explore that as well.

Vendors

Getting where to get stuff from for a good price will be crucial. I have a list of vendors which I will be sharing. If you are a vendor and reading this, let me know and maybe I can add you to the list.

• Fiber: Aksh Optifiber, Sterlite or any good provider near your area. Transporting fiber is expensive. Do keep that in mind. I got a quote of ₹7.8/m for 4 strand, ₹8.4/m for 6 strand, ₹9/m for 8 strand and ₹10.5/m for 12 strand from a quality local supplier.
• Fiber splicer: Simimoto or Fujikura are good options. Cost: ₹1.5-2.5 lakhs. I don’t remember the name of the vendor in India.
• PON equipment/general Mikrotik and Ubiquiti products: Tara Consulting Pvt. Ltd., JRS communications and Multilink Computers Pvt. Ltd.. Tara is better with the pricing initially, while JRS is better if you can negotiate. Tara’s OLT/ONUs are very good from what a friend told me. The pricing is nice too. Syrotech also deals with GEPON equipment. For Huawei OLT/ONUs importing would be your only good option.
• IP transit: Smaller players for upto 25 Mbps will cost you like ₹1000-1100/Mbps if you negotiate well. With a license STM-1(155 Mbps) should cost you ₹700-800/Mbps. The pricing is dropping all the time. If you do have a license, Airtel and Vodafone are good options for a few STMs of capacity. You may get a better deal with other Class A ISPs. Don’t bother with Tata unless you have >1 Gbps of capacity. As you grow, keep two pipes from large players and get rest of the capacity from peering. If you can’t find a good deal, do contact me. Maybe I can help.
• IP addresses: IRINN or APNIC. Maybe your upstream provider can give you some but generally this is not a good practice.
• NLD: Powergrid is cheap. Also talk to the telecos. Cost: ₹200-300.
• Peering locations: GPX Mumbai, if you are close to Mumbai. Otherwise look for something regional. NIXI is a good option too. Iifon is operating an exchange at Agartala I think.
• Torrent caches: Xtraband’s XCache2 and Extreme Peering. Cost: For XCache2: ₹60000/year for 40 Mbps of capacity. I also have XCache2’s brochure here. For Extreme Peering: ₹300/Mbps for 100 Mbps, ₹128/Mbps for 1 Gbps of “effective saving” per month. Either is not recommended if you do not have a router with ASIC to do forwarding.
• Server equipment: Build your own from commodity hardware or buy ready made hardware. Some AMD motherboards support ECC RAM so that should be nice. You can also try importing from Ebay as well. Run VMs on something like Proxmox to save power and money.
• Billing: Synnefo, Log2space and SSRMS. SSRMS is what most larger players use.
• ISP license: I think Multilink can help you out in this. I don’t know about anyone else.

Some tips

Do keep in mind that larger telecos have plans of outsourcing last mile to LCOs and making plans/doing billing themselves. Also you need to have good financing since this is a business where you have to pour in a lot of money for building the network, before you start to see any real kind of revenue flowing in.

For India last mile ISPs are very important. The remaining infra is already there. The landing stations are in place and the telecos have laid fiber till the city. Now getting fiber to the homes is the next step. As more people use the internet and more capacity is being used, bandwidth prices will keep dropping.

Also I would suggest you to use some forums for reference:

• Ubiquiti forums: The business talk section is excellent for asking questions/looking at what problems people are solving.
• DSL reports forum: Another excellent site for generally exploring. Their ISP section is nice.

Well that’s it for me. This ended up being longer than I expected. Happy network building to you! 😀

Feel free to contact me here if you have any queries or simply leave a comment down below.