August 20, 2005

PAKISTAN’S WORST internet connectivity nightmare became a reality on June 27 when its sole link to the outside world developed a fault, bringing to a halt all the online activities of businessmen as well as home users. Full connectivity was restored 12 days later.

This is just too long for an internet breakdown to last, especially when we want to promote our country as an attractive outsourcing and software development destination. I agree that a lot has already been written on the subject, but there are still some questions that need to be asked.

However, before I move on to the lessons that should be drawn from the fiasco, let’s discuss its reasons. As you already know, the undersea cable called SEA-ME-WE-3 (South-East Asia-Middle East–Western Europe-3) is managed and looked after by a consortium of 50-odd companies or countries. The 39,000km fibre-optic cable is the world’s longest and has about 39 landing points in 33 countries, in Europe, Asia-Pacific and Australia.

From this undersea cable, a segment extends towards Pakistan which servers as the country’s only terrestrial link and which is supposed to meet our entire internet needs. The SEA-ME-WE consortium is headed by Singapore Telecom (SingTel) and the Pakistan Telecommunications Company Limited (PTCL) is one of the operations and maintenance contractors for this cable system.

Interestingly, Pakistan made an initial investment of $35 million in the project and it still pays a hefty monthly maintenance payment of about Rs12.5 million. Where does this money come from? Needless to say, this amount comes from the citizens of this not-so-rich country.

Shouldn’t Pakistanis then expect an uninterrupted access to the internet? Or should we settle for the poor Quality of Service (QoS) that we usually get from the consortium? No, of course not. I think the consortium should be taken to task by the PTCL.

It has been reported that Internet Service Providers (ISPs), call centres, software houses and businesses specializing in providing international dialling facilities lost millions of dollars in current and potential contracts due to the cable fault. Just after the fault had been removed, the telecom and IT industries reportedly asked the authorities to pay $43 million in damages. But the question here is: Did the PTCL ask the international consortium to compensate for the losses incurred by businesses throughout the country?

This could be done only if effective Service Level Agreements (SLAs) were in place. In developed countries only those contracts that are backed up by effective and measurable SLAs, and which ensure a network availability of 99.99 per cent, are considered good. But, we largely don’t have such agreements.

Thus, PTCL seems to have a point when it says it is not legally bound to pay damages to the local users of the cable system. It follows then that our ISPs and other service seekers should first put in place a proper and clear SLA under which the provider agrees to offer a certain level of measurable service variables — uptimes, downtimes and minimum bandwidth, etc — before starting operations.

Only by using the agreed and discreetly quantifiable methods for the measurement of these variables, can a company legally claim the provider to be in breach of contract. And it should be the SLA which spells out the damages amount. However, we still live in a “demand draft age”. So, currently we merely have some connectivity agreements, and not service agreements.

Let us now look into some more issues which surfaced during the debacle.

Repair options

First of all, the repair work started off painfully slowly. It has been reported in the media that the consortium which runs SEA-ME-WE-3 had not asked SingTel, the consortium’s operating agent, to fix the problem but had instead contacted Emirates Telecommunications and Marine Services FZE (E-Marine) — a submarine cable installation, maintenance, and repair company based in the UAE — for the task.

Although the problem was reported on June 27, the repair ship reached the scene of the fault on July 3. That’s six full days. It is evident that E-Marine’s ship was not prepared to undertake the task in the prevailing climatic conditions as the monsoon hindered its operations. The PTCL’s president, Junaid Khan, had said at the time that weather was a “major challenge” for E-Marine. A question that should be asked here is: Why do we have a weather-dependent emergency maintenance support?

This leads us to another important question. A report put out by the BBC quoted a PTCL chief engineer as saying, “This is a highly sophisticated operation and only specialized companies can undertake such a delicate task under the sea. Pakistan has no such ability.” Isn’t it time that instead of focusing only on software and telecom services, we also develop indigenous repair capabilities. When Pakistan can build submarines, why can’t we invest in cable repair and maintenance too?

Local link

One of the questions that baffled internet users was: How and why did the cable rupture, located outside Pakistani lands, cause local traffic to get affected? In other words, why was the local email system also affected?

The answer lies in the way our emails etc travel. Apart from the traffic which goes through the Pakistan Internet Exchange (PIE), all the data are routed by servers located outside Pakistan, mostly in South East Asian countries and some in Europe and the US. Effective use of the PIE system could have dealt with the problem. (This was actually done to an extent during the crisis.)

The PIE system works like a telephone exchange, making it possible for two parties, which are on the same system, to communicate easily and switching the call to another exchange only when required. But, please don’t get confused with the notion of an inter-city email system.

So, questions like, “Why my email from Karachi to Lahore didn’t get through when only the international system was down” serve little purpose. As long as emails use domains that are served by providers outside the country, even an email sent to yourself will take a global trip!

Backup links

Where do we stand in terms of backup links and what are the options? This is an extremely important question. Well, let’s explore three immediate solutions – backup through satellite, through a redundant sea cable and a land route passing through Indian territories.

Satellite: This was the only backup we had — three 34mbps links initially providing just over 100mbps in connectivity. The capacity was later increased, mostly catering to the needs of call centres so that they may resume operations on a priority basis.

In the presence of just one terrestrial link, additional leased transponders would have certainly helped increase the percentage of backup capability. The initial backup was only 13 per cent of the total, as SEA-ME-WE-3 provided 775 megabytes per second of bandwidth. This percentage needs to be increased, keeping in mind the need and the cost. Note that satellite connectivity is characterized with small delays and occasional bursts, which is bad for call centres.

Sea: We will soon be acquiring our share in the SEA-ME-WE-4 to reduce dependence on a single link. Pakistan has already invested $40 million in the project and expects that additional backup connectivity will be available by the end of the year.

Work on SEA-ME-WE-4 was started after 16 international telecommunications companies had signed the construction and maintenance agreements for a new fibre-optic submarine cable system linking South-East Asia to Europe via the subcontinent and the Middle East. Measuring about 20,000km, the cable is capable of carrying telephone, internet and various other broadband data streams.

It’s among the latest generations of submarine cables, which use state-of-the-art Terabit DWDM technology to achieve ultra-fast connectivity. Major work, involving the laying down of the cable, has already been completed. But still, no timetable has been made public mentioning when this link would be available for use.

Land: Pakistan has been eyeing a land-based fibre-optic link with India through the Wagah border. There have been statements from government officials, detailing the merits of such a system. It has been said that after a feasibility report is approved it would take about six months for this system to become operational.

In a meeting between the Indian commerce secretary, S.N. Menon, and the Pakistani acting commerce secretary, Syed Asif Shah, in New Delhi recently the two countries expressed the desire to set up the proposed link at the earliest. An Indian company owned by Tata group, Videsh Sanchar Nigam Ltd, is said to have offered the link to Pakistan.

A network security specialist, experienced IT auditor and the head of Karachi University’s computer science department, Ayaz Ahmed, says this link will have to be monitored closely once it is operational. He also proposed the possibility of exploring a land link with China. Meanwhile, there’s no official word and timetable on this one too.

It is no surprise that today we are no better equipped than we were on July 27. This is rather understandable, as agreements on links having national importance cannot be signed overnight. Having said that, it should be clear that only when the proposed links are in place will the local users feel more secure about connectivity.

Lessons identified

There’s always a lesson or two that can be learnt for a crisis, any crisis. But since we cannot say for sure if any lessons have been learnt from this one, let’s discuss the lessons that can be identified:

— Don’t be reactive – plan in advance and then act accordingly.

Reaction to all such scenarios are ad hoc and need-based in nature. It’s time to make sure that we come up with planned activities consistently. Unless we come up with investment justification and plan properly, reliability of our systems will be in the doldrums.

Officials in high positions should become more conscious of issues in reliability. Risk management is another forgotten area. Do not blame the PTCL alone, the software houses and call centres should also analyse the requirements and create risk matrices, listing down the relevant issues, likelihood and potential severity.

This will enable them to see beforehand whatever looms large. For instance, didn’t we all know that we had just one link to the outside world which acted as our internet lifeline? Also, operators need to create their own plans. As one IT company representative told this writer, “Our disaster recovery plan was based on another ISP’s connection.”

— Use technology effectively.

What the tools of the huge Network Management System were doing when backup plans were formulated by the bandwidth provider of the country? Had we been using such tools to the best of their abilities, our satellite backup would not have been termed “too little, too late” by those who had stakes in connectivity.

— Allow and foster partnerships with private infrastructure setups.

Private VSAT and other satellite options should be allowed. As somebody commented that even though the breakdown hit the country badly, some service providers, using their illegal satellite connectivity meant for VoIP, managed to remain up and running – albeit at lower speeds.

— Create effective SLAs and use them to the benefit of the consumers.

Proper SLAs between the PTCL and international consortiums, between the PTCL and local ISPs and between the ISPs and user companies should be in place. And this should be the chain of damages disbursement.

In this manner, the people who are responsible to pay damages will pay up and we may see actual amounts getting disbursed under legal obligations. Surely this will also help improve the QoS, since there will be no escape routes for those who violate rules.

— Translate losses properly.

Accurate loss calculation is not possible in the absence of proper risk management and analysis. Due to this, we do not know how much loss was incurred during the crisis.

A detailed mechanism for cost management, associated with various services and their downtimes, should be put in place in a way that losses are calculated in a transparent manner, making estimates acceptable to all parties concerned. This will also facilitate informed decision making.

— Communicate properly.

Even elementary network management courses tell us how to use the media effectively in case of a hack or any other disaster. It’s extremely important to let the world know of the actual situation in a way that saves face and decreases chances of allegations.

In the case under discussion, the PTCL management was indeed answering questions put forth by the media on a regular basis, but there was no official word on the PTCL website. The website’s news section just listed six items, pertaining to the period between March 1 and June 14, and there was no mention of the unfolding crisis.

Presence of up-to-the-minute news items on official pages would have resulted in increased customer confidence and, possibly, lesser number of queries customer services ultimately had to deal with.

Similarly, there was no advert in newspapers to tell the weary customers about what had transpired and how the crisis was going to be resolved. Organizations should, in the best interest of its customers, endeavour to reach out and reassure.

— Treat the crisis as a wake-up call, for other services as well.

The internet has had its turn. What about the other services? Mobile telecom, banks, ATM networks and stock exchanges all should now avoid single-point failures. Regulatory authorities of all the sectors should provide sufficient backup facilities and develop effective disaster recovery plans.

— Lastly, while committing to international agreements, the authorities should properly weigh the merits and demerits of each proposal.

Whenever a crisis unfolds, a blame game soon begins. In the ongoing mud-slinging, one point is invariably missed — that there is always a regulatory body the sole purpose of which is to ensure compliance with standards so that the interest of stakeholders is safeguarded.

Secondly, internet is not an isolated phenomenon. Information and Communication Technologies (ICT) should be viewed as an integral part of society. Investment, maintenance, regulation and accountability of ICT will impact on society as a whole. Until we start taking IT in this manner, we will continue to forget our past fiascos, only to wake up from slumber some day.

The writer works as an analyst for a software house in Karachi

Please Visit our Sponsor (Ads open in separate window)