[General] Internet Slowdown

Wed Jan 30 23:22:35 +03 2008

Funny how we are all slaves to the might Internet and its mistress 
Bandwidth. Does anyone remember the days of 2600 BBS? :)

A ship's anchor is not a light beast; depending on the size of the boat, 
it might have one, two or four of these giant anchors. Anyway; what 
happened is that normally submarine pipes are marked in navigational 
maps so things like what happened in Egypt don't happen.

However, by some stroke of luck, the authorities shifted the locations 
where boats may berth (ie, park) and drop anchor. So what happened is 
one of these anchors damaged the covering over these submerged fiber 
optic cable 'tunnels'. As I'm sure you know, fiber optic uses light 
pulses to transfer information and the cables are actually very fine 
shards of glass -- the point being, not the most sturdy of objects.

The problem with the backbone is that it takes a lot of investment to 
plan, build and execute a submerged 'master link'; and most of the 
tier-1 bandwidth providers (like UUNet and AT&T) have already invested 
in linking major hubs via submerged cables. The reason? Its cheaper and 
faster than terrestrial (ie, above ground) and sat links -- not to 
mention fast; to make matters worse, these bandwidth providers are not 
the people that actually build these links, they are other companies so 
you can imagine the phone calls that are flying around right now.

Everyone buys upstream bandwidth -- even telcos like AT&T, Du, Etisalat, 
MoC in Kuwait, QualityNET, FT, etc. So when a link like this is 
compromised, it affects everyone from top down.

The reason the net is very slow right now is because the providers have 
shifted to their backup network (you will probably see some marketing/pr 
laced "report" mentioning how _____ telco is able to provide access 
because they have invested in next generation backup links blah blah 
blah)...anyway, the point is -- when you can't go underwater, you go 
above -- as in sat links with terrestrial base stations.

Satellite bandwidth has two problems -- giant latency (its not good for 
latency sensitive operations); its expensive (because of the nature of 
the technology) and requires additional systems -- terra stations; like 
the ones you see at the MoC centers (where there is a giant dish that 
moves depending on the time of day).

Satellite bandwidth is sold based on what's called a 'contention ratio' 
-- how many other people are sharing the access with you. To give you a 
comparison, typical home DSL is on a 10:1 contention ratio, meaning that 
any given time, there are 10 other people fighting over the same down/up 
pipe. Satellite bandwidth *starts* at 20:1 ratio, and the lower the 
ratio, the more expensive it gets. Of course, the lowest is 1:1 -- which 
means its you and some other lucky duck sharing the pipe (channel) at 
any given time.

So what is happening now is that routes are shifting (slowly) to 
satellite outbound and inbound. You will see lots of dropped packets 
because the channels are flooded; and eventually -- what goes up, must 
come down meaning that the signal must travel over (or under) ground to 
reach the DC where the server you are trying to access is.

Its a snowball effect that will affect systems in ME, Asia and 
eventually Europe and main net "hubs" like NY.

To fix such a problem, they have to send a repair ship, a dive team to 
assess the damage, then to do the actual repairs (underwater, mind you), 
then test connectivity between the two booster stations, and then 
restore bandwidth slowly. Even if they manage to fix it in one day, it 
will be flooded from all the 'back traffic' that is queued.

In summary -- a few days of patience are required :)

bashar abdullah wrote:
> Thanks very much Burhan for this informative, though bad, news. I 
> noticed it was too slow could kill you. I am just wondering, why do we 
> have to be under the mercy of those stupid people in Egypt where they 
> park there ships. Shouldn't first of all the cable be more preventive to 
> damages, as this is seriously not the first kind of problem we faced in 
> last 2 years. 2nd, Shouldn't the whole backbone of the internet inside 
> the middle-east have more robust redundant and high availability 
> infrastructure?
> 
> It's funny we talk about disaster side at work when the whole business 
> is pointless when a simple idiot mistake like that can harm the whole 
> middle-east, or in fact, the world!
> 
> Network is not my area, but if you can shed some light on those things, 
> I'd be more than grateful. I know high cost of redundancy under water is 
> one thing, but losing the whole internet is definitely more serious. I 
> can hardly call this internet really. A dial-up account works better 
> than my 512 right now.
> 
> Thanks again
> 
> On Jan 30, 2008 9:24 PM, Burhan Khalid <burhan at kuwaitnet.net 
> <mailto:burhan at kuwaitnet.net>> wrote:
> 
>     Salaam Everyone:
> 
>     As I'm sure you all have noticed; the Internet has been anemic today,
>     and for good reason. Two submarine fiber optic cables were damaged
>     affecting the Middle East and most of Asia. You can quote the following
>     in case you are being hammered by complaints. The local ISPs are aware
>     of the issue (well, at least FT with whom I spoke).
> 
>     "A submarine cable in the Mediterranean was cut earlier today, resulting
>     in a dramatic slowdown in internet access for people in India, Sri
>     Lanka, Pakistan and much of the Middle East.
> 
>     A spokesman for Flag Telecom, the owner of the severed cable, told the
>     Reg: "It is a problem off the coast of Alexandria in Egypt. For some
>     reason ships were asked to anchor in a different place to normal - 8.3km
>     from the beach. One of the ship's anchors cut our cable but there are
>     multiple cuts - we're not the only company having problems."
> 
>     He said they were in the process of getting a repair ship out to assess
>     the damage but he warned the whole process could take 12 to 15 days even
>     though the ship was in the Med. He said users in India would have a
>     slower internet access as a result.
> 
>     Such major damage to the internet backbone can cause major problems
>     despite redundancy which allows some re-routing. The loss of so much
>     bandwidth is likely to have an impact.
> 
>     A Reg reader told us: "We've got some connectivity to our India office,
>     but it's very flakey (currently losing half the packets) which could be
>     a result of overloading. Is very similar to a couple of Christmas' ago
>     when there was a earthquake near Taiwan and it severed undersea cables
>     causing major bottlenecks on what was left to most of Asia for a couple
>     of weeks."
> 
>     Apart from being serious for the region, the cable break could also hit
>     large UK and US enterprises which have offshored business processes and
>     backoffice functions to companies in India, Pakistan or the Middle
>     East." @
>     http://www.theregister.co.uk/2008/01/30/india_mideast_lose_internet/
> 
>     "International phone calls from the UAE were disrupted and Internet
>     service crawled to a halt Wednedsay when two submarine fibre-optic cable
>     systems "snapped" in the Mediterranean.
> 
>     The cables link the Middle East with Europe, but the slowdown was felt
>     the world over, according to local telecom operators. The disruption,
>     whose cause remains unknown, has wreaked havoc among millions of
>     internet and phone users, etisalat and du said in a statement.
> 
>     Du said two submarine fibre-optic cables, FLAG Telecom and SEA-ME-WE 4,
>     were "damaged".
> 
>     "We apologise to our customers for the inconvenience and assure them
>     that we will keep them informed of the situation on a continuous basis,"
>     said etisalat in a statement.
> 
>     A company spokesperson said they are working closely with the concerned
>     operators for urgent repair work "to ensure the links with major global
>     internet hubs are restored at the earliest."
> 
>     The company said it has activated backup links which meant that normal
>     Internet browsing and essential traffic will continue to work "at
>     reasonable speeds".
> 
>     At the Dubai Media and Internet free zones and the Knowledge Village,
>     web access crawled to halt.
> 
>     "It's a day gone to waste. They're not telling us what time it's coming
>     back.
> 
>     And they're not even sure whether it's coming back tomorrow," a Media
>     City executive said.
> 
>     A du spokesperson told XPRESS the degradation of internet services and
>     international voice calls only occurred for some customers during peak
>     times.
> 
>     "We are working actively with the submarine cable system operators to
>     ascertain the reasons for the cables being cut, and they are working to
>     give affected operators a recovery schedule."
> 
>     Etisalat said it has already started migrating internet and
>     international voice traffic through other cable systems that have not
>     been affected.
> 
>     "Some congestion may be expected at peak times until this issue is
>     resolved. We will keep our customers updated on its progress."" @
>     http://www.xpress4me.com/news/uae/national/20005528.html
> 
> 
>     _______________________________________________
>     General mailing list
>     General at oskw.org <mailto:General at oskw.org>
>     http://mail.oskw.org/mailman/listinfo/general_oskw.org
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> General mailing list
> General at oskw.org
> http://mail.oskw.org/mailman/listinfo/general_oskw.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3249 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://oskw.org/pipermail/general_oskw.org/attachments/20080130/01e05a53/attachment.bin>