above.net's co-location service

reviewed by Philip Greenspun for Web Tools Review

One of my photo.net readers offered to help my attempts to support a local animal charity by hosting a Web site for them out of his house.

I thought "oh great, a pathetic loser with his girlie-man 56 Kb modem thinks he can host a Web site from his basement." So I asked "What kind of bandwidth do you have to your house?" "Just a T1 line. It goes to my friend's house. He has two T1s plus a high-speed RF link."

Then I noted the "cisco.com" domain on Javier's email address and realized that the West Coast is pulling away from Massachusetts at a faster rate than I'd expected.

Eventually Javier and I got to talking and he said "you should park your server with my friend's co-location service. Those guys really understand networking."

When an engineer at the world's leading router manufacturer tells you that someone understands networking, that's a pretty strong recommendation.

So I looked into the unfortunately named "WebCondo" service at above.net and talked to their tech people. What kind of connections did they have?

"We have a 100 Mbit connection to MAE-east and, as I'm sure you know, MAE-west is split into two halves and the halves are only connected by a congested OC-3 [150 Mbit] wire. We don't want our traffic getting stuck in that OC-3 so we have two 100 Mbit connections, one to each half of MAE-west and we route packets accordingly. In the long run, though, the name of the hosting game is private peering so we're setting up as many of these arrangements as we can."

I decided that I would never again try to impress anyone with MIT Net's puny T3 connection to the Internet and 100% reliance on BBN Planet.

Can you maintain a machine without physical access?

Anyone who asks this question, e.g., me, is obviously out of touch with modern technology. If you're running Unix and your machine is healthy enough to talk to the network then it doesn't really matter where it is. But if your hardware itself gets sick then what you want is a computer with an on-site service contract (e.g., HP, DEC, or Sun) and some way to talk to "the console". If you have a wimpy computer, you're probably used to talking to the console via the built-in video card and monitor. If you have a real server (e.g., my HP K460), you'll find that the machine has no video card at all. The only way you can talk to it is via a serial line (my K460 came thoughtfully provided with a genuine HP-brand ASCII terminal).

The above.net people very cleverly figured out that they can use a terminal concentrator in reverse so give their customers access to the backs of their machines. Normally terminal concentrators are used by ISPs serving modem customers. You dial in and then you telnet out to the Internet. But above.net runs these in reverse. You telnet in and then connect to the serial cable plugged into the back of your Unix box.

For people with really flakey operating systems (e.g., NT), the above.net people have a service whereby their equipment will automatically power cycle a machine that won't respond to an ICMP ping.

Finally, the above.net folks have someone physically present in their facility 24x7. In theory these people could type something on a keyboard attached to your server but if this is happening then IMHO you've already failed in myriad ways. These folks are potentially useful to load CD-ROMs and change backup tapes (they'll do it every day or once/week).

Putting them to the test

I parked a Sun Ultra 2 at above.net. The machine has a nasty habit of catastrophically failing every month or two. Sun service comes in and replaces the motherboard, the power supply, the internal disk drives, etc., and walks away pronouncing the machine cured. Then it dies again 6 weeks later. Sun service policy precludes them from swapping out an entire machine even if they haven't got the slightest clue as to what caused a failure in the first place. This policy is kind of annoying if you're depending on a Sun server. But it is perfect if you want to stress test a co-location service like above.net!

One month after we parked our machine at above.net, our Sun reliably came crashing down. Within about 30 seconds, we got email from above.net's monitors complaining that our machine wasn't pingable. It wasn't until two minutes later that my Uptime service started complaining. We were able to telnet in and use above.net's remote console facility to get the machine back up. We were able to schedule a visit by Sun service to come into above.net's facility and replace another big pile of hardware. (Then the next day, the machine failed again and was down for 13 hours until Sun finally came by to swap out the parts they'd swapped the day before. But that's another story in Chapter 6 from my Database Backed Web Sites book.)

What about the Bandwidth?

Try tracerouting or pinging to www.arsdigita.com and see what kind of delays you get.

What does it all cost?

Prices vary depending on rack space and bandwidth consumed. I think you could get away with spending as little as $400/month for a small machine and a really huge honker (4-8 CPUs, 20 or 30 hard drives) would probably cost no more than $1500 or $2000/month.

What about the competition?

Above.net only has one real competitor: Exodus. Exodus has a very good network and, if you have the money to pay them, offers a broader range of services than above.net. For example, Exodus will set up a firewall for you, help with Unix and Oracle administration, things that simply aren't part of above.net's business. As far as I can tell, above.net has automated more of their procedures, e.g., power cycling. That makes that they can charge lower prices and still make a profit. It also gives me confidence that their service won't degrade as they expand and hire new people.

What about other options?

If you're interested in a larger view of the general problem of configuring and maintaining a Web server, you might wish to read the server chapter of my book on Web publishing.


philg@mit.edu

Reader's Comments

It's my understanding that Above.NET has continued to improve its connectivity, and now have a private OC-3 across the country all the way to their own router peering at MAE-EAST, as well as more high speed links within the Bay area to other major peering points, such as CIX.

-- Javier Henderson, December 12, 1997
Above.net has raised their prices since you wrote this page (not that its your business to track them). It currently costs ~$400/mo. for a 12" shelf, plus charges for tape changing, bandwidth, power cycle ports, and power (!).

-- Lee Schumacher, July 6, 1998
Having an OC3 to any of the MAEs or NAPs is a lot like having a 45 lane superhighway to your local mall parking lot a christmas.

A good host will have multiple connections from the primary backbone providers (MCI/C&W - UUNet - Sprint) and use MAE/NAP connections for that occasional rogue packet to a second or third tier provider.

All of the primary backbone providers now pull direct access to/from each other to avoid the bottlenecks of the MAEs/NAPs that were causing a majority of thier routing problems.

-- Jamie Curtis, October 22, 1998

For NT4 with IIS4, ICMP ping is not enough to determine if the web server is running, neither is querying the service running state. If you are hosting multiple virtual servers on the same machine it is possible for IIS to stop responding on any address that you are using independently of the others. You should always test the HTTP connection, not just the machine networking.

-- John Donnelly, May 20, 1999
I just tracerouted ArsDigita as Phil suggested, and it produced an interesting result... aD has jumped AboveNet's ship and switched to the Exodus bandwagon (which seems to house every major website in the world these days). Tell us it ain't so, Phil! Or at least tell us why, if you get the chance to update this page... to the layperson (i.e. me), it would seem that AboveNet (having 380 peering agreements to Exodus' 250) would be the superior solution on a pure connectivity basis. Is aD using some of the "value-added" services that Exodus offers and AboveNet does not?

On an unrelated note, I wanted to point out to this community that the AboveNet colocation service starts at around $1300 now, as a result of a new pricing structure that doesn't dip below 1 mbps.

-- Ben Lunsford, April 20, 2000

Big Hosts may help anyone who doesn't have the $1000+/m to pay AboveNet or Exodus. It is a brand new resource and hasn't caught on with all the advertisers yet (which make the spotlights work), but there are some good articles from a layperson perspective for one to peruse. Tell me what you think by emailing me. Good luck with your search for a good colocation (or dedicated server) provider.

-- Ben Lunsford, June 29, 2000
Actually the person that commented about Above.net in May of 1999 not being less than $1300/month because of new pricing structure is wrong. Unless because of being an old client from early 1997 until late 1998, moved servers for a few reasons at that point and then 9/20/1999 moved back to Above.net, the min. cost was in 9/20/1999 ~$400/month, 12" high x 19" rack, power, and 128k of 95% average bandwidth usage, billed at $1.50/1kbps used over the 95% usage all measured via MRTG doing the averaging (ie: they cut of the top 5% of your bandwidth peaks, and the next highest peak is your 95% usage level.)

You may want to directly contact Above.net if you have needs of less than an average peak usage of 1mbps (now remember these peaks are also a 5 min average bandwidth usage.... NOT true bandwidth usage patterns too... alittle confusing.) Great service, great support, and SUPER network infrastructure... I would talk to them if you can, as I really think you can get in for way less than $1300/month if you need less bandwidth. As a side note: Our servers are still there, as we add more and more, and the server by their tech staff is as good now as it was in early 1997 when it was such a small operation compared to the huge operation they are now... just keeps getting better.

-- Nevin Lyne, March 23, 2001

Nevin, I made that comment and I made it in April 2000, not May 1999. You would have been right as of May 1999, but as of April 2000 I was right and you were actually mistaken in your "correction". :o( AboveNet used to start at about $480/m at the date you rejoined, but was $1300/m minimum as of April 2000 and about that as of summer 2001 as well. It's not that they increased their prices across the board, but rather that they won't dip down to 64 or 128 kbps anymore for smaller customers... they simply start their services at no less than 1mbps. Which is certainly their choice as a private business. BTW, I imagine that old competitors like AboveNet and certainly the "Colo 2.0" folks like Equinix will benefit from the recent financial collapse of Exodus. Time will tell! Good luck to every entity in the industry.

-- Ben Lunsford, September 27, 2001
That's a nice resource (Big Hosts). Very satisfied with dedicated webhosting I found through there last month (from Servers On Demand). AboveNet (along with SiteSmith) is now MFN, by the way. Still a good colo service, but still runs expensive (not as much as before).

-- Matt Walsh, October 3, 2002
Actually we jumped ship from above.net, for a number of reasons, but we are now at www.he.net. Super people working there, the Fremont datacenter is 43,000 sq ft. They are privately owned, profitable, (ie: they expand when needed and cash allows not blow billions only to file bankruptcy...cough MFN...) they have smaller plans down to I think a couple of U of rack space + 128k 95% percentile bandwidth usage for $200/month.. but for the best in security and if you have a few servers at least... a 7' locked/private cabinet is $400/month + the more bandwidth you need the cheaper it gets... I think its $200/month for 128k but $500/month for 1 megabit... need more than one 7' cabinet? if you get 2 or more the price drops to $300 each/month... So you could have say 2 x 7' private cabinets + 1mbps of 95% bandwidth for $1600/month. Was very impressed by the staff, the support, the data center, their network, and the pricing. All told we spend more than 5 years at Above.net but could not justify their pricing over what others are offering for actually in my mind better options as well...

Just my opinion...

-- Nevin Lyne, October 4, 2002

I've been colocating at AboveNet for about 10 years and I never had any problem. I started back in 1986 with 1 ft. of rack space for $400. Now I have one rack that I share with a few friends. I have extra space, if you need to colocate very cheap at one of the best, drop me an email or check http://colo7.com/


Image: AboveNet4.gif

-- Mihai Beffa, February 2, 2006
Add a comment

Related Links

Add a link