PDA

View Full Version : Simply amazing! (google)



nAz
12-10-2004, 09:38 AM
The magic that makes Google tick
By Matt Loney, ZDNet UK
02 December 2004


The numbers alone are enough to make your eyes water.

# Over four billion Web pages, each an average of 10KB, all fully indexed.
# Up to 2,000 PCs in a cluster.
# Over 30 clusters.
# 104 interface languages including Klingon and Tagalog.
# One petabyte of data in a cluster -- so much that hard disk error rates of 10-15 begin to be a real issue.
# Sustained transfer rates of 2Gbps in a cluster.
# An expectation that two machines will fail every day in each of the larger clusters.
# No complete system failure since February 2000.

It is one of the largest computing projects on the planet, arguably employing more computers than any other single, fully managed system (we're not counting distributed computing projects here), some 200 computer science PhDs, and 600 other computer scientists.

And it is all hidden behind a deceptively simple, white, Web page that contains a single one-line text box and a button that says Google Search.

Full story (http://www.zdnet.com.au/insight/software/0,39023769,39168647,00.htm)

Wally_in_Cincy
12-10-2004, 09:57 AM
I wish I understood all that /ccboard/images/graemlins/smile.gif

I had heard they had 100,000 servers. One Saturday morning CCB had about 60 unregistered users, which is unheard of on Saturday. I wonder if they were crawlers. Is that possible? Or likely?

Like I said I don't understand all the nuts and bolts.

Tom_In_Cincy
12-10-2004, 10:07 AM
'crawlers'?
If you mean that people on google or some other search engine doing a search on 'billiards', I guess that would be considered a 'crawler' hit on this site.

I've seen as many as 30 'non-member' veiwers on this site and on Inside Pool's site. All with the same 1st two IP Address numbers. IIRC, Google searches start with a 68.xx address.

eg8r
12-10-2004, 10:25 AM
That is amazing. Thanks for the link.

eg8r

Cueless Joey
12-10-2004, 10:30 AM
That is just mind boggling.
Tagalog is Philippines' main dialect.
There are a few dozen dialects down there.

Singlemalt
12-10-2004, 12:43 PM
<blockquote><font class="small">Quote Tom_In_Cincy:</font><hr> 'crawlers'?
If you mean that people on google or some other search engine doing a search on 'billiards', I guess that would be considered a 'crawler' hit on this site.
<hr /></blockquote>

Crawlers basically scan a web site for information on what is contained on the site. Now, sites can use a robots.txt file to disable search engines from getting to certain material. This would be a "Disallow: /site folder/info".

None the less it is an impressive operation they have built. Now, you really want some mind boggling cluster information then read some articles on what little bit the NSA allows out on the super computing they do.