Published On: Wed, Aug 19th, 2015

How Google’s Networking Infrastructure Has Evolved Over The Last 10 Years

Google currently announced that it wants to repair your home Wi-Fi, though internally, it has prolonged been operative on distant some-more formidable networking issues. To bond a hundreds of thousands of machines that make adult a Google information center, we can’t only use a few basic routers and switches. So to conduct all a information issuing between a servers, Google has been building a possess hardware and software, and today, it’s pulling behind a screen and giving us a demeanour behind the expansion of a networking infrastructure.

Google’s stream setup, a so-called Jupiter network, has 100x some-more ability than a first-gen network and can broach a overwhelming 1 petabit/sec of sum bisection bandwidth. The association says this kind of speed allows 100,000 servers to review all of a scanned information in a Library of Congress in reduction than a tenth of a second.

“Such network opening has been tremendously lenient for Google services,” Google Fellow Amin Vahdat writes today. “Engineers were expelled from optimizing their formula for several levels of bandwidth hierarchy. For example, primarily there were unpleasant tradeoffs with clever information locality and chain of servers connected to a same tip of shelve switch contra correlated failures caused by a singular switch failure.”


Ten years ago, however, Google didn’t have anywhere nearby this capacity. This was before it had acquired YouTube and shortly after it launched products like Gmail, Google Earth and Google Maps, so a company’s needs were expected changing really quickly.

Here is what those early 2005 machines looked like:

  1. fh10_board

  2. firehose1.1_chassis

  3. Copy of FH_cabling (1)

As Google writes in a paper today, a association fundamentally still deployed customary server clusters in 2004, and these 2005 machines are a initial instance of a networking it deployed in a Firehose 1.0 information core architecture. The idea of a 2005 machines was to grasp 1 Gbps of bisection bandwidth between 10,000 servers. To get there, Google attempted to confederate a switching fabric right onto a homebuilt servers, though it turns out that “the uptime of servers was reduction than ideal.”

With Firehose 1.1, Google afterwards deployed a initial tradition data-center cluster fabric. “We had schooled from FH1.0 to not use unchanging servers to residence switch chips,” a company’s engineers write. Instead, Google built tradition enclosures and changed to a supposed Clos design for a information core networks.

By 2008, Firehose 1.1 had developed into WatchTower, that changed to regulating 10G fiber instead of normal networking cables. Google rolled this chronicle out to all of a information centers around a globe.

Here is what those racks looked like:


A year later, WatchTower afterwards morphed into Saturn. While a WatchTower fabric could scale to 87 Tbps, Saturn scaled adult to 207 Tbps on denser racks.

  1. Saturn_chassis

  2. WTbundles

  3. Pluto.png

Saturn clearly served Google well, since a association stranded with it for 3 years before a needs outstripped Saturn’s capabilities.

“As bandwidth mandate per server continued to grow, so did a need for uniform bandwidth opposite all clusters in a information center. With a appearance of unenlightened 40G able businessman silicon, we could cruise expanding a Clos fabric opposite a whole information core subsuming a inter-cluster networking layer,” Google’s engineers write.

It’s that kind of design that now allows Google to provide a singular information core as one hulk computer, with program doing a placement of a discriminate and storage resources accessible on all of a opposite servers on a network.

The Jupiter hardware certain looks opposite from Google’s beginning attempts during building a tradition networking gear, though in many ways, it’s also a fact that a association fast adopted Software Defined Networking that authorised it to fast innovate.

Google currently expelled 4 papers that sum several aspects of a networking setup. Because Google tends to strike a boundary of normal hardware and program architectures prolonged before others, identical papers have mostly spawned a new spin of creation outward of a company, too.

It’s doubtful that all that many startups need to build their possess information centers, though other information core operators will certainly investigate these papers in fact and maybe exercise identical solutions over time (which in spin will advantage their users, too, of course).


About the Author

Leave a comment

XHTML: You can use these html tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>