exadata single and two rack IB Fabric

this blog provide IB fabric diagram for single and two rack IB Fabric

single rack

single racktwo rack

two racks

 

Observations:

single rack

  • leaf switch1 11 to port1(db1-4,cell1-7) and 11 to port2 (db5-8, cell8-14)
  • leaf switch2 11 to port1 (db5-8,cell8-14) and 11 to port2(db1-4,cell1-7)
  • leaf switch has single connection to  spine and 7 links to each other
  • keep in mind that current pci-2 x8 can only support QDR with active/passive with two port HCA
  • assume that we do LB  between two leaf switches, the blocking is (1-(8/11)), latency is 7 links with single hub and single link with two hubs,
  • these means 4 DBs and 7 cells talk to each other with non blocking and single hub latency but talks to each others  with two or three hub latency and  .27% blocking

two racks

  • remove links between leaf switches
  • each leaf switch has 4 links to spines
  • 4 group of ( 4 db and 7 cell) just like single rack talk within group with one hub  and to each other with three hubs latency, block still (1-8/11)=.27%

 

 

About laotsao 老曹

HopBit GridComputing LLC Rockscluster Gridengine Solaris Zone, Solaris Cluster, OVM SPARC/Ldom Exadata, SPARC SuperCluster
This entry was posted in exadata, Infiniband, oracle. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s