Some observation of IB Fabric@exalogic

Oracle release the exalogic system (EL x2-2)  on the web, quick read and some observations on the IB Fabric (some may be in complete or in correct):

  • This is a very complex system
  • involved many new features, e.g. IB gateway
  • IB topology is a mesh network, the default with copper cables could create messy cabling and short distance between switches means many cable bending
  • even in single rack, IB connections could involved 1, 2 or 3  switches
  • from single rack to multiple rack, spline switch is added,  the IB connections is even more complex and could involve >3  switches
  • when connection involve multiple switches, it increase latency even it does encounter blocking in fabric
  • from the attached diagram , the storage server port share the same switch with servers 1-16, but need at least two hops for server 17-23
  • IMHO,

  1. the storage port 2 should connect to leaf#3 and the server distribution should be 15 and 15, not 14 and 16
  2. should create two zpools, one  for 1-15 and one for 16-30  to reduce the latency and blocking

About laotsao 老曹

HopBit GridComputing LLC Rockscluster Gridengine Solaris Zone, Solaris Cluster, OVM SPARC/Ldom Exadata, SPARC SuperCluster
This entry was posted in exalogic, Infiniband, oracle, Oracle EM, Oracle Enterprise Linux, Oracle OC, solaris 11 express. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s