Posts Tagged ‘Backplane Design’
Backplane High Level Design –the Secret to Success
In a previous design note on Backplane Architecture and Design, I touched briefly on the concept of a Backplane High Level Design (HLD). In this design note, I will touch on key aspects that go into this process, using a simple fictitious system architecture as a straw-man, to demonstrate the principle.
For any new backplane design, I always recommend starting with a HLD. It helps you capture your thoughts, in an organized manner, and later provides the road map to follow for detailed design of the backplane. It also facilitates concurrent design of the rest of the system by the rest of the design team.
I like to use PowerPoint to capture the HLD information, but any other graphical based tool could be used. Later on in the design process, the drawings in the HLD document are reused in a more formal design specification document.
One of the first things I do, when coming on board a project, is capture the system architecture in a series of functional block diagrams starting from the high-level system block diagram, as shown in Figure 1. This is an example of what you might receive from the system architect at the beginning of a project.
Each block diagram details how the respective circuit packs, or other components of the system, interconnect to one another; complete with the number of signal I/Os for that function. For example, Figure 2 below shows the system data path and system control plane block diagrams. It illustrates one possible way of how you would arrange the circuit pack blocks, as they would appear in a shelf, when viewed from the front. Whenever possible, I like to arrange the blocks this way, because it presents a consistent look and feel throughout the documentation; from mechanical views, to connector placement, and route planning.
Preliminary Route Planning:
After all the functional block diagrams are completed, I usually go through a preliminary route planning exercise. The idea here is to gain some intuition for the final routing strategy, and to uncover any hidden issues that may surface down the road.
This is the most crucial step in any backplane design. Usually at this stage of the project, the system packaging architect is busy developing the shelf packaging concept, and is looking for feedback on connectors and card locations, so he (or she) can complete the common features drawing. The common features drawing defines all the x-y coordinates for all connectors and other mechanical parts on the backplane.
An example of a preliminary routing plan strategy diagram is shown in Figure 3. Each color represents two routing layers; for a total of 6 layers. The heavy black lines represent the high-speed serial link bundles of the data path; routed completely from SW1 and SW4 to LC1-10. The partially routed heavy red and blue lines, follow the exact same route plan as the heavy black lines, except they terminate to the respective color-coded SW cards. The beauty of this comes later, when the actual routing of the backplane takes place. Because the routing is identical, except for the source and destinations, it is a simple copy and paste exercise to replicate the routing on 5 of the 6 layers. The only editing required is at each end of the links. As you can appreciate, this is a huge time saver in completing the final layout!
When the preliminary route plane is complete, a pin-list summary for each circuit pack is compiled using an Excel spreadsheet. The pin-list summarizes the minimum number of pins needed per circuit pack for the function. Later on, it helps to drive the selection and number of connectors.
After completing the preliminary route planning exercise, and pin-list summary, you will gain a sense for:
-
the number of routing layers you will need
-
circuit pack connector signal grouping and partitioning
-
connector selection criteria for density
-
minimum vertical routing channel space needed between connectors
-
worst case topologies for signal integrity analysis
Backplane Connector Selection:
Large companies invest a lot of money and time to qualify a connector family. There is always strong pressure to reuse connectors from one system design to another because of cost. Qualifying a new connector is no trivial task. It takes a significant development effort to model, characterize and test the connectors. If you try to qualify a new connector, at the same time as designing a new system, you run the risk of delaying the overall program if serious issues develop along the way. Sometimes though, reusing the same connector just won’t cut it. For whatever the reason, one day you will be forced to look at other connectors.
Choosing the right connector for any new system is the most important aspect for any backplane design; regardless if it is reuse of a previous connector, or looking at new ones. The connector is the lifeblood of the backplane because it ultimately drives minimum slot pitch and circuit board height. It must be capable of supporting current and next generation high-speed signaling standards, and be robust enough to withstand multiple insertions. Factors such as pin density, pin pitch, pairs per row, overall size, skew, and crosstalk are examples to consider in this process.
Preliminary Stack-up:
In any high-speed serial link architecture, the data plane links are the most critical signals. They are the ones that usually define the total number of routing layers for the final PCB stack-up. When we include 4 layers, for redundant power distribution, to the 6 routing layers, the minimum number of layers for the backplane will be 18 layers as shown in Figure 4.
The right half of the figure gives counter-bore details. Another name often used is back-drilling. It is a common procedure done on backplanes to minimize via stubs, which is a killer for multi-gigabit serial links.
Detailed Route Plan:
Usually, around this time in the project schedule, the mechanical architect has put together a preliminary common features drawing, showing the preliminary connector placement. We use this drawing as a template to do a more detailed routing plan analysis.
By studying the preliminary route plan and pin-list, we can come up with a strategy to organize and partition the signals within the connector, and perform a more detailed routing analysis. This process can take a few iterations before it is optimum, but eventually, we end up with a more detailed routing plan as summarized in Figure 5. Each illustration here represents two routing layers per drawing. One layer is for Tx and the other is for Rx.
Vertical Routing Channel Analysis:
Before we sign-off on connector placement and route plan though, we need to verify there is enough space between connectors for the vertical routing channels. Otherwise, this may be a deal breaker for the chosen connector; slot pitch; total number of layers; or even the whole system packaging concept. If you do not have enough space here, there will be compromises needed somewhere else to accommodate it. The worst case scenario is having to double the number of layers, or having to choose a higher cost connector.
Signal Integrity Analysis:
Finally preliminary channel simulations must be done before we can sign-off on the backplane physical architecture concept. Now that all the detailed routing analysis is complete, we can easily establish several topologies to analyze.
One example of a worst case reference topology is highlighted in Figure 6. During this stage, we use Manhattan distance to estimate trace lengths.
After procuring the connector models, and developing circuit models to represent the via structures, I like to use Agilent ADS to capture and simulate the topologies. An example of the circuit topology, and simulation results are summarized in Figure 7.
Here, the topology was simulated at 10GB/s. The S-parameters are compared against the IEEE 802.3 10BaseKR spec. You would normally do this for every topology of interest. Later on, during the detailed design phase of the program, I would get 3D models of the vias built and use actual routed lengths from the backplane and circuit pack cards to confirm the design.
Summary:
Hopefully by now, you can appreciate the backplane architecture and design can be a complex beast to tame, and get right the first time. There are many complex interrelated steps that require the due diligence and meticulous planning to be successful. We have only scratched the surface here. You can download the full white paper titled, “ Backplane Architecture High-Level Design” , from which this design note is based upon, and an example of the PowerPoint HLD document from our website at: www.lamsimenterprises.com.
If you would like more information on our signal integrity and backplane services, or how we can help you achieve your next high-speed design challenge, email us at: info@lamsimenterprises.com.
Backplane Architecture and Design
According to Wiktionary, an Architect is: “A person who plans, devises or contrives the achievement of a desired result.” Because the backplane is the key component in any system architecture, the sooner you consider the backplane’s physical architecture near the beginning of a project, the more successful the project will be. If you think about it in the same way as designing a building, you would never consider building it without first engaging a building architect to plan and oversee the detailed design. Likewise, the backplane architect plans and oversees the physical backplane design before any layout is ever started. He or she works closely with a system-packaging engineer to satisfy the system requirements before any concept becomes final. Sometimes the original system architecture needs revisions due to physical limitations the backplane imposes. This can only be established with due diligence and planning during the high-level design stage.
Unlike other circuit pack designs used in the system, the backplane is much like the keel of a ship of which the rest of the ship’s construction depends on for support and structural integrity throughout its lifetime. Backplanes need to be right the first time so that circuit packs can interoperate together day one and be capable of supporting future system upgrades as technology advances. Once the system has been deployed into the field, it is next to impossible to change the backplane to correct any deficiencies or to upgrade for performance like you can by redesigning the plug-in circuit packs.
The seasoned backplane architect is a unique individual usually tasked to turn the system architect’s ideas and dreams, like the system block diagram example shown to the left, into reality. An often-misunderstood profession, backplane architects wear many hats to accomplish their goals. Often they must juggle the design requirements from many disciplines and decide on the best trade-offs for the final design. They must converse fluently with system architects, mechanical designers, circuit pack designers, connector suppliers, PCB layout designers, ASIC/FPGA and software engineers. They must be organized and meticulous in their documentation and design. But, most importantly, they must have a sound knowledge of mechanical, PCB layout/fabrication, signal integrity, power and EMC issues.
The greatest danger in leaving the backplane design as an afterthought is the connector selection and pin-out definition. If left to system packaging engineers and board designers to define, they may not be optimum for either performance or system cost. Many times system architects and packaging engineers will merely take the total number of signals and choose a connector with the highest pin density per inch without considering PCB routing or signal integrity implications. Inefficient routing of the traces leads to an increase in layer count and results in a thicker board. Thicker boards leads to higher hole aspect ratios and longer vias affecting high speed performance. Additional layer count impacts common equipment cost.
The high-level design stage is where the physical backplane architecture starts to take shape. It uncovers potential layout routing issues and gives you the confidence the design will work the first time. The importance of this stage cannot be overstated. It primarily drives these key activities:
-
Sanitizes the system architecture.
-
Defines the final selection of appropriate connectors.
-
Defines the connector signal partitioning and circuit pack pin-outs.
-
Provides the routing plan and design rules for layout.
-
Defines the net topologies for signal integrity analysis and link budgeting.
-
Facilitates the mechanical design of shelf and system packaging.
-
Defines the minimum slot pitch for optimum routing channels.
-
Facilitates early circuit pack floor planning and final card size.
-
Facilitates ASIC and FPGA pin selection for optimum routing to backplane connectors.
-
Estimates PCB layer count and board thickness.
-
Establishes an estimate for system cost of goods to support the business case.
Proper route planning and connector pin-out definition is vital for optimum performance. When done correctly, the final schematic capture and actual PCB layout will flow smoothly with no surprises. As an example, the left half of the figure (labeled HLD Plan) shows a sample of an inner layer high-level design route plan I did using Framemaker as the drawing tool on a design before any schematic was ever captured or pin-outs defined. Everything was planned from the number of layers to how the tracks needed to break out of the connector fields. The right half of the figure is the actual layout done in Cadence Allegro showing the inner layer routing of the artwork. The due diligence done in the high-level design stage made the actual layout fairly trivial. If you forgo this step, the worst-case scenario is the project will need to be reset to redesign shelf mechanicals or redefine card pin-outs causing delay in meeting time to market objectives and ballooning R&D costs. It’s a classic case of pay me now or pay me later.
At Lamsim Enterprises Inc., we can help you with these or any other design challenges you may have by providing innovative signal integrity and backplane solutions. Visit us at our web site at: lamsimenterprises.com .
Backplane Architecture Terms and Definitions
The following is a list of common terms and definitions associated with system architecture and backplane design:
Backplane
A backplane is a multi-layered printed circuit board assembly serving as the backbone of a system. Its purpose is to interconnect several printed circuit board assemblies called circuit packs or cards using plug in connectors to form a complete system. These cards plug into one side of the shelf assembly. Usually in mission critical system applications like central office telco or data centers, the backplane is passive meaning it does not contain active semiconductor devices permanently attached as part of the final assembly. Usually only connectors are the only components, but occasionally capacitors and resistors are also used. Active backplanes on the other hand, contains active components and often found in enterprise or consumer grade applications
Midplane
A midplane is similar to a backplane in function except that the circuit packs plug into both sides of the shelf assembly. In these systems, cards with I/O cabling from the faceplate plug into one side of the shelf, while non-I/O circuit pack plug in on the other side. Some midplane architectures have the front card plugged in orthogonally to the rear cards for high speed applications.
Parallel Bus Topologies
Parallel bus topologies carry data words in parallel on multiple traces from card-slot to card-slot across a backplane or from chip to chip on a circuit pack. Up until the late 1990’s, most system architectures used this form of interconnect. Due to signal integrity and timing issues associated with some parallel bus architectures with 10 to 16 card slots, the speed of the bus was limited to 25-66 MHz Two popular industry standard systems still using parallel busses today are CompactPCI and VMEbus.
The main issue with a parallel bus topology is fault tolerance where a single point of failure on the bus can bring down the entire system. Mission critical systems often had to employ redundant busses to guard against single point failures.
As performance demand increased, newer high speed system architectures were designed using serial technology in a point-to-point or point-to-multi-point switched fabric topologies.
Switched Fabric
Switched fabric, or just plain fabric, is the term most popular used in telecommunications and high-speed networks, including InfiniBand, Fiber Channel, PCIe, ATCA and other proprietary fabric based architectures. In these architectures, all data passes through the fabric before continuing to its destination. It offers better total throughput than parallel busses because traffic is spread across multiple physical links. It manages and controls all functions of the network and acts as a repeater for the data flow.
Single Star Topology
Star topologies are one of the most common high-speed serial topologies used in networks today. The advantage is it reduces the chance of network failure by connecting all of the systems to a central node. A failure of a link from any peripheral node to the central node results in the isolation of that peripheral node from all others. As a result, the rest of the systems remain unaffected.
In its simplest form, a single star topology consists of one central hub node interconnected point-to-point to other peripheral nodes resembling a spoke wheel or star configuration. When implemented in a backplane, the central node is usually the switched fabric card and the peripheral nodes are line cards. The fabric card switches messages between the other line cards in the network. The line cards usually have faceplate I/O connectors to connect to other shelves in a network.
The main disadvantage with a single star topology is high dependence of the system on the functioning of the central fabric. Failure of the fabric card can bring down the entire system. Because of this, mission critical systems employ two fabric cards for redundancy in a dual star topology configuration.
Dual Star/Multi-star Topology
The dual star or multi-star topology is similar to the star network topology except it has two or more central hub nodes interconnected point-to-point to other peripheral nodes. When implemented in a backplane application, these central nodes are usually the switched fabric cards and peripheral nodes are the line cards. The additional fabric(s) provides redundancy in mission critical system applications in case of failure, or for upgrading fabric card hardware.
Fully Connected Mesh Topology
A fully connected mesh topology, when applied to a backplane application, does not have one central fabric node(s) as in the case of star topologies. Instead, each line card node connects with all other line card nodes forming a mesh. Its major disadvantage is the number of connections grows significantly with the number of nodes. This requires additional backplane connector pins and layers to interconnect them. Because of this, it is impractical for large systems and only used when there are a small number of cards needing to be interconnected.
Introduction
Welcome to my Blog! I am Bert Simonovich, founder and president of Lamsim Enterprises Inc. I graduated in 1976 from Mohawk College of Applied Arts and Technology in Hamilton, Ontario, Canada as an Electronic Engineering Technologist. I started my consulting business after working 32 years at Bell Northern Research/Nortel. Throughout my career, I have held a variety of hardware design engineering positions and pioneered several advanced technologies into products. Currently I offer innovative signal integrity and backplane solutions as a consultant.
From as far back as I can remember, I was always interested in how things worked. I would often take things apart just to see what was inside; -not always successful in putting them back together again though ;-o. I was always fascinated with electricity and electronics. When I was about 10 or 11, I was mystified with how telephones worked. After reading about Alexander Graham Bell in a booklet published by The Bell Telephone Company of Canada, I became inspired to buy a pair of old push to talk handsets from a local army surplus store. I experimented with them using a drycell battery and lamp cord wire. When I finally was able to get two-way communications, it seemed like magic. I knew right then what my career choice would be.
I have been fortunate throughout my career to have been a part of and contribute to some of the technology that enable the gadgets we enjoy today. I have met and worked with many smart and talented individuals who took the time to unselfishly share their knowledge and experience.
And now, after all this time, the passion I had as kid to learn and understand new things is still there. Except now, like cradling a fine glass of wine, I am able to slowly swirl it around, sip it and savor the taste. This blog is about sharing some of that passion. It will cover a range of topics from signal integrity, PCBs, backplane design, circuit modeling, simulation tools and other practical engineering solutions. I hope you find my posts interesting and get inspired to explore them further on your own.
Thanks for visiting. I invite you to constructively comment and share your own thoughts and experiences as well.
Bert Simonovich's Design Notes 