An informal description of the MCH board
For the foregoing discussion, please refer to the attached pdf (very informal and sketchy). The MCH is in fact made of a sandwich of 4 PCBs:
- The uplink board (aka the timing board) contains the uplink PHYs, the timing FPGA and its associated gear (DMTD, PLLs, clock fanouts...). The data links from the PHYs go directly to the main board through the sandwich connector, as do the RS232 debugging signals. The function of the timing FPGA is to take the rx_clk, massage it and generate the compensated clock. Currently we're doing the phase shifting in WR slaves (i.e. in the uplink ports of switches). In the future, we intend to do the shifting in the masters, so as to reduce the complexity of slaves. Another important function of the timing board is to generate the DMTD clock, a clock offset in frequency by a tiny bit wrt the 125 MHz frame clock, and which is a key ingredient of DMTD-based phase detection and shifting. This clock is also distributed everywhere, including other AMC cards in the uTCA crate.
- The main board. Packets hit the main FPGA from either uplink ports
in the timing board or downlink ports in other AMCs. For reasons of
clearness and laziness the connections to the backplane are not
shown. In the main board there are eight PHYs sitting between the
main FPGA and the !GbE star in the backplane. All the routing logic
is in the main FPGA. For storing routing tables, a ZBT RAM is used.
The ARM9 embedded Linux CPU performs several tasks:
- General service of the switch through its Fast Ethernet (100 mb/s) RJ45 connection. So you can ping, ssh, etc.
- Stuff too complicated for the main FPGA to handle (RSTP, PTP, etc.)
- Configuration of FPGAs.
- Other misc functions.
- Two other boards to distribute signals to the backplane. The reason for this is that there are not enough contacts in a uTCA backplane connector to send all we want. The limitation is most severe in the MCH slot, where all stars converge. So uTCA deals with this problem by having 4 backplane connectors in the MCH slot, separated by a certain distance. By sandwiching PCBs and user appropriate spacers, we can send more signals to the backplane. So these extra two boards take signals from the other two boards through the sandwich connectors and put them on the backplane. These are mainly SMI (to be described later) and REF/DMTD clocks.
In addition to the routing logic, the main FPGA also has the so-called SMI (Switch management Interface) links in LVDS to each slot in the backplane. This is used for things such as clock distribution, UTC, keeping routing tables up to date in all cards, etc. SMI operates synchronously to 125 MHz backplane reference clock and can be used to distribute event information such as PPS signal.
The Watchdog CPU is another 32-bit microcontroller (ARM7 from Atmel) with two main tasks: uTCA crate management (acting as an !I2C hub for controlling all slots using IPMI) and making sure we can't loose contact with the ARM9 or the Cyclone 3 after configuring with a broken binary. This is done with a watchdog mechanism, hence the name of this CPU.
-- Main.JavierSerrano - 19 Nov 2009
-- Main.TomaszWlostowski - 22 Nov 2009