MCH main board (a.k.a. MCH_MAIN)
Note!* This page refers to the newest revision (rev 1.1) of MCH hardware which is currently manufactured and (when everything goes well) will be available to all interested participants in mid-January 2010.
The image below shows the PCB layout of MCH mainboard with most important design blocks highlighted.
1. Main FPGA* schematics:
link
link
It's Altera's biggest Cyclone-3 series FPGA (EP3C120F780C8), containing
~110K logic cells and 3.8 Mbits of built-in RAM. It:
- is connected directly to all PHYs (both uplink and downlink). When a packet comes to one of switch ports, it parses its header, searches for entry matching address/type/QoS properties in the routing table, and sends it to appropriate destination port(s), eventually buffering it if the destination port is busy at the moment. This however doesn't apply to HP packets which are broadcasted to avoid time-consuming routing decision.
- talks to the CPU via memory-mapped IO using 32-bit asynchronous CPU-FPGA bus
- talks to the clocking FPGA via SPI interface
- implements the SMI master
- implements DMTD phase shift measurement for downlink ports
2. Main CPU* schematics:
link
,
link
We've chosen Atmel's ARM9 chip -
AT91SAM9263.
The CPU is responsible for:
- General service of the switch through its Fast Ethernet (100 mb/s) RJ45 connection. So you can ping, ssh, etc.
- Stuff too complicated for the main FPGA to handle (RSTP, PTP, etc.)
- Configuration of FPGAs.
- Other misc functions (playing Doom ;-)
3. 64 MB SDRAM* schematics:
link
64 Mbytes of SDRAM memory running at 133 MHz with 32-bit data bus (2x
32M 16-bit chips).
4. MicroSD card slot and 8 MB serial flash* schematics:
link
An optional slot for microSD card, allowing for easy switch firmware
changes (no debug cables required - just copy the firmware image using
card reader). I've put it mainly to ease the development (I've screwed
up the NAND once playing with the kernel/u-boot. It's easier to replace
a card than to resolder the flash chip). It's very likely to be removed
in the final HW revision. There is also another 8MB serial flash for
fail-safe boot when main firmware is damaged.
5. 100 Mbit Ethernet port* schematics:
link
Standard twisted-pair Ethernet port, allowing for SSHing, pinging,
SNMP-ing, anythingelseyouwant-ing the switch from external
(non-WhiteRabbit network). Done using AT91SAM9263's internal MAC and
DM9161A PHY chip.
6. Debug/JTAG connector* schematics:
link -
bottom-left corner
Single 26-pin 2mm-pitch IDC connector containing all interfaces required
to debug the board:
- FPGA JTAG (MAIN + CLKB chained, 2.5V supply)
- Main CPU JTAG (3.3V)
- Watchdog CPU JTAG (3.3V MP)
- Main CPU DBGU serial port (LVTTL-levels RS232)
- Main CPU DBGU serial port (LVTTL-levels RS232)
- CPU & watchdog resets
- uTCA PWRON line (to keep the main power up regardless of what the watchdog says)
7. MCH_MAIN to MCH_CLKB sandwich connectors* schematics:
link - top
Three 50-pin JAE KX15-series connectors, connecting two big MCH boards.
Contains:
- uplink PHY signals
- clocks: REFCLK, DMTDCLK, PHY RBCLK
- CMI (Clocking Mezzanine Interface) lines (SPI + IRQ + few general-purpose lines between main and clocking FPGAs)
- frontpanel RS232 signals
- JTAG & configuration signals for CLKB FPGA
- Configuration signals for AD9516 (to main CPU - the PLL has to be programmed before we boot up the FPGAs, as it's the source of REFCLK)
8. Power supply* schematics:
link
Main power supply, delivering +3.3V @ 4A, +2.5V & 4A, +1.2V @ 3A and
auxillary +1.2V for CPU core (which has to go up before +3.3V to avoid
CPU latchup)
9. Front panel LEDs* schematics:
link
8 dual color LEDs + 4 mandatory uTCA LEDs currently making the front
panel look more pleasant by performing the Knight Rider ;-)
10. 512 Kbyte ZBT memory*
schematics:link
IDT's 71V3557 chip (Zero-bus turnaround synchronous memory). Used for
the routing table storage, connected directly to main FPGA.
*11. Downlink PHY*s schematics:
link -
right,
link
MB_downlink_phy
8 GbE downlink ports connected to the uTCA backplane (fabric A).
Currently we are using TI's TLK1221 chips.
12. REFCLK fanout* schematics:
link
10-port fanout, delivering REFCLK to 8 downlink PHYs, main FPGA and ZBT
memory. Built using TI's CDCVF310 chip.
13. Watchdog MCU* schematics:
link
32-bit ARM7 microcontroller (Atmel's
[[http://www.atmel.com/dyn/products/Product_card.asp?part_id=3755
AT91SAM7X256 ) responsible for the IPMI-related tasks:
- MCH board initialization
- IPMI stuff
14. SMI connector* schematics:
link -
bottom right
40-pin board-to-board connector, passing the SMI interface signals to
MCH tongues 3/4 (Fabric E).
15. MCH Tongue 1* schematics:
link -
bottom right
170-pin standard MCH board edge connector. Contains:
- Fabric A !GbE links
- All IPMI busses
- Power supply (+12V_BULK, +3.3V MP)
- uTCA control signals (power-on, reset, etc.)
-- Main.TomaszWlostowski - 05 Dec 2009