Review Instructions
Altium project: EDA-04571-V1-0__20-02-2023_17-46-12_.zip, Xilinx checklist
Please ignore few non-critical issues like DRCs (mostly silkscreen related) and few unrouted lines (they will be connected once placement is fixed).
Please attach a txt or add your comments directly
Meeting for a discussion and additional clarifications: https://indico.cern.ch/event/1287844/
Review Outcome
https://ohwr.org/project/wr-switch-hw-v4/wikis/layout-review-summary-20230516
Raw feedback
GregD
Schematics:
- [TODO] power_supply: would be good to add a graph with intended power sequencing - just for clarity
- [TODO] PS_MIOS: check how's Linux driver support for I2C mux (IC20). Maybe for some of the stuff that needs to be frequently accessed (e.g. OLED?) would make sense to connect to PL and use PL-instantiated IIC ipcore?
- [TODO] USB_interfaces: with proper EEPROM configuration, you can use FTDI chip not only for UART, but also as JTAG for the FPGA. We have it sorted out in DI/OT. Then, when working with Vivado you don't need any special JTAG cable, just a regular USB cable connected to your FTDI. This requires also connecting a configuration EEPROM to pins EECS, EECLK, EEDATA
Layout:
- [to be seen] I don't think routing SD card or eMMC lines requires backdrilled vias.
- [TODO] Some traces are not routed, e.g. SATA_RX_P/N or SD_CTRL.SD_D0/2
- [TODO] Add GND vias for return current close to high speed signals, or even better, spread uniformly GND vias on PCB to connect well GND layers together and to ensure return paths for high speed signals.
- [TODO][X:165mm, Y: 150mm] in L2 there is thick (2mm) P3V8 trace that connects to other layers through very tiny vias (0.3mm hole size). Increase vias dimension.
- [done at the end] DEM adds teardrops in all designs recently, but it was not done for this PCB.
- [done at the end] missing licensing and URL in the silkscreen
GregK
/!\ Power planes
-
Report of VCCINT PDN analysis at 10A shows failures
- ideally the analysis should be done at 30A for such a big SoC
- [TODO] P3V3, P0V85 power supplies underrated
- [should be ok] P3V3 to be at least 8..10A
- [TODO, IC32 is the problem, WREN-like power supply, unify] P0V85 to be at least 30A.
- Proposed next steps:
- Good power budget with necessary margins; I use Excel sheets and ADI online tool
- Re-design of power supplies
- SI analysis of all 10G links using Hyperlynx and eye diagrams/BERR and PCB update
- SI analysis of the SDRAM using SI-DDR and PCB update
- PDN analysis for every rail. I did it for two rails and found critical issues so far
- PCB fixes – it’s necessary to add more polygons for VCCINT and 3.3V rails
NOTE: ATX standard/PSU - remote sense on pin 11, useful if we do not have enough copper on power planes
SCH
- some designators are rotated (I know, there is an option in settings)
- enable PN view of ICs, like ESD7016MUTAG
- [leave the footprint] are you sure ESD7016MUTAG would work here? It's clamping voltage is far too high to protect the FPGA bank
- [TODO for expansion conne] I'd use unipolar TVS for FMC status lines instead of PESD3V3L1BA
- [leave for the time being, we have the component] LMX2594RHAT is hard to get
- [leave as is, works fine for RF] we have TR2, Tr3, TR4, don't we want to use galvanic isolation (ground loop removal)
- [it is 10MHz input, TODO + check the input stage and TR2] I miss annotation about power level at P5 input
- [TODO: use standard coils] L74, L76, L75, L77 CMC windings are connected in parallel. That won't work. CMCs are very fragile to current impalance. To make them work, use common ferrite core or multi-windong CMCs
- [to be checked, D22-to low voltage rating] what's the purpose of IC49? Are we going to connect any other PSU? NOTE: behavior of PSU might depend on design, check
- [TODO: change fan connectors] please add annotation about fan PN, I cannot verify the connector wiring
- [TODO] add series 4k7 protection resistor in series with FANx_PRESENT input, We don't want FPGA damage when somebody inserts fan with different wiring
- [TODO] I'd connect D26 pin 5 to 3V3 and add series PWM input resistors. We don't want fan controller damage when somebody inserts fan with different wiring
- [TODO] isn the voltage rail nomenclature correct? P3AV3 looks weird, don't we use P3V3A in all OHWR designs?
- [to be checked] SFP I2C interface is not protected with TVS while the MGT IFC is.
- [TODO] connect D5, D6 pin 5 to 3V3. It would lower the clamping voltage
- [TODO] USBC requires 5k1 pulldown resistors on CC lines to work as USB device https://www.allaboutcircuits.com/technical-articles/introduction-to-usb-type-c-which-pins-power-delivery-data-transfer/
- [SATA is ok, keying to be checked] why SATA? PCIe SSDs are much faster, the M2 keying is different. I miss annotation about M2 keying
PCB
- missing return vias for MGT signals (critical) and all differential signals
- MGT vias below FPGA seem to be not impedance matched - there is random polygon pour and NFPs are not removed. Was there SI simulation performed? I'd like to see the eye diagram for 10G
- random GND pour around SDRAM traces breakes impedance. Did you run SI-DDR simulation? It doesn't look so
- errors in impedance profiles in stackup editor
- several accute corners on negative layers, especially polygons under FPGA
- no defined polygon pour sequence, anyway, GND polygon should not cover other polygons! This may lead to serious issues during re-pouring of all polygons
- no PDN analysis were performed. Otherwise one would discover that single piece of polygon cannot supply the core of such big FPGAs. I usually have to use 3 polygons in parallel to distribute VCCINT
- [Have thermal on all SMD pads] no thermal bridges for SMD pads on top layer. This makes solering challenging. No reason to do this way for all pads.
- [to be changed] are you sure 0V85 consumes less than 12A? I used much stronger DC/DC converters for much smaller chips
- [check ! good idea] why not to use VCCINT below FPGA as the DC/DC feedback?
- [TODO] 3 vias that connect L51 to power plane is just a joke :D They will burn a few moments after power on.
- [acute angles -> to be fixed] a lot of acute copper on top layer and other layers - it will affect reliability
- [leave for initial debugging] ferrite beads before and after DC/DC do not make much sense. Their impedance for such low frequency is mostly pure resistance. They only cause DC drop. They makes sense only during initial debugging.
- [TODO] DDR4_DQS8_x routed over split polygon
- [it is guard-ring, should be connected to vias, a debugging feature, leave as is] what is this strange unconnected trace around clocking ? whhy not use keepout lines?
- those beefy CMCs don't make any sense - they won't perform any function
- ZL9101M is EOL
- SI and PDN analysis at this stage does not make much sense.
- not enough power for 3.3V rail. If we assume each SFP taking 0.7W they will already use 5.3A. The DC/DC is rated for 5A and there is much more stuff supplied from that rail. Note that some SFPs, especially long range take much higher power
- I did initial PDN analysis for VCCINT and for existing layout at 10A we will get 688mV at the centre of FPGA
- there is current density of over 250A/mm2 while it should not exceed 50A/mm2.
- I'd recommend performing airflow simulation. The heatsink used seem to be too small.
Maciej
Some Clarifications:
- Reg GregK's "why SATA? PCIe SSDs are much faster", relevant issues where M.2 choice was discussed: #23 (closed), #39 (closed)
Feedback/review;
- J11/J13: replace edge fan connectors (PCB too thick, cannot find mating for that) with surface-mount connectors, possible replacements from molex catalogue, e.g. PCB side: this one, could be also this; fan side: this one, or this
- J8: "M.2 PCIe" -> if we decide to keep M.2 over SATA, replace "PCIe" with "SATA"
- Feedback from the mock-up prototype:
- J4: the USB3.1 connector needs to extend beyond PCB by min 0.5cm, probably need to find a new connector. Currently the connector is hidden behind the front panel and cannot be used (see pic )
- J12: Move PSU-present/alarm connector closer to the front
- J10: consider using angled connector (with the cables, it barely fits in the enclosure, some danger to strain the cables.
- SMA connectors P6, P4, P5, P2, P3 - I'd make the order of the I/Os different (similar to what we have in the current WRS and divided into INs and OUTs), i.e., from left: PPS_OUT, 10MHz_OUT, AUX_ABSCAL, 10MHz_IN, PPS_IN
- We seem to be missing PMUbus connectivity to the Power Supply (Data, Clock, Control, SMBALERT signals)
- License note missing and WR logo of bad quality ;-)
Paul
- Quiescent current for vccint is already 2.5A. Any power consumption estimate from vivado with a preliminary gateware? On spexi7u I use the DC/DC from Infineon with 30A output current capability on vccint. To be considered to update the design with a similar solution.
- IMHO, no need for ferrite beads on the DC/DC outputs
- A lot of missing power vias near decoupling capacitors
- Is it allowed to put via in pads for decoupling capacitors located under the fpga?
- Tracks for decoupling capacitors are often too small, I’m used to set the track width equal to the diameter of the vias
- Design checklist XTP427 from Xilinx is missing