We use Veryx ATTEST WRPTP Conformance Test Suite (open hardware, available from repo) to perform regression tests of WR Switch releases. These tests are grouped into:
- Inter Operability Group (IOG)
- Message Format Group (MFG)
- WRPTP Configuration Group (WCG)
- WRPTP State Machine Group (WSMG)
All tests are expected to pass.
Tests that a release must pass (65 tests for IEEE802.3 and 65 tests for UPD mapping)
TEST Name | Test Purpose Description | IEEE802.3 and/or UDP | v5.0.1 | v6.0 |
---|---|---|---|---|
IOG_001 | To verify that a WRPTP enabled device in WR_MASTER mode moves to standard PTP Master mode when it is connected to non-WR device. | both | passed | passed |
IOG_002 | To verify that a WRPTP enabled device in WR_SLAVE mode moves to standard PTP Slave mode when it is connected to non-WR device. | both | passed | passed |
MFG_001 | To verify that a WRPTP enabled device sends WRPTP Announce message in correct format when transport is over IEEE 802.3/Ethernet. | both | passed | passed |
MFG_002 | To verify that a WRPTP enabled device sends WRPTP Signaling messages (LOCK, CALIBRATE, CALIBRATED and WR_MODE_ON) in correct format when its port is WR Master and transport is over IEEE 802.3/Ethernet. | 802.3 | passed | passed |
MFG_003 | To verify that a WRPTP enabled device sends WRPTP Signaling messages (SLAVE_PRESENT, LOCKED, CALIBRATE and CALIBRATED) in correct format when its port is WR Slave and transport is over IEEE 802.3/Ethernet | 802.3 | passed | passed |
MFG_004 | To verify that a WRPTP enabled device sends WRPTP Announce message in correct format when transport is over IPv4/UDP | UDP | passed | passed |
MFG_005 | To verify that a WRPTP enabled device sends WRPTP Signaling messages (LOCK, CALIBRATE, CALIBRATED and WR_MODE_ON) in correct format when its port is WR Master and transport is over IPv4/UDP. | UDP | passed | passed |
MFG_006 | To verify that a WRPTP enabled device sends WRPTP Signaling messages (SLAVE_PRESENT, LOCKED, CALIBRATE and CALIBRATED) in correct format when its port is WR Slave and transport is over IPv4/UDP. | UDP | passed | passed |
WCG_001 | To verify that a WRPTP enabled device stores all attributes with default initialization values. Checking that the following attributes have correct default values. | both | passed | passed |
WCG_002 | To verify that a WRPTP enabled device supports to configure wrConfig data set member (allowable values: NON_WR, WR_S_ONLY, WR_M_ONLY and WR_M_AND_S). | both | failed | passed |
WCG_003 | To verify that a WRPTP enabled device with it's port in PTP Master sends CALIBRATED message with configured knownDeltaTx (allowed range: UInteger64). | both | passed | passed |
WCG_004 | To verify that a WRPTP enabled device with it's port in PTP Slave sends CALIBRATED message with configured knownDeltaTx (allowed range: UInteger64). | both | passed | passed |
WCG_005 | To verify that a WRPTP enabled device with it's port in PTP Master sends CALIBRATED message with configured knownDeltaRx (allowed range: UInteger64). | both | passed | passed |
WCG_006 | To verify that a WRPTP enabled device with it's port in PTP Slave sends CALIBRATED message with configured knownDeltaRx (allowed range: UInteger64). | both | passed | passed |
WSMG_001 | To verify that port of a WRPTP enabled device shall be in the IDLE state when WR Link Setup is not being performed. | both | passed | passed |
WSMG_002 | To verify that a WRPTP enabled device with it's port in PTP Master state sends LOCK message and transitions it's port WR state from IDLE to M_LOCK state upon receiving SLAVE_PRESENT message. | both | passed | passed |
WSMG_003 | To verify that a WRPTP enabled device with it's port in PTP Master state sends LOCK message and re-enters to it's port WR state M_LOCK on expiry of WR_M_LOCK_TIMEOUT. | both | passed | passed |
WSMG_004 | To verify that a WRPTP enabled device with it's port in PTP Master state sends CALIBRATE message and transitions it's port WR state from M_LOCK to CALIBRATION state on the reception of LOCKED message before EXC_TIMEOUT_RETRY occurs. | both | passed | passed |
WSMG_005 | To verify that a WRPTP enabled device with it's port in PTP Master transitions it's port WR state from M_LOCK to IDLE state after EXC_TIMEOUT_RETRY occurs. | both | passed | passed |
WSMG_006 | To verify that a WRPTP enabled device with it's port in PTP Master sends CALIBRATED message and transition it's port WR state from the CALIBRATION to CALIBRATED state | both | passed | passed |
WSMG_007 | To verify that a WRPTP enabled device with it's port in PTP Master sends CALIBRATED message and re-enters to it's port WR state CALIBRATED on expiry of WR_CALIBRATED_TIMEOUT. | both | passed | passed |
WSMG_008 | To verify that a WRPTP enabled device with it's port in PTP Master transitions it's port WR state from CALIBRATED to RESP_CALIB_REQ state on the reception of CALIBRATE message before EXC_TIMEOUT_RETRY occurs. | both | passed | passed |
WSMG_009 | To verify that a WRPTP enabled device with it's port in PTP Master transitions it's port WR state from CALIBRATED to IDLE state after EXC_TIMEOUT_RETRY occurs | both | passed | passed |
WSMG_010 | : To verify that a WRPTP enabled device with it's port in PTP Master re-enters to it's port WR state RESP_CALIB_REQ on expiry of WR_RESP_CALIB_REQ_TIMEOUT when otherPortCalPeriod is 0x0. | both | passed | passed |
WSMG_011 | To verify that a WRPTP enabled device with it's port in PTP Master re-enters to it's port WR state RESP_CALIB_REQ on expiry of RESP_CALIB_REQ_TIMEOUT when otherPortCalPeriod and otherPortCalRetry are greater than 0x0 (otherPortCalPeriod and otherPortCalRetry should be ignored by DUT). | both | passed | passed |
WSMG_012 | To verify that a WRPTP enabled device with it's port in PTP Master sends WR_MODE_ON message and transitions it's port WR state from RESP_CALIB_REQ to WR_LINK_ON state on the reception of CALIBRATED message before EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are 0x0. | both | passed | passed |
WSMG_013 | To verify that a WRPTP enabled device with it's port in PTP Master transitions it's port WR state from RESP_CALIB_REQ to IDLE state after EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are 0x0. | both | passed | passed |
WSMG_014 | To verify that a WRPTP enabled device with it's port in PTP Master sends WR_MODE_ON message and transitions it's port WR state from RESP_CALIB_REQ to WR_LINK_ON state on the reception of CALIBRATED message before EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are greater than 0x0 (otherPortCalPeriod and otherPortCalRetry should be ignored by DUT). | both | passed | passed |
WSMG_015 | To verify that a WRPTP enabled device with it's port in PTP Master transitions it's port WR state from RESP_CALIB_REQ to IDLE state after EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are greater than 0x0 (otherPortCalPeriod and otherPortCalRetry should be ignored by DUT). | both | passed | passed |
WSMG_016 | To verify that a WRPTP enabled device with it's port in PTP Master transitions it's port WR state from WR_LINK_ON to IDLE state upon successful completion of WR link setup process. | both | passed | passed |
WSMG_017 | To verify that a WRPTP enabled device with it's port in PTP Master stores otherPortDeltaTx, otherPortDeltaRx, otherPortCalPeriod, otherPortCalRetry and otherPortCalSendPattern received in CALIBRATE message. | both | passed | passed |
WSMG_018 | To verify that a WRPTP enabled device with it's port in PTP Slave sends SLAVE_PRESENT message and transitions it's port WR state from IDLE to PRESENT state. | both | passed | passed |
WSMG_019 | To verify that a WRPTP enabled device with it's port in PTP Slave sends SLAVE_PRESENT message and re-enters to it's port WR state PRESENT on expiry of WR_PRESENT_TIMEOUT. | both | passed | passed |
WSMG_020 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from PRESENT to IDLE state after EXC_TIMEOUT_RETRY occurs. | both | passed | passed |
WSMG_021 | To verify that a WRPTP enabled device with it's port in PTP Slave sends a LOCKED message and transitions it's port WR state from S_LOCK to LOCKED state. | both | passed | passed |
WSMG_022 | To verify that a WRPTP enabled device with it's port in PTP Slave sends LOCKED message and re-enters to it's port WR state LOCKED on expiry of WR_LOCKED_TIMEOUT. | both | passed | passed |
WSMG_023 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from LOCKED to RESP_CALIB_REQ state on the reception of CALIBRATE message before EXC_TIMEOUT_RETRY occurs. | both | passed | passed |
WSMG_024 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from LOCKED to IDLE state after EXC_TIMEOUT_RETRY occurs | both | passed | passed |
WSMG_025 | To verify that a WRPTP enabled device with it's port in PTP Slave re-enters to it's port WR state RESP_CALIB_REQ on expiry of WR_RESP_CALIB_REQ_TIMEOUT when otherPortCalPeriod is 0x0. | both | passed | passed |
WSMG_026 | To verify that a WRPTP enabled device with it's port in PTP Slave re-enters to it's port WR state RESP_CALIB_REQ on expiry of WR_RESP_CALIB_REQ_TIMEOUT when otherPortCalPeriod is greater than 0x0 (otherPortCalPeriod and otherPortCalRetry should be ignored by DUT). | both | passed | passed |
WSMG_027 | To verify that a WRPTP enabled device with it's port in PTP Slave sends CALIBRATE message and transitions it's port WR state from RESP_CALIB_REQ to CALIBRATION state on the reception of CALIBRATED message before EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are 0x0. | both | passed | passed |
WSMG_028 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from RESP_CALIB_REQ to IDLE state after EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are 0x0. | both | passed | passed |
WSMG_029 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from RESP_CALIB_REQ to CALIBRATION state on the reception of CALIBRATED message before EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are greater than 0x0 (otherPortCalPeriod and otherPortCalRetry should be ignored by DUT). | both | passed | passed |
WSMG_030 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from RESP_CALIB_REQ to IDLE state after EXC_TIMEOUT_RETRY occurs when otherPortCalPeriod and otherPortCalRetry are greater than 0x0 (otherPortCalPeriod and otherPortCalRetry should be ignored by DUT). | both | passed | passed |
WSMG_031 | To verify that a WRPTP enabled device with it's port in PTP Slave sends CALIBRATED message and transitions it's port WR state from the CALIBRATION to CALIBRATED state. | both | passed | passed |
WSMG_032 | To verify that a WRPTP enabled device with it's port in PTP Slave sends CALIBRATED message and re-enters to it's port WR state CALIBRATED on expiry of WR_CALIBRATED_TIMEOUT. | both | passed | passed |
WSMG_033 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from CALIBRATED to IDLE state after EXC_TIMEOUT_RETRY occurs. | both | passed | passed |
WSMG_034 | To verify that a WRPTP enabled device with it's port in PTP Slave transitions it's port WR state from WR_LINK_ON to IDLE state upon successful completion of WR link setup process | both | passed | passed |
WSMG_035 | To verify that a WRPTP enabled device with it's port in PTP Slave stores otherPortDeltaTx, otherPortDeltaRx, otherPortCalPeriod, otherPortCalRetry and otherPortCalSendPattern received in CALIBRATE message. | both | passed | passed |
Historical data
The following table gives a summary of the ATTEST-CTS-WRPTP tests performed for different versions of the WR switch firmware. Detailed results are available in ATTEST-CTS_WRPTP_TestResults.xlsx
Firmware version | Test report | Date | Test cases | Passed | Failed | Aborted |
---|---|---|---|---|---|---|
wr-switch-sw-v6.0-beta1-10-gb3b066a ppsi-v2016.12-374-g90c5320 |
report (Raw Ethernet) | 28-04-2020 | 49 | 46 | 0 | 3 (*) |
wr-switch-sw-v6.0-beta1-10-gb3b066a ppsi-v2016.12-374-g90c5320 |
report (UDP) | 28-04-2020 | 49 | 46 | 0 | 3 (*) |
wr-switch-v5.0.1 ppsi-v2016.12-175-gc854572-dirty |
report (Raw Ethernet) | 31-01-2019 | 49 | 45 | 1 | 3 (*) |
wr-switch-v5.0.1 ppsi-v2016.12-175-gc854572-dirty |
report (UDP) | 31-01-2019 | 49 | 45 | 1 | 3 (*) |
(*) NOTE: The 3 tests aborted for Raw Ethernet are UDP-only, similarly the 3 tests aborted for UDP are Raw Ethernet-only