Veryx Tests
We use Veryx ATTEST PTP Conformance Test Suite to perform regression tests of WR Switch releases. Out of 87 tests grouped into 6 groups, we have selected tests cover the features implemented by the WR Switch. These test groups are:
- Message Format Group (MFG)
- Message Handling Group (MHG)
- Message Transaction Group (MTG)
- PTP BMC Election Group (BEG)
- PTP Configuration Group (PCG)
- STATE MACHINE GROUP (SMG)
The tests that we expect the WR Switch to pass are listed below. There is a number of tests that do not pass due to known bugs, these are listed below as well.
Tests that a release must pass (65 tests for IEEE802.3 and 65 tests for UPD mapping)
TEST Name | Test Purpose Description | IEEE802.3 and/or UDP | v5.0.1 | v6.0 |
---|---|---|---|---|
BEG_001 | Verify that a boundary clock is elected as the Master based on Priority1. | both | passed | passed |
BEG_002 | Verify that in a boundary clock, the Master state is elected based on the clockClass value, when an Announce message with same Priority1 is received from the neighbour. | both | passed | passed |
BEG_003 | Verify that in a boundary clock, the Master state is elected based on the clockAccuracy value, when an Announce message with same Priority1 and clockClass value is received from the neighbour. | both | passed | passed |
BEG_004 | Verify that in a boundary clock, the Master state is elected based on the Variance value, when an Announce message with same Priority1, clockClass and clockAccuracy value is received from the neighbour. | both | passed | passed |
BEG_005 | Verify that in a boundary clock, the Master state is elected based on the Priority2 value, when an Announce message with same Priority1, clockClass, clockAccuracy and Variance value is received from the neighbour. | both | passed | passed |
MFG_001 | Verify that the boundary clock sends the UDP/IPv4 Announce message with following frame format. | UDP | passed | passed |
MFG_002 | Verify that the boundary clock sends the UDP/IPv4 Sync message with following frame format. | UDP | passed | passed |
MFG_003 | Verify that the boundary clock sends the UDP/IPv4 Follow_Up message with following frame format. | UDP | passed | passed |
MFG_004 | Verify that the boundary clock sends the UDP/IPv4 Delay_Req message with following frame format. | UDP | passed | passed |
MFG_005 | Verify that the boundary clock sends the UDP/IPv4 Delay_Resp message with following frame format. | UDP | passed | passed |
MFG_006 | Verify that the boundary clock sends the UDP/IPv4 PDelay_Resp message with following frame format. | UDP | passed | passed |
MFG_007 | Verify that the boundary clock sends the UDP/IPv4 PDelay_Resp_Follow_Up message with following frame format. | UDP | passed | passed |
MFG_008 | Verify that the boundary clock sends the IEEE 802.3 Announce message with following frame format. | IEEE802.3 | passed | passed |
MFG_009 | Verify that the boundary clock sends the IEEE 802.3 Sync message with following frame format. | IEEE802.3 | passed | passed |
MFG_010 | Verify that the boundary clock sends the IEEE 802.3 Follow_Up message with following frame format. | IEEE802.3 | passed | passed |
MFG_011 | Verify that the boundary clock sends the IEEE 802.3 Delay_Req message with following frame format. | IEEE802.3 | passed | passed |
MFG_012 | Verify that the boundary clock sends the IEEE 802.3 Delay_Resp message with following frame format. | IEEE802.3 | passed | passed |
MFG_013 | Verify that the boundary clock sends the IEEE 802.3 PDelay_Resp message with following frame format. | IEEE802.3 | passed | passed |
MFG_014 | Verify that the boundary clock sends the IEEE 802.3 PDelay_Resp_Follow_Up message with following frame format. | IEEE802.3 | passed | passed |
MFG_015 | Verify that the boundary clock sends the IEEE 802.3 PDelay_Req message with following frame format . | IEEE802.3 | passed | passed |
MFG_016 | Verify that the boundary clock sends the UDP/IPv4 PDelay_Req message with following frame format. | UDP | ||
MHG_001 | Verify that the boundary clock discards Pdelay_Req message when it is configured with delay request-response mechanism. | both | passed | passed |
MHG_002 | Verify that the boundary clock discards Delay_Req message when it is configured with peer delay mechanism. | both | failed | passed |
MHG_003 | Verify that the boundary clocks does not accept messages with different domain number than configured. | both | passed | passed |
MHG_004 | Verify that the boundary clock terminates all multicast Announce messages received on the port. | both | passed | passed |
MHG_005 | Verify that the boundary clock terminates all multicast Sync messages received on the port. | both | passed | passed |
MHG_006 | Verify that the boundary clock terminates all multicast Follow_Up messages received on the port. | both | passed | passed |
MHG_007 | Verify that the boundary clock terminates all multicast Delay_Req messages received on the port. | both | passed | passed |
MHG_008 | Verify that the boundary clock terminates all multicast Delay_Resp messages received on the port. | both | passed | passed |
MHG_009 | Verify that the boundary clock terminates all multicast Pdelay_Req messages received on the port. | both | passed | passed |
MHG_010 | Verify that the boundary clock ignores the Delay_Req message received on the port from which it was issued. | both | passed | passed |
MHG_011 | Verify that the boundary clock transmits Delay_Resp message as Multicast on reception of multicast Delay_Req messages. | both | passed | passed |
MHG_013 | Verify that the boundary clock terminates all multicast Pdelay_Resp messages received on the port. | both | passed | passed |
MHG_014 | Verify that the boundary clock terminates all multicast Pdelay_Resp_Follow_Up messages received on the port. | both | passed | passed |
MHG_015 | Verify that the boundary clock ignores the Pdelay_Req message received on the port from which it was issued. | both | passed | passed |
MTG_001 | Verify that the boundary clock transmits Delay_Req message on receiving Sync/ Sync and Follow_Up based on clock step on its slave port. | both | passed | passed |
MTG_002 | Verify that the boundary clock uses incremental sequence id for consecutive Announce messages. | both | passed | passed |
MTG_003 | Verify that the boundary clock uses incremental sequence id for consecutive Sync messages. | both | passed | passed |
MTG_004 | Verify that the boundary clock uses incremental sequence id for consecutive Delay_Req messages. | both | passed | passed |
MTG_005 | Verify that the boundary clock port supporting Delay Request-Response mechanism acting as master has alternateMasterFlag set to FALSE in Announce,Sync, Follow_Up,Delay_Resp PTP messsages | both | passed | passed |
MTG_006 | Verify that the boundary clock transmits Announce messages for every announce interval. | both | passed | passed |
MTG_007 | Verify that the boundary clock transmits Sync messages for every sync interval. | both | passed | passed |
MTG_008 | Verify that the boundary clock transmits Delay_Resp message on receiving Delay_Req on its master port. | both | passed | passed |
MTG_009 | Verify that the sequenceID and SourcePortIdentity field of received Follow_Up message shall match the sequenceId and SourcePortIdentity field of the prior Sync message respectively. | both | passed | passed |
MTG_010 | Verify that the requestingSequenceId and requestingSourcePortIdentity field of Delay_Resp message shall match the sequenceId and SourcePortIdentity field of the associated Delay_Req message respectively. | both | passed | passed |
MTG_012 | Verify that the boundary clock transmits Pdelay_Resp/ Pdelay_Resp and Pdelay_Resp_Follow_up message on receiving Pdelay_Req on its master port. | both | passed | passed |
MTG_013 | Verify that the sequenceId field, requestingSourcePortIdentity and domain number of Pdelay_Resp message shall match the sequenceId,SourcePortIdentity and domain number field of the associated Pdelay_Req message. | both | passed | passed |
MTG_014 | Verify that the correctionField, sequenceId field, requestingSourcePortIdentity and domain number of Pdelay_Resp_Follow_Up message shall match the correctionField, sequenceId, SourcePortIdentity and domain number field respectively of the associated Pdelay_Req message. | both | passed | passed |
MTG_015 | Verify that the boundary clock uses incremental sequence id for consecutive PDelay_Req messages. | both | passed | passed |
MTG_016 | Verify that the boundary clock transmits Pdelay_Req messages for every pdelay_request interval. | both | failed | passed |
MTG_017 | Verify that the boundary clock in multicast mode transmits Delay_Req message periodically at an interval between 0 and twice 2^portDS.logMinDelayReqInterval (where portDS.logMinDelayReqInterval is the logMessageInterval of last Delay_Resp message received). | both | passed | passed |
PCG_001 | Verify that the boundary clock allows to configure PTP on multiple ports. | both | passed | passed |
PCG_005 | Verify that the boundary clock configured for multicast transmission in Master has the unicastFlag field as FALSE in Delay Request-Response | both | passed | passed |
PCG_006 | Verify that the boundary clock configured for multicast transmission has the unicastFlag field as FALSE in Peer Delay messages. | both | passed | passed |
PCG_009 | Verify that the boundary clock configured for Two-Step clock has the twoStepFlag field as TRUE in Sync messages. | both | passed | passed |
PCG_010 | Verify that the boundary clock configured for Two-Step clock has the twoStepFlag field as TRUE in PDelay_Resp messages. | both | passed | passed |
PCG_012 | Verify that the boundary clock supports to configure priority2 to any value in range 0-255. | both | failed | passed |
PCG_018 | Verify that the boundary clock configured for multicast transmission in Slave has the unicastFlag field as FALSE in Delay Request message. | both | passed | passed |
SMG_002 | Verify that the boundary clock does not transmit any PTP messages on its port in DISABLED state. | both | passed | passed |
SMG_003 | Verify that the boundary clock discards any PTP messages that are received on its port in DISABLED state. | both | passed | passed |
SMG_004 | Verify that the boundary clock port in LISTENING state shall wait for the Announce receipt timeout to expire and transit to MASTER state. | both | passed | passed |
SMG_005 | Verify that the boundary clock port in LISTENING state shall wait to receive an Announce message to transit to MASTER/ SLAVE state. | both | passed | passed |
SMG_006 | Verify that the boundary clock port in MASTER state sends Announce messages periodically. | both | passed | passed |
SMG_007 | Verify that the boundary clock port in SLAVE state sends Delay_Req messages. | both | passed | passed |
SMG_008 | Verify that the boundary clock remains in SLAVE state if the same master continues | both | passed | passed |
SMG_010 | Verify that the boundary clock in LISTENING state moves to the SLAVE state when the master clock is selected. | both | passed | passed |
SMG_011 | Verify that the boundary clock in LISTENING state moves to DISABLED state when PTP is disabled at port. | both | passed | passed |
SMG_013 | Verify that the boundary clock in SLAVE state moves to DISABLED state when PTP is disabled at port. | both | passed | passed |
SMG_014 | Verify that the boundary clock in MASTER state moves to DISABLED state when PTP is disabled at port. | both | passed | passed |
SMG_015 | Verify that the boundary clock in SLAVE state moves to FAULTY state when a fault is detected. | both | failed | passed |
SMG_016 | Verify that the boundary clock in MASTER state moves to FAULTY state when a fault is detected. | both | failed | passed |
SMG_018 | Verify that the boundary clock in FAULTY state moves to DISABLED state when PTP is disabled at port. | both | aborted | passed |
SMG_019 | Verify that the boundary clock in DISABLED state moves to the LISTENING state when PTP is enabled at port. | both | passed | passed |
Test that do not pass because of known bugs (5 tests)
TEST Name | Test Descrition | Comment |
---|---|---|
MHG_012 | Verify that the boundary clock transmits Delay_Resp message as unicast on reception of unicast Delay_Req messages. | Unicast not implemented. |
MTG_011 | Verify that boundary clock configured for unicast transmission shall send follow_up message to same unicast address as the associated sync. | Unicast not implemented. |
PCG_002 | Verify that the boundary clock supports to configure domain number to any value in range 0 to 127. | Requires implementation of dynamic re-configuration. When we attempt to configure -1 as the domain number, DUT reverts to its default config and transmits Announce message with Domain number as 255 instead of retaining the domain number as 127 (the previous domain number configured by the test case). |
PCG_003 | Verify that the boundary clock configured for unicast transmission in Master has the unicastFlag field as TRUE in Delay Request-Response | Unicast not implemented. |
PCG_004 | Verify that the boundary clock configured for unicast transmission has the unicastFlag field as TRUE in Peer Delay messages. | Unicast not implemented. |
PCG_007 | Verify that the boundary clock configured for One-Step clock has the twoStepFlag field as FALSE in all Sync messages. | One-step configuration not supported/implemented |
PCG_008 | Verify that the boundary clock configured for One-Step clock has the twoStepFlag field as FALSE in Pdelay_Resp messages. | One-step configuration not supported/implemented |
PCG_011 | Verify that the boundary clock supports to configure priority1 to any value in range 0-255. | Requires implementation of dynamic re-configuration.When we attempt to configure 256 as priority1, DUT reverts to its default config and transmits Announce message with priority1 as 0 instead of retaining the domain number as 255 (the previous priority1 value configured by the test case) |
PCG_013 | Verify that the boundary clock does not allows to configure it as a Slave-only clock. | Requires implementation of dynamic re-configuration. |
PCG_014 | Verify that the DUT transmits announce message with logMessageInterval field which matches it's configured Announce Interval. | Requires implementation of dynamic re-configuration. When we attempt to configure -1 as Announce Interval, DUT reverts to its default config and transmits Announce message with logMessageInterval as 0 instead of retaining the logMessageInterval as 4 (the previous logMessageInterval value configured by the test case) |
PCG_015 | Verify that the DUT transmits multicast sync/ Sync and Follow_up message with logMessageInterval field which matches it's configured sync | Requires implementation of dynamic re-configuration. |
PCG_016 | Verify that the DUT transmits unicast sync/ sync and follow_up message with logMessageInterval field as 7F. | Unicast not implemented. |
PCG_017 | Verify that the boundary clock configured for unicast transmission in Slave has the unicastFlag field as TRUE in Delay Request message. | Unicast not implemented. |
SMG_017 | Verify that the boundary clock in FAULTY state moves to the LISTENING state when the fault is cleared. | To be investigated. |
Historical data
Firmware version | Test Mode | Test report | Date | Test cases | Passed | Failed | Aborted |
---|---|---|---|---|---|---|---|
v6.0-beta1-10-gb3b066a | 802.1 | Report | 29-04-2020 | 87 | 65 | 5 | 17 (*) |
v5.0-52-g2bfdf51 | 802.1 | Report | 5-12-2018 | 87 | 59 | 10 | 18 (*) |
v5.0-52-g2bfdf51 | UDP | Report | 5-12-2018 | 87 | 59 | 10 | 18 (*) |
(*) NOTE: Abandoned test cases are designed for UDP or Unicast while the test was run with Raw Ethernet multicast.