Proposal for new analyzer stream format

[martinling]

The current format used in the analyzer data stream is a 16-bit length for each packet, followed by the packet bytes. This format needs to be changed, at a minimum in order to add timestamp data.

Whilst we're changing the format, it would be good to look at how to include other features, and also minimize the number of bytes needed for each packet. There is considerable redundancy in the packet data, and by taking advantage of this, we may be able to extend the conditions under which LUNA can maintain a continuous capture, perhaps to the point of doing so indefinitely.

Possible optimisations

• After packets are validated, we can strip out fields used for error detection, to minimize the space needed to store them. The removed fields can be reconstructed at the host if required. Invalid packets must be stored in full exactly as received on the wire.

• Rather than prepending each packet with a length, we can begin each packet with its PID field. With the exception of data packets, all USB packets have a fixed length determined by their PID. We only need to include the length of data packets. Captured packets with the wrong length for their PID must be flagged as babble errors.

• One PID value is reserved and not valid on the wire. We can use this value to indicate that an invalid packet or other event follows. Received packets with this reserved PID value must be stored as invalid packets.

• The PID byte has 4 redundant bits. There are only 16 PID values, each of which can be uniquely identified by either their upper or lower 4 bits. The other bits are the inverse, and are used for error detection. When storing the PID of a validated packet we need only use 4 bits, and can use the rest of the PID byte for other fields. Received packets with invalid PID bytes must be stored as invalid packets.

• Normally, packets will have valid CRC fields. We can both save bytes, and reduce CPU time needed for validation on the host, if we validate CRCs in gateware and then strip out the CRC fields of valid packets, leaving just a valid flag. This frees up two bytes from data packets, and 5 bits from token packets. In the case of CRC failures, we must include the CRC as received.

• The length of a data packet is at most 1024 bytes, so requires 11 bits to store. We can fit this in one length byte plus the spare nibble of the PID byte.

Timestamping

Normally, the host will be sending start-of-frame (SOF) packets at an interval of either 1000 Hz (full speed) or 8000 Hz (high speed). It makes sense to optimize our timestamp storage around this pattern, and furthermore to do so specifically for the high speed case, as that is where we must be most concerned about overhead.

We can timestamp packets relative to the previous packet, which normally should be no longer ago than the last SOF packet. The analyzer runs on a 60MHz clock, so there are ~7500 clock cycles between HS microframes. A 13-bit timestamp with a range of 8192 cycles would be sufficient to timestamp any packet relative to the last microframe SOF.

A neat solution is possible for storing a 13-bit timestamp. For all token PIDs, which are the types that can start a transaction, there is a 5-bit CRC field. We do not need to retain validated CRCs, so we can replace this field with the first 5 bits of the timestamp. We then only need one additional byte to timestamp these packets.

For all other packets, there is a further timing constraint: they must follow the previous packet in a transaction within a specified time. The standard specifies the maximum end-to-end delay for HS in section 7.1.19.2. A response starting within 736 bit times must be accepted, and a response staring later than 816 bit times must be rejected. The latter corresponds to 102 cycles for us, so a 7-bit timestamp is sufficient. If a response is received any later than this, it must be treated as an invalid packet.

As such, only a single byte needs to be added to each valid packet for timestamping in the case of normal high-speed traffic.

Where longer times have elapsed, we can set a flag and include a wider timestamp. A 16-bit timestamp can represent up to ~1.09ms at a 60MHz clock, which is sufficient to span the time between SOF packets on a full speed bus. When more than this time elapses between packets, we can emit events for each time the 16-bit counter wraps, to allow the host to keep track of elapsed gaps between packets.

Non-packet events

In addition to packets, the analyzer may report other events, such as:

• Bus resets
• Speed changes
• Errors reported by PHY
• Buffer filled on the analyzer
• Trigger events

The stream format should allow for all currently required event types, with scope for other types to be added in future.

Implementation considerations

Key factors that need to be considered in the format:

• Although we can modify bytes already written to the buffer, we have a limited number of cycles in which to do so, so we should avoid any formats that would require moving data around in the buffer after it has arrived.
• There are two failure modes which are observable immediately when we see the first byte of a packet:
  1. Packet too late: applicable to data and handshake packets only.
  2. Invalid PID.
• There are two failure modes that we may discover only after we have started reading and buffering a packet:
  1. CRC failure: applicable to token and data packets only.
  2. Babble (excess length): applicable to any packet.

Proposed scheme

Token packets (SOF, SETUP, IN, OUT, PING, SPLIT)

Optimal token packet:

Byte	0	1	2	3
Data	pid[0:4], late=0, bad_crc=0, babble=0	payload[0:8]	payload[8:11], time[0:5]	time[5:13]

Additional bytes may follow if any of the following flags are set:

• The late flag indicates that the packet started too late for a 13-bit timestamp.
  • Note that this does not indicate the packet is invalid.
  • A 16-bit timestamp will be used.
• The bad_crc flag indicates that the CRC5 checksum did not match.
  • The CRC5 will be included as received.
  • A 16-bit timestamp will be used.
• The babble flag indicates that excess bytes followed the packet.
  • The babble length, bablen, will be included.
  • The babble bytes will follow at the end of the packet.

The following combinations are possible:

babble	bad_crc	late	Byte 2 [4:0]	Byte 3	Byte 4	Byte 5	Byte 6	Byte 7
0	0	0	time[0:5]	time[5:13]
0	0	1	0	time[0:8]	time[8:16]
0	1	X	crc[0:5]	time[0:8]	time[8:16]
1	0	0	time[0:5]	time[5:13]	bablen[0:8]	bablen[8:16]	babble...
1	0	1	0	time[0:8]	time[8:16]	bablen[0:8]	bablen[8:16]	babble...
1	1	X	crc[0:5]	time[0:8]	time[8:16]	bablen[0:8]	bablen[8:16]	babble...

Data packets (DATA0, DATA1, DATA2, MDATA)

Optimal data packet:

Byte	0	1	2 - N	N + 1
Data	pid[0:4], bad_crc=0, length[0:3]	length[3:11]	data bytes	time[0:8]

There is only one available flag bit in this format, but there is headroom in the length and timestamp fields to indicate the other errors.

• The bad_crc flag indicates that the CRC16 checksum did not match.
  • The CRC16 will be included as received.
  • This check is meaningless in the case of babble, so the babble and bad CRC conditions are mutually exclusive.
• Any length greater than 1024 indicates a babble error.
  • If length == 2047 then an xlen field is also included.
  • The number of babble bytes is given by length - 1024 + xlen.
  • The babble bytes will follow at the end of the packet.
• If time == 255 then an xtime field is also included.
  • The total cycle count since the last packet is given by time + xtime.
  • For high speed, any time greater than 102 indicates a late error.

The following combinations are possible:

length	bad_crc	time	Byte N + 2	Byte N + 3	Byte N + 4	Byte N + 5	Byte N + 6
0-1024	0	0-254
1025-2046	0	0-254	babble...
0-1024	0	255	xtime[0:8]	xtime[8:16]
1025-2046	0	255	xtime[0:8]	xtime[8:16]	babble...
0-1024	1	0-254	crc[0:8]	crc[8:16]
0-1024	1	255	crc[0:8]	crc[8:16]	xtime[0:8]	xtime[8:16]
2047	0	0-254	xlen[0:8]	xlen[8:16]	babble...
2047	0	255	xtime[0:8]	xtime[8:16]	xlen[0:8]	xlen[8:16]	babble...

Handshake packets (ACK, NAK, STALL, NYET, ERR)

Optimal handshake packet:

Byte	0	1
Data	pid[0:4], late=0, babble=0	time[0:8]

There is no CRC for a handshake packet, so only the late and babble cases need to be handled:

• The late flag indicates that a 16-bit timestamp is used.
  • For high speed, any timestamp greater than 102 indicates an invalid packet.
• The babble flag indicates that excess bytes followed the packet.
  • The babble length, bablen, will be included.
  • The babble bytes will follow at the end of the packet.

The following combinations are possible:

babble	late	Byte 1	Byte 2	Byte 3	Byte 4	Byte 5
0	0	time[0:8]
0	1	time[0:8]	time[8:16]
1	0	time[0:8]	bablen[0:8]	bablen[8:16]	babble...
1	1	time[0:8]	time[8:16]	bablen[0:8]	bablen[8:16]	babble...

Invalid packets and other events

Invalid packet:

Byte	0	1	2	3	4	5
Data	pid[0:4] = 0 (RSVD), flags=0	time[0:8]	time[8:16]	length[0:8]	length[8:16]	packet bytes...

The reserved PID can also be used for other types of events, with flag bits used to identify the event type and bytes to follow.

One event that will be required for the timestamping to function correctly is a timestamp counter wrap event, which should be emitted each time 2^16 cycles have elapsed since the end of the last packet. At a 60MHz clock this will occur every ~1.09ms.