[ixp1200] Fast Ports - sop_seq roll over

[Johnson, Erik J]

I guess I am the guilty party again =).  Comments are inline.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Erik J. Johnson
Intel Research & Development
erik.j.johnson at intel.com
NOT SPEAKING FOR INTEL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


-----Original Message-----
From: Yannis Haritakis [mailto:haritak at ics.forth.gr] 
Sent: Wednesday, September 04, 2002 10:46 AM
To: ixp1200 at CS.Princeton.EDU
Subject: [ixp1200] Fast Ports - sop_seq roll over



Hi,

 Based on the example of Fast Port usage of the book programming IXP1200
(intel press), I made the following observation. (If familiar please
respond).



 The example uses sop_seq field from rcv_ctl register in order to put the
MPKTs of a PACKET in the correct order.

<erik> No, it doesn't.  It uses the SOP_SEQ fields to order packets, not
mpackets.  Mpacket ordering is ensured through the interthread signals.
(See page 227, first paragraph).  I think the point was the this field could
be used to order mpackets, but the example does not do that.

 However, in general, using the sop_seq allows for only 16 different  MPKTS.
That is, the number rolls over after 0xF.  A 1500bytes PACKET results in 24
MPKTS.  If more than one thread handle these MPKTS in parallel, there is  a
possibility for two threads to receive the same sop_seq.

<erik> See answer above.  What is precluded is receiving more than 16
packets simultaneously, since then the packet ordering might be lost.
Again, that is an implementation issue.  Interthread signaling (or any kind
of ordered lock) could be used ensure packet ordering if you needed more
than 16 packets to be received simultaneously.


 Another relative issue is the next one :

 In the book it is written that the sop_seq field of rcv_cntl register,
follows the values of the sop_seq register. This is wrong.

 The sop_seq register changes only when an SOP-MPKT is received, while the
sop_seq field in rcv_cntl register behaves as described in the book. (Always
increases, and get the sop_seq value when a SOP-MPKT is received). To
support this behavior there are two counters (documented in hardware
reference manual)

<erik> Good point. I'll add the correction/clarification to the book's
errata.  The two fields only track for SOP mpackets.


 Now the question is this :

 If a thread, after the ctx_wait(start_receive) executes two back-to-back
csr_reads :
	csr_read(sop_seq, sync_none);
	csr_read(rcv_cntl, ctx_wait);

will end up having two sop_seq: The value of the register
and the field of rcv_cntl.

 Will the value of the register be the sop-seq number that was assigned to
the PACKET that this thread is working for ?

<erik> for the same reason you mentioned above.  Now, the more interesting
question is if the mpacket is an SOP, will the two values be the same?  In
that case, there still seems to me to be the possibility of them differing
(i.e., a race condition) if the SOP_SEQ register is updated based on the RSM
receiving the mpacket (instead of after the rcv control register is read).
I am not sure of the actual behavior of the hardware.

 I think the answer is "no necessarily"!

Yannis Charitakis
haritak at ics.forth.gr