[ixp1200] Fast Ports - sop_seq roll over

[Yannis Haritakis]

Erik,

 thanks a lot for your answer!

 I have some comments-questions to add:

> From: Yannis Haritakis [mailto:haritak at ics.forth.gr]
> Sent: Wednesday, September 04, 2002 10:46 AM
> To: ixp1200 at CS.Princeton.EDU
> Subject: [ixp1200] Fast Ports - sop_seq roll over
>
>
>
> Hi,
>
>  Based on the example of Fast Port usage of the book programming IXP1200
> (intel press), I made the following observation. (If familiar please
> respond).
>
>
>
>  The example uses sop_seq field from rcv_ctl register in order to put the
> MPKTs of a PACKET in the correct order.
>
> <erik> No, it doesn't.  It uses the SOP_SEQ fields to order packets, not
> mpackets.  Mpacket ordering is ensured through the interthread signals.
> (See page 227, first paragraph).  I think the point was the this field could
> be used to order mpackets, but the example does not do that.
>


Yes, you are right about that. I wanted to get to the below points. Sorry.
:-)


>  However, in general, using the sop_seq allows for only 16 different  MPKTS.
> That is, the number rolls over after 0xF.  A 1500bytes PACKET results in 24
> MPKTS.  If more than one thread handle these MPKTS in parallel, there is  a
> possibility for two threads to receive the same sop_seq.
>
> <erik> See answer above.  What is precluded is receiving more than 16
> packets simultaneously, since then the packet ordering might be lost.

Ok.


> Again, that is an implementation issue.  Interthread signaling (or any kind
> of ordered lock) could be used ensure packet ordering if you needed more
> than 16 packets to be received simultaneously.
>

Ok. I guess "packets" stands both for packets and mpkts.

>
>  Another relative issue is the next one :
>
>  In the book it is written that the sop_seq field of rcv_cntl register,
> follows the values of the sop_seq register. This is wrong.
>
>  The sop_seq register changes only when an SOP-MPKT is received, while the
> sop_seq field in rcv_cntl register behaves as described in the book. (Always
> increases, and get the sop_seq value when a SOP-MPKT is received). To
> support this behavior there are two counters (documented in hardware
> reference manual)
>
> <erik> Good point. I'll add the correction/clarification to the book's
> errata.  The two fields only track for SOP mpackets.
>
>
>  Now the question is this :
>
>  If a thread, after the ctx_wait(start_receive) executes two back-to-back
> csr_reads :
> 	csr_read(sop_seq, sync_none);
> 	csr_read(rcv_cntl, ctx_wait);
>
> will end up having two sop_seq: The value of the register
> and the field of rcv_cntl.
>
>  Will the value of the register be the sop-seq number that was assigned to
> the PACKET that this thread is working for ?
>
> <erik> for the same reason you mentioned above.  Now, the more interesting
> question is if the mpacket is an SOP, will the two values be the same?  In
> that case, there still seems to me to be the possibility of them differing
> (i.e., a race condition) if the SOP_SEQ register is updated based on the RSM
> receiving the mpacket (instead of after the rcv control register is read).
> I am not sure of the actual behavior of the hardware.
>
>  I think the answer is "no necessarily"!
>

I will agree with you, that the most probable is this:

the RSM updates SOP_SEQ and pushes the rcv_cntl info into the 4-entry
FIFO. That is, changing SOP_SEQ register before the thread reads the
corresponding rcv_cntl register.

This means that if 4 SOP-MPKTs arrive back to back (that is 4 PACKETS of
length 64Bytes arrive back to back), then when a thread reads the rcv_cntl
register from the top of the FIFO queue, the SOP_SEQ register will
already have changed three more times.


Moreover, assuming that a thread executes the above two commands (repeated
bellow):
>       csr_read(sop_seq, sync_none);
>       csr_read(rcv_cntl, ctx_wait);

Both commands will be queued in the same fbi-fifo (in the pull engine).
But before that the two commands will have to be
queued in the local (to the uengine fifo), and then the Bus Arbiter will
decide if both commands will go back to back on the shared bus to the pull
engine or if they will be split in order for another microengine to be
serviced.


If the Bus Arbiter is servicing a microengine until no more requests
exist, then we are ok. If not then we have no guarantee.


Am I right??? Does anyone have more information??


 Yannis Charitakis
 haritak at ics.forth.gr