[ixp1200] IXP1200 uEngine hangup problem

[shyam at tejasnetworks.com]

Hi all,

We are facing some microengine hangup problem in 
IXP1200 as detailed out below. Has anybody encountered these
problems in their projects. Thanks.

Regards
-Shyam

   It looks as if there is some problem with the Context Arbiter
of IXP1200 or in the signal event handling mechanism. For example,
for the RX scheduling portion in Rx threads, we do

For the below code, hangup occurs in the following scenarios ---

(1) In macro port_rx_receive() at the instruction ctx_arb[start_receive]..

    On examining the registers CTX_<N>_SIG_EVENTS ---- <N> is the
    context number & CTX_<N>_WAKEUP_EVENTS, it is found that the
    start_receive FBI signal event is reflected in both the registers
    (this clears up any problem being present in our recv_req issue). If
    context arbiter signal event event handling mechanism was OK, the
    uEngine should proceed forward. The strange fact is that reading
    RCV_CNTL register from the StrongARM core in a burst, results in
    things going forward. This implies that RCV_CNTL FIFO clears on 
    read from StrongARM, and that uEngines have not been forcibly
    paused and rules out any chance of there being a memory corruption.
    So clearly, the FBI signal event is not getting received by
    uEngine threads, even though their registers reflect it several
    times.
    
(2) Sometime the uEngine thread gets stuck in a
    ctx_arb[voluntary] instruction.

    It is found that other 3 uEngine threads keep on executing
    with context switching happening between them, ruling out
    any chance of a uEngine thread hogging the resources. Also
    the particular thread is not set into paused state as
    seen from CTX_ENABLES.

(3) In macro port_rxrdy_chk() at the instrcution
    csr[read, $rec_rdy, rcv_rdy_lo], ctx_swap

    Here again as in case (1) above, the registers CTX_<N>_SIG_EVENTS &
    CTX_<N>_WAKEUP_EVENTS ---- <N> is the context number, reflect a
    FBI event for the context, while implies that the Rx Ready flags
    have been read. All the other contexts in this uEngine get scheduled 
    one after the other in this interval, ruling out any chance of another
    context hogging the uEngine. All contexts are enabled to run as
    seen from CTX_ENABLES.

Clearly the above 3 scenarios point to some problem occuring in context
arbitration, particulary FBI events & the voluntary token. What could
be the reason for strange occurences?

-------------------Relevant MACROS-----------------------------------------
// critsect_init
#macro critsect_init(abs_reg)
	immed[abs_reg, 0xFFFFFFFF]
#endm

// critsect_enter
#macro critsect_enter(abs_reg)
begin#:
	alu[--, --, B, abs_reg]
	br<0[end#], guess_branch
	ctx_arb[voluntary]
	br[begin#]
	
end#:
	immed[abs_reg, 0]
#endm

// critsect_exit
#macro critsect_exit(abs_reg)
	immed[abs_reg, 0xFFFFFFFF]
#endm

// port_rxrdy_chk
// Description: Check if receive port is ready and wait till it becomes so.
// Outputs:
//     io_abs_rdyFlagsSem    : (@GPR) Semaphore to restrict ready flag
//                             reads to a single thread within a uEngine
// Inputs:
//     in_rdyFlagOfPortSvced : (GPR) Ready Flag of port being serviced
//                             by current context
#macro port_rxrdy_chk(io_abs_rdyFlagsSem, in_rdyFlagOfPortSvced)
	.local $rec_rdy
	check_port#:
		// Block other contexts from reading rec ready, otherwise
		// threads processing packets wont get to use the FBI
		critsect_enter[io_abs_rdyFlagsSem] 
		csr[read, $rec_rdy, rcv_rdy_lo], ctx_swap // Get port ready
		// Allow other contexts to read rec ready
		critsect_exit[io_abs_rdyFlagsSem]
		// Check if port has data
		alu[--, in_rdyFlagOfPortSvced, AND, $rec_rdy]
		br!=0[port_ready#] // If port has data then go process it
		ctx_arb[voluntary] // If port doesn't have data, loop
		br[check_port#]
	port_ready#:
	.endlocal
#endm

// issue_rx_request
// Description: Issues an assembled receive request.
// Inputs:
//     in_recReq : (GPR) Assembled receive request
#macro issue_rx_request(in_recReq)
	.local $recReqCsr
		alu[$recReqCsr, --, B, in_recReq]
		csr[write, $recReqCsr, rcv_req], ctx_swap
	.endlocal
#endm

// port_rx_receive
// Description: Waits till a receive request serviced by RSM, reads and
//              returns the RCV_CNTL register contents
// Outputs:
//     out_rcvCntl : (GPR) Assembled Receive Request
#macro port_rx_receive(out_rcvCntl)
	.local $rec_csr
		ctx_arb[start_receive]
	    csr[read, $rec_csr, rcv_cntl], ctx_swap
		alu[out_rcvCntl, --, B, $rec_csr]
	.endlocal
#endm

-------------------End Relevant MACROS--------------------------------------

----MAIN THREAD-------------------

       --------------------------------
       ---------------------------------

       while (1)

       ----------------------------------------
       -----Some Processing--------------------
       ----------------------------------------

	// Wait till port assigned to this context has data, create and issue
	// a request to service it, and wait for request to be serviced by RSM
	port_rxrdy_chk(@rdready_inflight, g_svcedPortRdyFlag)

	critsect_enter(@req_inflight)
	issue_rx_request(rcvReq)
	//critsect_exit(@req_inflight)
	port_rx_receive(rcvCntl)
	critsect_exit(@req_inflight)

       ----------------------------------------
       -----Some Processing--------------------
       ----------------------------------------

       end while

----END MAIN THREAD-------------------