[parsec-users] Is x264 really pipeline parallelism?

Rafael Asenjo Plaza asenjo at uma.es
Wed Aug 3 22:20:13 EDT 2011

Hi Nick,

From my point of view, x264 should be more precisely be classified as a wavefront algorithm instead of a pipeline one. We have recently implemented a TBB-based template to easily code H.264. This work is briefly described in http://www.ac.uma.es/~compilacion/publicaciones/UMA-DAC-11-01.pdf (a shorter and corrected version of this technical report has been accepted in the HiPC11 conf.).



El 02/08/2011, a las 14:44, nchen.dev at mac.com escribió:

> Hi
> According to the PARSEC technical report <http://parsec.cs.princeton.edu/doc/parsec-report.pdf>: dedup, ferret and x264 are parallelized using pipeline parallelism. With dedup and ferret it was pretty obvious what the pipelines were and what the stages/filters were. However, it wasn't so with x264.
> From the technical report:
> "The parallel algorithm of x264 uses the pipeline model with one stage per input
> video frame. This results in a virtual pipeline with as many stages as there are
> input frames. x264 processes a number of pipeline stages equal to the number of
> encoder threads in parallel, resulting in a sliding window which moves from the
> beginning of the pipeline to its end."
> When looking at the code, it is not obvious that the frames actually form stages of the pipeline. My understanding of a pipeline is that it is a structure of linear series interconnected stages/filters. Each stage/filter performs some operation on its input token and puts an output token for the next stage. For instance, in ferret the stages are:
> Input -> Segmentation -> Extraction -> Indexing -> Ranking -> Output
> Instead in x264, what is going on is that a thread is spawn for each frame. Each frame is determined to be an I, B or P frame. This thread will then perform some operations (residual calculation, quantization, etc) on the macroblocks in the frame and finally encode it. Because frames and macroblocks have inter and intra dependencies on one another, these threads are not embarrassingly parallel. From this description, it seems that the PThreads version of x264 is more like task parallelism.
> I don't see how stages == frame. A stage/filter is a node that performs some operation. It cannot be an object like a frame.  If you wanted to parallelize this with pipeline parallelism, the stages/filters would be something like:
> Input -> Predict -> Residual (macroblocks) -> Quantization (macroblocks) ->  Encode -> Output
> and the tokens would be frames coming in. However, because frames might have interdependencies with other frames, it is not easy to express this as a simple pipeline.
> Am I missing something? Perhaps someone could elaborate on why x264 was identified as pipeline parallelism.
> Thanks!
> --
> Nick
> This post has some relation to "x264 Threading Model" <https://lists.cs.princeton.edu/pipermail/parsec-users/2011-April/001089.html>
> _______________________________________________
> parsec-users mailing list
> parsec-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users

Rafael Asenjo Plaza
Dept. Arquitectura de Computadores      
Complejo Tecnologico Campus de Teatinos
Tel: +34 95 213 27 91
Fax: +34 95 213 27 90        

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20110803/28e56699/attachment.html>

More information about the parsec-users mailing list