[parsec-users] Segmentation fault for Dedup (uncompress)

Rafael Asenjo asenjo at uma.es
Fri Aug 14 06:15:34 EDT 2009


Hi Chris,

I found the problem and I have tested two solutions. After calling  
dedup with the -u option to uncompress a file, the function Decompress  
(in file decode.c) is called and it has this declaration:

   u_char tmpstr[UNCOMPRESS_BOUND];

which tries to allocate 10^7 bytes in the stack  
(UNCOMPRESS_BOUND=10000000, defined in dedupdef.h). This may be too  
much for some architectures and operating systems (actually for all  
the platforms I have tested, ranging from dual cores, quad-cores,  
itanium, SLES 9, 10, Ubuntu 8.10 and Mac OS Leopard).

I first changed the UNCOMPRESS_BOUND constant to 10^6 and that was  
enough to successfully run "dedup -u". If you further reduce this  
value you will impact in the decompress performance (I've tried a  
value of 100 which leads to really large uncompressing times).

The other approach I've followed is to keep the original value of  
UNCOMPRESS_BOUND, but allocating the tmpstr array in the heap:

   u_char *tmpstr;
   tmpstr= (u_char *)malloc(UNCOMPRESS_BOUND*sizeof(u_char));

Both approaches work fine, as far as I have tested, in the platforms  
previously mentioned. If you think there is a better alternative,  
please, just let me know. Thank you very much in advance,

Rafa.

P.S. A curiosity: the uncompress time increases with the size of the  
file more quickly than the compress time. Some numbers:

size			compress time		uncompress time
simsmall	1.5s					0.15s	
simlarge		15.75s				53.48s
native		58.28				1674.24s !!!

This numbers were obtained for the serial version of dedup. The  
decompressed file matches the original file (the one used to generate  
a compressed .ddp file) so decompressing is working fine, but too  
slow. Target machine is a SUSE LINUX 10.1 (X86-64) with 2 quad-core  
Xeon X5355  @ 2.66GHz without any other load apart from the dedup  
process.

El 11/08/2009, a las 21:23, Christian Bienia escribió:

> Hi Rafa,
>
> How do you invoke the program? It works fine for me. Here’s how  
> you’re supposed to decode a compressed file:
>
>         dedup -u -f -i archive.dat.ddp -o archive.dat
>
> It’s possible that you don’t call the program properly and we didn’t  
> catch an undefined state. Could you let me know whether the above  
> line works for you?
>
> - Chris
>
>
> From: parsec-users-bounces at lists.cs.princeton.edu [mailto:parsec-users-bounces at lists.cs.princeton.edu 
> ] On Behalf Of Rafael Asenjo
> Sent: Monday, August 10, 2009 6:29 AM
> To: parsec-users at lists.cs.princeton.edu
> Subject: [parsec-users] Segmentation fault for Dedup (uncompress)
>
> Hi,
>
> I've been using Parsec 1.0 and 2.0 for a while. I was testing the  
> uncompress option for Dedup (-u) so as to validate that  
> uncompressing a file gives you the original one. However I found  
> that calling dedup with -u produces a segmentation fault. I've  
> tested it in two platforms (intel Xeon, and Itanium 2), Linux/x86_64  
> and Linux/Itanium, with SLES 9 and 10, serial and parallel (2  
> threads), with simtest, simdev and simlarge, gcc 4.1 and 4.2 and  
> with parsec 1.0 and 2.0.
>
> Using gdb I found that segmentation fault arises when calling  
> "uncompress" function (from libz), at function Decompress in file  
> decoder.c. Does someone notice that before. Any help will be much  
> appreciated.
>
> Thanks,
>
> Rafa.
> __
> Dr. Rafael Asenjo Plaza
> Dept. Arquitectura de Computadores
> Complejo Tecnologico Campus de Teatinos
> E-29071 MALAGA (SPAIN)
> Tel: +34 95 213 27 91
> Fax: +34 95 213 27 90
> http://www.ac.uma.es/~asenjo
>
>
> _______________________________________________
> parsec-users mailing list
> parsec-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users

__
Rafael Asenjo Plaza
Dept. Arquitectura de Computadores
Complejo Tecnologico Campus de Teatinos
E-29071 MALAGA (SPAIN)
Tel: +34 95 213 27 91
Fax: +34 95 213 27 90
http://www.ac.uma.es/~asenjo


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090814/2f6dceb9/attachment.html>


More information about the parsec-users mailing list