[parsec-users] Segmentation fault for Dedup (uncompress)

Christian Bienia cbienia at CS.Princeton.EDU
Fri Aug 14 15:29:45 EDT 2009


Hi Rafael,

 

Thanks a lot for the fix. I’ll include it in the next version of PARSEC,
most likely the version that allocates the the buffer dynamically on the
heap.

 

It’s not surprising to me that the uncompress time is pretty big. We only
developed the decoding part to test the encoder, so performance was never a
goal. I’m not sure why it grows as fast as you observe though. However, it’s
on my list of things to improve.

 

Best,

Chris

 

 

From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of Rafael
Asenjo
Sent: Friday, August 14, 2009 6:16 AM
To: PARSEC Users
Subject: Re: [parsec-users] Segmentation fault for Dedup (uncompress)

 

Hi Chris,

 

I found the problem and I have tested two solutions. After calling dedup
with the -u option to uncompress a file, the function Decompress (in file
decode.c) is called and it has this declaration:

 

  u_char tmpstr[UNCOMPRESS_BOUND];

 

which tries to allocate 10^7 bytes in the stack (UNCOMPRESS_BOUND=10000000,
defined in dedupdef.h). This may be too much for some architectures and
operating systems (actually for all the platforms I have tested, ranging
from dual cores, quad-cores, itanium, SLES 9, 10, Ubuntu 8.10 and Mac OS
Leopard).

 

I first changed the UNCOMPRESS_BOUND constant to 10^6 and that was enough to
successfully run "dedup -u". If you further reduce this value you will
impact in the decompress performance (I've tried a value of 100 which leads
to really large uncompressing times).

 

The other approach I've followed is to keep the original value of
UNCOMPRESS_BOUND, but allocating the tmpstr array in the heap:

 

  u_char *tmpstr;

  tmpstr= (u_char *)malloc(UNCOMPRESS_BOUND*sizeof(u_char));

 

Both approaches work fine, as far as I have tested, in the platforms
previously mentioned. If you think there is a better alternative, please,
just let me know. Thank you very much in advance,

 

Rafa.

 

P.S. A curiosity: the uncompress time increases with the size of the file
more quickly than the compress time. Some numbers:

 

size                              compress time             uncompress time

simsmall          1.5s
0.15s    

simlarge                       15.75s
53.48s

native              58.28                                       1674.24s !!!

 

This numbers were obtained for the serial version of dedup. The decompressed
file matches the original file (the one used to generate a compressed .ddp
file) so decompressing is working fine, but too slow. Target machine is a
SUSE LINUX 10.1 (X86-64) with 2 quad-core Xeon X5355  @ 2.66GHz without any
other load apart from the dedup process.

 

El 11/08/2009, a las 21:23, Christian Bienia escribió:





Hi Rafa,

 

How do you invoke the program? It works fine for me. Here’s how you’re
supposed to decode a compressed file:

 

        dedup -u -f -i archive.dat.ddp -o archive.dat

 

It’s possible that you don’t call the program properly and we didn’t catch
an undefined state. Could you let me know whether the above line works for
you?

 

- Chris

 

 

From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of Rafael
Asenjo
Sent: Monday, August 10, 2009 6:29 AM
To: parsec-users at lists.cs.princeton.edu
Subject: [parsec-users] Segmentation fault for Dedup (uncompress)

 

Hi,

 

I've been using Parsec 1.0 and 2.0 for a while. I was testing the uncompress
option for Dedup (-u) so as to validate that uncompressing a file gives you
the original one. However I found that calling dedup with -u produces a
segmentation fault. I've tested it in two platforms (intel Xeon, and Itanium
2), Linux/x86_64 and Linux/Itanium, with SLES 9 and 10, serial and parallel
(2 threads), with simtest, simdev and simlarge, gcc 4.1 and 4.2 and with
parsec 1.0 and 2.0.

 

Using gdb I found that segmentation fault arises when calling "uncompress"
function (from libz), at function Decompress in file decoder.c. Does someone
notice that before. Any help will be much appreciated.

 

Thanks,

 

Rafa.

__

Dr. Rafael Asenjo Plaza
Dept. Arquitectura de Computadores      

Complejo Tecnologico Campus de Teatinos

E-29071 MALAGA (SPAIN)

Tel: +34 95 213 27 91
Fax: +34 95 213 27 90        

http://www.ac.uma.es/~asenjo

 

_______________________________________________
parsec-users mailing list
parsec-users at lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/parsec-users

 

__

Rafael Asenjo Plaza
Dept. Arquitectura de Computadores      

Complejo Tecnologico Campus de Teatinos

E-29071 MALAGA (SPAIN)

Tel: +34 95 213 27 91
Fax: +34 95 213 27 90        

http://www.ac.uma.es/~asenjo

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090814/30438c55/attachment.html>


More information about the parsec-users mailing list