[parsec-users] Dedup Memory Allocation

Christian Bienia cbienia at CS.Princeton.EDU
Tue Aug 4 21:26:52 EDT 2009

Hi Grishma,


Those buffers are needed to read in the input file. The program should also
run fine with smaller buffers, but this will increase the number of memory
copies necessary once the end of the buffer is reached and a data chunk is
not complete.


- Chris



From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of grishma
Sent: Tuesday, August 04, 2009 5:10 PM
To: parsec-users at lists.cs.princeton.edu
Subject: [parsec-users] Dedup Memory Allocation




Following is the definition of MAXBUF (defined in file “dedupdef.h” )


#define MAXBUF (600*1024*1024)     /* 128 MB for buffers */ 


In bracket it is written 128 MB but figure comes out to be 600MB. Is this a
bug or it means something else?


Because of this the malloc function in encoder.c fails. See the following


Enoder.c, Line 652


  u_char * src = (u_char *)malloc(MAXBUF*2);  à Succeeds

  u_char * left = (u_char *)malloc(MAXBUF);   à Memory allocation fails here

  u_char * new = (u_char *) malloc(MAXBUF); à Fails here . Not enough memory
to assign.


In 32 bit systems application gets max 2 GB memory. The above three
statements exceeds 2 GB limit.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090804/01649970/attachment.html>

More information about the parsec-users mailing list