[parsec-users] Dedup Memory Allocation

Kotecha, Grishma grishma.kotecha at intel.com
Fri Aug 7 13:11:35 EDT 2009


So is that the case that current implementation of Dedup (with the definition of MAXBUF as 600*1024*1024) will not run on 32 bit systems?


Grishma
________________________________
From: parsec-users-bounces at lists.cs.princeton.edu [mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of Christian Bienia
Sent: Tuesday, August 04, 2009 6:27 PM
To: 'PARSEC Users'
Subject: Re: [parsec-users] Dedup Memory Allocation

Hi Grishma,

Those buffers are needed to read in the input file. The program should also run fine with smaller buffers, but this will increase the number of memory copies necessary once the end of the buffer is reached and a data chunk is not complete.

- Chris


From: parsec-users-bounces at lists.cs.princeton.edu [mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of grishma kotecha
Sent: Tuesday, August 04, 2009 5:10 PM
To: parsec-users at lists.cs.princeton.edu
Subject: [parsec-users] Dedup Memory Allocation

Hi,

Following is the definition of MAXBUF (defined in file "dedupdef.h" )

#define MAXBUF (600*1024*1024)     /* 128 MB for buffers */

In bracket it is written 128 MB but figure comes out to be 600MB. Is this a bug or it means something else?

Because of this the malloc function in encoder.c fails. See the following comment:

Enoder.c, Line 652

  u_char * src = (u_char *)malloc(MAXBUF*2);  --> Succeeds
  u_char * left = (u_char *)malloc(MAXBUF);   --> Memory allocation fails here
  u_char * new = (u_char *) malloc(MAXBUF); --> Fails here . Not enough memory to assign.

In 32 bit systems application gets max 2 GB memory. The above three statements exceeds 2 GB limit.

Regards,
Grishma
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090807/205e86f4/attachment.html>


More information about the parsec-users mailing list