[parsec-users] Dedup Memory Allocation

Christian Bienia cbienia at CS.Princeton.EDU
Fri Aug 7 20:27:17 EDT 2009


Grishma,

 

I never tried running dedup on a 32-bit system, so I’m not sure whether
it’ll run or not. If you have details (other than the ones you already
published) then please send an email to parsec-users.

 

- Chris

 

 

From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of Kotecha,
Grishma
Sent: Friday, August 07, 2009 1:12 PM
To: PARSEC Users
Subject: Re: [parsec-users] Dedup Memory Allocation

 

So is that the case that current implementation of Dedup (with the
definition of MAXBUF as 600*1024*1024) will not run on 32 bit systems?

 

 

Grishma

  _____  

From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of Christian
Bienia
Sent: Tuesday, August 04, 2009 6:27 PM
To: 'PARSEC Users'
Subject: Re: [parsec-users] Dedup Memory Allocation

 

Hi Grishma,

 

Those buffers are needed to read in the input file. The program should also
run fine with smaller buffers, but this will increase the number of memory
copies necessary once the end of the buffer is reached and a data chunk is
not complete.

 

- Chris

 

 

From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of grishma
kotecha
Sent: Tuesday, August 04, 2009 5:10 PM
To: parsec-users at lists.cs.princeton.edu
Subject: [parsec-users] Dedup Memory Allocation

 

Hi,

 

Following is the definition of MAXBUF (defined in file “dedupdef.h” )

 

#define MAXBUF (600*1024*1024)     /* 128 MB for buffers */ 

 

In bracket it is written 128 MB but figure comes out to be 600MB. Is this a
bug or it means something else?

 

Because of this the malloc function in encoder.c fails. See the following
comment:

 

Enoder.c, Line 652

 

  u_char * src = (u_char *)malloc(MAXBUF*2);  à Succeeds

  u_char * left = (u_char *)malloc(MAXBUF);   à Memory allocation fails here

  u_char * new = (u_char *) malloc(MAXBUF); à Fails here . Not enough memory
to assign.

 

In 32 bit systems application gets max 2 GB memory. The above three
statements exceeds 2 GB limit.

 

Regards,

Grishma

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090807/3295bb92/attachment.htm>


More information about the parsec-users mailing list