[parsec-users] Cross Compiling Dedup for Alpha Arch using M5 Simulator

Christian Bienia cbienia at CS.Princeton.EDU
Sun Jul 19 18:08:53 EDT 2009


Ef,

 

The size of the hash table is independent of the input data size because (1)
the number of data chunks is data dependent and can thus not be determined
without actually computing all chunks and (2) the data is typically streamed
through the program so not even the amount of data is known. The hash table
size is therefore hardwired into the program and set to a very large value
by default.

 

I believe this is the part the causes trouble:

 

  cache = create_hashtable(65536, hash_from_key_fn, keys_equal_fn);

  if(cache == NULL) {

    printf("ERROR: Out of memory\n");

    exit(1);

  }

 

Interestingly, memory allocation does not fail during the hash table
allocation but at a later point in the encoder, as the error message
suggests. You should also note that you can't change the hash table size
without altering the memory access characteristics of the program. Try it
with several GB.

 

- Chris

 

 

From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of ef
Sent: Sunday, July 19, 2009 5:28 PM
To: PARSEC Users
Subject: Re: [parsec-users] Cross Compiling Dedup for Alpha Arch using M5
Simulator

 

Hello.
Hmmm, I am testing with simsmall I had initially had around 512MB for Dedup
Allocation I bumped it up to 1024MB and it still gives me the same error.
Any other ideas?

Thanks 
EF

2009/7/19 Christian Bienia <cbienia at cs.princeton.edu>

Hi Ef,

 

I believe this is another error unrelated to the first one. Dedup tries to
allocate a huge hash table to manage the data chunks from the input data.
You probably simply ran out of memory. Try to tell M5 to allow processes to
allocate more memory and dedup will probably run fine.

 

- Chris

 

 

From: parsec-users-bounces at lists.cs.princeton.edu
[mailto:parsec-users-bounces at lists.cs.princeton.edu] On Behalf Of ef
Sent: Sunday, July 19, 2009 4:22 PM
To: PARSEC Users
Subject: Re: [parsec-users] Cross Compiling Dedup for Alpha Arch using M5
Simulator

 

Hello,

I removed the flag O_LARGEFILE under encoder.c:
 /* src file open if((fd = open(conf->infile, O_RDONLY | O_LARGEFILE)) < 0)
      EXIT_TRACE("%s file open error TEST %s\n", conf->infile,
strerror(errno));

I got this error in M5:
[DataProcess] Memory allocation failed.


It sounds like I need to investigate my compiler and libc versions, as you
suggested.
Thanks,
EF

2009/7/19 Joel Hestness 

Hi EF,

  I encountered this in our first iteration of compiling PARSEC.  The
compiler and libc version you are using don't like the O_LARGEFILE flag used
when you open the file in encoder.c (perhaps the large file support is
standard on ALPHA?).  I can't remember if it fixed the problem, but I had
removed the flag in my current code base.  I would suggest trying that
first.  You might need to try a different version of the compiler/libc.

 

  Good luck,

  Joel

 

 

2009/7/19 ef 

Hello,
I have cross compiled Dedup for use in M5, however when I run the binary in
M5 I get the following output:

PARSEC Benchmark Suite Version 2.0
[Encode] /parsec/install/inputs/dedup/medias.dat file open error Not a
directory

Not sure where to even start fixing this, anyone have any clue? Any hints or
thoughts are appreciated.

Thanks,
EF

_______________________________________________
parsec-users mailing list
parsec-users at lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/parsec-users




-- 
 Joel Hestness
 PhD Student, Computer Architecture
 Dept. of Computer Science, University of Texas - Austin
 http://www.cs.utexas.edu/~hestness <http://www.cs.utexas.edu/%7Ehestness> 

_______________________________________________
parsec-users mailing list
parsec-users at lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/parsec-users

 


_______________________________________________
parsec-users mailing list
parsec-users at lists.cs.princeton.edu
https://lists.cs.princeton.edu/mailman/listinfo/parsec-users

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090719/4a0aca0d/attachment.htm>


More information about the parsec-users mailing list