[parsec-users] Cross Compiling Dedup for Alpha Arch using M5 Simulator

ef snorlaxgb at gmail.com
Thu Jul 30 20:09:03 EDT 2009


Hello,
As Rick Strong pointed out their needs to be 2GB of physical  memory to get
it to run, but beware you might encounter issues when M5 tries to boot strap
the kernel as some versions of the console cannot create 2GB >=.

Anyways as I run Dedup I get the following error:
PARSEC Benchmark Suite Version Test 2.0
Error: Compress hash error

Anyone have any idea? As Previously recommended by Chris Benia I created a
hash table of size 2^22 under dedup.c.

Thanks
EF

On Wed, Jul 22, 2009 at 2:32 AM, ef <snorlaxgb at gmail.com> wrote:

> Hello,
>
> That didn't seem to help remove the error, I increased the hash table to
> order of magnitude bigger but that didnt work. I am using the simsmall input
> (11MB). I will keep looking into the error and will update the mailing list
> if I find anything. If anyone has any other ideas I would appreciate it.
>
>
> Thanks
> EF
>
> 2009/7/19 Christian Bienia <cbienia at cs.princeton.edu>
>
>>  Ef,
>>
>>
>>
>> The size of the hash table is independent of the input data size because
>> (1) the number of data chunks is data dependent and can thus not be
>> determined without actually computing all chunks and (2) the data is
>> typically streamed through the program so not even the amount of data is
>> known. The hash table size is therefore hardwired into the program and set
>> to a very large value by default.
>>
>>
>>
>> I believe this is the part the causes trouble:
>>
>>
>>
>>   cache = create_hashtable(65536, hash_from_key_fn, keys_equal_fn);
>>
>>   if(cache == NULL) {
>>
>>     printf("ERROR: Out of memory\n");
>>
>>     exit(1);
>>
>>   }
>>
>>
>>
>> Interestingly, memory allocation does not fail during the hash table
>> allocation but at a later point in the encoder, as the error message
>> suggests. You should also note that you can’t change the hash table size
>> without altering the memory access characteristics of the program. Try it
>> with several GB.
>>
>>
>>
>> - Chris
>>
>>
>>
>>
>>
>> *From:* parsec-users-bounces at lists.cs.princeton.edu [mailto:
>> parsec-users-bounces at lists.cs.princeton.edu] *On Behalf Of *ef
>> *Sent:* Sunday, July 19, 2009 5:28 PM
>>
>> *To:* PARSEC Users
>> *Subject:* Re: [parsec-users] Cross Compiling Dedup for Alpha Arch using
>> M5 Simulator
>>
>>
>>
>> Hello.
>> Hmmm, I am testing with simsmall I had initially had around 512MB for
>> Dedup Allocation I bumped it up to 1024MB and it still gives me the same
>> error. Any other ideas?
>>
>> Thanks
>> EF
>>
>> 2009/7/19 Christian Bienia <cbienia at cs.princeton.edu>
>>
>> Hi Ef,
>>
>>
>>
>> I believe this is another error unrelated to the first one. Dedup tries to
>> allocate a huge hash table to manage the data chunks from the input data.
>> You probably simply ran out of memory. Try to tell M5 to allow processes to
>> allocate more memory and dedup will probably run fine.
>>
>>
>>
>> - Chris
>>
>>
>>
>>
>>
>> *From:* parsec-users-bounces at lists.cs.princeton.edu [mailto:
>> parsec-users-bounces at lists.cs.princeton.edu] *On Behalf Of *ef
>> *Sent:* Sunday, July 19, 2009 4:22 PM
>> *To:* PARSEC Users
>> *Subject:* Re: [parsec-users] Cross Compiling Dedup for Alpha Arch using
>> M5 Simulator
>>
>>
>>
>> Hello,
>>
>> I removed the flag O_LARGEFILE under encoder.c:
>>  /* src file open if((fd = open(conf->infile, O_RDONLY | O_LARGEFILE)) <
>> 0)
>>       EXIT_TRACE("%s file open error TEST %s\n", conf->infile,
>> strerror(errno));
>>
>> I got this error in M5:
>> [DataProcess] Memory allocation failed.
>>
>>
>> It sounds like I need to investigate my compiler and libc versions, as you
>> suggested.
>> Thanks,
>> EF
>>
>> 2009/7/19 Joel Hestness
>>
>> Hi EF,
>>
>>   I encountered this in our first iteration of compiling PARSEC.  The
>> compiler and libc version you are using don't like the O_LARGEFILE flag used
>> when you open the file in encoder.c (perhaps the large file support is
>> standard on ALPHA?).  I can't remember if it fixed the problem, but I had
>> removed the flag in my current code base.  I would suggest trying that
>> first.  You might need to try a different version of the compiler/libc.
>>
>>
>>
>>   Good luck,
>>
>>   Joel
>>
>>
>>
>>
>>
>> 2009/7/19 ef
>>
>> Hello,
>> I have cross compiled Dedup for use in M5, however when I run the binary
>> in M5 I get the following output:
>>
>> PARSEC Benchmark Suite Version 2.0
>> [Encode] /parsec/install/inputs/dedup/medias.dat file open error Not a
>> directory
>>
>> Not sure where to even start fixing this, anyone have any clue? Any hints
>> or thoughts are appreciated.
>>
>> Thanks,
>> EF
>>
>> _______________________________________________
>> parsec-users mailing list
>> parsec-users at lists.cs.princeton.edu
>> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>>
>>
>>
>>
>> --
>>  Joel Hestness
>>  PhD Student, Computer Architecture
>>  Dept. of Computer Science, University of Texas - Austin
>>  http://www.cs.utexas.edu/~hestness<http://www.cs.utexas.edu/%7Ehestness>
>>
>> _______________________________________________
>> parsec-users mailing list
>> parsec-users at lists.cs.princeton.edu
>> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>>
>>
>>
>>
>> _______________________________________________
>> parsec-users mailing list
>> parsec-users at lists.cs.princeton.edu
>> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>>
>>
>>
>> _______________________________________________
>> parsec-users mailing list
>> parsec-users at lists.cs.princeton.edu
>> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090730/4ccdd831/attachment.html>


More information about the parsec-users mailing list