[parsec-users] Cross Compiling Dedup for Alpha Arch using M5 Simulator

ef snorlaxgb at gmail.com
Wed Jul 22 03:32:46 EDT 2009


Hello,

That didn't seem to help remove the error, I increased the hash table to
order of magnitude bigger but that didnt work. I am using the simsmall input
(11MB). I will keep looking into the error and will update the mailing list
if I find anything. If anyone has any other ideas I would appreciate it.

Thanks
EF

2009/7/19 Christian Bienia <cbienia at cs.princeton.edu>

>  Ef,
>
>
>
> The size of the hash table is independent of the input data size because
> (1) the number of data chunks is data dependent and can thus not be
> determined without actually computing all chunks and (2) the data is
> typically streamed through the program so not even the amount of data is
> known. The hash table size is therefore hardwired into the program and set
> to a very large value by default.
>
>
>
> I believe this is the part the causes trouble:
>
>
>
>   cache = create_hashtable(65536, hash_from_key_fn, keys_equal_fn);
>
>   if(cache == NULL) {
>
>     printf("ERROR: Out of memory\n");
>
>     exit(1);
>
>   }
>
>
>
> Interestingly, memory allocation does not fail during the hash table
> allocation but at a later point in the encoder, as the error message
> suggests. You should also note that you can’t change the hash table size
> without altering the memory access characteristics of the program. Try it
> with several GB.
>
>
>
> - Chris
>
>
>
>
>
> *From:* parsec-users-bounces at lists.cs.princeton.edu [mailto:
> parsec-users-bounces at lists.cs.princeton.edu] *On Behalf Of *ef
> *Sent:* Sunday, July 19, 2009 5:28 PM
>
> *To:* PARSEC Users
> *Subject:* Re: [parsec-users] Cross Compiling Dedup for Alpha Arch using
> M5 Simulator
>
>
>
> Hello.
> Hmmm, I am testing with simsmall I had initially had around 512MB for Dedup
> Allocation I bumped it up to 1024MB and it still gives me the same error.
> Any other ideas?
>
> Thanks
> EF
>
> 2009/7/19 Christian Bienia <cbienia at cs.princeton.edu>
>
> Hi Ef,
>
>
>
> I believe this is another error unrelated to the first one. Dedup tries to
> allocate a huge hash table to manage the data chunks from the input data.
> You probably simply ran out of memory. Try to tell M5 to allow processes to
> allocate more memory and dedup will probably run fine.
>
>
>
> - Chris
>
>
>
>
>
> *From:* parsec-users-bounces at lists.cs.princeton.edu [mailto:
> parsec-users-bounces at lists.cs.princeton.edu] *On Behalf Of *ef
> *Sent:* Sunday, July 19, 2009 4:22 PM
> *To:* PARSEC Users
> *Subject:* Re: [parsec-users] Cross Compiling Dedup for Alpha Arch using
> M5 Simulator
>
>
>
> Hello,
>
> I removed the flag O_LARGEFILE under encoder.c:
>  /* src file open if((fd = open(conf->infile, O_RDONLY | O_LARGEFILE)) < 0)
>       EXIT_TRACE("%s file open error TEST %s\n", conf->infile,
> strerror(errno));
>
> I got this error in M5:
> [DataProcess] Memory allocation failed.
>
>
> It sounds like I need to investigate my compiler and libc versions, as you
> suggested.
> Thanks,
> EF
>
> 2009/7/19 Joel Hestness
>
> Hi EF,
>
>   I encountered this in our first iteration of compiling PARSEC.  The
> compiler and libc version you are using don't like the O_LARGEFILE flag used
> when you open the file in encoder.c (perhaps the large file support is
> standard on ALPHA?).  I can't remember if it fixed the problem, but I had
> removed the flag in my current code base.  I would suggest trying that
> first.  You might need to try a different version of the compiler/libc.
>
>
>
>   Good luck,
>
>   Joel
>
>
>
>
>
> 2009/7/19 ef
>
> Hello,
> I have cross compiled Dedup for use in M5, however when I run the binary in
> M5 I get the following output:
>
> PARSEC Benchmark Suite Version 2.0
> [Encode] /parsec/install/inputs/dedup/medias.dat file open error Not a
> directory
>
> Not sure where to even start fixing this, anyone have any clue? Any hints
> or thoughts are appreciated.
>
> Thanks,
> EF
>
> _______________________________________________
> parsec-users mailing list
> parsec-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>
>
>
>
> --
>  Joel Hestness
>  PhD Student, Computer Architecture
>  Dept. of Computer Science, University of Texas - Austin
>  http://www.cs.utexas.edu/~hestness <http://www.cs.utexas.edu/%7Ehestness>
>
> _______________________________________________
> parsec-users mailing list
> parsec-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>
>
>
>
> _______________________________________________
> parsec-users mailing list
> parsec-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>
>
>
> _______________________________________________
> parsec-users mailing list
> parsec-users at lists.cs.princeton.edu
> https://lists.cs.princeton.edu/mailman/listinfo/parsec-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/parsec-users/attachments/20090722/ea546ad8/attachment.html>


More information about the parsec-users mailing list