[IPOL discuss] Swap memory

[Pascal Getreuer]

Hi Miguel,

I have no knowledge about this problem myself, but here are some notes
that might be helpful.

According to [1], some memory managers use "optimistic memory allocation" :

> allocation doesn't actually happen until the last moment you really use it;
> for example, by writing data to the block. So, unless you touch the block,
> you can keep asking for more.

So with out-of-memory problems, it matters when the memory blocks are
being written.  Here is a modification of your snippet  that would be
interesting to compare with.

   // MALLOC
   int i = 0;
   double *data;
   do {
     cout << i << endl;
     data = (double*)malloc(1024 * 1024 * sizeof(double));
     if(data)
        memset(data, 1, 1024 * 1024 * sizeof(double));
     i++;
   } while (data != NULL);


My other comment is (if you haven't already) I recommend testing the
program with Valgrind [2].  The command line syntax is

   valgrind [valgrind options] myprogram [arguments for myprogram]

Valgrind is very useful in catching memory-related bugs, like
attempting to access the 257th element of a 256-element array or a
computation that depends on an uninitialized value.  This doesn't help
address the out-of-memory problem itself, but it would be useful to
resolve other memory bugs first.

I hope this helps.

[1] http://linuxdevcenter.com/pub/a/linux/2006/11/30/linux-out-of-memory.html
[2] http://valgrind.org/


Best,
Pascal


On Sun, Oct 23, 2011 at 12:09, Miguel Colom
<Miguel.Colom at cmla.ens-cachan.fr> wrote:
> Hello all,
> these days Marc Lebrun and I have been trying to find the cause of a
> memory bug in some denoising programs.
>
> These programs work properly in Marc's machine, but they fail on mine and
> also on the IPOL server.
>
> Finally we managed to find the cause of the problem. The programs allocate
> a lot of memory and when it runs out of a certain amount, "malloc" returns
> a NULL or a std::bad_alloc exception is raised by the "new" operator.
>
> In Marc's computer the program isn't crashing because when the available
> amount of free memory is low, it starts to use swap memory.
>
> In my computer, when the memory is low, the swap isn't used and the
> program fails to get more memory because the system isn't using swap.
>
> Does anyone know the cause why the system isn't using swap or how to
> configure it to change this behavior?
>
> Best,
> Miguel
>
> PS: to check if the swap is activated, you can use this command: sudo swap -s
>
> Filename                                Type            Size    Used    Priority
> /dev/sda1                               partition       5858300 200092  -1
>
> In my case, it is.
>
> PPS: to see the memory used by the process, you can use gnome-system-monitor
> . The following code portions can be used to make the process use lots of
> memory:
>
>    // MALLOC
>    int i = 0;
>    double *data;
>    do {
>      cout << i << endl;
>      data = (double*)malloc(1024 * 1024 * sizeof(double));
>      i++;
>    } while (data != NULL);
>
> ---
>    // NEW
>    int i = 0;
>    do {
>      cout << i << endl;
>      float *data = new float[1024 * 1024];
>      i++;
>    } while (true);
>
>
> _______________________________________________
> discuss mailing list
> discuss at list.ipol.im
> http://tools.ipol.im/mailman/listinfo/discuss
>