2011-04-18 10:37:46     Memory fragmentation or leak?

Document created by Aaronwu Employee on Aug 26, 2013
Version 1Show Document
  • View in full screen mode

2011-04-18 10:37:46     Memory fragmentation or leak?

Peter Gombos (HUNGARY)

Message: 99944   

 

Hi,

 

I use 2010R1RC5. Our board says there are 28M free after all of our application loaded. But after a while the system getting unstable and some of my application complains about cannot allocate memory.

 

I wrote a simple memory allocator application:

 

---------------------------------------------------------------------------------

 

char *p[500];

 

 

// Allocate as much memory as possible

int block_test(int ksize, int maxnr)

{

int i = 0;

int j;

 

for (j=0; j<maxnr; j++) p[j] = NULL;

while (i<maxnr)

  {

   p[i] = malloc(1024*ksize);

   if (p[i]==NULL) break;

   i++;

  }

for (j=0; j<maxnr; j++) if (p[j]!= NULL) free(p[j]);

printf("Allocating max %d pcs. %dk blocks: %d blocks (%d kbytes) could be allocated\n", maxnr, ksize, i, i*ksize);

return i;

}

 

 

//Main

int main(int argc, char *argv[])

{

printf("\nMemory tester application\n");

// block_test(256,100);

block_test(512,50);

printf("\n");

return 0;

}

 

---------------------------------------------------------------------------------

 

If I run this application at first time I can get 12Mbytes memory (depending on blocksize). Next time I can allocate fewer blocks. If I run it 10-20 times I run out of the memory and the application cannot allocate any block at all. The free memory is still 28M.

 

I don't see any problem in my application. What can be the problem?

QuoteReplyEditDelete

 

 

2011-04-18 23:53:23     Re: Memory fragmentation or leak?

Sonic Zhang (CHINA)

Message: 99952   

 

This is expected behavior on NOMMU architecture. If you want to allocate large continous menory, such as 12M bytes, do it once at the beginning of Linux boot up. The later you allocate a big memory, the bigger possibility you get a failure. Only the allocation less than 4k bytes is ensured on NOMMU at any time.

QuoteReplyEditDelete

 

 

2011-04-19 02:12:06     Re: Memory fragmentation or leak?

Peter Gombos (HUNGARY)

Message: 99962   

 

Hi Sonic,

 

I think you misunderstood my problem.

 

The limitation of the NOMMU systems is clear. But my test application tries to allocate medium sized blocks.

 

If I allocate 10 blocks with 512k size it is OK. I free the allocated blocks and I can reallocate them again as many times as I want.

 

But if I try to allocate more (30 * 512k) blocks I receive an OOM message (console dump).My application can handle this and I free all of the previously allocated blocks. But next time I can allocate less and less memory. Finally I can not allocate any memory.

 

Is it acceptable? Without dump everything is OK.

QuoteReplyEditDelete

 

 

2011-04-19 02:28:33     Re: Memory fragmentation or leak?

Sonic Zhang (CHINA)

Message: 99965   

 

You application is not the only process allocating memory in the system. Others may occupy some pages between you free and allocation and break the big continous free memory block. Such as VFS buffer, network SKB, network service daemon, etc.

QuoteReplyEditDelete

 

 

2011-04-22 07:08:50     Re: Memory fragmentation or leak?

Gilles Ganault (FRANCE)

Message: 100125   

 

This is expected behavior on NOMMU architecture. If you want to allocate large continous menory, such as 12M bytes, do it once at the beginning of Linux boot up. The later you allocate a big memory, the bigger possibility you get a failure. Only the allocation less than 4k bytes is ensured on NOMMU at any time.

 

---

 

Is there an application that we can launch at boot-time that will watch every application that is launched and stop/restart those that are making too many allocations, in order to prevent systemic consequences on other, innocent applications?

 

BTW, will all RAM be returned to Linux after an application ends/is killed, or are there times when the only way to recover RAM is by rebooting the device?

 

Thank you.

QuoteReplyEditDelete

 

 

2011-04-24 23:16:50     Re: Memory fragmentation or leak?

Sonic Zhang (CHINA)

Message: 100136   

 

Not all RAM is freed immediately after an application ends. Such as VFS file buffer, network buffer, kernel data structure, etc. Some kernel threads, kernel modules and service daemon won't exit and free memory allocated until you force to or reboot.

QuoteReplyEditDelete

 

 

2011-04-25 18:07:46     Re: Memory fragmentation or leak?

Gilles Ganault (FRANCE)

Message: 100150   

 

Thanks for the tip. I'll schedule nightly reboots to avoid memory leaks.

QuoteReplyEditDelete

 

 

2011-04-26 03:25:28     Re: Memory fragmentation or leak?

Gilles Ganault (FRANCE)

Message: 100161   

 

BTW, is there an application in uClinux-dist (or that can be compiled with the uClinux toolchain) that can monitor the system +  applications and send an alert in case RAM is becoming too low/fragmented?

QuoteReplyEditDelete

 

 

2011-04-27 11:23:58     Re: Memory fragmentation or leak?

Rob Maris (GERMANY)

Message: 100206   

 

Hm, Sonic - thanks for this information, however. As a backgrounder: a few months ago I also encountered memory leaks while experimenting with mpg321. Since I found that mpg321 was not the problem, I suspected the problem deeper (even alsa related). Meanwhile I informed my customer (who distributes blackfin-based VoIP systems to end customers), that no unnecessary app-program close and restarts should be executed, until a solution is presented. Some notes to the problem are attached.

 

Can you provide a documentation link to the NOMMU relationship to memory leaks?

 

-Rob

 

Memory Leak mpg321

QuoteReplyEditDelete

 

 

2011-04-27 22:51:15     Re: Memory fragmentation or leak?

Sonic Zhang (CHINA)

Message: 100219   

 

Memory leak is a bug, while memory fragment is not. Memory leak should be fixed in either MMU or NOMMU kernel. Memory fragmentation can not be shield on NOMMU architecture in application.

Attachments

Outcomes