If the code has run till Contour_Bottom2Top then you have provided enough memory during INIT (enough to allocate number of nodes provided in adi_contour_mem_alloc.c).
So your problem must be not enough nodes are allocated. 20 times also may not be sufficient if image you are trying to process has lot of noise. In order to know which type of node you are running out of, put breakpoint at all of the ADI_ITB_STATUS_INSUFFICIENT_MEM returns. Second parameter of pMemAlloc call will give you type of node you are running out. Hope this helps.
Thanks for the tips on debugging adi_contour_TraceContour. I did as you suggested and found that it's not actually insufficient memory that is causing the problem. Function AddSegmentToContourEndReversed and AddSegmentToContourBegReversed will run into infinite loop of allocating memory when xStart is 0 and this is the reason that causes the insufficient memory to be returned.
Thanks for pointing it out. It’s a bug and will be fixed in next release.
Meanwhile you can make the following changes in contours_ref.c and use it in place of library.
Change data type of “i” to int16_t in APIs AddSegmentToContourEndReversed and AddSegmentToContourBegReversed. This should resolve the issue.
Changing i from uint16_t to int16_t seems risky for me. On large contours, it may overflow the counter, especially when it's already reaching 10000 on small test images I am working with right now. So instead of changing the data type, I modified the condition to "i > xStart" and make a last check for i=0. It's working fine now! Thanks for the help.