2008-12-22 15:42:41     JPEG compression support for Blackfin

Document created by Aaronwu Employee on Aug 9, 2013
Version 1Show Document
  • View in full screen mode

2008-12-22 15:42:41     JPEG compression support for Blackfin

Ed Abbas (CANADA)

Message: 67045   

 

hello

 

I was wondering if there is any opensource JPEG compression algorithms support for

 

the Blackfin family prcocessors and if there is any , how fast there algorithms are and can they be

 

impelemented to have real time compression on a speed of 10 fbs link

 

 

 

Thanks

QuoteReplyEditDelete

 

 

2008-12-22 18:24:49     Re: JPEG compression support for Blackfin

Robin Getz (UNITED STATES)

Message: 67048   

 

Ed:

 

Yes - check out

 

http://docs.blackfin.uclinux.org/doku.php?id=uclinux-dist:libs:libjpeg

 

-Robin

QuoteReplyEditDelete

 

 

2008-12-23 11:24:53     Re: JPEG compression support for Blackfin

Ed Abbas (CANADA)

Message: 67072   

 

Hi Robin

 

Thanks for the reply , may be I didnt mention in my last post that the size of my image is a 5 mega pixel

 

I like to be able to connect a 5 mega pixel CMOS sensor to a blackfin processor which will capture those pixels

 

then convert them to jpeg and send them over the ethernet link in a rate around 10 fps ,

 

Do you this this ( :libs:libjpeg) by uClinux can handle the size of data I mentioned above

 

Thanks

 

Ed

QuoteReplyEditDelete

 

 

2008-12-23 14:35:26     Re: JPEG compression support for Blackfin

Mike Frysinger (UNITED STATES)

Message: 67074   

 

libjpeg can handle any sized image.  the only limitation is the memory in the system.

QuoteReplyEditDelete

 

 

2008-12-29 11:23:05     Re: JPEG compression support for Blackfin

Ed Abbas (CANADA)

Message: 67156   

 

HI Mike

 

Thanks for the answer , I was looking at the 537-stamp board and I found that we can expand the SDRAM chips

 

on that board to increase the RAM to 128M byte , my question is

 

did anybody try that path before and does the BSP uClinux support that increase in size.

 

and do you think the 64Mbyte SDRAM already on the STAMP board will be enough to handle 5 mega pixel image sensor data in real time , i mean collecting the data and running jpeg compression on it

 

we are talking here about 15 fps kind of speed

 

Thanks

 

 

QuoteReplyEditDelete

 

 

2008-12-29 15:25:13     Re: JPEG compression support for Blackfin

Mike Frysinger (UNITED STATES)

Message: 67159   

 

the only limitation in memory support is the chip itself.  whatever the hardware supports, Linux supports.

 

QuoteReplyEditDelete

 

 

2008-12-29 15:32:45     Re: JPEG compression support for Blackfin

Frank Van Hooft (CANADA)

Message: 67161   

 

Not to be negative, but I doubt you'll come close to your 15 FPS jpeg compression requirement unless you have some hardware elsewhere in your system helping things along.

 

I'm running a BF537 & 64 MB RAM with a WVGA (752 x 480) sensor. Using libjpeg I'm doing around 3 FPS. I'm using a colour sensor, so there's a little bit of overhead for the simple colour interpolation we're using. But you get the idea.

 

FFMPEG will do somewhat faster jpeg compression. However the problem I'd found was periodic out-of-memory errors due to ffmpeg's habit of constantly malloc'ing & free'ing large chunks of RAM, resulting in memory fragmentation & the eventual OOM failure. Libjpeg seems better in that regard. However, if you've got a lot of RAM, maybe that won't be an issue for you.

 

If you try ffmpeg, be careful that it actually gives you the framerate you think it's doing. I found that it will give you frames at any rate you specify, however if it can't keep up it'll simply duplicate previous frames to maintain the framerate. This can easily be seen because you'll see consecutive jpg images of identical size. Don't be fooled.

 

At it's simplest level, jpg compression works by breaking an image into blocks of 8x8 pixels, and performing a DCT on each block. So, jpeg compression duration scales pretty linearly with image size. Your image size is about 13 times ours, which means that all things being equal, you'll be achieving less than 1 FPS compression rate.

 

The ADI website shows optimised code for jpeg & mjpeg compression for the VisualDSP (non-linux) tools, which looks like it offers better performance.  But to the best of my knowledge, that code hasn't been ported to the uclinux environment. However, I've no idea if even that code would give you 15 FPS with that size of a sensor.

 

If you have any flexibility in your choice of image sensor, take advantage of it. If you can use a sensor that provides colour interpolation, or outputs pixels in YCrCb colour format, so much the better.

 

I hope this helps.

QuoteReplyEditDelete

 

 

2008-12-30 09:57:16     Re: JPEG compression support for Blackfin

Ed Abbas (CANADA)

Message: 67211   

 

Hi Frank

 

Thanks for the valuable information it is my first project in dealing with Blackfin and image sensors. I would like to ask you few questions about your project

 

1- our sensor is coloured also so does that mean that we need an interpolation algorithem before the compression?

 

2- I'm gussing your image sensor connected to BF537 using the PPI interface , may I ask how fast you are getting the data to the SDRAM , and are you using DMA also , does the uclinunix drivers for PPI need to be modified to run  , I'm thinking to base the design on a BF537 stamp board.

 

3- does the uclinux have a driver for the FFMPEG you mentioned in your reply.

 

Thanks

 

Ed

 

 

QuoteReplyEditDelete

 

 

2008-12-30 10:45:36     Re: JPEG compression support for Blackfin

Robin Getz (UNITED STATES)

Message: 67214   

 

Frank/Ed:

 

ffmpeg is what we had been focusing on -- since so many people wanted video (not jpeg) - however - the analysis of ffmpeg being a memory pig is correct - it wasn't really architected for a embedded system (although many people use it).

 

We have seen 20-30fps (QVGA 320x240 ) with an application like linphone (compressing one way video - H.264 - and doing 2 way voice - G.711 + RTP and the transport protocol) As things scale larger frame sizes, frame rate drops.

 

Since compression is a much more computationly complex operation - decompression (or H.264) was much better (2/3 of D1 - 480 x 320  H.264+mp3 audio) could do real time no problem.

 

It is pretty difficult to compare ffpeg to libjpeg to the optimized ones from ADI (which only compile with VDSP++, and don't run under an operating system) - so I'm not sure what the point is.

 

-Robin

QuoteReplyEditDelete

 

 

2008-12-31 16:07:20     Re: JPEG compression support for Blackfin

Frank Van Hooft (CANADA)

Message: 67262   

 

Hi Robin, Ed,

 

Robin, the sole reason I mentioned the optimised VDSP M/JPEG code was to make Ed aware of it, in case he had the option of running with or without uclinux. Either way, he'd be on blackfin...   :-)

 

Ed, to answer your questions:

 

1) Read the datasheet for your image sensor (& whatever hardware you have inbetween the sensor & the blackfin, if any). This will tell you what the pixel data looks like. Depending upon what your sensor gives you, you may or may not need to write  some interpolation code (although most sensors will require it). There are lots of interpolation algorithms to choose from. Googling "bayer color interpolation" will produce lots of information.

 

2) Our sensor is connected to the PPI port. DMA is used to receive the data & transfer it to the SDRAM. We can vary the clock, but basically the PPI clock is run at 10 to 20 MHz.

 

3) Yes there is a V4L (video for linux 1) driver that FFMPEG can use. Search the documentation wiki on this site for info. There are also schematics on this site which show how to connect some CMOS sensors to the PPI port connector on the BF537-STAMP board, which is handy for prototyping or proof-of-concept. Depending upon your sensor you will likely need to change the driver slightly (for image size if nothing else) but of course you have the source - you do whatever you like.

QuoteReplyEditDelete

 

 

2009-01-05 18:24:35     Re: JPEG compression support for Blackfin

Ed Abbas (CANADA)

Message: 67392   

 

 

 

Hi Frank

 

  Thanks very much for the details answers to my questions. The CMOS sensor that we are thinking to use puts the output as a Bayer pattern so Im guessing I need some kind of interpolation method here before the image compression.

 

1- if we want to compare the interpolation time and the JPEG compression time which one will take more time and what is the average ratio in your opionion between the two.

 

2- May I ask what kind of developing time frame we are looking at for getting the stamp board running with Linux and capturing the data from the CMOS sensor.

 

3- The BF537 , what might load this processor for exmaple if we want to cpature data , interpolation , compression then transfer through the ethernet port. do you think all these processes can run together or that might cause the BF537 to hang and may be crash. if BF537 cant handle that which one of the blackfin family is better to use and of course we want a BF which is supported by uClinux.

 

regards

 

Ed

 

 

QuoteReplyEditDelete

 

 

2009-01-05 23:00:36     Re: JPEG compression support for Blackfin

Robin Getz (UNITED STATES)

Message: 67407   

 

Ed:

 

Depending on what is going on - you might be better off using the BF548 - which has a pixel compositor - which you might be able to use for the Bayer pattern conversion. I have not done this before - but a quick read of the manual should tell you if it is possible to help or not.

 

-Robin

Attachments

    Outcomes