2008-12-22 15:42:41 JPEG compression support for Blackfin
Ed Abbas (CANADA)
Message: 67045
hello
I was wondering if there is any opensource JPEG compression algorithms support for
the Blackfin family prcocessors and if there is any , how fast there algorithms are and can they be
impelemented to have real time compression on a speed of 10 fbs link
Thanks
QuoteReplyEditDelete
2008-12-22 18:24:49 Re: JPEG compression support for Blackfin
Robin Getz (UNITED STATES)
Message: 67048
Ed:
Yes - check out
http://docs.blackfin.uclinux.org/doku.php?id=uclinux-dist:libs:libjpeg
-Robin
QuoteReplyEditDelete
2008-12-23 11:24:53 Re: JPEG compression support for Blackfin
Ed Abbas (CANADA)
Message: 67072
Hi Robin
Thanks for the reply , may be I didnt mention in my last post that the size of my image is a 5 mega pixel
I like to be able to connect a 5 mega pixel CMOS sensor to a blackfin processor which will capture those pixels
then convert them to jpeg and send them over the ethernet link in a rate around 10 fps ,
Do you this this ( :libs:libjpeg) by uClinux can handle the size of data I mentioned above
Thanks
Ed
QuoteReplyEditDelete
2008-12-23 14:35:26 Re: JPEG compression support for Blackfin
Mike Frysinger (UNITED STATES)
Message: 67074
libjpeg can handle any sized image. the only limitation is the memory in the system.
QuoteReplyEditDelete
2008-12-29 11:23:05 Re: JPEG compression support for Blackfin
Ed Abbas (CANADA)
Message: 67156
HI Mike
Thanks for the answer , I was looking at the 537-stamp board and I found that we can expand the SDRAM chips
on that board to increase the RAM to 128M byte , my question is
did anybody try that path before and does the BSP uClinux support that increase in size.
and do you think the 64Mbyte SDRAM already on the STAMP board will be enough to handle 5 mega pixel image sensor data in real time , i mean collecting the data and running jpeg compression on it
we are talking here about 15 fps kind of speed
Thanks
QuoteReplyEditDelete
2008-12-29 15:25:13 Re: JPEG compression support for Blackfin
Mike Frysinger (UNITED STATES)
Message: 67159
the only limitation in memory support is the chip itself. whatever the hardware supports, Linux supports.
QuoteReplyEditDelete
2008-12-29 15:32:45 Re: JPEG compression support for Blackfin
Frank Van Hooft (CANADA)
Message: 67161
Not to be negative, but I doubt you'll come close to your 15 FPS jpeg compression requirement unless you have some hardware elsewhere in your system helping things along.
I'm running a BF537 & 64 MB RAM with a WVGA (752 x 480) sensor. Using libjpeg I'm doing around 3 FPS. I'm using a colour sensor, so there's a little bit of overhead for the simple colour interpolation we're using. But you get the idea.
FFMPEG will do somewhat faster jpeg compression. However the problem I'd found was periodic out-of-memory errors due to ffmpeg's habit of constantly malloc'ing & free'ing large chunks of RAM, resulting in memory fragmentation & the eventual OOM failure. Libjpeg seems better in that regard. However, if you've got a lot of RAM, maybe that won't be an issue for you.
If you try ffmpeg, be careful that it actually gives you the framerate you think it's doing. I found that it will give you frames at any rate you specify, however if it can't keep up it'll simply duplicate previous frames to maintain the framerate. This can easily be seen because you'll see consecutive jpg images of identical size. Don't be fooled.
At it's simplest level, jpg compression works by breaking an image into blocks of 8x8 pixels, and performing a DCT on each block. So, jpeg compression duration scales pretty linearly with image size. Your image size is about 13 times ours, which means that all things being equal, you'll be achieving less than 1 FPS compression rate.
The ADI website shows optimised code for jpeg & mjpeg compression for the VisualDSP (non-linux) tools, which looks like it offers better performance. But to the best of my knowledge, that code hasn't been ported to the uclinux environment. However, I've no idea if even that code would give you 15 FPS with that size of a sensor.
If you have any flexibility in your choice of image sensor, take advantage of it. If you can use a sensor that provides colour interpolation, or outputs pixels in YCrCb colour format, so much the better.
I hope this helps.
QuoteReplyEditDelete
2008-12-30 09:57:16 Re: JPEG compression support for Blackfin
Ed Abbas (CANADA)
Message: 67211
Hi Frank
Thanks for the valuable information it is my first project in dealing with Blackfin and image sensors. I would like to ask you few questions about your project
1- our sensor is coloured also so does that mean that we need an interpolation algorithem before the compression?
2- I'm gussing your image sensor connected to BF537 using the PPI interface , may I ask how fast you are getting the data to the SDRAM , and are you using DMA also , does the uclinunix drivers for PPI need to be modified to run , I'm thinking to base the design on a BF537 stamp board.
3- does the uclinux have a driver for the FFMPEG you mentioned in your reply.
Thanks
Ed
QuoteReplyEditDelete
2008-12-30 10:45:36 Re: JPEG compression support for Blackfin
Robin Getz (UNITED STATES)
Message: 67214
Frank/Ed:
ffmpeg is what we had been focusing on -- since so many people wanted video (not jpeg) - however - the analysis of ffmpeg being a memory pig is correct - it wasn't really architected for a embedded system (although many people use it).
We have seen 20-30fps (QVGA 320x240 ) with an application like linphone (compressing one way video - H.264 - and doing 2 way voice - G.711 + RTP and the transport protocol) As things scale larger frame sizes, frame rate drops.
Since compression is a much more computationly complex operation - decompression (or H.264) was much better (2/3 of D1 - 480 x 320 H.264+mp3 audio) could do real time no problem.
It is pretty difficult to compare ffpeg to libjpeg to the optimized ones from ADI (which only compile with VDSP++, and don't run under an operating system) - so I'm not sure what the point is.
-Robin
QuoteReplyEditDelete
2008-12-31 16:07:20 Re: JPEG compression support for Blackfin
Frank Van Hooft (CANADA)
Message: 67262
Hi Robin, Ed,
Robin, the sole reason I mentioned the optimised VDSP M/JPEG code was to make Ed aware of it, in case he had the option of running with or without uclinux. Either way, he'd be on blackfin... :-)
Ed, to answer your questions:
1) Read the datasheet for your image sensor (& whatever hardware you have inbetween the sensor & the blackfin, if any). This will tell you what the pixel data looks like. Depending upon what your sensor gives you, you may or may not need to write some interpolation code (although most sensors will require it). There are lots of interpolation algorithms to choose from. Googling "bayer color interpolation" will produce lots of information.
2) Our sensor is connected to the PPI port. DMA is used to receive the data & transfer it to the SDRAM. We can vary the clock, but basically the PPI clock is run at 10 to 20 MHz.
3) Yes there is a V4L (video for linux 1) driver that FFMPEG can use. Search the documentation wiki on this site for info. There are also schematics on this site which show how to connect some CMOS sensors to the PPI port connector on the BF537-STAMP board, which is handy for prototyping or proof-of-concept. Depending upon your sensor you will likely need to change the driver slightly (for image size if nothing else) but of course you have the source - you do whatever you like.
QuoteReplyEditDelete
2009-01-05 18:24:35 Re: JPEG compression support for Blackfin
Ed Abbas (CANADA)
Message: 67392
Hi Frank
Thanks very much for the details answers to my questions. The CMOS sensor that we are thinking to use puts the output as a Bayer pattern so Im guessing I need some kind of interpolation method here before the image compression.
1- if we want to compare the interpolation time and the JPEG compression time which one will take more time and what is the average ratio in your opionion between the two.
2- May I ask what kind of developing time frame we are looking at for getting the stamp board running with Linux and capturing the data from the CMOS sensor.
3- The BF537 , what might load this processor for exmaple if we want to cpature data , interpolation , compression then transfer through the ethernet port. do you think all these processes can run together or that might cause the BF537 to hang and may be crash. if BF537 cant handle that which one of the blackfin family is better to use and of course we want a BF which is supported by uClinux.
regards
Ed
QuoteReplyEditDelete
2009-01-05 23:00:36 Re: JPEG compression support for Blackfin
Robin Getz (UNITED STATES)
Message: 67407
Ed:
Depending on what is going on - you might be better off using the BF548 - which has a pixel compositor - which you might be able to use for the Bayer pattern conversion. I have not done this before - but a quick read of the manual should tell you if it is possible to help or not.
-Robin