2009-05-04 05:47:44 Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73552

Hi all,

I'm using the ADV7183B with the BF537 - STAMP board. I would like to capture the video from a composite video signal and use it to make video calls with linphone. When I start the driver everithing is OK, ADV7183B is detected and also I have '/dev/video0'. Then I try to capture from this device and and make a video call to my Laptop (where is running linphone 3.1.1) to show the streamed video. The trace is something about this when I make video call:

root:/> modprobe blackfin_cam

i2c-bfin-twi i2c-bfin-twi.0: Blackfin BF5xx on-chip I2C TWI Contoller Driver, Version 1.8, regs_base@ffc01400

driver for ADV7183B get_camops

ADV7183B: detecting client on address 0x40

driver for ADV7183B init

ADV7183B: V4L driver Blackfin CMOS Camera now ready

ADV7183B: i2c driver ready

root:/> tftp -g -r .linphonerc 192.168.0.101

root:/> linphonec -C

NLS disabled.

Ready

linphonec> Registration on sip:192.168.0.101 successful.

linphonec> call Laptop

Contacting sip:portatil@192.168.0.101

linphonec> linphonec> Connected.

linphonec> ...using 176x144 window

[mpeg4 @ 0xa07344]bitrate above max bitrate

ortp-error-avcodec_open() failed: -1

And here it freezes and I cant see any video recived from the Blackfin in my Laptop. The camera I'm using is a Composite Video Camera of 380 Lines.

My first question is, here in the trace, linphone defaults to CIF window when sending video, ¿how can I change it?

On the other hand, I have tried to run ffmpeg and capture video as you can see in this page: docs.blackfin.uclinux.org/doku.php?id=linux-kernel:drivers:v4l_blackfin_camera

root:/> ffmpeg -f video4linux -r 10.0 -s 720x288 -i /dev/video0 -qscale 6 -f mjpeg foo

configuration: --prefix=/usr --source-path=/home/saaysa/Escritorio/uClinux-dist-2008R1.5-RC3/lib/ffmpeg/build/../ffmpeg-svn-11114 --target-os=Linux --cpu=bfin --arch=bfin --extra-ldflags= -mcpu=bf537-0.3 --enable-static --enable-shared --enable-gpl --enable-pp --disable-strip --cross-prefix=bfin-linux-uclibc-

libavutil version: 49.5.0

libavcodec version: 51.48.0

libavformat version: 52.1.0

built on Apr 30 2009 10:39:56, gcc: 4.1.2 (ADI svn)

Here freezes and don't capture nothing.

I have done the strace comand to show what is happening, because I have read in the forum that maybe it freezes in a ioctl call to control the '/dev/video0' file and yes...this is the final lines from the output of strace of the comand above:

root:/> strace ffmpeg -f video4linux -r 10.0 -s 720x288 -i /dev/video0 -qscale 6 -f mjpeg foo

...

open("/dev/video0", O_RDWR|O_LARGEFILE) = 3

ioctl(3, FS_IOC32_GETVERSION or FS_IOC_GETVERSION or VIDIOCGCAP, 0xd83d78) = 0

ioctl(3, VIDIOCGAUDIO, 0xcfecdc) = -1 EINVAL (Invalid argument)

ioctl(3, VIDIOCSAUDIO, 0xcfecdc) = 0

ioctl(3, VIDIOCGPICT, 0xcfed24) = 0

ioctl(3, SONYPI_IOCGBATFLAGS or VIDIOCSPICT, 0xcfed24) = -1 EINVAL (Invalid argument)

ioctl(3, SONYPI_IOCGBATFLAGS or VIDIOCSPICT, 0xcfed24) = 0

ioctl(3, VIDIOCGMBUF, 0xd83de0) = 0

mmap2(NULL, 829440, PROT_READ|PROT_WRITE, MAP_SHARED, 3, 0) = -1 ENODEV (No such device)

mmap2(NULL, 829440, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = 0x1000

gettimeofday({42812, 178771}, NULL) = 0

ioctl(3, VIDIOCMCAPTURE, 0xd83e68) = 0

mmap2(NULL, 118784, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS|0x4000000, 0, 0) = 0xca0000

gettimeofday({42812, 181921}, NULL) = 0

nanosleep({0, 96850000}, NULL) = 0

gettimeofday({42812, 283086}, NULL) = 0

mmap2(NULL, 417792, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS|0x4000000, 0, 0) = 0x2000000

ioctl(3, VIDIOCSYNC

And here it freezes...

Also, the v4l_test freezes at the ioctl call, exactly, line 155 of uClinux-dist-2008R1.5-RC3/user/blkfin-test/camera_test/v4l_test.c

while (ioctl(devfd, VIDIOCSYNC, &frame_id) < 0 &&

(errno == EAGAIN || errno == EINTR));

Maybe, PPI_DMA interrupts is not ok?

I have seen that in /proc/interrupts there is a 0 in the PPI_DMA while running the V4L_test...

root:/> v4l_test -r 15 -o foo &

242

root:/> capture rate: 15, height: 288, w d t. .52.using 352x288 window

device: /dev/video0, output: foo

open /dev/video0

found Blackfin CMOS Camera device. (maxsize=720x576)

default picture properties: brightness=27714,hue=25441,colour=26219,contrast=28265,depth=16, palette=7.

trying to set capture size to 352x288

capture size set to 352x288

root:/> cat /proc/interrupts

6: 400885 BFIN Timer Tick

11: 0 PPI_DMA

12: 235 SPORT RX Data

13: 85 SPORT TX Data

16: 59 i2c-bfin-twi

18: 1 BFIN_UART_RX

19: 629 BFIN_UART_TX

24: 229137 EMAC_RX

42: 0 PPI ERROR

45: 0 SPORT error

Err: 0

Is this wrong?

I don't know what is happening, I'm using the ADV7183B Driver from the uClinux-dist...

The Schematic of the ADV7183B Daughter board I'm using is right here:

blackfin.uclinux.org/gf/download/forummessage/73379/6220/ADV7183B_84_DAUGHTER_BOARD_V1.0.pdf

Thank you!

QuoteReplyEditDelete

2009-05-04 07:01:18 Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73557 It kind of 'freezes' because you don't receive PPI/DMA interrupts.

It's simply waiting for frames that don't get received.

Did you implement ADV7193 /OE switching?

You may need to add some defines in drivers/media/video/blackfin/adv7183b.h

-Michael

QuoteReplyEditDelete

2009-05-04 12:40:45 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73578

Thank you for your answet Michael.

I'm not sure of what do you mean with "Did you implement ADV7193 /OE switching?".I'm not sure, but I think this is a signal generated by the Blackfin that enables de output (PIXEL + HS + VS + FIELD) when you are capturing. I have noticed the port map of the adv7183B.h of the Output Enable bit was wrong for the BF537 - STAMP board, so I changed it:

I have changed this line:

# define ADV7183B_GPIO_OE GPIO_PF2

For this other line

# define ADV7183B_GPIO_OE GPIO_PF10

While I'm running the "v4l_test" I have measured the voltage in the /OE pin and still high, so DATA + HS + VS + FIELD are High Impedance. Then I have pulled down the /OE "manually" and now I can see digital signals in the PPI and HS, VS and FRAME signal, but THE V4L_TEST still freezing. But I have good news, if I pull down the /OE manually (connecting it to ground) and I try to use ffmpeg, it does not freeze! I get something like this:

root:/> ffmpeg -f video4linux -r 10.0 -s 720x288 -i /dev/video0 -qscale 6 -f mjpeg foo

libavutil version: 49.5.0

libavcodec version: 51.48.0

libavformat version: 52.1.0

built on Apr 30 2009 10:39:56, gcc: 4.1.2 (ADI svn)

Input #0, video4linux, from '/dev/video0':

Duration: N/A, start: 31974.363300, bitrate: 33177 kb/s

Stream #0.0: Video: rawvideo, yuyv422, 720x288 [PAR 0:1 DAR 0:1], 33177 kb/s, 10.00 tb(r)

Output #0, mjpeg, to 'foo':

Stream #0.0: Video: mjpeg, yuvj420p, 720x288 [PAR 0:1 DAR 0:1], q=2-31, 200 kb/s, 10.00 tb(c)

Stream mapping:

Stream #0.0 -> #0.0

Press [q] to stop encoding

frame= 114 fps= 10 q=6.0 size= 3758kB time=11.4 bitrate=2700.5kbits/s

But if I save the captured data to a file and then paly it in my computer it's only a GREEN image... any ideas?

QuoteReplyEditDelete

2009-05-05 03:37:12 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73619 >But if I save the captured data to a file and then paly it in my computer it's only a GREEN image... any ideas?

Are you sure you connected the video source to the right ADV7183 input terminal?

-Michael

QuoteReplyEditDelete

2009-05-05 06:12:34 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73646

Hi michael.

The camera was conncted to the correct input but I think yesterday I didn't supply the camera so this is the reason why I got the green screen. After supply the camera and test the composite video signal is on the AVIN5 of the ADV7183B I can see the video but without syncronization as you can see in the video atached where I have shown to the camera 3 colours (first red, second green and third blue) to see if the colour works and it seems to work correctly... I'm not sure, but maybe this is a problem of sync of the frames. I have measured with a oscope the HS, VS and FIELD signals and seems to be ok (atached description of the signals...) because I'm using a PAL 50HZ camera. If the signals are ok the only thing that it's possible is that blackfin is not 'reading' this syncs signals due to wrongs defines, for example of the pin of the FIELD, HS, VS... What do you think?

SIGNALS MEASURED FROM THE ADV7183B.pdf

test.avi

QuoteReplyEditDelete

2009-05-05 06:48:47 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73650 The ADV7183B driver uses ITU-R BT.656 input mode. This means the SYNC

signals are embedded into the data stream. Do you have a NTSC source

(most digital cameras feature a composite out signal with the option to

select between NTSC and PAL) to test with?

-Michael

QuoteReplyEditDelete

2009-05-05 10:50:31 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73670

Hi Michael!

You are right, HS, VS sync are not used by the Blackfin since the ITU-R BT.656 (digital video output from the ADV7183B) has the sync embedded. But if this info is included in the video, why I'm not having the frames with sync?The ADV7183B is extracting the sync correctly from the composite video signal? Has the composite video signal bad sync? I thougth maybe it will be due to a bad auto detection of the "input video standard", but I have set manually the "Input Video Standar" and I don't have succes, I'm still having bad video capture...

The camera we have to use in the project is the one I'm using now (with the camera I recorded the last video), maybe we can use other source of composite video but just for testing purposes, it's mandatoy in our project to use the camera I have used... If you have any other ideas, please let me know, I need some light...

QuoteReplyEditDelete

2009-05-05 17:21:21 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73714 The ADV7183B by default auto recognizes the video format supplied.

However there is no guarantee that the driver DMA matches up with the

supplied data, there might be some tweaks necessary.

I'm sure we can figure it out.

-Michael

QuoteReplyEditDelete

2009-05-06 08:05:13 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73750

Hi Michael,

I have tested the NTSC camera. As I don't have a NTSC camera, I have used the ADV7393 configured as NTSC 720x480.In order to clarify things, I explain, now, we have two BF537 - STAMP. First one with the ADV7393 wich will output the NTSC/PAL composite video signal of a static image created with jpegviewer for example. And second one with the ADV7183B composite video input recording the image sended by the other board.

But things still the same. I have attached the image sended in composite video from the ADV73933 and the recorded video in the ADV7183B.

>However there is no guarantee that the driver DMA matches up with the

supplied data, there might be some tweaks necessary.

Let me know what kind of tweaks are necessary. Where can I start to know how to solve it? Should I read the DMA Chapter in the Hardware Reference? Should I change DMA configuration uClinux?

test_display.jpg

test.avi

QuoteReplyEditDelete

2009-05-06 08:31:56 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73755 Let me try things here - on a BF561-EZKIT.

Will test both NTSC and PAL - and let you know how it goes on my side.

-Michael

QuoteReplyEditDelete

2009-05-06 09:33:48 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73759

See attached videos.

The driver is optimized for PAL - but it also captures NTSC.

I don't see any horizontal distortions, in both modes.

I bet it's a PPI_CLK issue. PPI_CLK is very sensitive to noise, over and undershoot.

The PPI clock needs to be very clean. Can you check with a scope?

root:/> ffmpeg -f video4linux -r 15.0 -s 720x288 -i /dev/video0 -qscale 6 -f mjp

eg foo_NTSC.mjpeg

configuration: --prefix=/usr --source-path=/home/michael/devel/svn/uclinux-dist-trunk/lib/ffmpeg/build/../ff-

libavutil version: 49.5.0

libavcodec version: 51.48.0

libavformat version: 52.1.0

built on May 6 2009 14:49:41, gcc: 4.1.2 (ADI svn)

Input #0, video4linux, from '/dev/video0':

Duration: N/A, start: 1167610488.384000, bitrate: 49766 kb/s

Stream #0.0: Video: rawvideo, uyvy422, 720x288 [PAR 0:1 DAR 0:1], 49766 kb/s, 15.00 tb(r)

File 'foo' already exists. Overwrite ? [y/N] y

Output #0, mjpeg, to 'foo':

Stream #0.0: Video: mjpeg, yuvj420p, 720x288 [PAR 0:1 DAR 0:1], q=2-31, 200 kb/s, 15.00 tb(c)

Stream mapping:

Stream #0.0 -> #0.0

Press [q] to stop encoding

frame= 160 fps= 20 q=6.0 Lsize= 2980kB time=10.7 bitrate=2288.8kbits/s

video:1438kB audio:0kB global headers:0kB muxing overhead 107.193180%

Received signal 2: terminating.

foo_PAL.mjpeg

foo_NTSC.mjpeg

QuoteReplyEditDelete

2009-05-07 06:32:19 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73816

Hi Michael.

Thank you for testing it in a BF561 kit. I have measured the PPI_CLK and i have a clear 27MHz clock signal of more or less 50% dutty cicle.

I tested the capture with the same comand line than you and I have a good capture of the image of the camera and the static image (atached files). I have noticed that If I use the input resolution of 720x288 the captures are correct (from the camera and from the static image of the ADV7393). I don't know how good are this news... Could you please change the resolution to QVGA and try it on your BF561? If this resolution (720x288) works OK, this means the DMA is not the problem? What's the problem of other resolutions?

static_image.mjpeg

camera.mjpeg

QuoteReplyEditDelete

2009-05-07 06:43:56 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73817 So far the driver in BT656 is 720x288.

In external frame sync mode it's possible to support other resolutions.

QuoteReplyEditDelete

2009-05-07 06:57:36 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73818

O.K, thank you for all. Sorry, but I 'didn't know this thing about the BT656. What I need is a 320x240 (or CIF) resolution of the input video, how can I use this external frame sync to achieve it? Using HS, VS and FRAME? How can I configure the driver to use this lines instead of embedded sync? Can I modify the BT656 driver to be compliant with 320x240? How?

Thank you another time Michael.

QuoteReplyEditDelete

2009-05-07 07:15:22 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73819 Well thinking about it a bit more - it's not that easy.

In order to use the FS mode you need to tweak some defines in the

ADV7183.h file.

But - think about this:

Right now we use just one field and therefore the 720x288 aspect ratio

is not correct.

In FS mode you can crop a window using PPI_DELAY and PPI_COUNT however

it's not going to help because the ADV7183 produces YCrCb 4:2:2 output.

So the color representation in Memory looks like this:

Y1,Cr12, Y2, Cb12, Y3, Cr34, Y4, Cb34,...

So cropping a window will still give you twice the horizontal pixels.

The only way to work around this is in software.

-Michael

QuoteReplyEditDelete

2009-05-07 08:17:56 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73820

Hi Michael. I have read some code (adv7183b.c, adv7183b.h, blackfin_cam.c and blackfin_cam.h) and the only reference to the ITU.BT-656 I have found is in the blakfin_cam.c and blackfin_cam.h. I don't know where start to make the camera works on a QVGA resolution. You said this is software, where can I start?

QuoteReplyEditDelete

2009-05-07 08:45:05 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73821 You need to do it in your v4l user space application (linphone).

Take a look in msv4l.c: You need some variant of crop_or_pad().

-Michael

QuoteReplyEditDelete

2009-05-07 12:02:18 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 73825

One question please...In order to get the video on the resolution I want (320x240), first I would like to make video calls with the 720x288 resolution and then change this resolution by using the function you told me (****_or_pad()). The problem is that I don't know where to configure the input resolution of the video in Linphone and I get this error when try to call other side because it seems that the default input resolution is QCIF...:

root:/> linphonec -C

NLS disabled.

Ready

linphonec> Registration on sip:192.168.0.101 successful.

linphonec> call portatil

Contacting sip:portatil@192.168.0.101

linphonec> linphonec> Connected.

linphonec> ...using 176x144 window

[mpeg4 @ 0x2207344]bitrate above max bitrate

ortp-error-avcodec_open() failed: -1

How can I change the window of 176x144 to 720x288? Where is the option? I have searched for it in the .linphonerc and in the mediastreamer source files, but I cant find it...

Thank you.

QuoteReplyEditDelete

2009-05-07 12:27:15 Re: Error capturing Video with ADV7183B

Michael Hennerich (GERMANY)

Message: 73827 I'm not a linphone expert - but linphone tries CIF and QCIF.

These resolutions are defined in some linphone header file as:

#define MS_VIDEO_SIZE_QCIF_W 176

#define MS_VIDEO_SIZE_QCIF_H 144

I think you need to look in the source code.

-Michael

QuoteReplyEditDelete

2009-05-25 08:12:28 Re: Error capturing Video with ADV7183B

Samuel Ayet (SPAIN)

Message: 74536

Hi again,

I have read the code of Linphone and I have tried to know how to configure the linphone to crop my 720x288 captured frame.

I have configured linphone to capture frames at 720x288 (got_vsize) pixels and crop them into a 352x288 (vsize) pixels to compress and send the video stream.

As you told me, I have used the crop_or_pad() function from msv4l.c, but this function doesn't crop correctly the frames. I explain, if I use this functions seems that in reception I have a smaller frame, but overlapped (I have a 352 pixel width frame with overlapping at both sides of the frame) and very poor quality (there are some white lines in the frame) and the worst thing, in the transmision side (the Blackfin who crops the frames) just works arround 5 seconds and then I get a Page Allocation Faillure (you can see the error trace in the attached file).

The way of cropping of the Linphone is just get the frame at 720x288, then copy part of the image into other buffer. If this way of cropping is OK, why I have overlapping? How can I solve the overlapping problem by cropping the frames?

As we are facing this problems, we think that maybe the crop solution is not as easy as we thought and maybe modifying the BT.656 driver to get frames at 352x288 instead of 720x288 will be easyer, what do you think?

Thank you!

error_trace.txt

QuoteReplyEditDelete

2009-06-16 23:39:29 Re: Error capturing Video with ADV7183B

bin zhang (CHINA)

Message: 75852

You can use MDMA to make a try.I think it's very useful for you.

TranslateQuoteReplyEditDelete

2009-10-27 15:30:04 Re: Error capturing Video with ADV7183B

Slava Zhuiko (RUSSIAN FEDERATION)

Message: 81792

Hi Samuel Ayet,