STM32MP257F-DK: Unable to collect images from MIPI camera

[rdmikkelsen]

I am trying to collect images from a MIPI camera that is not supported by default on the STM32MP257F-DK, but for which I have a driver. With that driver, it's seeing the camera as an attached device, and I can see it in the dmesg output and also as /dev/media2 in v4l2-ctl --list-devices. I have followed the steps at this link (modified to use the imx664 instead of imx335), and also steps I have found elsewhere (below). Is there anything I need to look for or change in order to actually see image frames from the camera, now that I can see the camera itself?

media-ctl -d /dev/media2 -r
media-ctl -d /dev/media2 -l "'48020000.csi':1 -> 'dcmipp_input':0 [1]"
media-ctl -d /dev/media2 -l "'dcmipp_input':2 -> 'dcmipp_main_isp':0 [1]"
media-ctl -d /dev/media2 -V "'imx664 0-0010':0 [fmt:SRGGB12_1X12/2704x1540]"
media-ctl -d /dev/media2 -V "'48020000.csi':1 [fmt:SRGGB12_1X12/2704x1540]"
media-ctl -d /dev/media2 -V "'dcmipp_input':2 [fmt:SRGGB12_1X12/2704x1540]"
media-ctl -d /dev/media2 -V "'dcmipp_main_isp':1 [fmt:RGB888_1X24/2704x1540]"
media-ctl -d /dev/media2 -V "'dcmipp_main_postproc':0 [fmt:RGB888_1X24/2704x1540 compose:(0,0)/2704x1540]"
media-ctl -d /dev/media2 -V "'dcmipp_main_postproc':1 [fmt:RGB888_1X24/2704x1540]"
v4l2-ctl -d /dev/video1 --set-fmt-video=width=2704,height=1540,pixelformat=RGBP --stream-mmap --stream-to=test.raw --stream-count=1 --verbose

I either see no output, or an output file such as test.raw which is an empty file. The v4l2-ctl command above hangs forever at this point, and I have to Ctrl+C to stop it:

VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
    Width/Height : 2704/1540
    Pixel Format : 'RGBP' (16-bit RGB 5-6-5)
    Field : None
    Bytes per Line : 5408
    Size Image : 8328320
    Colorspace : Rec. 709
    Transfer Function : Default (maps to Rec. 709)
    YCbCr/HSV Encoding: Default (maps to Rec. 709)
    Quantization : Default (maps to Full Range)
    Flags : 
        VIDIOC_REQBUFS returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_QUERYBUF returned 0 (Success)
        VIDIOC_G_FMT returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_QBUF returned 0 (Success)
        VIDIOC_STREAMON returned 0 (Success)

I have tried a few things, including changing the link frequency from 594MHz to be a multiple of the link frequency mentioned in the driver, adding a clock-noncontinuous; line in the device tree, changing the clock frequency, and changing the size and format used in the media-ctl and v4l2-ctl commands (2704x1540 and 640x480 seem to be default sizes for this sensor). It was changing the clock frequency to match the driver's expected value that let me see the device in the first place. 

Attached are the image showing the camera topology as specified in the link above, and also a screenshot showing an ever-increasing number of CSI interrupts and an unchanging number of 0 DCMIPP interrupts taken while the capture command was hanging forever. 

Note: this is the same sensor as used here, although those issues were resolved. 

[Gregory PLANCHON]

Hi @rdmikkelsen ,

First of all, have you tried launching the camera demo to see if it works?

Have you tried adding some tracing to see if you have any issues? Refer to this chapter for more information: https://wiki.st.com/stm32mpu/wiki/STM32MP2_V4L2_camera_overview#How_to_debug? 

Regards,

Grégory 

[rdmikkelsen]

Hi @Gregory PLANCHON , 

I have tried the camera demo, and it does not detect an attached camera. 

I have tried some of the debugging steps there and others noted in the question to see what the problem might be and haven't been able to find anything so far. I have tried to edit the commands at that link to check for the IMX664:

cd ~ # go to $HOME directory
echo "#!/bin/bash" > dtdumpentry.sh;echo "hexdump -e '\"=\"' -e '20/1 \"%c\"\"\t\"' -e '20/1 \"%02x\"\"\n\"' \$1" >> dtdumpentry.sh;chmod +x dtdumpentry.sh
echo "#!/bin/bash" > dtdump.sh;echo "find \$1* -type f -print0 -exec ./dtdumpentry.sh {} \;" >> dtdump.sh;chmod +x dtdump.sh

cd ~ # go to $HOME directory
rm ~/devicetree.txt
echo "[devicetree]" >> ~/devicetree.txt
echo "|-[dcmipp]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc@0/bus@42080000/dcmipp@48030000 | sed 's/\/proc\/device-tree\/soc@0\/bus@42080000\//| |-/' >> devicetree.txt
echo "|" >> devicetree.txt
echo "|-[camera:" | tr -d "\n" >> devicetree.txt
cat /proc/device-tree/soc@0/bus@42080000/i2c@40130000/imx664@10/compatible >> devicetree.txt
echo "]" >> devicetree.txt
./dtdump.sh /proc/device-tree/soc@0/bus@42080000/i2c@40130000/imx664@10 -type f -print0 -exec ./dtdump.sh {} \; | sed 's/\/proc\/device-tree\/soc@0\/bus@42080000\//| |-/' >> devicetree.txt
cat ~/devicetree.txt # or cat -t ~/devicetree.txt

and I don't see anything related to the camera in the output her, although I definitely see IMX664-related output elsewhere as in the body of the question. 

[devicetree]
|-[dcmipp]
| |-dcmipp@48030000/power-domains= 00000019 
| |-dcmipp@48030000/clock-names=kclkmclk 6b636c6b006d636c6b00 
| |-dcmipp@48030000/port/endpoint/remote-endpoint=U 00000055 
| |-dcmipp@48030000/port/endpoint/bus-type= 00000004 
| |-dcmipp@48030000/port/endpoint/phandle=T 00000054 
| |-dcmipp@48030000/port/endpoint/name=endpoint 656e64706f696e7400 
| |-dcmipp@48030000/port/name=port 706f727400 
| |-dcmipp@48030000/resets=P 0000001200000050 
| |-dcmipp@48030000/interrupts=� 00000000000000c600000004 
| |-dcmipp@48030000/clocks=�" 00000012000000d00000001200000122 
| |-dcmipp@48030000/compatible=st,stm32mp25-dcmipp 73742c73746d33326d7032352d64636d69707000
| |-dcmipp@48030000/status=okay 6f6b617900 
| |-dcmipp@48030000/reg=H 4803000000001000 
| |-dcmipp@48030000/phandle=& 00000126 
| |-dcmipp@48030000/access-controllers=W 0000001800000057 
| |-dcmipp@48030000/name=dcmipp 64636d69707000 
|
|-[camera:]

Is there something else I should be looking for?

[Gregory PLANCHON]

Hi @rdmikkelsen ,

To help us understand better, could you share the link to your drivers and your device tree?

Regards,

Grégory 

[Gregory PLANCHON]

There are a few more tests to run as well,

First, enable dynamic debug to get a bit more trace :

echo "module stm32_csi +p" > /sys/kernel/debug/dynamic_debug/control

You should get output like this:

[ 95.184783] stm32-csi 48020000.csi: Starting the CSI2
[ 95.184814] stm32-csi 48020000.csi: Computed Mbps: 1188
[ 95.184826] stm32-csi 48020000.csi: PHY settings: (1200 Mbps, 11 HS FRange, 460 OSC Freq)
[ 95.185466] stm32-csi 48020000.csi: VC0: enable DT0(0x2c)/DT0FT(0x4)
[ 97.162197] stm32-csi 48020000.csi: Stopping the CSI2

Then you have a lot of interrupts; this isn’t normal. Normally, you should only get one when launching the preview.
You should check the logs using the v4l2 log-status option

First, you need to find the correct subdev to get, for example:

$> cat /sys/class/video4linux/v4l-subdev6/name
48020000.csi

and then you can share the output of the command with us:

$> v4l2-ctl -d /dev/v4l-subdev6 --log-status

Regards,

Grégory 

[rdmikkelsen]

@Gregory PLANCHON Thank you for the list of tests to run. I got an error (sh: write error: Invalid argument) running the command to enable dynamic debug, so maybe the output doesn't have as much info as it could. There is definitely already a /sys/kernel/debug/dynamic_debug/control file with a fair amount in it. 

I have just seen the huge and always-increasing number of interrupts when I run the command to capture an image form the command line and it stalls forever in the middle of execution as in the first post. 

The output for cat /sys/class/video4linux/v4l-subdev6/name is 48020000.csi, as in your example. These seem to match up exactly with graph.png in the original post: v4l-subdev7/name is imx664 0-0010, and so on. 

The v4l2 log-status command doesn't give much information, unfortunately. Here is the output: 

Status Log:

[ 365.338391] 48020000.csi: ================= START STATUS =================
[ 365.339855] 48020000.csi: ================== END STATUS ==================

I can't share the driver source because it was provided to me by another company, but I can share the relevant device tree files, which I have included aobj-$(CONFIG_VIDEO_IMX664) += imx664.os attachments here. 

In the i2c drivers folder, there is a Makefile, in which I added this line in with the other IMX cameras:

obj-$(CONFIG_VIDEO_IMX664) += imx664.o

and a Kconfig file, in which I added this section in with the existing IMX camera blocks:

config VIDEO_IMX664
    tristate "Sony IMX664 sensor support"
    depends on OF_GPIO
    default m
    help
      This is a Video4Linux2 sensor driver for the Sony
      IMX664 camera.

      To compile this driver as a module, choose M here: the
      module will be called imx664.

[Gregory PLANCHON]

Hi @rdmikkelsen ,

Can you test this graph to dump the raw Bayer data coming from the sensor (example for the IMW335—adapt it for your sensor):

 media-ctl -d platform:48030000.dcmipp -r
media-ctl -d platform:48030000.dcmipp -l '"48020000.csi":1->"dcmipp_input":0[1]'
media-ctl -d platform:48030000.dcmipp -l "'dcmipp_input':1->'dcmipp_dump_postproc':0[1]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'imx335 0-001a':0[fmt:SRGGB10_1X10/2592x1944]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'48020000.csi':1[fmt:SRGGB10_1X10/2592x1944]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'dcmipp_input':1[fmt:SRGGB10_1X10/2592x1944 field:none]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB10_1X10/2592x1944]"
export dump_capture_dev=$(media-ctl -d "platform:48030000.dcmipp" -e "dcmipp_dump_capture")

then 

v4l2-ctl -d $dump_capture_dev --set-fmt-video=width=2592,height=1944,pixelformat=RG10 --stream-mmap --stream-count=1 --stream-to=grab-2592x1944-rggb10le.raw

To launch the capture 

And for dynamic debug you can follow this wiki page 

Regards,

Grégory 

[Gregory PLANCHON]

Hi @rdmikkelsen ,

I update my comment with the dynamic debug command.

It seems there was a typo in the command

Regards,

Grégory 

[rdmikkelsen]

@Gregory PLANCHON 

Thank you for the commands for a raw data dump as well as the updated dynamic debug command. I do see more info now with the log st6atus command, but the v4l2-ctl capture command still hangs forever in the same way as before. It does make a .raw file, but it has a size of 0. I let it run for a little bit before ending it with Ctrl-C, which may be the reason for the high number of error events. 

Log status output below:

Status Log:

[ 496.797756] 48020000.csi: ================= START STATUS =================
[ 496.799272] stm32-csi 48020000.csi: Corrected ECC error events: 2004978
[ 496.805913] stm32-csi 48020000.csi: ECC error events: 2121
[ 496.811446] stm32-csi 48020000.csi: L1: D-PHY control error events: 1
[ 496.817882] stm32-csi 48020000.csi: L0: D-PHY control error events: 1
[ 496.824363] 48020000.csi: ================== END STATUS ==================

Here are the commands that I ran, adjusted for the sensor I am using:

media-ctl -d platform:48030000.dcmipp -r
media-ctl -d platform:48030000.dcmipp -l '"48020000.csi":1->"dcmipp_input":0[1]'
media-ctl -d platform:48030000.dcmipp -l "'dcmipp_input':1->'dcmipp_dump_postproc':0[1]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'imx664 0-0010':0[fmt:SRGGB12_1X12/2704x1540]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'48020000.csi':1[fmt:SRGGB12_1X12/2704x1540]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'dcmipp_input':1[fmt:SRGGB12_1X12/2704x1540 field:none]"
media-ctl -d platform:48030000.dcmipp --set-v4l2 "'dcmipp_dump_postproc':1[fmt:SRGGB12_1X12/2704x1540]"
export dump_capture_dev=$(media-ctl -d "platform:48030000.dcmipp" -e "dcmipp_dump_capture")

v4l2-ctl -d $dump_capture_dev --set-fmt-video=width=2704,height=1540,pixelformat=RG12 --stream-mmap --stream-count=1 --stream-to=grab-2704x1540-rggb12le.raw

Does any of the new output show anything helpful?

[Gregory PLANCHON]

Hi @rdmikkelsen ,

Thanks for your update. We are currently analysing the logs to understand your issue.

Could you please share the results of this command :

echo "module stm32_csi +p" > /sys/kernel/debug/dynamic_debug/control

 You should see this kind of output in dmesg :

[ 95.184783] stm32-csi 48020000.csi: Starting the CSI2
[ 95.184814] stm32-csi 48020000.csi: Computed Mbps: 1188
[ 95.184826] stm32-csi 48020000.csi: PHY settings: (1200 Mbps, 11 HS FRange, 460 OSC Freq)
[ 95.185466] stm32-csi 48020000.csi: VC0: enable DT0(0x2c)/DT0FT(0x4)
[ 97.162197] stm32-csi 48020000.csi: Stopping the CSI2

Regards,

Grégory