Revision as of 08:51, 30 November 2017

NOTE: Under construction

Overview

The document below provide basical informations about the rockchip-isp1 driver driver and Image Signal Processing block on Rockchip SoC with examples and details.

Hardware

ISP Details

Block diagram

The datasheet about ISP is currently only available under NDA, so we can't paste the completed block diagram from datasheet to here.

Below diagram is a abstract version:

MIPI Details

Software

Driver

rockchip-isp1 is a V4L2 based driver for Image Signal Processing block on Rockchip SoC. Compared to the earlier driver rk-isp10, it use Media Controller framework, which it's different from plain v4L2. While the plain v4L2 had a view of the device as a plain DMA based image drabber which connects the input data to the host memory, the Media Controller takes into consideration the fact that a typical video device might consist of multiple sub-devices 

Media Controller Basics

Please read below link carefully, especially if you don't have use it before.

https://linuxtv.org/downloads/v4l-dvb-apis/uapi/mediactl/media-controller-intro.html

Linux

User applications 

The applcation for plain v4L2 capture device still can be used(like gstreamer v4l2src plugins, v4l2ctl), but the media controller must be setup in a proper state before starting capture.
For your own applications for Video Capture with rockchip-isp1 driver, it should do things like the following commands:

media-ctl -l '"rockchip-sy-mipi-dphy":1->"rkisp1-isp-subdev":0[1]'
media-ctl -l '"rkisp1-input-params":0->"rkisp1-isp-subdev":1[1]'
media-ctl -l '"rkisp1-isp-subdev":2->"rkisp1_selfpath":0[1]'
media-ctl -l '"rkisp1-isp-subdev":2->"rkisp1_mainpath":0[1]'
media-ctl -l '"rkisp1-isp-subdev":3->"rkisp1-statistics":0[1]'
# Above parts of media pipeline have been initializated when kernel driver probing, but please note that they can be changed in runtime
# Set pad formats
## rkisp1 source pad
media-ctl -set-v4l2 '"rkisp1-isp-subdev":2[fmt:YUYV2X8/800x600]'
## rkisp1 sink pad == sensor source [ad]
media-ctl --set-v4l2 '"rkisp1-isp-subdev":0[fmt:SRGGB10/800*600]'
media-ctl --set-v4l2 '"ov2685 7-0010":0[fmt:SRGGB10/800*600]'
# Set link(Assume there are two sensors in one mipi-phy)
## Enable sensor1
media-ctl -l '"ov2685 7-0010":0->"rockchip-sy-mipi-dphy":0[1]'
## Disable sensor2
media-ctl -l '"ov5695 7-0036":0->"rockchip-sy-mipi-dphy":0[0]'

Integrate 3A algorithm(TO BE DONE)

Multi sensors(TO BE DONE)

Multi paths (TO BE DONE)

Record Preview, Still Capture in different resolution at a time

Gstreamer

We have write a gstreamer plugin, rkcamsrc, which is a modified v4l2src plugin.

Please refer to readme in github for usage.

Android

Camera Hal3

https://source.android.com/devices/camera/camera3?hl=zh-cn

Reproduce

The driver currently can run on ASUS Tinker Board and Firefly-RK3288-Reload.

Source Code

Kernel

1. 4.15 + FORMLIST Patches
2. Rockchip 4.4 BSP

Sensor Driver

1. IMX219
2. OV5647
3. TC35874X

Device Tree

1. Tinker Board
2. Firefly-rk3288-Reload

Test Command

cat /sys/class/video4linux/video*/name
# get self_path videox and main_path videox+1
gst-launch-1.0 rkcamsrc device=/dev/videox+1 io-mode=4 disable-3A=true ! videoconvert ! video/x-raw,format=NV12,width=640,height=480 ! kmssink

Others

Topology

Media device information
------------------------
driver          rkisp1
model           rkisp1
serial          
bus info        
hw revision     0x0
driver version  0.0.0

Device topology
- entity 1: rkisp1-isp-subdev (4 pads, 5 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SBGGR10_1X10/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                <- "rockchip-sy-mipi-dphy":1 [ENABLED]
        pad1: Sink
                <- "rkisp1-input-params":0 [ENABLED]
        pad2: Source
                [fmt:SBGGR10_1X10/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                -> "rkisp1_selfpath":0 [ENABLED]
                -> "rkisp1_mainpath":0 [ENABLED]
        pad3: Source
                -> "rkisp1-statistics":0 [ENABLED]

- entity 2: rkisp1_selfpath (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "rkisp1-isp-subdev":2 [ENABLED]

- entity 3: rkisp1_mainpath (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video1
        pad0: Sink
                <- "rkisp1-isp-subdev":2 [ENABLED]

- entity 4: rkisp1-statistics (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video2
        pad0: Sink
                <- "rkisp1-isp-subdev":3 [ENABLED]

- entity 5: rkisp1-input-params (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video3
        pad0: Source
                -> "rkisp1-isp-subdev":1 [ENABLED]

- entity 6: rockchip-sy-mipi-dphy (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:SRGGB10_1X10/3280x2464 field:none]
                <- "imx219 2-0010":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/3280x2464 field:none]
                -> "rkisp1-isp-subdev":0 [ENABLED]

- entity 7: imx219 2-0010 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:SRGGB10_1X10/3280x2464 field:none]
                -> "rockchip-sy-mipi-dphy":0 [ENABLED]