View source for Rockchip-isp1


NOTE: Under Construction

&nbsp;

= Overview =

The document below&nbsp;provide&nbsp;basical informations&nbsp;about the&nbsp;rockchip-isp1 driver driver and&nbsp;Image Signal Processing block on Rockchip SoC&nbsp;with&nbsp;examples and details.

= Hardware =

== ISP Details ==

=== Block diagram ===

The datasheet about ISP&nbsp;is currently only available under&nbsp;NDA, so we can't&nbsp;paste the completed block diagram from&nbsp;datasheet&nbsp;to here.

Below&nbsp;diagram&nbsp;is a abstract version:

== MIPI Details ==

= Software =

== Driver ==

rockchip-isp1 is a V4L2 based driver for Image Signal Processing block on Rockchip SoC. Compared to the earlier driver&nbsp;rk-isp10, it use&nbsp;Media Controller&nbsp;framework, which it's different from plain v4L2. While the plain v4L2 had a view of the device as a plain DMA based image drabber which connects the input data to the host memory, the Media Controller takes into consideration the fact that a typical video device might consist of multiple sub-devices&nbsp;

=== Media Controller Basics ===

Please read below link carefully,&nbsp;especially if you don't have use it before.

[https://linuxtv.org/downloads/v4l-dvb-apis/uapi/mediactl/media-controller-intro.html https://linuxtv.org/downloads/v4l-dvb-apis/uapi/mediactl/media-controller-intro.html]

== Linux ==

=== User applications&nbsp; ===

The applcation for&nbsp;plain v4L2 capture device still can be used(like gstreamer v4l2src plugins, v4l2ctl), but the media&nbsp;controller must be setup in a&nbsp;proper state before starting capture.<br/> For your own applications for Video Capture with rockchip-isp1 driver, it should do things like the following commands:
<pre>media-ctl -l '"rockchip-sy-mipi-dphy":1->"rkisp1-isp-subdev":0[1]'
media-ctl -l '"rkisp1-input-params":0->"rkisp1-isp-subdev":1[1]'
media-ctl -l '"rkisp1-isp-subdev":2->"rkisp1_selfpath":0[1]'
media-ctl -l '"rkisp1-isp-subdev":2->"rkisp1_mainpath":0[1]'
media-ctl -l '"rkisp1-isp-subdev":3->"rkisp1-statistics":0[1]'
# Above parts of media pipeline have been initializated when kernel driver probing, but please note that they can be changed in runtime
# Set pad formats
## rkisp1 source pad
media-ctl -set-v4l2 '"rkisp1-isp-subdev":2[fmt:YUYV2X8/800x600]'
## rkisp1 sink pad == sensor source [ad]
media-ctl --set-v4l2 '"rkisp1-isp-subdev":0[fmt:SRGGB10/800*600]'
media-ctl --set-v4l2 '"ov2685 7-0010":0[fmt:SRGGB10/800*600]'
# Set link(Assume there are two sensors in one mipi-phy)
## Enable sensor1
media-ctl -l '"ov2685 7-0010":0->"rockchip-sy-mipi-dphy":0[1]'
## Disable sensor2
media-ctl -l '"ov5695 7-0036":0->"rockchip-sy-mipi-dphy":0[0]'</pre>

Integrate 3A&nbsp;algorithm(TO BE DONE)

==== Multi sensors(TO BE DONE) ====

==== Multi paths (TO BE DONE) ====

Record, Preview, Still Capture&nbsp;in different resolution&nbsp;at a time

==== Gstreamer ====

We have write a gstreamer plugin, [https://github.com/rockchip-linux/gstreamer-rockchip-extra rkcamsrc], which is modified from gstreamer v4l2src plugin.

Please refer to readme in github for usage.

== Android ==

=== Camera Hal3 ===

[https://source.android.com/devices/camera/camera3?hl=zh-cn https://source.android.com/devices/camera/camera3?hl=zh-cn]

= Reproduce =

The driver currently can run&nbsp;on ASUS Tinker Board and Firefly-RK3288-Reload.

== Source Code ==

=== Kernel ===

#4.15 + [https://patchwork.linuxtv.org/project/linux-media/list/?submitter=7231 FORMLIST Patches] 
#[https://github.com/rockchip-linux/kernel/tree/isp-early-access Rockchip 4.4 BSP] 

=== Sensor Driver ===

#[https://github.com/rockchip-linux/kernel/blob/isp-early-access/drivers/media/i2c/imx219.c IMX219] 
#[https://github.com/rockchip-linux/kernel/blob/isp-early-access/drivers/media/i2c/ov5647.c OV5647] 
#[https://github.com/rockchip-linux/kernel/blob/isp-early-access/drivers/media/i2c/tc35874x.c TC35874X] 

=== Device Tree ===

#[https://github.com/rockchip-linux/kernel/blob/isp-early-access/arch/arm/boot/dts/rk3288-miniarm.dts Tinker Board] 
#[https://github.com/rockchip-linux/kernel/blob/isp-early-access/arch/arm/boot/dts/rk3288-firefly.dtsi Firefly-rk3288-Reload] 

== Test Command ==
<pre>cat /sys/class/video4linux/video*/name
# get self_path videox and main_path videox+1
gst-launch-1.0 rkcamsrc device=/dev/videox+1 io-mode=4 disable-3A=true&nbsp;! videoconvert&nbsp;! video/x-raw,format=NV12,width=640,height=480&nbsp;! kmssink</pre>

= Others =

== Topology ==
<pre>Media device information
------------------------
driver          rkisp1
model           rkisp1
serial          
bus info        
hw revision     0x0
driver version  0.0.0

Device topology
- entity 1: rkisp1-isp-subdev (4 pads, 5 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
        pad0: Sink
                [fmt:SBGGR10_1X10/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                <- "rockchip-sy-mipi-dphy":1 [ENABLED]
        pad1: Sink
                <- "rkisp1-input-params":0 [ENABLED]
        pad2: Source
                [fmt:SBGGR10_1X10/800x600 field:none
                 crop.bounds:(0,0)/800x600
                 crop:(0,0)/800x600]
                -> "rkisp1_selfpath":0 [ENABLED]
                -> "rkisp1_mainpath":0 [ENABLED]
        pad3: Source
                -> "rkisp1-statistics":0 [ENABLED]

- entity 2: rkisp1_selfpath (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "rkisp1-isp-subdev":2 [ENABLED]

- entity 3: rkisp1_mainpath (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video1
        pad0: Sink
                <- "rkisp1-isp-subdev":2 [ENABLED]

- entity 4: rkisp1-statistics (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video2
        pad0: Sink
                <- "rkisp1-isp-subdev":3 [ENABLED]

- entity 5: rkisp1-input-params (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video3
        pad0: Source
                -> "rkisp1-isp-subdev":1 [ENABLED]

- entity 6: rockchip-sy-mipi-dphy (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:SRGGB10_1X10/3280x2464 field:none]
                <- "imx219 2-0010":0 [ENABLED]
        pad1: Source
                [fmt:SRGGB10_1X10/3280x2464 field:none]
                -> "rkisp1-isp-subdev":0 [ENABLED]

- entity 7: imx219 2-0010 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev2
        pad0: Source
                [fmt:SRGGB10_1X10/3280x2464 field:none]
                -> "rockchip-sy-mipi-dphy":0 [ENABLED]</pre>