Rockchip-isp1
Contents
Overview
The document below provide basical informations about the rockchip-isp1 driver driver and Image Signal Processing block on Rockchip SoC with examples and details.
Hardware
More detailed information could be found in TRM chapter "Image Signal Processing", "MIPI D-PHY" , but they are only available under NDA.
ISP Details
ISP comprises with:
- MIPI serial camera interface
- Image Signal Processing
- Many Image Enhancement Blocks
- Crop
- Resize
Block diagram
The completed block diagram can not be pasted from datasheet to here, below diagram is an abstract version:
MIPI Details
There are three D-PHY instances in rockchip SoC, their connection are shown as following figure:
Software
Driver
rockchip-isp1 is a V4L2 based driver for Image Signal Processing block on Rockchip SoC. Compared to the earlier driver rk-isp10, it use Media Controller framework, which it's different from plain v4L2. While the plain v4L2 had a view of the device as a plain DMA based image drabber which connects the input data to the host memory, the Media Controller takes into consideration the fact that a typical video device might consist of multiple sub-devices.
Media Controller Basics
Please read below link carefully, especially if you don't have use it before.
https://linuxtv.org/downloads/v4l-dvb-apis/uapi/mediactl/media-controller-intro.html
Block diagram
File view
V4l2 view
| Name | Type | Description |
|---|---|---|
| rkisp1_mainpath | v4l2_vdev, capture |
Format: YUV, RAW Bayer; Max resolution: 4416*3312; Support: Crop |
| rkisp1_selfpath | v4l2_vdev, capture |
Format: YUV, RGB; Max resolution: 1920*1080; Support: Crop |
| rkisp1-isp-subdev | v4l2_subdev |
Internal isp blocks; Support: source/sink pad crop. The format on sink pad should be equal to sensor input format, the size should be equal/less than sensor input size. The format on source pad should be equal to vdev output format if output format is raw bayer, otherwise it should be YUYV2X8. The size should be equal/less than sink pad size. |
| rockchip-sy-mipi-dphy | v4l2_subdev |
MIPI-DPHY Configure |
| rkisp1-statistics | v4l2_vdev, capture |
Provide Image color Statistics information. |
| rkisp1-input-params | v4l2_vdev, output |
Accept params for AWB, BLC...... Image enhancement blcoks |
Sensor Driver Requirement
The sensor driver should implement controls in the following table.
| Controls | Description | R/W | Needed by |
|---|---|---|---|
| V4L2_CID_VBLANK | Vertical blanking. The idle period after every frame during which no image data is produced. The unit of vertical blanking is a line. Every line has length of the image width plus horizontal blanking at the pixel rate defined by V4L2_CID_PIXEL_RATE control in the same sub-device.
|
R | Ae |
| V4L2_CID_HBLANK | Horizontal blanking. The idle period after every line of image data during which no image data is produced. The unit of horizontal blanking is pixels. | R | Ae |
| V4L2_CID_EXPOSURE | Determines the exposure time of the camera sensor. The exposure time is limited by the frame interval. Drivers should interpret the values as 100 µs units, where the value 1 stands for 1/10000th of a second, 10000 for 1 second and 100000 for 10 seconds. | R/W | Ae |
| V4L2_CID_ANALOGUE_GAIN | Analogue gain is gain affecting all colour components in the pixel matrix. The gain operation is performed in the analogue domain before A/D conversion. | R/W | Ae |
| V4L2_CID_DIGITAL_GAIN | Digital gain is the value by which all colour components are multiplied by. Typically the digital gain applied is the control value divided by e.g. 0x100, meaning that to get no digital gain the control value needs to be 0x100. The no-gain configuration is also typically the default. | R/W | Ae |
| V4L2_CID_PIXEL_RATE | Pixel rate in the source pads of the subdev. This control is read-only and its unit is pixels / second. | R | Ae |
| V4L2_CID_LINK_FREQ | Data bus frequency. Together with the media bus pixel code, bus type (clock cycles per sample), the data bus frequency defines the pixel rate (V4L2_CID_PIXEL_RATE) in the pixel array (or possibly elsewhere, if the device is not an image sensor). The frame rate can be calculated from the pixel clock, image width and height and horizontal and vertical blanking. While the pixel rate control may be defined elsewhere than in the subdev containing the pixel array, the frame rate cannot be obtained from that information. This is because only on the pixel array it can be assumed that the vertical and horizontal blanking information is exact: no other blanking is allowed in the pixel array. The selection of frame rate is performed by selecting the desired horizontal and vertical blanking. The unit of this control is Hz.
|
R | mipi-dphy |
Devicetree Bindings
Linux
User applications
The v4l-utils tool and applications
The v4l-utils tool is a V4L2 development kit maintained by Linuxtv[1] . It provides a set
of V4L2 and media framework related tools for configuring V4L2 sub-device properties,
testing V4L2 devices, and providing development libraries such as libv4l2.so and so on.
This chapter mainly introduces two command-line tools in v4l-utils: media-ctl and v4l2-
ctl
media-ctl, used to view and configure topology
v4l2-ctl, used to configure v4l2 controls, capture frames, set cif, isp, sensor
parameters
The format code of different versions of v4l-utils will be different, especially mbus-fmt
part. The version used in this document is v4l-utils-1.14.1 integrated in Linux SDK.
Android
Camera Hal3
https://source.android.com/devices/camera/camera3
Reproduce
The driver currently can run on ASUS Tinker Board and Firefly-RK3288-Reload + Rockchip Debian/Yocto SDK.
Source Code
Kernel
- 4.15 + FORMLIST Patches
- Rockchip 4.4 BSP
Sensor Driver
Device Tree
Test Command
cat /sys/class/video4linux/video*/name # get self_path videox and main_path videox+1 gst-launch-1.0 rkcamsrc device=/dev/videox+1 io-mode=4 ! videoconvert ! video/x-raw,format=NV12,width=640,height=480 ! kmssink
Others
Topology
Media device information
------------------------
driver rkisp1
model rkisp1
serial
bus info
hw revision 0x0
driver version 0.0.0
Device topology
- entity 1: rkisp1-isp-subdev (4 pads, 5 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SBGGR10_1X10/800x600 field:none
crop.bounds:(0,0)/800x600
crop:(0,0)/800x600]
<- "rockchip-sy-mipi-dphy":1 [ENABLED]
pad1: Sink
<- "rkisp1-input-params":0 [ENABLED]
pad2: Source
[fmt:SBGGR10_1X10/800x600 field:none
crop.bounds:(0,0)/800x600
crop:(0,0)/800x600]
-> "rkisp1_selfpath":0 [ENABLED]
-> "rkisp1_mainpath":0 [ENABLED]
pad3: Source
-> "rkisp1-statistics":0 [ENABLED]
- entity 2: rkisp1_selfpath (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "rkisp1-isp-subdev":2 [ENABLED]
- entity 3: rkisp1_mainpath (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "rkisp1-isp-subdev":2 [ENABLED]
- entity 4: rkisp1-statistics (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Sink
<- "rkisp1-isp-subdev":3 [ENABLED]
- entity 5: rkisp1-input-params (1 pad, 1 link)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Source
-> "rkisp1-isp-subdev":1 [ENABLED]
- entity 6: rockchip-sy-mipi-dphy (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SRGGB10_1X10/3280x2464 field:none]
<- "imx219 2-0010":0 [ENABLED]
pad1: Source
[fmt:SRGGB10_1X10/3280x2464 field:none]
-> "rkisp1-isp-subdev":0 [ENABLED]
- entity 7: imx219 2-0010 (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SRGGB10_1X10/3280x2464 field:none]
-> "rockchip-sy-mipi-dphy":0 [ENABLED]
