Infinite-ISP: An Open Source Hardware Image Signal Processor Platform for all Imaging Needs

By Taimur Bilal, Bakhtawar Amjad, Talha Iqbal, Bilal Zafar, Khurram Usman, Sohaib Imran Bhatti, Abdul Rahman Qureshi, Muqudas Rafiq, Maria Nadeem, Junaid Amjad, Sarfraz Shafi, Kamran Malik and Muhammad Abdullah (10xEngineers Inc.)

While traditionally not available in the public domain, access to the complete image signal processing (ISP) algorithmic pipeline and its hardware implementation are necessary to enable new imaging use cases and to improve the performance of high level deep learning vision networks. In this paper, we present  Infinite-ISP: A complete hardware ISP development suite, comprising of an ISP algorithm development platform, a bit-accurate fixed-point reference model, an ISP register transfer level development platform, an FPGA development workflow and an ISP tuning tool. To aid the hardware development process, we develop end-to-end reference designs for the KV260 vision AI starter kit (AMD Xilinx Kria SOM) and Efinix Ti180J484 kit, with support for 3 image sensors (Sony IMX219, Onsemi AR1335 and Omnivision OV5647), using the Infinite-ISP framework. These ISP reference designs support 10-bit 2592×1536 4 Megapixel (MP) Bayer image sensors with a maximum pixel throughput of 125 MP/s or 30 frames per second. We demonstrate that the image quality of Infinite-ISP is comparable to commercial ISPs found in Skype Certified Cameras and also performs competitively with digital still cameras for most imaging scenarios in terms of the perceptual IQ metrics BRISQUE, PIQE, and NIQE. We envision the Infinite-ISP under the permissive open-source license to streamline the ISP development from algorithmic design to hardware implementation and to foster community building and further research around it for both academia and industry.

To read the full article, click here