Why not integrate the image signal processor into the image sensor?

Have you ever wondered why image sensors don't integrate ISPs? Sensor manufacturers such as Sony, OmniVision and others don't seem to consider adding value to their sensor products by integrating dedicated ISPs.

As we said earlier, ISPs are a key component of embedded camera systems because sensors only provide data in RAW format. The Image Signal Sensor (ISP) can convert RAW-format data into high-quality, actionable output data through processing such as noise reduction, correction, and white balance.

So, if ISP is so convenient, why isn't ISP integrated in image sensors?

Has ISP never been integrated into image sensors?

Before examining the question of why ISP is not integrated into image sensors, let's first explore whether it is the case that image sensors have not always been integrated with ISP?

The answer is clearly no. In the early days, it was very common practice for image sensors to be integrated with image signal processors (ISPs). This integration scheme provided an integrated solution for early Russian camera systems. However, with the advancement of technology and the diversification of market needs, this integration model is gradually replaced by more flexible designs. The last known and sold sensor on the market with an integrated ISP is OmniVision's OV5640, a 1/4-inch 5 MP camera.

Why do image sensors no longer come with ISPs?

In my opinion, there are roughly two main reasons why image sensors are no longer equipped with ISp:

1. The rise of microprocessors with built-in ISPs
2. Differences in ISP requirements by product developers

Let's take a closer look below.

The Rise of Microprocessors with Built-in ISPs

Previously, processors did not have built-in ISPs.Later, as technology evolved, modern microprocessors such as Qualcomm, NXP, and NVIDIA have begun to offer built-in ISP functionality. This built-in ISP not only provides the necessary image processing power, but also reduces system cost and complexity. As a result, sensor manufacturers are choosing not to integrate ISPs into their image sensors in order to avoid adding unnecessary costs and remain competitive in the marketplace.

Differences in ISP requirements for product developers

The second important reason why manufacturers are adopting the original Bayer filter sensors is that many product developers and design engineers want to choose an ISP based on their needs, which also depend on the features and interfaces supported by the ISP.
Different ISPs offer different levels of functionality and performance, so developers need to choose the most appropriate ISP for their application scenario to achieve the best image quality and system performance. one of Sinoseen Camera Module's key strengths is its extensive industry experience in delivering the most appropriate products for different types of applications.

Impact of Microprocessor Built-in ISP

The popularity of microprocessor-built ISPs has changed the design and marketing strategy of image sensors. This convergence of technologies reduces the latency and bandwidth requirements for data transmission, making image processing more efficient. Built-in ISPs are often optimized for the microprocessor on board to provide better performance and faster processing.

This design also significantly reduces the total system cost, making the product more flexible to meet the needs of different customers. This flexibility allows manufacturers to introduce customized products for different market segments to meet the needs of specific application scenarios. In addition, microprocessors with built-in ISPs offer better integration and compatibility.

Of course, the built-in ISP might limit the design engineer's flexibility in selecting an ISP because the built-in ISP's features and performance may not meet the needs of all application scenarios. In addition, the built-in ISP may not offer advanced features and customization options comparable to those of top-tier standalone ISPs.

External ISP vs. Internal ISP

Although image processors now have built-in ISPs, there are still some scenarios that require the use of an external ISP.

Let's be clear from the outset that USB cameras require an external ISP. So the question of whether to choose an external ISP or an internal ISP will only arise with cameras other than USB cameras.
Although ISPs are integrated into today's image processors, internal ISPs are still less complex than external ISPs, which offer more flexibility and more features than internal ISPs. In applications where multiple cameras need to be synchronized, we still recommend using an external ISP for better image output.

Also, some developers using NVIDIA processors don't like to use the internal ISP because of the extra bandwidth it takes up on the GPU, so they prefer to use an external ISP for independent algorithmic processing.

Anyway, I hope you can understand why today ISPs are integrated in the processor and not in the image sensor. What's more, the choice between internal and external ISPs depends a lot on your application. The more complex your application, the greater the need for an external ISP.

Sinoseen, as a camera module manufacturer in China for more than 10 years, is committed to providing the most suitable camera module solutions for our customers. If you have any need for ISP, please feel free to ask Sinoseen for help.