Under the impetus of various needs, it is no longer a novelty for system manufacturers to develop their own chips.

Syed Alam, Global Semiconductor Director at Accenture, pointed out in an interview with the media: "More and more companies want to customize chips to meet the specific requirements of their applications, rather than using the same general-purpose chips as their competitors." "This allows them to better control the integration of software and hardware, while distinguishing themselves from competitors," Alam added.

In addition, in the view of analysts, self-developed chips can also give system manufacturers advantages in power consumption and cost that competitors do not have.

It is precisely because of so many advantages that more and more system manufacturers have joined the ranks of self-developed chips. The same situation has also occurred in the smartphone market, which has contributed half of the semiconductor market. Taking Apple as an example, they are also creating their own differentiated competitive advantages through more and more self-developed chips.

Advertisement

The local mobile giant vivo is also advancing into the deep waters of technology through self-developed chips. Recently, vivo announced at the imaging strategy conference that it will upgrade a brand-new architecture in the next generation of self-developed chip designs - from the traditional ISP architecture to the AI ISP architecture. This is a leap in technological innovation.

The logic behind self-developed chips

For vivo, it is indeed time for them to start thinking deeply about how to break through in the future, because in terms of mobile phone shipments alone, they have dominated the domestic market. According to data from the market research institution Canalys, in the third quarter of this year, vivo once again led the domestic market in shipments with 14.1 million units, accounting for a market share of 20%, further widening the gap with the second place.

From their performance, it can be seen that the excellent market performance is closely related to the high-quality products that vivo and iQOO mainly promoted in the second quarter this year. In addition to choosing the leading main chip, the company has spared no effort in investing in appearance, photography, and software optimization. In addition, vivo's continuous efforts in self-developed chips have also allowed the company to repeatedly set new heights in the fiercely competitive smartphone market.

In September 2021, vivo first disclosed the company's first self-developed professional imaging chip to the outside world.As Du Yuanjia, the Chief Image Algorithm Officer of vivo, said at the product introduction meeting, the company has made efforts in this area because it has seen that consumers not only want their mobile phones to take good photos in conventional scenes, but also hope that they can take good photos in challenging scenes such as low-light and sports scenes. This undoubtedly puts forward higher demands for the image processing capabilities and computing power of mobile phone chips. And as a mobile device, power consumption is an inevitable topic.

Based on such considerations, vivo has invested more than 300 people and spent 24 months in developing the independent image ISP vivo V1.

The so-called ISP is short for Image Signal Processor, which is the familiar image signal processor. According to the introduction, the main function of the ISP is generally to process the output data of the Image Sensor, such as AEC (Automatic Exposure Control), AGC (Automatic Gain Control), AWB (Automatic White Balance), color correction, Lens Shading, Gamma correction, removing bad points, Auto Black Level, Auto White Level, and other functions.

Looking back at the development of ISP on smartphones, after the integration of the industry, this function has basically been integrated into the SoC of mobile phone main control manufacturers such as Qualcomm, MediaTek, and Spreadtrum. However, in recent years, due to the increased requirements for image processing, manufacturers have had a trend of splitting it out, which is the reason why vivo chose to enter from this path.

"V1 is a professional image chip developed by vivo, serving high-speed computing imaging, and is a fully customized special specification integrated circuit chip," Du Yuanjia emphasized.

According to vivo, the V1 chip has the characteristics of high computing power, low latency, and low power consumption. Its innovation lies in optimizing the storage architecture of data inside the chip and the high-speed read and write circuit, achieving an equivalent of 32MB of ultra-large cache, while the cache of flagship-level desktop computer processors at that time was 16MB and 24MB. V1's read and write speed is as high as 35.84Gbps, with the ability to perform real-time noise reduction and frame interpolation at 1080P 60PFS.

Thanks to such hardware design, coupled with vivo's self-developed algorithms, and in cooperation with leading giants in the global imaging industry such as Zeiss, the flagship mobile phone X70 equipped with the self-developed image chip V1 has been highly recognized in the field of imaging.

Seven months later, vivo released its second self-developed chip V1+, which further expanded more application scenarios, including game frame insertion, and significantly improved the quality of night scene imaging. The X80 series equipped with V1+ is still the benchmark product of domestic flagship image mobile phones to date.

Now, vivo has developed the next generation of chips based on a new AI ISP architecture, which will also become another milestone in vivo's self-developed chips.

The leapfrog innovation brought by AI ISPIf we were to select a few buzzwords from the technology sector in recent years, artificial intelligence would undoubtedly occupy a place of honor. Due to its multifaceted advantages, artificial intelligence has transformed numerous industries over the past few years, and the ISP domain is no exception.

According to relevant data, to "clearly see" the raw data input from image sensors, traditional ISPs require the integration of nearly a hundred digital signal algorithms. To adapt to different cameras and lenses, thousands of parameters need to be adjusted to ensure their universality. At the same time, we also need to coordinate different algorithms to ensure that the ISP can work efficiently. This complex process undoubtedly poses a huge challenge to developers.

Therefore, artificial intelligence with "learning" capabilities has become the focus of attention for ISP developers, and AI ISP has emerged as a result.

As the semiconductor industry observation stated in the recently published article "A New Direction for ISP!", introducing AI into ISP is the best solution for night vision, scene recognition, face detection, super-resolution, and more. The article further points out that in order to incorporate AI elements into ISP, it is necessary to make corresponding adjustments to ISP to integrate with artificial intelligence. For example, to meet the needs of ISP, there needs to be a module capable of efficiently processing AI calculations (AI engine) to meet the needs of ISP. Of course, latency and power consumption should also be paid attention to simultaneously.

The new generation of vivo's AI ISP architecture was also born under the guidance of such thinking. This leapfrog technological innovation is first reflected in its self-developed AI computing unit. According to vivo, this AI ISP integrates vivo's self-developed AI computing unit, bringing ultra-high energy efficiency ratio.

Different from the industry's commonly used NPU, vivo chose the DLA accelerator, which is more suitable for complex imaging calculations, on the AI computing unit. AI algorithms are data-driven. Because a large amount of data comparison and fitting are needed in the calculation process, in order to further break through the energy efficiency ratio, it is necessary to solve the problems of latency and power consumption.

In response to this issue, vivo embedded dedicated on-chip SRAM for its DLA accelerator, which is another highlight of the AI ISP architecture design. Moreover, because it is equipped with a system cache on the shared self-developed memory unit, this greatly reduces data access latency.

According to vivo, the commonly used DDR architecture memory unit in the industry adopts an off-chip storage form, which has the disadvantages of high latency and high power consumption, limiting the data throughput rate, and thus limiting the inference calculation efficiency of video data. However, vivo insists on using the expensive DDR-Less on-chip near-memory computing architecture in the memory unit design, combined with the hardening of the ultra-large SRAM on the chip, to achieve the characteristics of fast computation and low latency, and its peak data throughput rate has reached an incredible 1.3 trillion bits per second (bit/s).

By jointly customizing the design of algorithms and DLA, vivo has achieved software and hardware collaborative optimization, solving the problems of power consumption and latency, while also making the peak energy efficiency ratio of the DLA accelerator reach 16.3 trillion operations per watt (Tops/w), and this data has also made vivo's AI ISP architecture reach the industry's ceiling level.Thanks to the upgrade of vivo's self-developed image processing unit, it will bring a brand new algorithm effect to its next generation of self-developed chips. According to the introduction, combined with the new hardware capabilities, vivo has upgraded the NR algorithm of V1+, bringing AI-NR better night scene details and enhancing the noise reduction effect of night scenes by 20%; through HDR tone mapping algorithm, combined with movie-level 3DLUTs color style, vivo can meet the needs of professional users for first-level overall color adjustment and second-level local color adjustment. Under the city night scene, the dynamic range has been theoretically increased by 4EV compared to before; the MEMC frame insertion algorithm based on new hardware further improves the delay problem and enhances the effect.

At the press conference, vivo introduced that, from the actual video shooting effect, the 1080p 60 frames (fps) night scene video shot based on the new generation of self-developed chips, compared with the previous generation of models, has brand new surprises in terms of sensitivity and dynamic range. In the extreme night environment, thanks to the upgrade and optimization of hardware and software, the picture quality and brightness of the video have been greatly improved compared to the previous generation. Vivo revealed that, based on the new hardware, the next generation of flagship image products of vivo (perhaps the rumored X90 series) can even shoot the stars in the sky under the light of camping.

From the above description, we can see the strength of vivo's self-developed chips. Coupled with the past experience in dual-core tuning, it will inevitably bring more possibilities to vivo in the creation of new flagship smartphones, and this will also ensure that vivo continues to maintain a leading position in the field of dual-core tuning and self-developed chips.

Go deep into the technical waters

After the release of V1, vivo achieved a breakthrough in the chip field; from V1 to V1+, it consolidated the strength of vivo in the chip field. Now, based on the new AI ISP architecture, the next generation of self-developed professional image chip - perhaps it is V2, has completed another transformation in the chip field for vivo. This also allows them to take root in the chip industry and have the confidence to continue to move forward in the deep technical waters.

Looking at the layout of several major domestic mobile phones, developing self-developed chips is a necessary path for vivo. Because if the competitors are all building their own differentiated advantages, it is difficult for the enterprise to win the next round of technical competition by relying solely on the brand. Especially in the current highly homogenized era of smartphones, vivo continues to invest in this self-developed track and develops the various businesses of the enterprise in conjunction, which is a guarantee for the enterprise to continue to win.

Yang Qing, the product manager of the vivo X series, also said in an interview with the media in April this year: "At present, vivo's resources are mainly deployed in algorithms, IP transformation, and chip architecture design, and processes such as IC circuit design, tape-out, production, etc., will be entrusted to partners, and both parties will use their respective expertise and advantages to cooperate reasonably and divide the work."

Hu Baishan, Executive Vice President and Chief Operating Officer of vivo, emphasized that, overall, vivo's R&D logic is based on four tracks: design (including appearance, interaction, etc.), image (photography, video, etc.), system (including underlying system and AI technology use, etc.), and performance (game scenarios, etc.). Among them, the ones that are closely tied to chip capabilities are image and performance. This is also the reason why vivo defined the chip at the beginning of these two tracks.

However, in my view, AI ISP may be just the "appetizer" for vivo's next generation of self-developed professional image chip V2. Vivo is bound to bring more innovation in the chip field, which will inevitably bring some unique leading advantages to its smartphones. In the current era of smartphones lacking attractiveness, these investments by vivo are worth looking forward to by consumers.