Design and FPGA Implementation of an Interpolative Neural Network for Digital Image Zooming

In this paper, the VLSI implementation of a novel image interpolator for the real-time applications is presented. It is based on Probabilistic Neural Network (PNN) that adjusts automatically the smoothing parameters of interpolative model for varied smooth/edge image region. We used MIAT methodology to design the algorithm and implemented on Altera Cyclone FPGA technology.

1.    Introduction

Image interpolation is a popular technique for image manipulation and processing. It’s most common application is to provide better visual effect after resizing a digital image for display or printing. In recent years, due to the consumer multimedia products being in vogue, imaging and display device become ubiquitous, and image interpolation is becoming more and more important.

The size and resolution of image display device, like LCD, increase constantly. The display systems require full screen and high quality visual effect. The image interpolation becomes an important process. Besides, digital zooming of DSC (Digital Still Camera) also relies on image interpolation technique.

Divers interpolation methods have been proposed. The simplest and fastest one is Nearest Neighbor (NN) method in which the value of the pixel point the closest is given as the value at the interpolating position [1]. This method is efficient, but its disadvantage is the blocking effect. Another generic method is bilinear interpolation. The interpolative value is given by the weighting average of neighboring pixels. This method considers that the variation value between the neighbor pixels of image is always smooth, despite the shape features like edges in images. It conduce blurry effect on image visualization.

Better interpolative image quality can be produced by bicubic interpolation [2]. Its principle is similar as bi-linear interpolation, but the linear spatial model is replaced by cubic-spline model. Consequently, it provides higher precision. Inevitably, it still has obvious artifact and blurry interpolative effect.

Aiming at above problems, some researchers proposed various methods to reduce blurry interpolative effect at edge region [3] [4] [5] [6]. These methods can be classified into two categories: one is to carry out edge detection or sharpness estimation before interpolation. This kind of interpolations consumes a great deal of computational resources. Besides, it may produce discontinuities at edge transition region and degrade therefore the visual quality of images. The category resides in the execution of edge enhancement after normal interpolation procedure. In addition to the consumption of extra computing time, it also has the drawback of introducing more noises as a result of twofold image processing stages.

An innovative approach of image interpolation is introduced in this paper. We propose an interpolative PNN (Probabilistic Neural Network) model and present the VLSI implementation with a high performer to support the real-time applications.

2.    Interpolative PNN model

In this section, we present the algorithm of intelligent image interpolator. Our idea comes from Specht's Probabilistic Neural Network (PNN) model[7,8]. It’s well adapted to manipulate pattern recognition and classification problem. PNN is constructed on the basis of Bayes theorem and Parzen probability density function estimation. It is consisted of three layers of neurons. The middle layer is kernel function layer which uses Gaussian kernel.

2.1 Interpolative PNN model

The interpolative PNN model that we propose is shown in figure 1. It is consisted of five layers neurons: Euclidian layer, Gaussian layer, weighting layer, summation layer and division layer.
 

Fig. 1 proposed interpolative PNN

In Fig. 1, the Euclidian distance
 
(1)

U is interpolation position,  for two dimensional image spaces. 
 , N is neighboring region of interpolation position U. g(X) denotes gray level at X_i. p(U) is interpolated value.

For an interpolative position U = (u, v), the positions Xi, i = 0,...,N, of N neighbor pixels in N_U are adopted as connection weights of Euclidian layer. The output is then feedforward to Gaussian neuron for obtaining the probability density f(d, ó). The gray level of neighbor pixel is used as weighting coefficient in weighting layer. Finally the resulting interpolated value is obtained by rule of gravity center:
 
(2)

Where
 
(3)

2.2 Estimation of smoothing parameter by single neuron

Due to the adoption of Gaussian function as kernel function of interpolative PNN, the interpolation will produce good smoothing result in flat image region. But at edge region, it may cause undesired blurring effect.

To overcome this blurring effect, we have elaborated a mechanism to make interpolative PNN adapt the region smoothing/sharpness. We use a single neuron [9] to adjust the parameter ó of interpolative PNN. The edge feature at each pixel is measured as input vector of single neuron, and the output of neuron is smoothing parameter ó.

Than we use PSO[10, 11] for training the single neuron to optimize smoothing parameter estimation model. We encode the single neuron by (N+3) parameters, including N weights, 1 bias and 2 adjustable parameters of transfer function. PSO, in offline mode, searches the best set of parameters in the (N+3) dimensional space.

Since the conventional PSNR does not precisely reflect human’s visual perception of a quality deviation between two images, we use the percentage edge error (PEE), proposed in[12] as the fitness function to evaluate the quality of PSO’s solution. The PEE measures how close the interpolated image sharpness is from the original image. The smaller PEE signifies a better edge preserving during interpolation process.
The complete system architecture of the proposed intelligent image interpolator is shown in figure 2.


Fig. 2 System architecture of intelligent image interpolator

3.    FPGA Implementation

We adopted our previous methodology for system design and hardware high-level synthesis [12] to achieve a hierarchical and modular in functional architecture.

The system is decomposed from coarse to fine into a set of independent functional modules, which is convenient for collaborative design. At the second stage, for every independent functional module, we use the graphical tool GRAFCET to construct a discrete event system model. At the third stage, according to a set of synthesis rules, all GRAFCET models can be translated into controller hardware architecture in VHDL. At the last stage, by connecting several general-purpose combinational logic building blocks, system hardware high-level synthesis can be completely realized. The functional architecture of the proposed intelligent image interpolator is shown in figure 3. We use a pipelined controller to control interpolator kernel, including edge adaptation module, Gaussian module and interpolator module. The 3-stage pipelined flow can increase the circuit performances.
 

Fig. 3 The functional architecture

In order to simplify the project and to increase the circuit performances, all the operations can be done using fixed point arithmetic. In such a case all the inputs and the output of the circuit can be quantized, as usually, in 256 levels using 8 bit buses for data transmission. A hardware implementation to obtain high speed performances is shown in Fig. 4, where the main algorithm has been subdivided into three different sub-blocks.

 

Fig. 4 A hardware implementation

Due to the complexity of the realization of some operations like multiplications, divisions and power elevation, and also due to the fact that these operators cannot reach high speed performances if realized with standard sequential methods, we propose to use a look-up table (LUT) to group most of these operators.

On the other side, to limit memory requirements, it can be useful to first compute 
 and   and successively to use these 8-bit values as addresses for the LUT memory.

In such a way, we can also avoid the use of a two’s complement arithmetic in the realization, because all the internal signals of the circuit are realized with positive integer numbers.

A remarkable simplification can be obtained noting that the values of the coefficient SIGMA  are between 0.1 and 1. Due to the fact that these values are weights used to keep the interpolated pixel similar either to its left or right neighbor, a coarse quantization of these values does not have a great influence on the result, while, on the other hand, it can produce an important simplification in the hardware implementation. In particular we obtained a very result using a quantization SIGMA composed of 16 steps

  (4)

The system is a large design as it contains 3 modules and a pipelined controller, hence the targeted FPGA device is the largest in the Cyclone family, the EP1C6. The implementation utilizes 3980 Logic Element (LE) slices which is 52% of the total number of LE slices available on this device. The critical path of the design lies in the function f component and the design uses a system clock of 20.5 MHz. The data rate achieved is 20fps in XGA format.

4.    Conclusion

This paper introduces a novel image interpolator which takes into consideration both smoothness (flat region) and sharpness (edge region) at the same model and presents the VLSI implementation with a high performer to support the real-time applications.

Acknowledge

This work has been supported by National Science Council of Taiwan Project NSC-92-2213-E-214-015 “Intelligent image interpolator”.

References

[1]    P. Thevenaz, T. Blu, and M. Unser, "Interpolation revisited", IEEE Trans. Medical Imaging, vol. 19, no. 7, pp.739-758, July 2000.

[2]    R. Keys, “Cubic Convolution Interpolation for Digital Image Processing,” IEEE Trans. Acoustics, Speech, Signal Processing, vol. 29, no. 6, 1981, pp. 1153-1160.

[3]    S. Thurnhofer and S. Mitra, “Edge-Enhanced Image Zooming,” Optical Engineering, vol. 35, no. 7, July 1996, pp. 1862-1870.

[4]    S. Battiato, G. Gallo, F. Stanco, A locally adaptive zooming algorithm for digital images, Image and Vision Computing, vol. 20, 2002, pp.805–812.

[5]    Dai, Dao-Qing; Shih, Tsi-Min; Chau, Foo-Tim, “Polynomial preserving algorithm for digital image interpolation”, Signal Processing Volume: 67, Issue: 1, May 29, 1998, pp. 109-121. 

[6]    Arandiga, F.; Donat, R.; Mulet, P., “Adaptive interpolation of images”, Signal Processing Volume: 83, Issue: 2, February, 2003, pp. 459-464.

[7]    Specht, D.F., “Probabilistic neural networks for classification, mapping, or associative memory”, IEEE International Conference on Neural Networks, vol.1, pp.525 -532,Jul 1988.

[8]    Specht, D.F., “Enhancements to probabilistic neural networks”, International Joint Conference on Neural Networks , Volume: 1 , pp.761 -7687,1992.

[9]    Ching-Han CHEN, 1998, "Automatic Design of Neural Networks Based on Genetic Algorithms", ICS'98 Workshop on Artificial Intelligence, pp.8-13.

[10]    Kennedy, J. and Eberhart, R. C., “Particle swarm optimization”, in Proc. IEEE Int. Conf. Neural Networks, Perth, Australia, Nov. 1995, pp. 1942–1948.

[11]    Kennedy, J., “The particle swarm: Social adaptation of knowledge”, in Proc. 1997 Int. Conf. Evolutionary Computation, Indianapolis, Apr. 1997, pp. 303–308.

[12]    Ching - Han CHEN, Jia - Hong DAI, “A Methodology for System Design and Hardware High-Level Synthesis” , ICSS 2005 , 2005 , pp.D3-1-6.