Object Recognition Using Symbolic Similarity Analysis
A Study on Object Recognition Using Symbolic Similarity Analysis and Zernike Moments
by Nyamatulla M. Patel*,
- Published in Journal of Advances in Science and Technology, E-ISSN: 2230-9659
Volume 12, Issue No. 25, Dec 2016, Pages 270 - 278 (9)
Published by: Ignited Minds Journals
ABSTRACT
Object recognition is a processof extracting information such as size, position, pose and functions related to the object. Var-ious approaches have been developed using invariant, 3D-CAD models (K. Roh and I. Kweon 1998, M. Stevens and J. Beveridge, 1997. D. Keren,M. Osadchy and C. Gotsman, 2001) .Object recognition methods based on local image such as local differential in-variants and SIFT are developed but these approaches have shown limited success to some problems (David.G.Lowe.1999, C. Schmid,A. Zesserman and R. Mohr, 1998). In this paper object recognition system is developed using Zernike moments, and symbolic simi-larity analysis. In this paperZernike moments are used,as these moments are invariant to general affine transformations. Here the concept of symbolic object and similarity measure are used for better object recognition under rotation and scale changes. The system consists of two parts, in first part is training phase and second part is testing phase. In training phase the object from COIL-100 database are trained and values are stored in the form of symbolic object in MATLAB database. In testing phase a query image is given as input for which a symbolic object is created using Zernike moments and recognition of the query image is carried out by doing similarity analysisbetween test object and knowledge base. The system is developed on MATLAB 7.6.0 and run on Pen-tium machine.
KEYWORD
object recognition, symbolic similarity analysis, 3D-CAD models, local image, Zernike moments, affine transformations, rotation, scale changes, COIL-100 database, MATLAB
1. INTRODUCTION
Object recognition in cluttered real-world scenes requires local image features that are unaffected by nearby clutter or partial occlusion. The features must be at least partially invariant to il-lumination, 3D projective transforms, and common object varia-tions. On the other hand, the features must also be sufficiently distinctive to identify specific objects among many alternatives. The difficulty of the object recognition problem is due in large part to thelack of success in finding such image features. How-ever, recent research on the use of dense local features (Schmid & Mohr 1997) has shown that efficient recognition can often be achieved by using local image descriptors sampled at a large number of repeatable locations. An object can be best described by a set of features. For object recognition there is a need to extract features from the object, the important features of any object are edges and the corners which represent the entire object. An object is always subjected to illu-mination variation, 3D-projective transforms and common object variations, under such conditions edges and corners are not suffi-cient for object recognition. In order to overcome such problems various methods have been developed in last few years for effi-cient recognition of the object, such as scale invariant feature transforms, Similarity-Measure Segmentation, mean shift & tex-tural moments , feature pose, geometric blur and Zernike moment and so on. Out of various methods available for feature extrac-tion, in this paperZernike moments are used. 3-d or 2-d objects are generally recognized with the help of their shapes and most of the real time objects have irregular shapes. Hence they cannot be properly described with the help of regular shape descriptors like circularity, linearity and so on. Hence in this paper Zernike momentsare used. Zernike moments are robust to various environmental changes such as illumination changes and pose changes (Sungho Kim, Inso Kweon Incheol Kim, 2003). The moments are higher space feature vector and are generally of order N. The more order of moments are considered, the better the recognition probability. Using these Zernike mo-ments in this work symbolic object is created. Symbolic objects are extensions of classical data types. In conventional data sets, the objects are “individualized” whereas in
Nyamatulla M. Patel* 2
and loaded with features extracted from Zernike moments then efficient object recognition can be done using similarity measure. The overall recognition process is as follows.
Figure 1.1 Block Diagram for Object Recognition Using Symbol-ic Similarity Analysis.
A) STEPS FOR GENERATING KNOWLEDGE BASE OF SYMBOLIC OBJECT
1. Edge Detection. 2. Corner Detection using Curvature Scale space. 3. Calculate Zernike moments. 4. Create Symbolic object using Quantitative features. 5. Store all the symbolic object values in knowledge base.
B) STEPS FOR RECOGNITION OF QUERY IMAGE
1. Edge Detection. 2. Corner Detection using Curvature Scale space. 3. Calculate Zernike moments. 4. Create Symbolic object using Quantitative features. 5. Compare symbolic object of the query image with symbolic objects of the knowledge base using similarity analysis algo-rithm.
A) EDGE AND CORNER DETECTION
Edgesand corners are regions of interest where there is a sudden change in intensity. These features play an important role in ob-ject identification methods used in machine vision and image processing systems. In this papera novel method for edge detec-tion in images is used. The approach used here is extracting Edg-es of the input image using morphological operators. Here mor-phological edge detector is used which returns a one pixel thick m-connected binary boundary image. The algorithm works on all types of images (i.e. binary, gray level and color images). Since this method is based on morphological operations, this is very simple, efficient and fast. This work of edge and corner detection is being adopted from the work ofNeeta Nain, Vijay Laxmi, Ankur Kumar Jain & Rakesh Agarwal and X. C. He & N. H. C. Yung (2008)respectively.
B) CURVATURE SCALE SPACE ALGORITHM
1. Detect edges using the likes of a Canny edge detector to obtain a binary edge map. 2. Extract edge contours from the edge map. When the edge reaches an end point, fill the gap and continue the extraction if the end point is nearly connected to another end point, or mark this point as a T-junction corner if the end point is nearly con-nected to an edge contour, but not to another end point 3. After contour extraction, compute the curvature at a fixed low scale for each contour to retain the true corners, and regard the local maxima of absolute curvature as corner candidates. 4. Compute athreshold adaptively according to the mean curva-ture within a region of support. Round corners are removed by comparing the curvature of corner candidates with the adaptive threshold. 5. Based on a dynamically recalculated region of support, evalu-ate the angles of the remaining corner candidates to eliminate any false corners. 6. Finally, consider the end points of open contours, and mark them as corners unless they are very close to another corner.
2
3-d or 2-d objects are generally recognized with the help of their shapes and most of the real time objects have irregular shapes. Hence they cannot be properly described with the help of regular shape descriptors like circularity, linearity and so on. Hence in this paperZernike moments are used. Zernike moments are ro-bust to various environmental changes such as illumination changes and pose changes. The moments are higher space feature vector and are generally of order N. The more order of moments are considered, the better the recognition probability (Sungho Kim, Inso Kweon Incheol Kim 2003). If any image is assumed to be a object, its descriptors are known as feature vectors. When an object is represented by more than one type of feature vectors combines, is known as synthetic objects. Zernike moment of order n andrepetition m is defined as Where: f (x,y) is the image intensity at (x,y) in Cartesian coordinates, Eqn (2)
3. SYMBOLIC OBJECT AND SIMILARITY
MEASURE
Symbolic objects are extensions of classical data types (K. Chidananda Gowda and E. Diday 1992). In conventional data sets, the objects are “individualized” whereas in symbolic data sets, they are more “unified” by means of relationships. They are more complex than conventional data in following ways. 1) All objects of a symbolic data set may not be defined on the same variables. or even an interval of values. 3) In complex symbolic objects, the values that the variables take may include one or more elementary objects. 4) The description of a symbolic object may depend on the relations existing between other objects. 5) The values that the variables take may have typicality values that indicate frequency of occurrence, relative likelihood, level of importance of the values, and so on. Based on the complexity, the symbolic objects can be of Asser-tion, Hoard, or Synthetic type.
A) DESCRIPTION OF SYMBOLIC OBJETCS
Various definitions and descriptions of symbolic objects are giv-en by Diday (1989). Symbolic objects are defined by a logical conjunction of events linking values and variables in which the variables can take one (including none) or more values and all the objects need not be defined on the same variables. Below are the nonformal description of symbolic objects of the type Asser-tion, Hoard, and Synthetic (K. Chidananda Gowda and E. Diday 1992).
EVENT
An event is a value-variable pair that links feature variables and feature values of objects. The following are two examples for events: E1= [height = [1.5 –2.0]] E2= [color = {white, blue}] Here, El is an event that indicates that the variable height takes a value between 1.5 and 2.0; and E2is an event that indicates that the variable color takes a value either white or blue.
ASSERTION OBJECTS
An assertion object is a conjunction of events pertaining to a particular object. Following is an example for an Assertion object:
Nyamatulla M. Patel* 2
It means thesynthetic object s consists of two Hoard objects hland h2where hlis a Hoard of roads and it consists of two ele-mentary objects. rl: It is an Expressway with 2 vehicles. r2: It is a mainroad with 1 vehicle. h2is a Hoard of vehicles and it consists of three elementary ob-jects. vl: It is a car of blue color and it is on r l. v2: It is a truck of red color and it is on rl. u3: It is a bus of green color and it is on r2.
B) FEATURE TYPES
Two Symbolic objects A and B are written as the Cartesian prod-uct of features Akand Bkas: A = A1 X A2 X……XAd B = B1 X B2 X…….XBd Let Ukdenote the domain of the kth feature. Then the feature space can be written as a Cartesian product: U(d) = U1 X U2 X…………….XUd Eqn. (3) The feature values may be measured on different scales resulting in the following types. 1) Quantitative features a) Continuous ratio values, e.g., length, velocity, height, etc. b) Discrete absolute values, e.g., persons, children, houses, etc. c) Interval values, e.g., duration, spread, etc 2) Qualitative features a) Nominal (unordered), e.g., color, sex, blood-type, etc. c) Combinational,e.g., road-crossing (highway 1, highway 2), vehiclesin-same-direction (car, bus), etc. 3) Structured variables (tree ordered or graph oriented sets) Table1 Typical Symbolic Object containing various Zernike mo-ment feature values. Structured variables are tree-ordered sets where the parent nodes represent the generalizations of the children nodes. For example, a parent node called “vehicle” may be a generalization of cars of the type “Ford,” “Fiat,” “Renault,” “Benz,” and so on. Out of these three types in this workAssertion type symbolic object with quantitative features is created for object recognition.
C) STRUCTURE OF SYMBOLIC OBJECT
A symbolic object is conceptually a small model which can rep-resent a large model also a symbolic object is a symbolic structure containing feature points/feature functions and a match-ing technique. So for object recognition here a novel symbolic object is created of assertion type with quantitative features. The structure of the symbolic object contains four different variables such as magnitude of the Zernike moments, angle of the Zernike moments, interest points, and Zernike moments of order 10. Such a structure is useful for efficient recognition of the objects using symbolic similarity analysisas in Table 1.
D) SIMILARITY MEASURE
1) S(A,A) = S(B,B) > S(A,B) 2) S(A,B) = S(B,A)
Similarity between A and B is written as 3) S(A,B) = S(A1,B1) + S(A2,B2)+…….+ S(Ak,Bk) Eqn. (4) 4) For the kth feature, S (Ak,Bk) is defined using the follow-ing three components: 1) SP(Ak,Bk) due to position p 2) Ss(Ak,Bk) due to span s 3) Sc(Ak,Bk) due to content c
2
when the feature type is quantitative. It indicates the relative positions of two feature values on real line. The similarity component due to “span” indicates the relative sizes of the feature values without referring to common parts between them. The component due to “content” is a measure of the common parts between two feature values. The components SP, Ss and Sc are defined such that their values are normalized between 0 and 1. Quantitative Interval Type of Ak and Bk: The definition of similarity between two quantitative intervalsis important since ratio and absolute type of quantitative features are special cases of the former: Let al= lower limit of interval Ak au= upper limit of interval Ak bl= lower limit of interval Bk bu= upper limit of interval Bk inters= length of intersection of Ak and Bk ls= span length of Ak and Bk =|max(au,bu)-min(a1,b1)| where max(.) and min(.) represent maximum and minimum val-ues, respectively. The three similarity components are defined as follows: Similarity components due to position is 4) SP(Ak,Bk) = 1-al-bl/|Uk| Eqn (5) where uk denotes the length of the maximum interval of kth fea-ture. Similarity component due to span is, 5) Ss (Ak,Bk) = (la + lb)/2.ls Eqn (6) Where la = |au-al|, and lb = |bu-bl| Similarity component due to content is Sc(Ak,Bk) = inters/ls To match two shape feature from contour images which are the j-th feature of the i-th shape for contours c1and c2the similarity function is given as The disadvantage with this is that it does not provide with maxi-mum deviation of one feature with respect to other feature over the entire training knowledge base. Therefore in this project symbolic object based technique is used for recognizing the test sample. The similarity between the two objects due to content and span is found using below equation Where
4. SIMULATION AND RESULTS
The proposed work has been simulated to evaluate the performance of our system which is developed using Zernike moments as object features. Symbolic object is created for every object and values are stored in MATLAB database and recognition of the object is carried by using symbolic similarity analysis. The pro-posed system is developed on MATLAB 7.6.0 version by writing MATLAB code and Pentium Dual Core Processor.
A) SIMULATION MODEL
Simulation of the proposed work is carried out on MATALB 7.6.0 version. Firstly in this system training of the objects is car-ried out, here COIL-100 database is used which is having 100 different classes of the objects. Each object class is having around 72 objects which are taken at different angles from 0 to 360 degree spaced 5degrees apart, so a total of 7200 object are available out which 400 objects are used for training. In the train-ing process 400 objects are given as input to system. The pro-posed system trains each object one by one.For each object first edges are detected using morphological operations and then from this edge extracted object the next step is to extract interest points (corner points). Corner detection for each object is done using curvature scale space algorithm which detects true corners while eliminating false and round corners. One true corners are detect-ed then Zernike moments at these interest points are calculated. The next step is to create symbolic object for each object and is created by using Zernike moments as
Nyamatulla M. Patel* 2
trained and values are stored in the form of sym-bolic object which is called as the knowledge base for object recognition. Once knowledge base is generated then recognition of the query object is done using symbolic similarity analysis. Before symbolic similarity analysis is applied a symbolic object for the query object is also created. Then to which class the query object belongs is recognized by applying symbolic similarity analysis and this analysis recognizes the appropriate class of the object by querying the knowledge base.
B) RESULTS AND DISCUSSION
Figure 1 Object models used for training phase
Figure 1 shows the 4different classes of the objects out 100 clas-ses of the object from CIOL-100 database such as dolls, cars fruits, cups, jugs, bottles etc can be found around us are trained and tested on this work. In the first experiment the performance of the system is evaluated by training a total of 400 objects of 100 different classes of four different views and scale changes by a factor of 1 or 2. The sys-tem recognizes almost all the objects at an average efficiency of 93.43%. In second experiment again a total of 400 objects were trained of 50 different classes of eight different views and scale change by a factor of 1 or 2. The system is able to recognize all the object of 50 different classes with an average efficiency of 93.98%. In third experiment a total of 1000 objects were trained of 50 different classes of 20 different views and scale change by a factor of 1 or 2. The system is able to recognize all the object of 50 different classes with an average efficiency of 96.60%.
Table 2 Results of the Object Recognition
Figure 2 Percentage of efficiency for different classes when tested on 1000 objects of 50 different classes of 20 different views.
Figure 3 Percentage of efficiency for the experiments carried out on 25, 50 and75 classes of different objects of 4 views each.
2
Table 3 Comparison with other methods for object recognition. From the above comparison table it is found that the results obtained for object recognition using Zernike moments and Symbolic Similarity Analysis are better and appreciable.
Figure 4 (a) Input Object of class-43 Figure 4 (b) Edge Detected Object Timefor detecting edge = 0.0077Sec Fig. 4(d) Figure 4(c)Object with Detected Corners.
Figure 4(d) Object with Curve Point Time for detecting corner = 0.1071Sec Time for extracting curve = 1.9354Sec. Figure 4 (a) Shows the input object. Edges from the input object are calculated using Morphological edge detector which requires a time0.0077Sec as shown in figure4 (b). The corner points and curve point are detected which require a time of 1.9354Secand 0.1071 Sec respectively as shown in figure 4 (c) and (d) respectively. Table 1shows the typical values of Zernike moments calculated for the above object which are stored in the form of symbolic object.
Output result after query Fig. 5 (a) Figure 5 (a) Object with Detected Corners
Nyamatulla M. Patel* 2
Fig. 5(b) Figure 5(b) Object with Curve Points.
The System recognizes the query object effectively and gives the result that this object belongs to class-43 with an query time of 7.60sec.
5. CONCLUSIONS AND FUTURE WORK
Object recognition is the process of extracting local object fea-tures such as name, size, position, edges, corners which are help-ful for better object recognition without requiring full infor-mation related to the object. In this worka object recognition system is developed using Zernike moments, symbolic object and similarity measure. A total of 100 different classes of the objects are tested on this system and the system is able to recognize all the classes of objects with an efficiency of 94.67%, whereas object recognition systems developed using Harris and Hessian corner detector which is 60% (Meritxell Vinyals, Arnau Ramisa and Ricardo Toledo) efficient, MSER and DOG‟S detectors which are 80% (Meritxell Vinyals, Arnau Ramisa and Ricardo Toledo) efficient and it is also efficient than the system devel-oped using object feature Geometric Blur which is48% efficient on Caltech-101 dataset (Alexander C. Berg, Tamara L. Berg and Jitendra Malik)from this it is concluded that the system devel-oped in this project using Zernike moments as object features and symbolic similarity analysis works well and able to classify 100 different classes of the objects. In this workobject recognition is carried out on the trained ob-jects of COIL-100 database, which is stored in the knowledge base of the system and objects are of small size, the future work on this project can be extended by training system for large ob-jects and a huge knowledge base and further a digital camera can be attached to the system by which online object recognition can be done which is helpful for motion tacking and Robotics appli-cations.
REFERENCES
Retriev-al”, In International Workshop on Shape, Contour and Grouping in Computer Vision, 1998. D. Keren, M. Osadchy, C. Gotsman.Antifaces: A Novel, Fast Method for Image Detectio. IEEE Tras. on Pattern Recognition and Machine Intelligence, vol.23, No.7, pp.747-761, 2001. David. G. Lowe. 1991. Fitting parameterized three-dimensional models to images. IEEE Trans. on Pattern Analysis and Machine Intelligence, 13(5):441-450. David.G.Lowe.1999. Object recognition from local scale-invariant features. International Conference on Computer Vision, Corfu, Greece, pp. 1150-1157. David G. Lowe.2001.Local feature view clustering for 3D object recognition. IEEE conference on Computer Vision and Pattern Recognition, Kauai, Hawaii. David G. Lowe. 2004 Distinctive Image Features from Scale-Invariant Key points. Computer Science Depart-ment University of British Columbia Vancouver, B.C., Canada lowe@cs.ubc.ca D. Keren, M. Osadchy, C. Gotsman.Antifaces: A Novel, Fast Method for Image Detectio. IEEE Tras. on Pattern Recognition and Machine Intelligence, vol.23, No.7, pp.747-761, 2001. E. Diday, Ed. Data Analysis, Learning Symbolic and Numeric Knowledge.Antibes, France: Nova Science, 1989. Harris, C. and Stephens, M. 1988. A combined corner and edge detector. In Fourth Alvey Vision Conference, Manchester, UK, pp. 147-151. J. Canny. A Computational Approach to Edge Detection. PAMI, 8(6):679-698, 1986. K. Chidananda Gowda and E. Diday, Symbolic Clustering Using a New Similarity Measure. IEEE transaction on systems, man, and cybernetics, VOL. 22, NO. 2, MARCWAF‟RIL 1992. K. Roh and I. Kweon. 2-D Occluded and Curved Object Recognition using Invariant Descriptor and Projective Refinement, IEICE
2
E81-D, No. 5, 1998. Moravec, H. 1981. Rover visual obstacle avoidance.In International Joint Conference on Artificial Intelligence, Vancouver, Canada, pp.785-790. M. Brown and David G. Lowe. 2001 Unsupervised 3D Object Recognition and Reconstruction in Unordered Datasets, proceedings of the 2001 IEEE computer society. M. Stevens, J. Beveridge, “Precise Matching of 3-D Target Models to Multisensor Data”, CSU, Jan. 20, 1997 Neeta Nain, Vijay Laxmi, Ankur Kumar Jain & Rakesh Agarwal. Morphological edge detection and corner de-tection algorithm using chain coding. Department of Computer Engineering ,Malaviya National Institute of Technology, Jaipur-302017Rajasthan, India Pope, A.R and Lowe, D.G. 2000. Probabilistic models of appearance for 3-D object recognition. International Journal of Computer Vision, 40(2):149-167. Schmid, C., and Mohr, R. 1997. Local grayvalue invar-iants for image retrieval. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(5):530-534. Sungho Kim, Inso Kweon Incheol Kim September 14-19 2003. Robust Model-based 3D Object Recognition by Combining Feature Matching with Tracking. Proceed-ings of the 2003 IEEE International Conference on Robotics & Automation Taipei, Taiwan.
Corresponding Author Nyamatulla M. Patel*
Assistant Professor, Department of Electronics and Communication Engineering, Hirasugar Institute of Technology, Nidasoshi
E-Mail – nyamatp@gmail.com