Report Keri

download Report Keri

of 49

Embed Size (px)

Transcript of Report Keri

  • 7/25/2019 Report Keri


    Honours Project Report

    Genetic Algorithms:Colour Image Segmentation

    Keri Woods

    [email protected]

    Supervised by: Audrey Mbogho

    Department of Computer Science

    University of Cape Town


  • 7/25/2019 Report Keri



    Image segmentation has great importance in many image processing applications,

    and yet no general image segmentation exists. Image segmentation is complicated

    task, often with many parameters needing to be tuned to get good results. This

    report researches and discusses the concepts of image segmentation, genetic algo-

    rithms and segmentation evaluation. Due to the flexibility of genetic algorithms andtheir ability to effectively explore large search spaces, it may be viable to use them

    to improve existing image segmentation methods. A region merging algorithm was

    implemented and evaluated using quantitative means. These segmentation results

    were compared to those of two other image segmentation methods: region growing

    and watershed segmentation. Our region merging method was shown to produce

    average results. A genetic algorithm was implemented in an attempt to improve

    the segmentation results by evolving the segmentation parameters. A fitness func-

    tion needing neither human input nor a ground truth segmentation comparison was

    proposed. The results on the effect of the genetic algorithm on the performance of

    the genetic algorithm are inconclusive. However, even if the genetic algorithm does

    offer an improvement, it has the major drawback of running very slowly. There are

    many possible way of improving this system, and it seems to be an area where much

    research is needed.


  • 7/25/2019 Report Keri



    Abstract i

    List of Figures iv

    1 Introduction 11.1 Pro ject Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

    1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

    1.3 Project Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

    1.3.1 Limited Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

    2 Background and Related Work 4

    2.1 General Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

    2.2 Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    2.2.1 Colour Segmentation . . . . . . . . . . . . . . . . . . . . . . . 5

    2.2.2 Methods of Image Segmentation . . . . . . . . . . . . . . . . . 5

    2.2.3 Difficulties with Image Segmentation . . . . . . . . . . . . . . 7

    2.3 Segmentation Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 7

    2.4 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

    2.4.1 Applications of Genetic Algorithms for Image Segmentation . 9

    2.4.2 Fitness Function . . . . . . . . . . . . . . . . . . . . . . . . . 122.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

    3 System Design and Implementation 14

    3.1 Region Merging Image Segmentation . . . . . . . . . . . . . . . . . . 16

    3.1.1 Region Merging Module . . . . . . . . . . . . . . . . . . . . . 18

    3.2 Genetic Algorithm Module . . . . . . . . . . . . . . . . . . . . . . . . 23

    3.2.1 Chromosome Structure . . . . . . . . . . . . . . . . . . . . . . 23

    3.2.2 Reproduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

    3.2.3 Selection and Termination . . . . . . . . . . . . . . . . . . . . 26

    3.2.4 Fitness Function . . . . . . . . . . . . . . . . . . . . . . . . . 27


  • 7/25/2019 Report Keri


    4 Experimental Design 29

    4.1 Evaluation of Region Merging Image Segmentation . . . . . . . . . . 29

    4.2 Effect of Genetic Algorithm on Segmentation Quality . . . . . . . . . 30

    4.2.1 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

    4.2.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

    4.3 Modification of Genetic Algorithm Parameters . . . . . . . . . . . . . 31

    4.3.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

    4.3.2 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

    5 Results 32

    5.1 Region Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

    5.2 Genetic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

    5.2.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

    5.3 Alteration of Genetic Algorithm Parameters . . . . . . . . . . . . . . 36

    6 Future Work 38

    7 Conclusions 39


  • 7/25/2019 Report Keri


    List of Figures

    3.1 Module Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    3.2 UML Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

    3.3 Module Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    3.4 RegionList Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213.5 Region Class . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

    3.6 Genetic Algorithm Module . . . . . . . . . . . . . . . . . . . . . . . . 24

    3.7 Parameter Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

    3.8 Chromosome Structure . . . . . . . . . . . . . . . . . . . . . . . . . . 26

    5.1 Image Segmentation Example . . . . . . . . . . . . . . . . . . . . . . 33

    5.2 Comparison of Segmentation Methods using RMSE . . . . . . . . . . 34

    5.3 This table shows the comparison of the RMSE values using genetic

    algorithms vs. traditional algorithms . . . . . . . . . . . . . . . . . . 34

    5.4 The Effect of Scale Parameter on Segmentation Results . . . . . . . . 35


  • 7/25/2019 Report Keri


    Chapter 1


    1.1 Project Description

    In this project we review image segmentation methods and look into the use ge-

    netic algorithms for colour image segmentation. We base the segmentation part of

    our implementation on a region merging method described in Baatz and Schape[8].

    For our genetic algorithm, we follow a later improvement by Feitosaet al[22] which

    evolves the parameters for region merging to give visibly better results. The key

    to this improvement is a well-chosen fitness function, which captures the require-

    ments of good segmentation without needing a comparison with a ground truth

    segmented image. However, Feitosas fitness function relies on manual (user-based)

    segmentation to test the success or failure of the genetic algorithm. Our approach,

    in contrast, does not need user feedback, making it useful to real-world, automated

    segmentation applications.

    1.2 Motivation

    Image segmentation is an important process and its results are used in many image

    processing applications. However, despite its importance, there doesnt to seem to be

    any general method of image segmentation that works well on all images[42]. Image

    segmentation involves a lot of uncertainty, often with many parameters that need to

    be tuned to provide optimal results. For example, the Phoenix image segmentation

    method has 14 adjustable parameters[12]. This large number of parameters create a

    very large search space. Colour images have even more information than grey-scale

    images, and this information can be used to create higher quality segmentation. It

    does, however, increase the complexity of the problem. A way of handling the large

    search space is to use a directed search method, such as genetic algorithms.

    Genetic algorithms have many qualities that make them well suited to the problem

    of image segmentation, such as the ability to forego a local optimum to reach a global


  • 7/25/2019 Report Keri


    optimum and the ability to efficiently find an optimal solution from within a large

    search space[9].

    Genetic algorithms could allow an image segmentation process that usually requires

    manual input to become unsupervised.

    Genetic algorithms have been used to successfully colour segment images[12]. Dueto their flexibility, it seems feasible to be able to use them to come up with a general

    segmentation method.

    1.3 Project Scope

    The main aims of this investigation are:

    1. Research image segmentation

    2. Research genetic algorithms, especially those used as part of an image seg-

    mentation process

    3. Research segmentation evaluation

    4. Implement a method of segmenting colour images

    5. Implement a genetic algorithm to improve the segmentation algorithm imple-


    6. Evaluate whether the use of the genetic algorithm is an improvement over the

    original segmentation method

    It will be advantageous if the segmentation method using the genetic algorithm is

    able to segment general images and enables unsupervised colour image segmentation.

    1.3.1 Limited Scope

    Due to time limitations, some of the aims mentioned in the project proposal were

    not met. These include:

    1. Only one segmentation method was implemented and experimented with,

    rather than comparing a number of different ones

    2. The genetic algorithm implemented wasnt compared with those developed by


    3. No experimentation on genetic algorithm parameters was done


  • 7/25/2019 Report Keri


    The reasons for these aims not being completed include:

    1. Difficulties in the implementation of the region merging segmentation, includ-

    ing memory leaks

    2. Proposal objectives being too ambitious

    3. Slow running times of the region merging-genetic algorithm system


  • 7/25/2019 Report Keri


    Chapter 2

    Background and Related Work

    In this section, we start by looking at the concept and importance of image segmen-

    tation and mention the requirements for good image segmentation. The implication

    of using colour in image segmentation is explored and methods for image segmen-

    tation are briefly discussed. Problems with existing image segmentation methods

    are mentioned. The evaluation of image segmentation is briefly discussed. Genetic

    algorithms are then introduced and their suitability for use in image segmentation

    is examined. We explore various applications of genetic algorithms to the problem

    of image segmentation. Genetic algorithm fitness functions that have been used by

    others with image segmentation as discussed. Finally, the feasibility of the use of

    genetic algorithms for general colour image segmentation is considered and designissues for such an algorithm are discussed.

    2.1 General Overview

    Image segmentation is an important process and its results are used in many image

    processing applications. However, there is no general way to successfully segment all

    images[42]. Colour images have more information than grey-scale images, and this

    information can be used to create higher quality segmentation. It does, however,

    increase the complexity of the problem. A way of handling this complexity is to use

    a directed search method, such as genetic algorithms. Genetic algorithms, which

    mimic the process of evolution, have many qualities that make them well suited to

    the problem of image segmentation, such as the ability to forego a local optimum

    to reach a global optimum[9] and the ability too efficiently find an optimal solution

    from within a large search space[6].

    The main uses of genetic algorithms in image segmentation are for the modification

    of parameters in existing segmentation algorithms and pixel-level segmentation[21].

    Various algorithms that successfully apply genetic algorithms to image segmentation


  • 7/25/2019 Report Keri


    have been developed. Though these results are promising, none of them solve this

    open problem.

    2.2 Image Segmentation

    Image segmentation is the process of dividing an image into homogeneous regions.

    This is equivalent to finding the boundaries between the regions. Segmentation is

    the first step for many higher level image processing and computer vision opera-

    tions, including shape recognition, medical imaging[24], locating objects in satellite

    images[33], face detection[4] and road sign recognition[48].

    2.2.1 Colour Segmentation

    Until recently, most image segmentation has been performed on grey-scale im-

    ages. Processing colour images requires much more computation than the pro-

    cessing of grey-scale ones, but now with the increasing speed and decreasing cost

    of computation, colour image processing has been much researched in the last


    Colour images contain far more information than monochrome images. Each pixel

    in a colour image has information about brightness, hue and saturation. There

    are many models to represent the colours, including RGB (red, green, blue), CMY

    (cyan, magenta, yellow), HSV (hue, saturation, intensity), YIQ, HSI and many

    others. Several colour spaces have been used for image segmentation[19] and no

    general advantage of one colour space has yet been found[42].

    Many of the colour image segmentation algorithms are derived from methods of

    grey-scale image segmentation. However, colour creates a more complete represen-

    tation of an image and exploiting this fact can result in a more reliable segmenta-

    tion. Specialised techniques suited to the nature of colour information have been


    2.2.2 Methods of Image Segmentation

    Image segmentation is an old and important problem, and there are numerous image

    segmentation methods. Most of these methods were developed to be used on a

    certain class of images and therefore arent general image segmentation methods[9].

    Bhanu and Lee[10] divide the image segmentation algorithms into three major


    1. Edge Based


  • 7/25/2019 Report Keri


    2. Region Based

    3. Clustering Based

    4. Model Based1

    Edge Based Techniques

    Edge detection involves the detection of boundaries between different regions of

    the image. These boundaries correspond to discontinuities between pixels of the

    chosen feature (eg colour, texture, intensity).

    Region Based Techniques

    Region splitting is an image segmentation method whereby pixels are classified

    into regions. Each region corresponds to a range of feature values, with thresholds

    being the delimiters. The choice of these thresholds is very important, as it greatly

    affects the quality of the segmentation. This method tends to excessively split

    regions, resulting in over segmentation[10].

    Region growing joins neighbouring pixels with similar characteristics to form

    larger regions. This continues until the termination conditions are met. Most of the

    region growing algorithms focus on local information, making it difficult to get good

    global results. This method tends to excessively add to regions, resulting in under


    Region merging recursively merges similar regions. It is similar to region grow-

    ing, except that two whole regions are combined, rather than one region combining

    with individual pixels[14].

    Region splitting and mergingtries to overcome the weaknesses of region grow-

    ing and region splitting by combining the two techniques. Initially the image is

    divided into arbitrary regions. Region splitting and region merging occur until thetermination conditions are met[10].

    The Phoenix image segmentation algorithm is a region splitting method

    for segmentation that has been widely used and tested on colour images. It uses

    histogram analysis, thresholding and connected component analysis to partially seg-

    ment the image. Each region then has the same process applied to it recursively,

    until termination conditions are met and the image is fully segmented. The algo-

    rithm uses 17 parameters, 14 of which are adjustable[12][30].


    This isnt discussed here as it is used to match particular objects and so isnt relevant togeneral image segmentation


  • 7/25/2019 Report Keri


    Clustering Based Techniques

    Clustering separates the image into various classes without any prior knowledge.

    This method is based on the assumption that objects within each class should have

    a high degree of similarity, while those in different classes should be dissimilar. It is

    considered an unsupervised image segmentation technique[33][40].

    2.2.3 Difficulties with Image Segmentation

    Image segmentation is easy when objects have distinct colours and are well sep-

    arated, but can be a problem if there are many complex objects with less dis-

    tinct colour. Gradual variation in colour[41], illumination[43], shading[49] and

    textures[40] are also possible problems.

    A brute force method of dealing with image segmentation would be to enumerate

    all possible partitions of the image and evaluate each one. This creates an extremely

    large search space, and so this method is not feasible[9].

    Even once a segmentation method has been chosen, there are usually many param-

    eters that need to be tuned to create high quality segmentation. For most methods,

    it is not feasible to perform an exhaustive search of these parameters.

    Despite the many methods for image segmentation, there is no general algorithm

    that works well for all images. Because of the wide variety of images, a generalalgorithm needs to be adaptable. Only then can a segmentation algorithm cope

    with a wide variety of images[10].

    Many adaptive methods have been used for image segmentation, including ge-

    netic algorithms[11], neural networks[20], self-adaptive regularisation[47], ant colony

    optimization[37], fuzzy clustering[2] and simulated annealing[51].

    2.3 Segmentation Evaluation

    Methods of evaluating image segmentation are divided into two main categories[50]:

    1. Analytical Methods

    2. Empirical Methods

    Analytical methods look at the actual segmentation algorithm itself, rather than

    its results, while empirical methods evaluate the segmentation algorithm by lookingat its results. In this case, it is the empirical methods that we are more interested

    in. Empirical methods can be further sub divided into:


  • 7/25/2019 Report Keri


    1. Goodness Methods

    2. Discrepancy Methods

    Goodness methods evaluate the quality of the segmentation by looking at its de-

    sirable properties, and dont compare it to any other segmentation. It is this type of

    method that we wish to use as a fitness function for the genetic algorithm, because

    they dont need any prior knowledge of the image segmentation or ground truth

    segmented image and so are advantageous for an unsupervised algorithm. Discrep-

    ancy methods compare the segmentation results to an ideally segmented image, to

    see how much its segmentation differs from the target segmentation. It is this class

    of method that we will use to evaluate the results of our segmentation algorithm

    during experimentation.

    2.4 Genetic Algorithms

    Genetic algorithms are an optimization technique that can be used in image segmen-

    tation [11]. It mimics natural selection, allowing an algorithm to adapt. Solutions

    are represented by a population of individual chromosomes, usually represented as

    binary strings. A chromosome is made up of genes, each of which can represent a

    particular characteristic. Each individual in the population is evaluated and given

    a fitness score based on how well they solve the particular problem. The higher

    the individuals fitness score, the greater their probability of breeding. Breeding

    creates the next generation through crossover and mutation. Crossover combines

    the chromosome of two individuals, creating a new individual which is unlike either

    of the parents. Mutation, which occurs only a small percent of the time, randomly

    alters a new individuals chromosome. Since the more optimal individuals have a

    greater chance of breeding, the population tends to evolve and reach an optimal


    Genetic algorithms have been used to solve a wide variety of problems, including

    numerical and combinatorial optimisation, circuit design and cellular automata rule

    design[23]. In image processing, genetic algorithms have successfully been used for

    feature extraction, object recognition[44], knowledge based segmentation[36] and

    image classification[18].

    Image segmentation is easily and naturally formulated as on optimisation problem.

    It can either be seen as finding the optimal segmentation amongst all candidate seg-

    mentations, or as finding the optimal parameters for an existing image segmentation

    algorithm. In both cases, this creates an extremely large search space, indicating

    the use of genetic algorithms[6].

    Genetic algorithms are advantageous in that they are able to forego local optima

    in an attempt to reach the global optimum[9]. This makes them far less likely to


  • 7/25/2019 Report Keri


    get caught in a local optimum than deterministic optimization techniques, such as

    local hill-climbing and gradient descent. Though more computationally expensive

    than these methods, genetic algorithms are less computationally expensive than ex-

    haustive searches and other adaptive techniques, such as simulated annealing, which

    is theoretically guaranteed to find a global optimum[35]. While, genetic algorithms

    cannot guarantee finding a global optimum, they usually give a good approximation.

    This makes genetic algorithms a good compromise between accuracy and computa-

    tional intensity.

    Many image segmentation problems have large search spaces but need only an

    approximate global optimum. In this case, genetic algorithms using a directed search

    have proven useful[10]. A disadvantage of genetic algorithms is that they can take

    a long time to converge. Though this is the case, they are still much more efficient

    than performing an exhaustive search[23] [51].

    Many images, particularly natural scenes, are complex and noisy. A characteristic

    of genetic algorithms is their effectiveness and robustness in dealing with uncertainty,

    insufficient information and noise. Combined with the fact that no matter how it is

    posed, the image segmentation problem involves a very large search space, making

    genetic algorithms well suited to the problem[10][3].

    One of the major challenges for designing genetic algorithms is defining a fitness

    function. The only information available to the population of chromosomes is the re-

    sult of the fitness function evaluated every generation. This makes an appropriatelydefined fitness function essential for successful genetic algorithms. In the context of

    image segmentation, the fitness function should evaluate the resulting segmentation.

    There is, however, no generally accepted unsupervised method of evaluating image


    2.4.1 Applications of Genetic Algorithms for Image Segmen-


    Farmer and Shugars[21] divide the genetic algorithms used for image segmentation

    into two major classes:

    1. Parameter selection, where genetic algorithms are used to modify the param-

    eters of an existing image segmentation method to improve its output.

    2. Pixel-level segmentation, where genetic algorithms are used to perform region


    Most image segmentation methods have many parameters that need to be opti-

    mised, and therefore the first method is used more often[11]. Many such methods are


  • 7/25/2019 Report Keri


    discussed, as well as a few methods employing pixel-level segmentation. Modified

    genetic algorithms and hybrid genetic algorithms have also been used for segmenta-


    Parameter Modification

    Most image segmentation methods have many parameters, constants and thresholds

    that need to be adjusted to produce optimal segmentation results. This creates a

    very large search space. Since the parameters typically interact in complex and non-

    linear ways, an analytic solution is not generally possible. With a reasonable amount

    of computation, genetic algorithms are able to find good approximations of a global

    optimum within a large search space. They are therefore well suited to problems

    involving parameter optimization. Most of the applications of genetic algorithms to

    image segmentation involve the optimisation of various parameters[44][11][42][40].

    Bhanu et al[11] pose image segmentation as an optimisation problem. They de-

    fine a general segmentation method, whereby genetic algorithms are applied to the

    parameters of various well known image segmentation methods. They advocate the

    use of genetic algorithms to adapting the parameters of knows segmentation meth-

    ods in order to be applicable to general images. They used outdoor colour imagery

    and adapted 4 parameters of the Phoenix segmentation algorithm with genetic al-

    gorithms. They had successful results, producing high quality image segmentation

    with a reasonable amount of computation. Even though they perform well on out-door scenes, these algorithms have not been proved to be able to cope with general

    images. The fact that these algorithms can be modified to adapt the parameters of

    other segmentation methods makes this method very promising.

    Feitosa et al[22] adopt a very similar approach and use genetic algorithms to modify

    the parameters of a region merging segmentation algorithm. They use a fitness

    function that measures the similarity of resulting segments to a target segmentation

    provided by a user. Though computation is straight forward and intuitive, manual

    segmentation is still necessary beforehand. This method can easily be adapted to

    modify parameters of other segmentation methods.

    Zingaretti el al[52] propose using genetic algorithms in unsupervised colour image

    segmentation. This is another case of parameters of an existing image segmentation

    method being tuned by genetic algorithms. A key difference in this method is

    that it performs multi-pass thresholding. Different thresholds are adapted during

    each pass of genetic algorithms. An important advantage of this method over the

    previous one is that segmentation is performed totally unsupervised, without any

    manual segmentation. It also doesnt rely on any prior information regarding the

    type of image that is being processed or the task for which the segmentation results

    will be used. This approach successfully segmented a wide variety of images, with


  • 7/25/2019 Report Keri


    the exception of images that were highly textured.

    Pignalberi et al[39] use genetic algorithms for the optimisation of parameters in an

    image segmentation algorithm. In this case, they focused on range images, where a

    pixel is coloured depending on the distance between the object and a sensor. This

    method segments out surfaces of 3D objects, but could be applied to segmentation

    of 2D images.

    Pixel-Level Segmentation

    In pixel-level segmentation, genetic algorithms are used for region labelling. A pixel

    is classed in a particular region depending on its characteristics[21]. Peng el al[38]

    take this approach. Each pixel in the image is represented by a chromosome, which

    is a region label.

    Ramos and Muge[40] reformulate image segmentation as a clustering problem and

    use genetic algorithms to find the optimal clusters. The chromosomes encode which

    pixels are members of which regions. The major disadvantage of this method is that

    the number of clusters must be given as an input, and so cant be an unsupervised,

    general approach. Chun and Yang[16] take a similar approach, but use a fuzzy

    fitness function.

    Modified Genetic Algorithms

    Gong and Yang[25] represent the image and the segmentation results by quadtrees.

    In a similar way to Zingaretti el al[52], they define a two pass system, genetic

    algorithms being used for optimization in both passes. In the first pass, genetic

    algorithms are used to minimise an energy function. In the optional second pass,

    a parameter defining how coarse or fine the segmentation is modified by genetic

    algorithms to obtain optimal segmentation results. The chromosomes encode the

    quadtrees, making it inefficient to apply the usual crossover and mutation operations.

    To cope with this, a new crossover method and three mutation methods are defined.Aoyagi and Tsuji[7] use modified genetic algorithms for pixel-level segmentation.

    They approach image segmentation as a feature clustering problem and like Gong

    and Yang[25] use an energy function as a fitness function. They found it difficult to

    get ideal segmentation using traditional genetic algorithms, and so introduced four

    special types of mutation. They also propose a new method for creating individuals

    of the population.

    Hybrid Genetic Algorithms

    Grenfenstette[26] mentions that genetic algorithms can be combined with local

    search techniques, creating a high performance search algorithm. The following


  • 7/25/2019 Report Keri


    are examples of successful implementation of hybrid genetic algorithms for image


    Genetic algorithms have been combined with other evolutionary methods to tackle

    the image segmentation problem. Bhandarkar and Zhang[9] combine genetic algo-

    rithms with simulated annealing based techniques - which mimic the physical pro-cess of cooling - to approach the problem of grey-scale image segmentation. They

    do this is an attempt to get rid of the weaknesses of each class. This resulted in

    better performance than traditional genetic algorithms. Zhijun et al[51] combined

    genetic algorithms and simulated annealing to evolve a neural network, resulting in

    less computation than if genetic algorithms were used on there own. Melkemi et

    al[35] combine genetic algorithms with extremal optimization to perform pixel-level


    2.4.2 Fitness Function

    As mentioned previously, the choice of an appropriate fitness function is very im-

    portant. It is the fitness function that is largely responsible for the quality of the

    image segmentation obtained. The fitness function is what the algorithm "aims" to

    optimise. When dealing with image segmentation, one needs a fitness function that

    indicates how well the image has been segmented.

    Feitosaet al[22] calculate fitness by quantifying the difference between the segmen-tation obtained using the individuals parameters and a target segmentation. The

    target segmentation is obtained by manual segmentation, making this a discrepancy

    method of evaluation. This works fine for experimental purposes - it shows that

    genetic algorithms can be used to improve the quality of segmentation. However,

    this fitness function makes the segmentation program of little practical use. If one

    needs to segment the image beforehand, no benefit is obtained from the program.

    Many other implementations also used discrepancy methods for calculating fitness,

    including [29][39]. These methods arent relevant for use as a fitness function in this

    implementation, as we need an unsupervised method of image segmentation.

    Instead of using comparing the segmentation to a target segmentation, Zingaretti

    el al[52] compare it to an edge map obtained by applying a Roberts edge operator

    to the image. This makes the application more useful, but using an edge map as

    "perfect" segmentation means that their program tries to obtain this segmentation.

    It doesnt seem that segmentation better than that obtained with the edge operator

    can be obtained, and using an edge filter is much more efficient.

    As discussed previously, good image segmentation meets certain requirements:[33][9][10][27]

    1. Every pixel in the image belongs to a region


  • 7/25/2019 Report Keri


    2. A region is connected: any two pixels in a particular region can be connected

    by a line that doesnt leave the region

    3. Each region is homogeneous with respect to a chosen characteristic. The char-

    acteristic could be syntactic (for example, colour, intensity or texture) or based

    on semantic interpretation

    4. Adjacent regions cant be merged into a single homogeneous region

    5. No regions overlap

    One needs a fitness function that quantifies the quality of segmentation and it

    would be ideal if a fitness function could evaluate the segmentation based on the

    above segmentation requirements.

    2.5 Conclusion

    The use of genetic algorithms in image segmentation shows promising results. Ge-

    netic algorithms are a commonly used approach to optimising the parameters of

    existing image segmentation algorithms The major decisions are choosing a method

    of segmentation to which genetic algorithms will be applied, finding a fitness func-

    tion that is a good measure of the quality of image segmentation and finding a

    meaningful way to represent the chromosomes.


  • 7/25/2019 Report Keri


    Chapter 3

    System Design and Implementation

    This system is composed of two modules:

    1. Region Merging Module

    2. Genetic Algorithm Module

    The Region Merging Module performs a region merging method of image segmen-

    tation on the input image. The Genetic Algorithm Module optimises the region

    merging parameters. varies the region merging parameters, in an attempt to im-

    prove the segmentation.

    The Region Merging Module can be totally independent of the Genetic Algorithm

    Module, with the parameters being set manually rather than letting them be altered

    by the genetic algorithm. The Genetic Algorithm Module only calls the Region

    Merging Module to calculate the fitness value. The interaction between the modules

    is shown below in Figure 3.1:

    The Region Merging Module receives and stores the input image. Whenever the

    Genetic Algorithm Module requires a fitness value, it sends the segmentation pa-

    rameters that it is currently working with to the Region Merging Module, requestinga fitness value. The Region Merging Module uses these parameters, segments the

    image, evaluates the fitness and sends the fitness value to the Genetic Algorithm

    Module. This process is repeated until the genetic algorithm terminates. The Ge-

    netic Algorithm Module then sends the parameters that corresponded to the best

    fitness value to the Region Merging Module. The Region Merging Modules uses

    these parameters to segment the image and outputs the results of this segmenta-


    These modules are loosely coupled, with the region merging module being able tobe replaced with a different segmentation method that needs its parameters tuned.

    The fitness function can also be easily replace with different fitness function. This


  • 7/25/2019 Report Keri


    Region Merging


    Genetic Algorithm


    Input Image

    Output Result

    Send Parameters

    (Request Fitness)

    Return Fitness

    Final Parameters

    Figure 3.1: Module Interaction

    allows the system to be more adaptable when experimenting with the use of genetic

    algorithms for image segmentation.

    This system was implemented in C++, using Microsoft Visual Studio 2005. Image

    handling was done using the CImg Library[46], which is an open source C++ toolkit

    for image processing. This library is very easy to set up, is well documented and

    easy to use. It also offers much useful functionality for image processing.

    A genetic algorithm library, GALib[1] was used for the implementation of the

    Genetic Algorithm Module. It is well-documented, is very easy to use and offers

    many different options for customising and configuring your genetic algorithm.

    This implementation uses genetic algorithms modify the parameters of a region

    merging algorithm. This particular region merging method was designed by Baatz

    and Schape[8]. Feitosaet al[22] used genetic algorithms to modify the parameters of

    this algorithm in order to optimise the segmentation obtained. This implementation

    follows Feitosa et als ideas, except uses a different fitness function. The problemwith the fitness function used by them is that it relies on manual segmentation as

    measure of good segmentation, which makes the segmentation method only useful

    to prove that a genetic algorithm can be used to improve segmentation. This means

    that their implementation isnt useful as a segmentation application.

    The article[22] didnt mention key implementation details, leaving implementation

    open to interpretation.


  • 7/25/2019 Report Keri


    3.1 Region Merging Image Segmentation

    The idea behind region merging[8][22] is to divide the image up into many small

    regions. Adjacent regions can be merged together, based on whether certain criteria

    are met. This continues until no regions meet the merging criteria.

    This particular method of region merging looks at the heterogeneity of the various

    regions in the image when deciding whether or not regions will be merged. The

    goal of the merging procedure is to minimise the weighted heterogeneity. There are

    two main classes of heterogeneity: spectral heterogeneity and spatial heterogeneity.

    The spectral heterogeneity measures how varied the colour within the region is.

    The spatial heterogeneity measures the deviation of the region from a compact,

    smooth shape. This is separated into compactness and smoothness components.

    Compactness looks at the ratio of the perimeter to the square root of the area,

    while smoothness looks at the ratio of the perimeter to the length of a bounding

    box of the region, parallel to the image borders.

    Initially, each pixel is initialised as a region. Regions are then visited in turn,

    to decide whether that region will merge with another region. The order in which

    regions are visited is discussed later.

    When considering whether a region is going to be merged, each of the regions ad-

    jacent to the region are considered. A fusion factor between the region and each

    particular adjacent region is calculated. The fusion factor takes into account thevarious heterogeneity components:

    Fusionfactor= wcolour hcolour+ (1 wcolour) hshape


    hcolour is the spectral heterogeneity component

    hshape is the spatial heterogeneity component

    wcolour is the weight given to the spectral heterogeneity component, which gives its

    relative importance in comparison to the spatial heterogeneity component

    A colour heterogeneity of each of the colour channels (red, green and blue) is calcu-

    lated separately, and added to get the spectral heterogeneity component (modified

    from the version presented in the paper):

    hcolour = cwc(nObj3 cObj3 nObj1 cObj1 nObj2 cObj2)


    Obj1 is the region selected for merging

    Obj2 is the region adjacent to Obj1

    Obj3 is the result of merging Obj1 and Obj2


  • 7/25/2019 Report Keri


    c is the particular colour channel being considered

    c is the standard deviation of the pixels in the colour channel c 1

    The spatial heterogeneity component is a combination of compactness and smooth-

    ness components:hshape=wcompact hcompact+ (1 wcompact hsmooth)


    wcompact is the weight given to the compactness component relative to the smooth-

    ness component

    hcompact is the compactness component

    hsmooth is the smoothness component

    The formula for the compactness component is:

    hcompact=nObj3 lObj3nObj3 (nObj1 lObj1


    + nObj2 lObj2nObj2 )where: l is the perimeter of the region

    The formula for the smoothness component is:

    hsmooth= nObj3 lObj3bObj3 (nObj1 lObj1bObj1

    + nObj2 lObj2bObj2 )where: b is the perimeter of the regions bounding box

    Once the fusion factor for each adjacent region has been calculated, the region with

    the minimum fusion factor is considered. If its fusion factor is less than a certain

    threshold, then that adjacent region will be merged to the region being initially

    considered. The threshold value used here to the square of a scale parameter. This

    is known as a scale parameter, as adjusting this will result in differences in the

    resulting region size. The larger the scale parameter, the more merges that will

    happen, resulting in larger regions.

    This process is then repeated, with another region being selected and the fusionfactor between it and its neighbouring regions being calculated. This process is

    repeated until no more regions can be merged.

    Baatz[8] mentions that one of the problems with their method of region merging

    is that the order in which regions are visited in an attempt to merge them makes

    a difference in the segmentation results. They recommend that when choosing a

    sequence of starting regions that sequential merges should be distributed as far

    from each other as possible.

    1In this case, =



    i=1x2i xbar2 =



    i=1(xi x)2 where N is the number of

    pixels, xi is the pixel value and xis the average value of the pixels


  • 7/25/2019 Report Keri


    They mention that an approximate solution is to handle each region in a random

    sequence, but this isnt optimal. They didnt give details of the method they actually


    We tried to select random regions and process them, but found that processing

    regions in order


    was more successful. This also provides a deterministic result,which makes it easier for a genetic algorithm to work out. If the fitness generated by

    the segmentation algorithm isnt deterministic, it is difficult for the genetic algorithm

    to optimise it.

    3.1.1 Region Merging Module

    The system only depends on the CImg[46] class, with its accompanies cimg_library

    namespace, as well as standard C++ templates and functions.We make broad use of the Standard Template Library (STL) to create containers

    for the point, edge and region data structures. In each case, we use the container

    that minimises the segmentation algorithms running time.

    The system is composed of:

    1. classRegion

    2. classRegionList

    3. structPoint

    Their interactions can be seen in the UML diagram, shown below in Figure 3.2.

    RegionList and Region are the two main classes, with Point being just a simple

    struct. Along with this, various typedefs were defined:

    typedef unsigned long int Label

    typedef std::map < Label, Region >RMaptypedef cimg_library::CImg < Label >RImage

    typedef cimg_library::CImg < Label >Image

    typedef std::multimap < Label, Region >EdgeMaptypedef std::pair < Label, Region >EdgePairtypedef std::list < Point >PointList

    2Initially, the pixels are given region labels left to right, top to bottom. Region ordering is anordering of these regions, with regions that no longer exist being ignored


  • 7/25/2019 Report Keri


    RegionList( );~RegionList();

    getSize();mergeAll ();mergeRandom();saveResults();saveRegionMap();

    addRegion();getRegion();removeRegion();getFusionFactor ();getHCompact ();getHSmooth();getHColour();getHShape();tryMerge ( );coordToLabel ();

    RImage rimage;Image image;RMap* regions;unsigned int imageWidth;unsigned int imageHeight;float wRed;float wGreen;float wBlue;float wColour;float scale;float wCompact;







    void addEdge();unsigned int getArea();Label getLabel();float getAverageR();float getAverageG();float getAverageB();unsigned int getPerimeter();

    unsigned int getPerimeterCombined();unsigned int getPerimeterBB();unsigned int getPerimeterBBCombined();float getStdDev();float getStdDevCombined();void splice( );

    EdgeMap* getEdges();PointList* getPixels();void removeAllEdgesTo ( );void changeAllEdgesTo ( );

    RImage* rimage;Image* image;PointList* pixels;EdgeMap* edges;Label label;unsigned int maxX;unsigned int maxY;unsigned int minX;unsigned int minY;Label sumR;Label sumG;Label sumB;float averageR;float averageG;float averageB;unsigned int area;unsigned int perimeter;







    Point( int a, int b)

    int xint y


    Figure 3.2: UML Diagram


  • 7/25/2019 Report Keri


    Region Merging


    Genetic Algorithm


    Input Image

    Output Result

    Send Parameters

    (Request Fitness)

    Return Fitness

    Final Parameters

    Figure 3.3: Module Interaction

    RegionList Class

    The RegionList class represents the collection of all the regions in the segmented

    or partially segmented image. The structure of the class is shown in Figure 3.4.

    The main components of this class are:

    1. RImage rimage

    2. Image image

    3. RMap* regions

    The Image object, represents the original image, represented in RGB colour space.

    Image is a type of CImg object and allows access to the red, green and blue compo-

    nents of each pixel. The RImage object (region map) is also a type of CImg image

    object, with the same dimensions as the original image.

    The segmentation algorithm working on a collection of regions must often select

    and adjust a specific region. In this case, the STL map data structure is the natural

    choice (RMap). As a container, a map has the advantage of a logarithmic size

    complexity for key-based insertions, deletions, and searching. To allow keyed access

    to our collection of regions, we assign a unique integer (Label) to each region based

    on its initial position in the starting image. Subsequently, when merging two regions,

    the target region will keeps its key and adjusts all neighbouring regions to reflect

    this change. In this way, all possible changes to the region collection, like adding,removing, or altering region contents, and searching for a region, run in logarithmic

    complexity in the size of collection.


  • 7/25/2019 Report Keri


    RegionList( const char* filename, float r, float g, float b, float col, float s, float com );~RegionList();

    int getSize();void mergeAll ();

    bool mergeRandom();void saveResults( const char* filename );void saveRegionMap( const char* filename );void addRegion( const Label& label, const Point& p );Region* getRegion( const Label& label );void removeRegion( const Label& label );float getFusionFactor ( Region* region1, Region* region2 );float getHCompact ( Region* region1, Region* region2 );float getHSmooth( Region* region1, Region* region2 );float getHColour( Region* region1, Region* region2 );float getHShape( float hCompact, float hSmooth );bool tryMerge ( Region* r );Label coordToLabel ( unsigned int x, unsigned int y );

    RImage rimage;Image image;RMap* regions;unsigned int imageWidth;

    unsigned int imageHeight;float wRed;float wGreen;float wBlue;float wColour;float scale;float wCompact;


    Figure 3.4: RegionList Class


  • 7/25/2019 Report Keri


    Region( Label lbl, const Point& p, RImage* regionImage, Image* originalImage );


    void addEdge( Label lbl, Region* otherRegion );unsigned int getArea();Label getLabel();float getAverageR();float getAverageG();float getAverageB();unsigned int getPerimeter();unsigned int getPerimeterCombined( const Region* otherRegion );unsigned int getPerimeterBB();unsigned int getPerimeterBBCombined ( const Region* otherRegion );float getStdDev( unsigned int channel );float getStdDevCombined( const Region* otherRegion, unsigned int channel );

    void splice( Region* otherRegion );EdgeMap* getEdges();PointList* getPixels();void removeAllEdgesTo ( Label lbl );void changeAllEdgesTo ( Label lblOldRegion, Region* newRegion );

    RImage* rimage;Image* image;PointList* pixels;EdgeMap* edges;Label label;unsigned int maxX;unsigned int maxY;unsigned int minX;unsigned int minY;Label sumR;Label sumG;Label sumB;float averageR;float averageG;float averageB;unsigned int area;unsigned int perimeter;


    Figure 3.5: Region Class

    The image height and width are stored, as they are accessed very frequently and

    their storage prevents numerous function calls to the Image and RImage objects. Allthe other attributes are segmentation parameters and are passed in as arguments to

    the constructor. These are either set manually or are set by the genetic algorithm.

    Region Class

    TheRegionclass represents a region in a segmented or partially segmented image.

    The structure of the class is shown in Figure 3.5.

    The main components of the Regionclass are:

    1. PointList* pixels


  • 7/25/2019 Report Keri


    2. EdgeMap* edges

    3. Label label

    Pixels represent the pixels making up the image. They are Points, stored as a list

    (PointList). A list was used because insertion by pushing on to the front of thelist runs in constant time. Deletion of pixels only occurs when all the pixels in the

    region are deleted3 are deleted, so at each pixel delete takes constant time. (Deletion

    runs in linear time if the algorithm wasnt already iterating through the list.)

    Edges are stored as a multimap (EdgeMap). A boundary pixel in a region can

    point to one or more other regions. However, edges are stored separately from pixels,

    since actual pixel positions are unimportant when using edges to check connectivity.

    Insertion and find are both logarithmic time operations on a multimap. When

    erasing, all the elements with the same key are deleted. This is an logarithmic timeoperation, plus linear time is required for deleting nequal elements.

    Regions are assigned unique integer values (Label) based on their initial positions

    in the image.

    Pointers to both the original image and the region map are included as this class

    needs access to them. All the other attributes are stored, and updated every time

    the region is altered. These values are needed for the fusion factor calculation. It

    is worth using a little extra memory to store these values and save the numerous

    computations that would be required otherwise.

    3.2 Genetic Algorithm Module

    The Genetic Algorithm Module is a simple module, with much of the functionality

    being provided by the GALib library[1]. Its basic behaviour and interaction with

    the Region Merging Module is shown in Figure 3.6.

    The genetic algorithm is represented by a GASimpleGA object, provided by GALib.This class is used for genetic algorithms with non-overlapping populations, and in-

    herits from the general GAGeneticAlgorithm class.

    3.2.1 Chromosome Structure

    Each chromosome is represented by a GABin2DecGenome object (again provided

    by GALib) and inherits from the general GAGenome class. This type of genome

    stores its genes in binary, but converts them into decimal values to be used in thefitness calculation.

    3Regions are only every merged or destroyed


  • 7/25/2019 Report Keri


    Region Merging Module


    Mutate Offspring

    Insert Offspringinto Population

    Are stoppingcriteria satisfied

    Select Indiviualsfor Mating

    Select Indiviualsfor Mating

    Mate individulalto produce offspring

    Genetic Algorithm Module

    request fitnes

    return fitness

    Figure 3.6: Genetic Algorithm Module


  • 7/25/2019 Report Keri


    0.1 410wBlue 0-1

    0-1 40.1 10wGreen0-1 4wRed 0.1 10


    No. ofBits





    No. ofValues











    Figure 3.7: Parameter Details

    In the Genetic Algorithm Module, a chromosome represents the various segmenta-

    tion parameters of the region merging algorithm. These are:

    1. Scale parameter

    2. Weight given to the spectral heterogeneity

    3. Weight given to the compactness component relative to the smoothness com-


    4. Weight of the red channel

    5. Weight of the green channel

    6. Weight of the blue channel

    Each of these is encoded in binary, as a separate gene. The scale parameter canvary from 0 to 100, while all the other parameters are vary from 0 to 1. The scale

    parameter and the weight of the various colour bands each have a precision of0.1,

    while spectral and compactness weights have a precision of 0.01. All the weight

    parameters are encoded as positive integers, with the maximum value of1 divided

    by the particular precision factor (0.01 0r 0.1). To access the actual fractional

    values of the weights, one converts the parameter from binary to decimal and then

    multiplies it by the precision factor.

    To encode each gene in binary, one looks at the number of possible values thatparameter can take on. For example, the scale parameter can take on(10010)+1 =1001 different values. The binary representation of that gene therefore has to be

    able to encode 1001 different values. 1001 is then encoded using the same number

    of bits that would be used to encoded the smallest power of2 greater than 1001.

    This is 1024 = 210, and so log21024 = 10 bits is used to encode this.

    As can be see from figure, there are about1000100100101010 = 1010 differentparameter combinations! It would be infeasible to try each of these combinations to

    find the ideal combination of them, indicating the use of a genetic algorithm.

    The only other complication is that the sum of the weights of the colour bands

    needs to add up to 1, so their values are normalised. To get the normalised value


  • 7/25/2019 Report Keri


    Scale wColour wCompact wRed wGreen wBlue

    10 bits 7 bits 7 bits 4 bits 4 bits 4 bits

    Figure 3.8: Chromosome Structure

    of of band i, one takes the weight of band i and divides it by the sum of all three

    band weights.

    3.2.2 Reproduction

    In this implementation, one point crossover is used. In one point crossover, a random

    position on the chromosome is select and the two individuals exchange the genes

    on one side of that point. The mutation used is flip mutation, which is the typicalmutation operation used on binary strings: bits in the chromosome are flipped with

    a given probability.

    Elitism is used, with the best individual in each generation being carried over to

    the next generation.

    3.2.3 Selection and Termination

    Roulette wheel selection is used to select individuals to take part in reproduction. Inroulette wheel selection, an individuals probability of taking part in reproduction

    is proportional to their fitness. In this case, the fitness values are normalised: in

    this fitness evaluation (see the following section later), a low fitness corresponds to a

    good quality segmentation. The fitness values values are adjusted so that individuals

    with low fitness value end up with a high adjusted fitness, which corresponds to a

    high probability of taking place in reproduction.

    Termination takes place when the maximum number of generations is reached.

    This was chosen to be the termination criterion because then one can be sure thatthe genetic algorithm will terminate. On the other hand, if convergence was used

    as the termination criterion, it could continue for very many generations with the

    population never converging. Also, because of the large amount of computation

    required for each fitness computation - an image has to be both segmented and

    evaluated - one wants to make sure not too many generations are evolved. The

    image segmentation parameters dont have to be exact to get a good segmentation

    result, so it isnt worth waiting for the population to converge.


  • 7/25/2019 Report Keri


    3.2.4 Fitness Function

    As mentioned previously, one would like a fitness function that evaluates the follow-

    ing requirements of good image segmentation:

    1. Every pixel in the image belongs to a region

    2. A region is connected

    3. Each region is homogeneous

    4. Adjacent regions cant be merged into a single homogeneous region

    5. No regions overlap

    Requirements 1, 2 and 5 are always obtained when using this region merging tech-nique, therefore the important characteristics to quantify is that each region is ho-

    mogeneous and that adjacent regions cant be merged into one homogeneous region.

    Various different fitness function were. Liu[31] proposed goodness method to be

    used for the quantitative evaluation of the performance of image segmentation, which

    could be used as a fitness measure. The evaluation function Fon segmented image

    I is defined as:

    F(I) = 11000N2RRi=1 e2


    where: N2 is the size of the image R is the total number of regions Ai is the area

    of region i ei is the colour error, Euclidean distances in colour space between the

    values of the pixels in region i in the original image and the values of the pixels in

    the segmented image, defined as:

    ei= Aij=1

    (C1jR C2


    2 + (C1jG C2jG)

    2 + (C1jB C2jB)


    where: R, G and B are the colour channels C1j is the colour of pixel j in the

    original image C2j is the assigned colour of the same pixel in the segmented image

    Lower values ofF(I) correspond to better segmentation. Fevaluates the results

    both locally and globally.

    R is global measure and penalises over-segmentation.

    The e2


    portion is a local measure which penalises small regions or regions with a

    large colour error.

    Borsottiet al[13] propose an improvement, which was implemented:

    Q(I) = 11000N2



    1+logAi+ (R(Ai)



  • 7/25/2019 Report Keri


    We propose an alternative fitness function4, aiming to minimise weighted colour

    heterogeneity. The colour heterogeneity is based on Feitosas[22] concept.

    E(I) = ((Ri=1Hi) + 1) R


    E(I) is the fitness evaluation of segmented image I

    R is the number of regions in the image

    Hi is the colour heterogeneity of the ith region:

    Hi=Ai (R+ G+ B)

    where: Ai is the area of region i

    j is the standard deviation of colour channel j

    Hiis the weighted colour heterogeneity of region i. It is weighted by the size of the

    region so that larger regions contribute more to the overall heterogeneity. The colour

    heterogeneity of all the regions are added together to get the total heterogeneity of

    the image. 1 is added to this as when each region has area of 1, the standard

    deviation of that image is zero. When the regions are initialised, they are have

    an area of1 pixel, and so would each have a standard deviation of zero. If the 1

    wasnt added, the fitness of the initial segmentation would be 0, meaning its the

    ideal segmentation, which of course isnt the case.

    The sum of the colour heterogeneities as multiplied by R to penalise small regions.

    This is done because smaller regions tend to have a smaller standard deviation, for

    example a single pixel has a standard deviation of1.

    When evaluating this method against the segmentation requirements, one can seethat it is a good measure of the fact that each region is homogeneous. The fact

    that adjacent regions cant be merged isnt evaluated directly in this measure, but

    multiplying by the number of regions indirectly evaluates this: if two adjacent regions

    are very similar and are merged, their combined standard deviation doesnt increase

    much (ie Ai i +Aj j) is not much greater than (Ai +Aj) (i +j)), thetotal colour heterogeneity is multiplied by 1 less5. This results in a lower fitness

    value. This indirectly implies that if fitness is at a minimum, adjacent regions cant

    be merged into a homogeneous regions, otherwise the fitness value would decrease -

    which is not possible if the fitness value is the minimum one.

    4Once again, a low fitness value corresponds to good segmentation5Because the number of regions decreases


  • 7/25/2019 Report Keri


    Chapter 4

    Experimental Design

    The main areas that need to be tested are:

    1. Evaluation of region merging image segmentation

    2. Segmentation improvement provided by genetic algorithms

    3. Improvement of genetic algorithm by altering genetic algorithm parameters

    4.1 Evaluation of Region Merging Image Segmen-


    Various images from the Berkeley Image Dataset[34] will be chosen to be segmented.

    These represent a wide range of images, from those that are easy to segment to

    those that are much more difficult to segment, such as a camouflaged animal. These

    results will be evaluated subjectively. For example, it can be seen whether or not a

    camouflaged animal is segmented out.

    Dorin Comaniciu[17] has images on his website with the corresponding successfully

    segmented images. These segmented images will be used as ground truths when

    evaluating our system. It would have been better if images manually segmented

    by humans were used as the ground truth, as they represent "ideal" segmentation.

    However, no manually segmented images in the correct format were found. The

    region merging algorithm will be run on each of the images to produce the resulting

    segmented images. One needs to find an objective way of evaluating the performance

    of this segmentation in comparison to the ground truth images.

    This evaluation will be done using ImageMagicks[32] compare function, using the

    MSE (mean squared error) option. The difference between the values of each pixel of

    the ground truth segmentation and the corresponding pixel in the segmented image

    that is being evaluated. This value is then squared, to get rid of negative values


  • 7/25/2019 Report Keri


    and penalise more heavily where the pixel value difference is large. These values are

    then summed over the whole image to get the MSE. The root of this is taken to get

    , the RMSE (root mean squared error). This value is then divided by the number

    of pixels in the image to obtain the normalised RMSE, an evaluation measure that

    is size independent.

    This same procedure will be repeated with the region growing and watershed al-

    gorithms implemented by Marco Gallotta. The results of all three of these image

    segmentation algorithms will be compared.

    4.2 Effect of Genetic Algorithm on Segmentation


    4.2.1 Hypothesis

    The use of the genetic algorithm will improve the quality of the image segmentation.

    4.2.2 Method

    Each image to be evaluated will be segmented using the region merging algorithm.

    The segmentation parameters will be tweaked to attempt to obtain the best results.The best results will be evaluated using a normalised RMSE, as discussed above,

    and recorded. Due to the fact that the results from genetic algorithms arent deter-

    ministic, the images will be segmented five number times1 by the genetic algorithm

    version. Each run will be evaluated and the average score for each image recorded.

    The average score for each algorithm type will calculated and the results will be


    The population size used in the genetic algorithm should be made as large as

    computation time allows. The other genetic algorithm parameters should be tweakedto try and obtain the best possible results. Note of the values used should be made,

    so that the experiment can be repeated2.

    15 was an arbitrary choice. The more times the segmentation is run, the more accurate theresults, so it should be repeated as many times as is feasible

    2It may be preferable to perform the next part of the experimentation first, examining the effectof modifying the parameters of the genetic algorithm, to determine the optimal parameters


  • 7/25/2019 Report Keri


    4.3 Modification of Genetic Algorithm Parameters

    4.3.1 Purpose

    The main purpose of this section of the experimentation is to see what effect changing

    the genetic algorithm parameters and other factors - such as reproduction operators

    and termination criteria - will have on the quality of the segmentation results. This

    knowledge can then be used to find the optimal settings for the genetic algorithm.

    4.3.2 Method

    Various settings can be modified, such as:

    1. Population size

    2. Chance of mutation taking place

    3. Termination criteria

    4. Number of individuals passing unchanged to the next generation

    5. Crossover operators used

    6. Mutation operators used

    7. Selection method

    8. Whether elitism is used and how many individuals are passed unchanged to

    the next generation

    The setting to be examined is chosen. If it is a numerical value, it will be tested

    at equally spaced intervals throughout its range or part of its range, otherwise the

    various options are tried. It is very important that only one of these factors is

    changed at a time, so that one knows what is causing the change in the segmentation

    quality (if any change).

    The segmentation will be performed 53 times will each option or value, and the....

    measure calculated. The average of these values will be taken as the result at that

    particular value or option. If the parameter being set is numerical, the results will

    be graphed and analysed, if not the various options will be compared. Conclusions

    will be made about what options produce the best segmentation results.

    3again an arbitrary choice


  • 7/25/2019 Report Keri


    Chapter 5


    5.1 Region Merging

    The segmentation program outputs two result images. The first is the segmented

    image, with each region being shown in the average colour of all its pixels. The

    second is the region map, where each region is given a different value, and therefore

    a different shade in the image. This is done to make the distinction between regions

    clearer. If two adjacent regions have the same or very similar average values, the

    fact that they werent merged (and probably should have been) wouldnt show up

    in the segmented image. An example of the results are shown in Figure 5.1.

    The results of the comparison of the region merging implementation with the region

    growing and watershed algorithms is shown in Figure 5.2.

    As can by seen, this implementation had rather average results. The watershed

    method performed the best, and ran much faster than either of the other algorithms.

    5.2 Genetic Algorithm

    Two different fitness functions were implemented. Q(I), as defined previously, could

    be subjectively seen to perform badly, appearing to produce worse segmentation

    results than the original region merging algorithm. No further evaluation using this

    fitness function was performed.

    E(I) subjectively seemed to perform better than the original region merging algo-

    rithm. To confirm or refute this, further evaluation was performed. This evaluation

    was objective, being deterministic and calculated by the computer.

    The results obtained are shown below in Figure 5.3:

    As can be seen, the genetic algorithm only showed an improvement over the original

    region merging implementation for one of the images. Reasons for this could include


  • 7/25/2019 Report Keri





    Figure 5.1: Image Segmentation Example(a)Original Image

    (b)Segmented Image(c)Image Map


  • 7/25/2019 Report Keri








    5076 3324.65






    Figure 5.2: Comparison of Segmentation Methods using RMSE









    GA avg 4947











    Non GA

    4852 3414

    5195 3408





    Figure 5.3: This table shows the comparison of the RMSE values using geneticalgorithms vs. traditional algorithms

    a poorly chosen fitness function, using a population size that is too small to offer

    enough variation for the individuals to evolve or not running the genetic algorithm

    for enough generations for the population to evolve sufficiently.

    It may seem totally infeasible to find the ideal parameter combination for the

    region merging algorithm. However, the fact that there are1010, different parameter

    combinations is deceptive. From experimenting with varying the parameters, it

    was found1 that the only parameter that made a significant different was the scale

    parameter. Even the scale parameter only had any great effect on the segmentation

    when altered by a large amount. A feasible solution would be to give each colour

    channel an equal weighting, setting wRed, wGreen and wBlue all to 0.33; letting

    compactness and smoothness have the same influence, setting wCompact to 0.5.

    The effect of changing the scale parameter is shown below in Figure 5.4.

    The effect of changing the scale parameter by 10 isnt that great. Suppose, to be

    on the safe side, we allow the scale parameter to vary by increments of 5. This

    means that there are only 20 different combinations of parameters to try - which

    should be slightly faster than running 2 generations of a genetic algorithm with a

    population size of10.

    With a bit of prior knowledge, one doesnt even have to try all of this limited range

    of scale factor. If the various objects in the image appear distinct (such as the flowers

    shown below), one can set the scale parameter high. This high scale parameter will

    mean that more region merging occurs. This means that small colour variations,

    such as shadows or reflections, within an object hopefully wont be recognised as

    separate objects, while the separate objects are distinct enough from each other not

    to be merged together. However, if one is looking to distinguish objects of similar

    1These are just general observations, and havent been formally tested


  • 7/25/2019 Report Keri





    Figure 5.4: The Effect of Scale Parameter on Segmentation Results(a)Original Image

    (b)Scale parameter 30(c)Scale parameter 70


  • 7/25/2019 Report Keri


    colour, such as the camouflaged crocodile, one needs to set the scale parameter

    much lower. The negative aspect of doing this, however, is that it means that the

    segmentation isnt unsupervised and does require user interaction.

    A major problem in using this region merging- genetic algorithm system is that

    it is very slow to run. When processing a 481 by 321 pixel image, using a geneticalgorithm with a population size of 10 and running 20 generations, the take taken is

    about an hour2. This makes getting results a rather tedious process, especially since

    because of the random nature of genetic algorithms, one needs to take the average

    performance over a few trials when evaluating the performance of the segmentation.

    Marco Gallotta came up with a solution to this problem and ran this algorithm on

    a grid.

    5.2.1 Conclusion

    It can be concluded that we are unable to reject the hypothesis that the use of

    a genetic algorithm would improve the quality of the image segmentation. The

    genetic algorithm could provide an improvement in segmentation quality, though

    this hasnt been shown. Because of the time taken to run the genetic algorithm, its

    full potential hasnt been explored.

    Although there is a chance it could offer an improvement in segmentation quality,

    the major drawback is that running the genetic algorithm is extremely slow. Thiscan be improved by running it on a grid, but not many people have access to a grid,

    and so in most cases this isnt a feasible solution. Other options would be to limit

    the range through which the parameters are allowed two vary or to provide a totally

    different underlying image segmentation algorithm. An example of an algorithm that

    could be used would by the watershed segmentation algorithm. In the experiments

    done with Marco Gallottas implementation of this algorithm, it was found to run

    far quicker than either the region growing or region merging methods.

    5.3 Alteration of Genetic Algorithm Parameters

    It was not feasible to run these experiments due to the extremely long execution

    time of the region merging segmentation with the genetic algorithm3. Not only does

    one have to wait a long time to get results for the performance of various parameter

    settings, but it makes it totally infeasible to increase the maximum number of gener-

    ations or increase the population size. This is unfortunate, as one predicts that the

    segmentation performance would increase the as the number of generations increase2Running on a laptop with a Core Duo 1.8 GHz with 1gig RAM3It takes exactly an hour to run the algorithm for 20 generations, each with ten individuals


  • 7/25/2019 Report Keri


    (because there is more time to evolve) and also increase with increasing population

    size (a larger population provides larger variation, which makes finding an optimum

    solution more likely).


  • 7/25/2019 Report Keri


    Chapter 6

    Future Work

    Improvements on the system could include using different merging criteria (instead

    of the fusion factor) or using a different colour space. As mentioned the fitness

    function is very important to a genetic algorithm. Various fitness functions were

    suggested, though only one was experimented with. These fitness functions, as

    well as others could be experimented with. A fitness function is just a function,

    so there is the possibility that genetic programming1 could be used to evolve a

    fitness function. Manually segmented images could be used as ground truths and a

    discrepancy measure could be used as a fitness function. If this training set of images

    is a good representation of the type of images that one wishes to segment, this could

    be successful. The evolved fitness function could be used in a program, such as theone implemented in this paper to provide unsupervised image segmentation. The

    main challenge of doing this would be to provide suitable terminal and function sets.

    1for a description of genetic programming see [28]


  • 7/25/2019 Report Keri


    Chapter 7


    We successfully implemented region merging segmentation method. We improved

    on this by using a genetic algorithm to modify the segmentation parameters. In

    the process of doing this, we discovered the importance of a well chosen fitness

    function and a novel fitness function was defined. A major advantage of this fitness

    function is that it requires no comparison with a manually segmented image. This

    has a major advantage over many other fitness functions requiring such input, this

    method useful to real-world, automated segmentation applications.

    The effect of the genetic algorithm on the image segmentation results is inconclu-

    sive, its major drawback is that it is extremely slow to run. This means that the

    small improvement in segmentation quality probably isnt worth the extra compu-

    tation time. A way of getting around this drawback is to run the genetic algorithm

    on a grid.

    A major problem with the development of a general image segmentation method

    is that different applications and different users require different results from image

    segmentation. In reality, there is no ideal segmentation and the quality of the

    segmentation depends on required use of the output.


  • 7/25/2019 Report Keri



    [1] Galib: A c++ library of genetic algorithm components. WWW.

    [2] P. D. Acton, L. S. Pilowsky, H. F. Kung, and P. J. Ell. Automatic segmenta-

    tion of dynamic neuroreceptor single-photon emission tomography images using

    fuzzy clustering. European Journal of Nuclear Medicine, 26(6):581590, June


    [3] B. Ahrens. Genetic algorithm optimization of superresolution parameters. In

    Proceedings of the 2005 conference on Genetic and evolutionary computation,

    GECCO, pages 20832088, June 2005.

    [4] A. Albiol, L. Torres, and E.J. Delp. An unsupervised color image segmentation

    algorithm for facedetection applications. In Proceedings of 2001 International

    Conference on Image Processing, volume 2, pages 681684, October 2001.

    [5] M. Alfonseca. Genetic algorithms. InProceedings of the international conference

    on APL, pages 16, 1991.

    [6] P. Andrey. Selectionist relaxation: Genetic algorithms applied to image seg-

    mentation. In Image and Vision Computing, volume 17, pages 175187, 1999.

    [7] M. Aoyagi and K. Tsuji. A modified genetic algorithm for image segmentation

    based on feature clustering. InInternational Symposium on Information Theory

    and its Applications, 2004.

    [8] M. Baatz and A. Schape. Multiresolution segmentation - an opti-

    mization approach for high quality multi-scale image segmentation. In

    Strobl/Blaschke/Griesebner (editors): Angewandte Geographische Infoma-

    tionsverarbeitung XII, pages 1223.

    [9] S.M. Bhandarkar and H. Zhang. Image segmentation using evolutionary com-

    putation. InIEEE Transactions on Evolutionary Computation, volume 3, pages

    121, April 1999.

    [10] B. Bhanu and S. Lee. Genetic Learning for Adaptive Image Segmentation.

    Springer, 1994.


  • 7/25/2019 Report Keri


    [11] B. Bhanu, S. Lee, and J. Ming. Adaptive image segmentation using a genetic

    algorithm. InIEEE Transactions on Systems, Man and Cybernetics, volume 25,

    pages 15431567, December 1995.

    [12] B. Bhanu and J. Peng. Adaptive integrated image segmentation and object

    recognition. In IEEE Transactions on Systems, Man and Cybernetics, Part C,volume 30, pages 427441, November 2000.

    [13] M. Borsotti, P. Campadelli, and R. Schettini. Quantitative evaluation of color

    image segmentation results. Pattern Recogn. Lett., 19(8):741747, 1998.

    [14] A. Bouman. Ee637 digital image processing i course notes. Purdue University

    School of Electrical and Computer Engineering, Video and Image Processing

    Program, January 2007.

    [15] H. Chen, W. Chien, and S. Wang. Contrast-based color image segmentation.

    IEEE Signal Processing Letters, 11(7):641644, July 2004.

    [16] D.N. Chun and H.S. Yang. Robust image segmentation using genetic algorithm

    with a fuzzy measure. In Pattern Recognition 29(7), pages 11951211, 1996.

    [17] D. Comaniciu. Color image segmentation results. WWW, 2007.

    [18] J. M. Daida, T. F. Bersano-Begey, and J. F. Vesechy. Computer-assisted design

    of image classification algorithms: Dynamic and static fitness evaluations ina scaffold genetic programming environment. In Genetic Programming 1996:

    Proceedings of the First Annual Conference, July 1996.

    [19] K. S. Deshmukh and G. N. Shinde. An adaptive color image segmentation.

    Electronic Letters on Computer Vision and Image Analysis, 5(4):1223, 2005.

    [20] G. Dong and M. Xie. Color clustering and learning for image segmentation

    based on neural networks. In IEEE Transactions on Neural Networks, vol-

    ume 16, pages 925936, July 2005.

    [21] M.E. Farmer and D. Shugars. Application of genetic algorithms for wrapper-

    based image segmentation and classification. In IEEE Congress on Evolutionary

    Computation, pages 13001307, July 2006.

    [22] R.Q. Feitosa, G.A.O.P. Costa, and T.B. Cazes. A genetic approach for the

    automatic adaptation of segmentation parameters. In OBIA06, 2006.

    [23] S. Forrest. Genetic algorithms. ACM Computing Surveys, 28(1):7780, March



  • 7/25/2019 Report Keri


    [24] P. Ghosh and M. Mitchell. Segmentation of medical images using a genetic

    algorithm. InGECCO 06: Proceedings of the 8th annual conference on Genetic

    and evolutionary computation, pages 11711178, New York, NY, USA, 2006.

    ACM Press.

    [25] M. Gong and Y.H. Yang. Genetic-based multiresolution color image segmenta-tion. In Vision Interface 2001, pages 141148, June 2001.

    [26] J.J. Grefenstette. Genetic algorithms and machine learning. In COLT, pages

    34, 1993.

    [27] C. Hampton, T. Persons, C. Wyatt, and Y. Zhang. Survey of image segmenta-

    tion, 1998.

    [28] J.R. Koza. Genetic Programming. MIT Press, 1992.

    [29] L.F. Tian L. Tang and B.L. Steward. Color image segmentation with genetic

    algorithm for in-field weed sensing. Accepted for publication by Information

    and Electrical Technologies Division of ASAE.

    [30] K.I. Laws. The phoenix image segmentation system: Description and evalua-

    tion. In SRI AICenter, 1982.

    [31] J. Liu. A multiresolution method for distributed parameter estimation. SIAM

    Journal on Scientific Computing, 14(2):389405, 1993.

    [32] ImageMagick Studio LLC. Imagemagick: Convert, edit, and compose images.

    WWW, 1999.

    [33] L. Lucchese and S. K. Mitra. Color image segmentation: A state-of-the-art sur-

    vey. InProceedings of Indian National Science Academy (INSA-A), volume 67,

    pages 207221, March 2001.

    [34] D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented

    natural images and its application to evaluating segmentation algorithms andmeasuring ecological statistics. In Proc. 8th Intl Conf. Computer Vision, vol-

    ume 2, pages 416423, July 2001.

    [35] K. E. Melkemi, M. Batouche, and S. Foufou. A multiagent system approach for

    image segmentation using genetic algorithms and external optimization heuris-

    tics. Pattern Recongition Letters, 26:12301238, 2006.

    [36] K. Otobe, K. Tanaka, and M. Hirafuji. Knowledge acquisition on image process-

    ing based on genetic algorithms. In Proceedings of the IASTED InternationalConference on Signal and Image Processing, October 1998.


  • 7/25/2019 Report Keri


    [37] S. Ouadfel and M. Batouche. MRF-based image segmentation using ant colony

    system.Electronic Letters on Computer Vision and Image Analysis, 2(2):1224,


    [38] H. Peng, F. Long, Z. Chi, and W. Su. A hierarchical distributed genetic algo-

    rithm for image segmentation. In Proceedings of the 2000 Congress on Evolu-tionary Computation, volume 1, pages 272276, 2000.

    [39] G. Pignalberi, R. Cucchiara, L. Cinque, and S. Levialdi. Tuning range im-

    age segmentation by genetic algorithm. EURASIP Journal on Applied Signal

    Processing, 8:780790, 2003.

    [40] V. Ramos and F. Muge. Image colour segmentation by genetic algorithms. In

    Proceedings of the 11th Portuguese Conference on Pattern Recognition, May


    [41] S. J. Sangwine and R. E. N. Horne. The Colour Image Processing Handbook.

    Springer, 1998.

    [42] W. Skarbek and A. Koschan. Colour image segmentation - a survey, 1994.

    [43] I. Stainvas and D. Lowe. A generative model for separating illumination and

    reflectance from images. Journal of Machine Learning, 4:14991519, 2003.

    [44] D. L. Swets, B. Punch, and J. Weng. Genetic algorithms for object recognitionin a complex scene. In Proceedings of the 1995 International Conference on

    Image Processing (ICIP95), 1995.

    [45] S.M. Thede. An introduction to genetic algorithms. InJournal of Computing

    Sciences in Colleges, volume 20, pages 115123, October 2004.

    [46] David Tschumperl. C++ template image processing library. WWW, 2000.

    [47] W. Vanzella and V. Torre. A versatile segmentation procedure. In IEEE Trans-

    actions on Systems, Man and Cybernetics, Part B, volume 36, pages 366378,April 2006.

    [48] S. Vitabile, G. Pilato, and F. Sorbello. Road signs recognition using a dynamic

    pixel aggregation technique in the hsv color space. InProceedings of the 11th

    Internation Conference on Image Analysis and Processing (ICIAP01), 2001.

    [49] S. Wesolkowski, S. Tominaga, and R.D. Dony. Shading and highlight invariant

    color image segmentation. SPIE, 4300:229240, 2001.

    [50] Y.J. Zhang. A survey on evaluation methods for image segmentation. InPattern

    Recognition, volume 29, pages 13351346, 1996.


  • 7/25/2019 Report Keri