Keywords

1 Introduction

Skin has an important immunologic role, and for that reason the correlation between skin tumours development and immunologic mechanisms are intensely studied. Non-immunologic risk factors like individual predisposition, sun and environmental exposure are all contributing to the skin neoplasia incidence. Early accurate detection of all skin tumour types is essential for patients’ morbidity or survival.

Generally, in medical practice, the most important skin tumours, could be classified either as malignant tumours or benign lesions. The aim of this research is classification of skin tumours on malignant tumours and benign lesions for skin tumour dermoscopy images. In this research, only malignant melanomas will be analyzed. Melanomas are the most malignant skin tumours. They grow in melanocytes, the cells responsible for pigmentation. This type of skin cancer is rapidly increasing, but its related mortality rate is increasing more modestly. The critical factor in assessment of a patient prognosis in skin cancer is early diagnosis. More than 60,000 people in the United States (US) were diagnosed with invasive melanoma in 2000, and more than 8,000 died of the disease [1]. In 2011, an estimated 70,230 adults in the US were diagnosed with melanoma. It is estimated that 8,790 deaths from melanoma will occur within a year. Almost 10,130 deaths from melanoma will appear over this (2019) year [2]. Its frequency is rising in many other countries, for example, 10 cases were reported each year in Algeria [3].

Currently, experienced dermatologists use dermoscopy analysis and observation based on dermatology criteria to pose a working diagnosis of a type of a skin tumour. Availably of an objective system able to classify moles helps physicians-dermatologists in diagnosis and early detection of melanoma. In development of such a system it is not necessary to classify every mole but to achieve such a precision that all malignant tumours are classified as dangerous, and none is overlooked. The evaluation of the clinical exams’ accuracy showed that without use of a dermatoscope, dermatologists can detect only 65–80% of cases of melanoma. The dermoscopy increases the diagnostic accuracy rate for about 10–27% [4].

This paper presents hybrid automatic classification model for skin tumour images. The proposed classification system for skin tumour images is based on golden rule based on dermatology criteria, named ABCDE rule: Asymmetry, Border, Colour, Dimension, and Evolution. The proposed hybrid automatic classification model uses dermatology criteria ABC – Asymmetry, Border, and Colour, in this research. The Principal Component Analysis (PCA) has been used in analysis of Asymmetry; for Border irregularity – analysis of edge – outlining of a convex polynomial is measured; and for Colour – analysis of colours – standard deviation for all three red-green-blue (RGB) channels on pixels which are included on skin tumour images are measured. In general, three artificial intelligence techniques are used for classification, particularly: Support Vector Machine (SVM), k-nearest neighbors (k-NN) and artificial neural network – Multilayer perceptron (ANN-MLP). The implemented model is tested on the part of the skin images of HAM10000 Data Set. Also, this paper continuous the authors’ previous research in different applications on medical domain presented in [5,6,7,8].

The rest of the paper is organized in the following way: Sect. 2 provides an overview of the basic idea on image classification and related work. Section 3 presents modeling the automatic classification for skin tumour. The Preliminary experimental results testing with well-known HAM10000 Data Set are presented in Sect. 4. Section 5 provides conclusions and some points for future work.

2 Image Classification and Related Work

Classification is a process of assigning a new item or observation to its proper place in an established set of categories. Classification is used mostly as a supervised learning method, and the goal of classification is predictive. General references regarding data classification are presented in [9], and the very good contemporary hybrid classification techniques can be found in the textbook [10].

2.1 Related Work in Image Skin Tumour Classification

In the past three decades, many approaches have been proposed to solve skin tumour image classification problem to help physicians to make decision regarding this particular illness and future patient treatments. In the paper [1] the discrete wavelet transforms (DWT) for feature extraction is used, and skin images have been reduced using PCA. In the classification stage two classifiers have been developed. The first classifier is based on feed forward back propagation artificial neural network (FP-ANN) and the second classifier is based on k-nearest neighbor (k-NN). The features hence derived are used to train a neural network based binary classifier, which can automatically infer whether the image is that of a normal skin or a pathological skin, suffering from skin tumour. According to the classification results of both classifiers, in decision phase, normal or abnormal skin lesion is defined.

Several dermatoscopic rules for automatic detection of melanoma in order to generate new high-level features – ABCD rule, 7-point checklist, Menzies method and CASH (colour, architecture, symmetry, homogeneity) algorithm – allowing semantic analysis are shown in [4]. The extracted features are based on shape, colour and texture features. 206 images of skin lesions are used. They were all extracted from the two online public databases Dermatology Information System and DermQuest, 119 are melanomas, and 87 are not melanoma. A neural network classifier is used for decision making, but it is not noted which type of ANN is used.

The paper [11] presents a unified method for histopathology image representation learning, visual analysis interpretation, and automatic classification of skin histopathology images as either having basal cell carcinoma or not. The novel approach is inspired by ideas from image feature representation learning and deep learning (DL) and yields a DL architecture that combines an autoencoder learning layer, a convolutional layer. A softmax classifier for cancer detection, a generalization of a logistic regression classifier [12], and visual analysis interpretation are used.

Smartphone applications are readily accessible and potentially offer an instant risk assessment of the likelihood of malignancy so that the people could immediately seek further medical attention from a clinician for more detailed assessment of the lesion. There is, however, a risk that melanomas might be missed and treatment delayed if the application reassures the user that their lesion is low risk [13].

3 Modeling the Automatic Classification for Skin Tumour

The proposed flow-diagram for modeling the automatic classification for skin tumour image is presented in Fig. 1. It presents one of the standard steps: (1) Preprocessing; (2) Segmentation; (3) Analysis in three sub-steps – (a) Analysis of asymmetry; (b) Analysis of edge; (c) Analysis of Colours; (4) Extraction the Set of Features; (5) Classification with three classifiers – (a) SVM; (b) k-NN; (c) ANN-MLP. Therefore, before such an examination, it is necessary to start by preprocessing and segmenting the skin tumour image.

Fig. 1.
figure 1

Flow-diagram of the proposed hybrid techniques

Full size image

Preprocessing is the first step in skin tumour images. Many skin features may have impact on digital images like hair and colour, and other features such as brightness, and the type of the scanner or digital camera. In the preprocessing step, the border detection procedure, colour space transformation, contrast enhancement, and artifact removal are treated. The idea is that if there is a transaction on edge detection of a source noised image, it can be located with other additional edges due to the presence of noise. Therefore, filtering the noised image is necessary.

3.1 Preprocessing and Segmentation - Filtering the Noised Image

Some images include artefacts, mostly hair; these artefacts can be misleading for the segmentation algorithm. The DullRazor technique, an artefact removal preprocessing technique, deals well with hair and other artefacts. It tends to erase the details of the image by making the pigmented network unclear, and separate only skin tumor.

figure a

The basic steps of the DullRazor algorithm proposed in [14], are summarized by the pseudo code shown in Algorithm 1. The DullRazor technique has fully addressed the problem of human hairs including the imaged lesions by designing an automatic segmentation program to differentiate skin lesions from the normal healthy skin. This technique performed well with most of the images, and with those where hairs, especially dark thick hairs, cover part of the lesions as shown on Fig. 2.

Fig. 2.
figure 2

Hair removal by the DullRazor technique: (a) original image, (b) image after removal of hair

Full size image

3.2 Analysis of Asymmetry

Asymmetry is an essential parameter in differentiating malignant tumours from benign lesions. It is generally evaluated by dermatologists through observation by comparing the two halves of the lesion according to the principal axis. The usage method proposed in [15] calculates the index of symmetry by the differences between the areas defined with main axes and areas defined with main axes rotated by the 180 axes, compared to the centre of gravity of the skin tumour [16].

Therefore, it can be concluded that asymmetry is a quantifiable property and that parameter can be used for discriminating and characterizing the melanomas. The main axes is defined by classical technique PCA and one of the most common methods used to achieve reduction without losing too much information. The central symmetry can be determined by a rotation of 180° around the centre of gravity which is presented in Fig. 3. The axial symmetry around the principal and the secondary axis of inertia are considered. It is focused on two formulations: minimizing the residual sum of squares, and maximizing the variance captured which is in detail discussed in [17].

Fig. 3.
figure 3

Calculating the symmetry following the two principal axes: (a) image after filtering, (b) binary mask, (c) detection axis of inertia

Full size image

3.3 Analysis of Edges

Analysis of Edges (Border) is measured by circumscribed convex polygon around the contours of skin tumour and skin tumour comparisons surface contours and polygons. Circumscribed convex polygon is designed usage of convex hull algorithm. The completely convex hull algorithm is in detail described in [18]. The segmentation of an image into a complex of edges is a useful prerequisite for object identification. However, although many low-level processing methods can be applied for this purpose, the problem is to decide which object boundary each pixel in an image falls within and which high level constraints are necessary (Fig. 4).

Fig. 4.
figure 4

Extraction Border from skin tumours images: (a) benign, (b) malignant [15]

Full size image

3.4 Analysis of Colours

The pigmentation of skin tumours can be characterized by several colours – five to six colours may be present in a malignant lesion. The skin tumour Colour – Analysis of Colours – is measured by mean of standard deviation for all three red-green-blue (RGB) channels on pixels which are included in skin tumour images. Description of applied algorithm is summarized by the pseudo code and is shown in Algorithm 3.

figure b

The last equation computes the entire image mean of variance for all three (RGB) channels.

3.5 Classification – Support Vector Machine

Support Vector Machine (SVM) [19] is mathematical computational unit just like neural network that constructs hyperplanes defining decision boundary of classification. Main intuition behind contusions of SVM is to maximize separation between classified labels. SVM is applicable for binary classification and multi-class classification. Original SVM was stated for linear classification but non-linear classification can be obtained by using kernel function. Kernel function maps low dimension data into high dimensional space where a linear separation of data is possible. SVM derived from statistical learning and uses supervised learning model for training. On solving it gets converted into quadratic programming problem solving quadratic equation with linear constraints. There are infinite numbers of possible decision boundaries but SVM selects the best decision boundary. Associated hyperplane is mathematically formulated by following:

$$ \begin{aligned} X_{i} * w + b \ge 1\;\;\;\;if\;\,y_{i} = + 1 \hfill \\ X_{i} * w + b \le 1\;\;\;\;if\,\;y_{i} = - 1 \hfill \\ \end{aligned} $$

where, xi = feature vector, yi = class label, w = weights, b = Threshold/bias value, d+ = the shortest distance to the closest positive point, d− = the shortest distance to the closest negative point. H1 and H2 are “supporting hyperplane”. The points on the planes H1 and H2 are “support vectors”. In order to get optimum results, the margin has to be maximized. Thus, the optimization problem ψ can be defined by: ψ = min (1/2 wT w); subjected to yi (wT xi + b) ≥ 1, i = 1, 2, … m (m - total number of training instances). The quadratic optimization problem of SVM is solved using Sequential Minimal Optimization algorithm, in which two parametric values are solved at an instant keeping rest constant. The values are updated at every iteration till the convergence is reached.

3.6 Classification – k-Nearest Neighbors

One of the most straightforward instance-based learning algorithms is the nearest neighbor algorithm – k-Nearest Neighbors (k–NN)). The k–NN is based on the principle that the instances within a dataset will generally exist in close proximity to other instances that have similar properties [20]. If the instances are tagged with a classification label, then the value of the label of an unclassified instance can be determined by observing the class of its nearest neighbors. The k-NN locates the k nearest instances to the query instance and determines its class by identifying the single most frequent class label.

figure c

Description of the algorithm summarized by the pseudo code is shown in Algorithm 3. The training phase for k-NN consists of simply storing all known instances and their class labels. A tabular representation can be used, or a specialized structure such as a kd-tree. When it is necessary to tune the value of k and/or perform feature selection, n-fold cross-validation can be used on the training dataset. The testing phase for a new instance t, is given a known set I.

3.7 Classification – Artificial Neural Network - Multilayer Perceptron

A Multilayer perceptron can be viewed as a logistic regression classifier where the input is first transformed using a learnt non-linear transformation \( {\mathbf{\upvarphi }} \). A Multilayer perceptron (MLP) (or Artificial Neural Network – ANN-MLP) with a single hidden layer can be represented graphically as in Fig. 5.

Fig. 5.
figure 5

Artificial Neural Network – Multilayer perceptron

Full size image

Formally, a one-hidden-layer MLP is a function ƒ: ℜD → ℜL, where D is the size of input vector x and L is the size of the output vector ƒ(x), such that, in matrix notation:

$$ f(x) = G(b^{(2)} + W^{(2)} (s(b^{(1)} + W^{(1)} x))) $$

with bias vectors b(1), b(2); weight matrices W(1), W(2) and activation functions G and s.

The vector h(x) = \( {\mathbf{\upvarphi }} \)(x) = s(b(1) + W(1) x) constitutes the hidden layer. W(1) ∈ ℜD x Dh is the weight matrix connecting the input vector to the hidden layer. Each column W.i (1) represents the weights from the input units to the i-th hidden unit. Typical choices for s include Rectified Linear Unit (ReLU), with f(x) = 0 for x < 0; f(x) = 1 for x ≥ 0; and f’(x) = 0 for x < 0; f’(x) = 1 for x ≥ 0. In this research this function is used, because it typically yields to faster training and sometimes also to better define local minima. The output vector is then obtained as: o(x) = G(b(2) + W(2) h(x)).

4 Experimental Results and Discussion

The proposed hybrid automatic classification system for skin tumour images which combines following techniques: DullRazor technique, PCA, convex hull algorithm, and classifiers SVM, k–NN, and ANN-MLP was further on, in our research, tested on tumour images from well-known repository HAM10000 Data Set [21]. HAM10000 Data Set is collection of 10015 dermatoscopic images from different populations, acquired and stored by different modalities.

4.1 Data Set

This research puts stress on recognition of the first three characteristics of an ABCD(E) method: Asymmetry, Border and Colour (ABC), considering that this data set did not contain clinical dimensions of moles on the images, nor any other data that could help determine the size of the mole, it was not possible to follow the evolution procedure. From content of repository HAM10000 Data Set which consists of 10015 dermatoscopic images, the first images that are eliminated are images with vignette, because vignette makes skin image very asymmetric, and it is difficult to recognize differences between skin tumour and vignette. Then, according HAM10000 metadata, approximately the same number of benign and malignant skin tumours, 971 cases of benign skin tumour (Positives) and 1163 cases of malignant skin tumours (Negatives) are selected, which means total of 2134 skin tumour images are used in this research.

4.2 Performance Evaluation

The performances of the proposed techniques are evaluated for the skin tumour images. The proposed techniques performance evaluated in terms of sensitivity, specificity, precision, recall, and accuracy are defined in the following way:

  1. 1.

    Sensitivity = Recall (true positive fraction): Sensitivity = TP/(TP + FN)

  2. 2.

    Specificity (true negative fraction): Specificity = TN/(TN + FP)

  3. 3.

    Precision (false predictive value): the result indicates a good measure to determine the costs of False Positive is high: Precision = TP/(TP + FP)

  4. 4.

    Accuracy: the probability that the diagnostic test is performed correctly:

    $$ Accuracy = \;\left( {{\text{TP}}\; + \;{\text{TN}}} \right)/\left( {{\text{TP}}\; + \;{\text{TN}}\; + \;{\text{FP}}\; + \;{\text{FN}}} \right) $$
  5. 5.

    F1 Score: the success rate of classification:

    $$ F1\;Score\; = \; 2\;*\;{\text{Precision}}\;*\;{\text{Recall}}/\left( {{\text{Precision}}\; + \;{\text{Recall}}} \right) $$

Where, TP (True Positives) – Correctly classified positive cases; TN (True Negative) – Correctly classified negative cases; FP (False Positives) – Incorrectly classified negative cases; FN (False Negative) – Incorrectly classified positive cases.

4.3 Experimental Results

Training the model and classifying skin tumours in categories is based on three characteristics – (a) Symmetry, (b) Border regularity, (c) Colour deviation mean; which are measured and normalized according to the description of the data themselves. The experimental results are presented in Table 1. Three different artificial techniques are used for classification: (a) SVM, (b) k-NN and (c) ANN–MLP classifiers.

Table 1. Experimental results for Symmetry, Border regularity, Colour deviation mean for Positives and Negatives skin tumours
Full size table

The data set is divided in two classes, 70% for training and 30% for testing classifiers. Therefore, there are 1493 training instances and 641 testing instances. Every three-classifier experiment is repeated one hundred (100) times.

Experimental clinical diagnostics performance Sensitivity, Specificity, Precision, Accuracy, and F1 Score for all three classifiers SVM, k-NN, and ANN-MLP are presented in Table 2. The best performance for Specificity, Precision, Accuracy, and F1 Score has SVM classifier, but it must be noticed that ANN-MLP classifier has near close performance with SVM classifier. Maybe some correction in hidden layer can be significantly improved in ANN-MLP performance. The experimental results for different skin images used by SVM classifier are presented in Table 3.

Table 2. Compare clinical diagnostics performance for three classifiers
Full size table
Table 3. Comparison of the original type defined in HAM10000 metadata of skin tumours and predicted type with Support Vector Machine classifier (T – True; F – False)
Full size table

4.4 Discussion

These experimental results could be compared with some other skin tumour images analysis. In research presented in [3] Sensitivity is 67.5%, Specificity is 80.5%, correct classifications Accuracy is 74%, but another data set which consists of 180 skin tumour images is used there. In our research best experimental results for Sensitivity is 78%, Specificity is 93.8%, Precision is 85.5%, Accuracy is 84.8%, and F1 Score is 79.6%. All performances in our research are better than in the mentioned research. But, on the other side, paper [1] presents system which got 100% for Sensitivity, 95% for Specificity, and 97.5% for Accuracy, but the data set used there consists of only 40 images (20 normal and 20 abnormal). The number of skin images in our research is 2134 which is at least ten times more in comparison to the other research.

5 Conclusion and Future Work

The aim of this paper is to propose the new hybrid strategy for automatic classification of skin tumour images. First, the algorithms are employed for the extraction of the set of features based on ABC dermatology criteria – Asymmetry, Border, and Colour. Different mathematical and artificial intelligence techniques are combined: DullRazor technique, PCA, convex hull algorithm, and three classifiers SVM, k–NN, and ANN-MLP. The system is tested on 2143 selected tumour images from repository HAM10000 Data Set which consists of 10015 dermatoscopic images.

Preliminary experimental results encourage the further research by the authors because the proposed strategy on experimental data set has: Sensitivity of 78%, Specificity of 93.8%, Precision of 85.5%, Accuracy of 84.8%, and F1 Score of 79.6%. But the improvements are possible in many aspects. Our future research will focus on: (1) improving segmentation and preprocessing, as well as analyzing the system that could lead to improving the robustness of the solution (2) certain corrections in ANN-MLP classifier which will improve its performance; (3) broadening the data set with new classes which exist in the meta data set (4) creating new hybrid model combined with novel artificial intelligence techniques which will efficiently solve real-world skin tumours data sets.