|Rosella Machine Intelligence & Data Mining|
| Home | DataMining & Machine Learning | Products & Downloads | Site Map | Contact |
<!- menu column -->
Skin Cancer Detection using Machine Learning - Deep Learning Approach
Skin cancer can be detected through machine learning techniques using deep learning algorithms with very high accuracy. However it is not possible to detect 100% accurately. There are a number of issues with machine learning methods. This is explained in the section Limitations.
Skin Cancer Detection Method
Skin cancer is "the abnormal growth of abnormal cells" in the outer skin layer. When you have malignant skin cancer, you will notice some of the following conditions in your skin lesions, moles, spots;
If some of these conditions occur in your skin lesions and conditions do not improve, it's safer to see a doctor or dermatologist for diagnosis and treatment before too late.
Skin Cancer Checker Machine Learning - Deep Learning Method
In combination with the above conditions, skin lesion photo images can be used to detect skin cancer using artificial intelligence machine learning techniques, specifically deep learning convolutional neural networks. To succeed machine learning, the followings should be noted;
It is noted that neural network training can take months of computation on high performance computer. Just executing Python code with a bunch of images won't produce good models. Systematic training data creation and methodological model training is needed. You need powerful dynamic tools with which you can guide training process like driving a car with dashboard information and handles. For example, CMSR Machine Learning Studio.
Luck is very important part of neural network training as it relies on randomizations: random initialization of network weights and random shuffling of training data. Exceptionally good models can come with exceptional luck! Try multiple times to get that luck.
About one million augmented images are needed for high accuracy on unseen images. But training such large data will be very time consuming. Efficient network configuration is needed.
Dealing with Neural Network Divergence
Neural network training is a very tricky business. Neural network divergence is a common problem. There are two network divergence cases. One is Not a Number (NaN). When this number is detected, network is dead. You will need to reinitialize network or abandon it. Another is that network accuracy drops suddenly and then unable to recover. Too high error correction ratio is the main reason of neural network divergence. By controlling error correction ratio, network divergence can be avoided.
CMSR Machine Learning Studio keeps best versions. When divergence happens, you can return back to the best version which does not have divergence and keep training with lower error correction ratio. In this way, divergence can be overcome. The following figure shows that accuracy drops to the bottom. After returning to the best previous version with smaller error correction ratio, the network recovers.
Software testing revealed the following three main sources of errors;
The first problem is for users to improve photo quality. The other two problems have no clear solutions at the moment. In these cases, use this system with these in mind.
[False negatives] Certain skin cancer looks similar to pimples or skin rash. In this case, this system may classify as low risk. Note that pimples and skin rash improve or disappear in days. If conditions don't improve in days, see a doctor or dermatologist.
[False positives] Some bad skin infections look similar to skin cancer. This system may make mistakes. If conditions don't improve, you may need to see a doctor for treatment.
Note that this doesn't work well with dark skin. This is a limitation of this approach.
Our Reference System and Accuracy
In our reference system Skin Cancer Checker, four neural network models were developed using very powerful machine learning modeling system CMSR Machine Learning Studio. Four deep convolutional neural network models, employing advanced proprietary algorithms and trained with 1 million photo images, are used. It has very high accuracy well over 90%. On 1 million training dataset images, 99.79% sensitivity and 99.35% specificity, 99.57% combined accuracy. (Note that sensitivity is the rate of detecting cancer as cancer. Specificity is the rate of detecting non-cancer as non-cancer.)
On training dataset, 90.94% skin cancer images score as 99% risk probability. 97.95% of skin cancer images score as 90% or higher risk probability. Further tests on 1,853 skin cancer photos that were mostly not used in training showed over 97.4% detection rates. With good quality photos, detection rates will be much higher. Errors were mostly with poor or confusing photo quality.
Notice that this is not 100% detection rate! So care should be taken. Accuracy of detecting not-cancer images as not-cancer is high. But less accurate than detecting cancer images. Note that some bad skin infection photos look alike cancer on visual level. So this system may rate as high risk. When conditions don't improve, always check with a doctor!
Our reference system is available for research and testing purpose only. It's not for general public use. If you are a developer or related researcher, try our referrence system Skin Cancer Checker.