New Preprint: How Adversarial attacks affect Deep Neural Networks Detecting COVID-19?
In short: I took five networks that classify COVID-19 chest X-rays and attacked them four different ways. Average accuracy fell from 96.7% to as low as 25.5%. After building an online COVID-19…
- published
- read time
- 2 min
- words
- 392
- lang
- en
- filed under
- papersmachine learningDeep learning
In short: I took five networks that classify COVID-19 chest X-rays and attacked them four different ways. Average accuracy fell from 96.7% to as low as 25.5%.
After building an online COVID-19 diagnosing system, I got interested in where machine learning in healthcare breaks. One of the biggest problems is how fragile these models are. So I wrote a preprint about it.
What I tested
Five networks: ResNet-18, ResNet-50, Wide ResNet-16-8 (WRN-16-8), VGG-19, and Inception v3. Four attacks: Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), Carlini and Wagner (C&W), and Spatial Transformations Attack (ST).
What happened
| Attack | Average accuracy |
|---|---|
| None (clean images) | 96.7% |
| FGSM | 41.1% |
| PGD | 25.5% |
| C&W | 50.1% |
| ST | 56.3% |
ResNet-50 and WRN-16-8 held up better than the rest. That makes them the sensible place to spend effort on defenses.
Abstract
Considering the global crisis of Coronavirus infection (COVID-19), the essence of utilizing novel approaches to achieve quick and accurate diagnosing methods is required. Deep Neural Networks (DNN) showed outstanding capabilities in classifying various data types, including medical images, in order to build a practical automatic diagnosing system. Therefore, DNNs can help the healthcare system to reduce patients waiting time. However, despite acceptable accuracy and low false-negative rate of DNNs in medical image classification, they have shown vulnerabilities in terms of adversarial attacks. Such input can lead the model to misclassification. This paper investigated the effect of these attacks on five commonly used neural networks, including ResNet-18, ResNet-50, Wide ResNet-16-8 (WRN-16-8), VGG-19, and Inception v3. Four adversarial attacks, including Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), Carlini and Wagner (C&W), and Spatial Transformations Attack (ST), were used to complete this investigation. Average accuracy on test images was 96.7% and decreased to 41.1%, 25.5%, 50.1%, and 56.3% in FGSM, PGD, C&W, and ST, respectively. Results are indicating that ResNet-50 and WRN-16-8 were generally less affected by attacks. Therefore using defense methods in these two models can enhance their performance encountering adversarial perturbations.
Full report: dx.doi.org/10.21203/rs.3.rs-763355/v1
related