Skip to content
This repository has been archived by the owner on Oct 3, 2024. It is now read-only.

Commit

Permalink
edit readme, plot names (#70)
Browse files Browse the repository at this point in the history
  • Loading branch information
@nkrusch
nkrusch authored Nov 24, 2023
1 parent 0c14bd9 commit 764c8b1
Show file tree
Hide file tree
Showing 6 changed files with 99 additions and 5 deletions.
8 changes: 4 additions & 4 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,15 @@ Check your environment using the following command, and install/upgrade as neces
python3 --version & make --version
```

### Reproducing experiments

Install dependencies
**Install dependencies**

```
pip install -r requirements.txt --user
```

![duration](https://img.shields.io/badge/%F0%9F%95%92%2024%E2%80%9448%20hours/each-FFFF00?style=flat-square) **Run attacks evaluations.**
### Reproducing experiments

![duration](https://img.shields.io/badge/%F0%9F%95%92%2024%E2%80%9448%20hours/each-FFFF00?style=flat-square) **Run attack evaluations.**
Run experiments for combinations of data sets x classifiers x attacks (24 experiment cases).

<pre>
Expand Down
2 changes: 1 addition & 1 deletion plot/table.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def fmt(r):
hdr = 'classifier,exp-name,attack,' \
'accuracy,evades,valid'
mat = [fmt(r) for r in self.raw_rata]
return self.make_table('plot', hdr, mat)
return self.make_table('evasion', hdr, mat)

def accuracy_plot(self):
def fmt(m, n):
Expand Down
25 changes: 25 additions & 0 deletions ref_result/__evasion_ref_result_attacks.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
classifier exp-name attack accuracy evades valid
Neural Network IoT-23 CPGD 97.7 50.2 0.0
Neural Network IoT-23 VHSJ 97.7 0.4 0.4
Neural Network IoT-23 VPGD 97.7 50.1 50.1
Neural Network IoT-23 VZoo 97.7 4.7 4.7
Neural Network LCLD CPGD 64.6 36.5 0.0
Neural Network LCLD VHSJ 64.3 19.4 19.4
Neural Network LCLD VPGD 64.2 64.2 64.2
Neural Network LCLD VZoo 64.5 29.0 29.0
Neural Network UNSW-NB15 CPGD 91.7 70.8 36.8
Neural Network UNSW-NB15 VHSJ 91.5 54.4 54.4
Neural Network UNSW-NB15 VPGD 91.3 91.3 91.3
Neural Network UNSW-NB15 VZoo 91.8 83.0 83.0
Neural Network URL CPGD 95.4 57.9 20.9
Neural Network URL VHSJ 95.4 43.5 43.5
Neural Network URL VPGD 95.4 95.4 95.4
Neural Network URL VZoo 95.5 90.0 90.0
XGBoost IoT-23 VHSJ 97.9 22.1 22.1
XGBoost IoT-23 VZoo 97.9 5.9 5.9
XGBoost LCLD VHSJ 65.0 16.0 16.0
XGBoost LCLD VZoo 65.2 21.8 21.8
XGBoost UNSW-NB15 VHSJ 93.7 61.5 61.5
XGBoost UNSW-NB15 VZoo 93.7 74.6 74.6
XGBoost URL VHSJ 96.0 90.1 90.1
XGBoost URL VZoo 96.1 78.3 78.3
25 changes: 25 additions & 0 deletions ref_result/__evasion_ref_result_original.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
classifier exp-name attack accuracy evades valid
Neural Network IoT-23 CPGD 97.7 97.7 12.6
Neural Network IoT-23 HSJ 97.7 50.0 10.0
Neural Network IoT-23 PGD 97.7 50.0 9.8
Neural Network IoT-23 Zoo 97.7 49.3 21.9
Neural Network LCLD CPGD 64.3 40.3 0.0
Neural Network LCLD HSJ 64.2 64.2 0.5
Neural Network LCLD PGD 64.5 64.4 0.0
Neural Network LCLD Zoo 64.7 61.5 33.9
Neural Network UNSW-NB15 CPGD 91.8 78.9 39.4
Neural Network UNSW-NB15 HSJ 91.5 91.5 0.5
Neural Network UNSW-NB15 PGD 91.3 88.9 0.2
Neural Network UNSW-NB15 Zoo 91.6 88.4 75.1
Neural Network URL CPGD 95.5 55.6 19.0
Neural Network URL HSJ 95.4 47.9 33.1
Neural Network URL PGD 95.4 72.7 28.1
Neural Network URL Zoo 95.4 92.4 87.1
XGBoost IoT-23 HSJ 97.9 97.9 12.5
XGBoost IoT-23 Zoo 97.9 96.4 52.2
XGBoost LCLD HSJ 65.2 65.2 0.5
XGBoost LCLD Zoo 65.3 65.3 31.7
XGBoost UNSW-NB15 HSJ 93.8 93.8 0.3
XGBoost UNSW-NB15 Zoo 93.8 81.9 70.5
XGBoost URL HSJ 96.2 96.1 78.3
XGBoost URL Zoo 96.1 95.9 80.3
19 changes: 19 additions & 0 deletions ref_result/__evasion_ref_result_perf_ubuntu.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
classifier exp-name attack accuracy evades valid
Neural Network PT-1-03 PGD 91.0 90.4 80.7
Neural Network PT-1-03 VPGD 91.7 91.7 91.7
Neural Network PT-1-03 CPGD 91.7 71.2 43.7
Neural Network PT-1-06 PGD 91.7 90.8 71.4
Neural Network PT-1-06 VPGD 91.6 91.6 91.6
Neural Network PT-1-06 CPGD 91.6 71.3 29.6
Neural Network PT-1-12 PGD 91.3 90.5 74.5
Neural Network PT-1-12 CPGD 91.7 61.8 3.9
Neural Network PT-1-12 VPGD 91.7 91.7 91.7
Neural Network PT-2-03 PGD 91.6 89.2 89.2
Neural Network PT-2-03 CPGD 91.9 70.9 0.0
Neural Network PT-2-03 VPGD 91.8 91.8 91.8
Neural Network PT-2-06 PGD 91.5 90.2 90.2
Neural Network PT-2-06 CPGD 91.8 69.3 0.0
Neural Network PT-2-06 VPGD 91.7 91.7 91.7
Neural Network PT-2-12 PGD 91.6 91.2 91.2
Neural Network PT-2-12 VPGD 91.5 91.5 91.5
Neural Network PT-2-12 CPGD 91.7 78.8 1.2
25 changes: 25 additions & 0 deletions ref_result/__evasion_ref_result_reset.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
classifier exp-name attack accuracy evades valid
Neural Network IoT-23 CPGD 97.7 50.2 0.0
Neural Network IoT-23 VHSJ 97.7 2.4 2.4
Neural Network IoT-23 VPGD 97.7 12.7 12.7
Neural Network IoT-23 VZoo 97.7 4.2 4.2
Neural Network LCLD CPGD 64.0 43.6 0.0
Neural Network LCLD VHSJ 64.3 15.2 15.2
Neural Network LCLD VPGD 64.1 16.2 16.2
Neural Network LCLD VZoo 64.2 22.5 22.5
Neural Network UNSW-NB15 CPGD 91.3 59.2 26.9
Neural Network UNSW-NB15 VHSJ 91.6 0.3 0.3
Neural Network UNSW-NB15 VPGD 92.0 20.2 20.2
Neural Network UNSW-NB15 VZoo 92.1 73.7 73.7
Neural Network URL CPGD 95.6 63.2 18.0
Neural Network URL VHSJ 95.5 32.7 32.7
Neural Network URL VPGD 95.5 95.5 95.5
Neural Network URL VZoo 95.2 87.2 87.2
XGBoost IoT-23 VHSJ 97.9 21.0 21.0
XGBoost IoT-23 VZoo 97.9 4.6 4.6
XGBoost LCLD VHSJ 65.4 10.1 10.1
XGBoost LCLD VZoo 65.3 20.2 20.2
XGBoost UNSW-NB15 VHSJ 93.6 0.4 0.4
XGBoost UNSW-NB15 VZoo 94.0 70.1 70.1
XGBoost URL VHSJ 96.1 77.7 77.7
XGBoost URL VZoo 96.1 79.6 79.6

0 comments on commit 764c8b1

Please sign in to comment.