BPMN_Detection / README.md
BenjiELCA's picture
Update README.md
311dd05 verified
---
license: mpl-2.0
language:
- en
metrics:
- f1
- accuracy
- recall
- precision
---
---
license: apache-2.0
# BPMN element detection
## Model description
This project aims to detect Business Process Model and Notation (BPMN) elements from hand-drawn diagrams using a machine learning model. The model is trained to recognize various BPMN elements such as tasks, events, gateways, and connectors from images of hand-drawn diagrams.
The dataset contains 15 target labels:
- **AGENT**
* `pool`
* `lane`
- **TASK**
* `task`
* `subProcess`
- **TASK_INFO**
* `dataObject`
* `dataStore`
- **PROCESS_INFO**
* `background`
- **CONDITION**
* `exclusiveGateway`
* `parallelGateway`
* `eventBasedGateway`
- **EVENT**
* `event`
* `messageEvent`
* `timerEvent`
- **FLOW**
* `sequenceFlow`
* `dataAssociation`
* `messageFlow`
## Results per type
It achieves the following results on the evaluation set with objects:
- Labels Precision: 0.97
- Precision: 0.97
- Recall: 0.95
- F1: 0.96
It achieves the following results on the evaluation set with arrows:
- Labels precision: 0.98
- Precision: 0.92
- Recall: 0.93
- F1: 0.92
- Keypoints Accuracy: 0.71
# Results per class
| Class | Precision | Recall | F1 |
|:-----------------:|:---------:|:--------:|:-------:|
| task | 0.9518 | 0.9875 | 0.9693 |
| exclusiveGateway | 0.9548 | 0.9427 | 0.9487 |
| event | 0.9515 | 0.9235 | 0.9373 |
| parallelGateway | 0.9333 | 0.9180 | 0.9256 |
| messageEvent | 0.9291 | 0.9365 | 0.9328 |
| pool | 0.8797 | 0.936 | 0.9070 |
| lane | 0.9178 | 0.67 | 0.7746 |
| dataObject | 0.9333 | 0.9565 | 0.9448 |
| dataStore | 1.0 | 0.64 | 0.7805 |
| eventBasedGateway | 0.7273 | 0.7273 | 0.7273 |
| timerEvent | 0.8571 | 0.75 | 0.8 |
| sequenceFlow | 0.9292 | 0.9605 | 0.9446 |
| dataAssociation | 0.8472 | 0.8095 | 0.8279 |
| messageFlow | 0.8589 | 0.7910 | 0.8235 |
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0176
- train_batch_size: 4
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50
### Example of Training results
| Epoch | Avg Loss | Test Loss | Classifier Loss | Box Reg Loss | Objectness Loss | RPN Box Reg Loss | Precision | Recall | F1 Score |
|:-----:|:--------:|:---------:|:---------------:|:------------:|:---------------:|:----------------:|:---------:|:------:|:--------:|
| 1 | 3.9451 | 2.0591 | 2.4416 | 0.5426 | 0.6502 | 0.3107 | 0.2763 | 0.0393 | 0.0689 |
| 2 | 2.7259 | 1.5387 | 1.6724 | 0.6697 | 0.1868 | 0.1969 | 0.5754 | 0.3358 | 0.4241 |
| 3 | 2.2004 | 1.1307 | 1.3860 | 0.5330 | 0.1216 | 0.1598 | 0.8657 | 0.6841 | 0.7643 |
| 4 | 1.8611 | 1.0110 | 1.1775 | 0.4172 | 0.1099 | 0.1565 | 0.7708 | 0.7790 | 0.7749 |
| 5 | 1.7461 | 0.9593 | 1.1202 | 0.3820 | 0.0971 | 0.1468 | 0.8542 | 0.8046 | 0.8287 |
| 6 | 1.5859 | 0.8956 | 0.9986 | 0.3590 | 0.0872 | 0.1412 | 0.8884 | 0.8002 | 0.8420 |
| 7 | 1.5621 | 0.9073 | 1.0214 | 0.3351 | 0.0776 | 0.1280 | 0.9435 | 0.8034 | 0.8678 |
| 8 | 1.5194 | 0.8695 | 0.9881 | 0.3261 | 0.0738 | 0.1314 | 0.9048 | 0.8246 | 0.8628 |
| 9 | 1.5449 | 0.9014 | 1.0105 | 0.3229 | 0.0769 | 0.1346 | 0.9478 | 0.8046 | 0.8704 |
| 10 | 1.5805 | 0.8134 | 1.0333 | 0.3338 | 0.0703 | 0.1431 | 0.8920 | 0.8920 | 0.8920 |
| 11 | 1.3838 | 0.8097 | 0.8743 | 0.3065 | 0.0653 | 0.1376 | 0.9634 | 0.8371 | 0.8958 |
| 12 | 1.3582 | 0.7362 | 0.8751 | 0.2909 | 0.0617 | 0.1306 | 0.9457 | 0.8596 | 0.9006 |
| 13 | 1.3126 | 0.7149 | 0.8347 | 0.2921 | 0.0593 | 0.1264 | 0.9152 | 0.9295 | 0.9223 |
| 14 | 1.3532 | 0.7775 | 0.9079 | 0.2783 | 0.0543 | 0.1128 | 0.9639 | 0.8508 | 0.9038 |
| 15 | 1.3188 | 0.6738 | 0.8986 | 0.2720 | 0.0434 | 0.1048 | 0.8856 | 0.9419 | 0.9129 |
| 16 | 1.2512 | 0.7478 | 0.7840 | 0.2784 | 0.0621 | 0.1268 | 0.9181 | 0.9101 | 0.9141 |
| 17 | 1.2909 | 0.6556 | 0.8425 | 0.2778 | 0.0547 | 0.1159 | 0.9012 | 0.9282 | 0.9145 |
| 18 | 1.2526 | 0.7003 | 0.8442 | 0.2607 | 0.0443 | 0.1034 | 0.9169 | 0.9020 | 0.9094 |
| 19 | 1.1980 | 0.7136 | 0.8062 | 0.2528 | 0.0361 | 0.1029 | 0.9520 | 0.9157 | 0.9335 |
| 20 | 1.1821 | 0.6308 | 0.7895 | 0.2517 | 0.0378 | 0.1030 | 0.9023 | 0.9513 | 0.9262 |
| 21 | 1.0843 | 0.6883 | 0.7168 | 0.2402 | 0.0316 | 0.0957 | 0.9348 | 0.9032 | 0.9187 |
| 22 | 1.1058 | 0.6192 | 0.7367 | 0.2336 | 0.0374 | 0.0981 | 0.9321 | 0.9513 | 0.9416 |
| 23 | 1.0699 | 0.5962 | 0.7119 | 0.2340 | 0.0306 | 0.0935 | 0.9353 | 0.9476 | 0.9414 |
| 24 | 1.0616 | 0.6674 | 0.7031 | 0.2367 | 0.0311 | 0.0908 | 0.9418 | 0.9301 | 0.9359 |
| 25 | 1.0784 | 0.6158 | 0.7275 | 0.2311 | 0.0295 | 0.0904 | 0.9176 | 0.9320 | 0.9247 |
| 26 | 1.0618 | 0.6483 | 0.7121 | 0.2283 | 0.0297 | 0.0916 | 0.9411 | 0.9182 | 0.9295 |
| 27 | 1.0530 | 0.5958 | 0.7139 | 0.2236 | 0.0279 | 0.0876 | 0.9477 | 0.9395 | 0.9436 |
| 28 | 1.0452 | 0.5964 | 0.7097 | 0.2223 | 0.0283 | 0.0849 | 0.9465 | 0.9494 | 0.9480 |
| 29 | 1.0966 | 0.6288 | 0.7795 | 0.2176 | 0.0203 | 0.0792 | 0.9558 | 0.9320 | 0.9437 |
| 30 | 1.0506 | 0.5956 | 0.7312 | 0.2142 | 0.0195 | 0.0856 | 0.9370 | 0.9370 | 0.9370 |
| 31 | 1.0030 | 0.6099 | 0.6777 | 0.2163 | 0.0204 | 0.0886 | 0.9506 | 0.9251 | 0.9377 |
| 32 | 0.9748 | 0.5976 | 0.6610 | 0.2098 | 0.0201 | 0.0839 | 0.9527 | 0.9313 | 0.9419 |
| 33 | 0.9540 | 0.5907 | 0.6402 | 0.2059 | 0.0216 | 0.0863 | 0.9536 | 0.9238 | 0.9385 |
| 34 | 0.9730 | 0.5809 | 0.6500 | 0.2076 | 0.0281 | 0.0873 | 0.9407 | 0.9413 | 0.9410 |
| 35 | 0.9894 | 0.5837 | 0.6831 | 0.2066 | 0.0202 | 0.0794 | 0.9451 | 0.9345 | 0.9397 |
| 36 | 0.9042 | 0.5534 | 0.5873 | 0.2096 | 0.0214 | 0.0860 | 0.9460 | 0.9519 | 0.9490 |
| 37 | 0.9546 | 0.5562 | 0.6400 | 0.2112 | 0.0216 | 0.0818 | 0.9260 | 0.9457 | 0.9358 |
| 38 | 0.9806 | 0.5792 | 0.6800 | 0.2031 | 0.0175 | 0.0800 | 0.9476 | 0.9363 | 0.9419 |
| 39 | 0.9294 | 0.5703 | 0.6247 | 0.2016 | 0.0204 | 0.0826 | 0.9401 | 0.9501 | 0.9450 |
| 40 | 0.9786 | 0.5880 | 0.6733 | 0.2010 | 0.0268 | 0.0775 | 0.9375 | 0.9170 | 0.9271 |
| 41 | 1.0026 | 0.5875 | 0.7073 | 0.2033 | 0.0179 | 0.0742 | 0.9476 | 0.9251 | 0.9362 |
| 42 | 0.9567 | 0.5724 | 0.6677 | 0.1992 | 0.0164 | 0.0734 | 0.9468 | 0.9332 | 0.9400 |
| 43 | 0.8747 | 0.5709 | 0.5794 | 0.1980 | 0.0159 | 0.0814 | 0.9557 | 0.9432 | 0.9494 |
| 44 | 1.0310 | 0.5497 | 0.7392 | 0.1956 | 0.0254 | 0.0709 | 0.9589 | 0.9313 | 0.9449 |
| 45 | 0.9526 | 0.5580 | 0.6598 | 0.1982 | 0.0185 | 0.0762 | 0.9401 | 0.9413 | 0.9407 |
| 46 | 0.8753 | 0.5548 | 0.5940 | 0.1939 | 0.0176 | 0.0698 | 0.9468 | 0.9438 | 0.9453 |
| 47 | 0.9328 | 0.5735 | 0.6493 | 0.1953 | 0.0163 | 0.0720 | 0.9534 | 0.9320 | 0.9426 |
| 48 | 0.9019 | 0.5605 | 0.6071 | 0.2002 | 0.0182 | 0.0765 | 0.9496 | 0.9413 | 0.9455 |
| 49 | 0.8335 | 0.5637 | 0.5459 | 0.1918 | 0.0175 | 0.0783 | 0.9588 | 0.9307 | 0.9446 |
| 50 | 0.9043 | 0.5617 | 0.6179 | 0.1933 | 0.0154 | 0.0776 | 0.9597 | 0.9370 | 0.9482 |