sgiordano commited on
Commit
64c8ed4
1 Parent(s): f7e57fb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -24
README.md CHANGED
@@ -91,12 +91,12 @@ pipeline_tag: image-segmentation
91
  <br>
92
 
93
  <div style="border:1px solid black; padding:25px; background-color:#FDFFF4 ; padding-top:10px; padding-bottom:1px;">
94
- <h1>FLAIR-INC_rgbie_15cl_resnet34-unet</h1>
95
- <p>The general characteristics of this specific model <strong>FLAIR-INC_rgbie_15cl_resnet34-unet</strong> are :</p>
96
  <ul style="list-style-type:disc;">
97
  <li>Trained with the FLAIR-INC dataset</li>
98
- <li>RGBIE images (true colours + infrared + elevation)</li>
99
- <li>U-Net with a Resnet-34 encoder</li>
100
  <li>15 class nomenclature : [building, pervious surface, impervious surface, bare soil, water, coniferous, deciduous, brushwood, vineyard, herbaceous, agricultural land, plowed land, swimming pool, snow, greenhouse]</li>
101
  </ul>
102
  </div>
@@ -119,18 +119,13 @@ The product called ([BD ORTHO®](https://geoservices.ign.fr/bdortho)) has its ow
119
  Consequently, the model’s prediction would improve if the user images are similar to the original ones.
120
 
121
  _**Radiometry of input images**_ :
122
- The BD ORTHO input images are distributed in 8-bit encoding format per channel. When traning the model, input normalization was performed (see section **Trainingg Details**).
123
  It is recommended that the user apply the same type of input normalization while inferring the model.
124
 
125
  _**Multi-domain model**_ :
126
  The FLAIR-INC dataset that was used for training is composed of 75 radiometric domains. In the case of aerial images, domain shifts are frequent and are mainly due to : the date of acquisition of the aerial survey (from april to november), the spatial domain (equivalent to a french department administrative division) and downstream radiometric processing.
127
  By construction (sampling 75 domains) the model is robust to these shifts, and can be applied to any images of the ([BD ORTHO® product](https://geoservices.ign.fr/bdortho)).
128
 
129
- _**Specification for the Elevation channel**_ :
130
- The fifth dimension of the RGBIE images is the Elevation (height of building and vegetation). This information is encoded in a 8-bit encoding format.
131
- When decoded to [0,255] ints, a difference of 1 should coresponds to 0.2 meters step of elevation difference.
132
-
133
-
134
  _**Land Cover classes of prediction**_ :
135
  The orginial class nomenclature of the FLAIR Dataset encompasses 19 classes (See the [FLAIR dataset](https://huggingface.co/datasets/IGNF/FLAIR) page for details).
136
  However 3 classes corresponding to uncertain labelisation (Mixed (16), Ligneous (17) and Other (19)) and 1 class with very poor labelling (Clear cut (15)) were desactivated during training.
@@ -141,15 +136,15 @@ As a result, the logits produced by the model are of size 19x1, but classes n°
141
  ## Bias, Risks, Limitations and Recommendations
142
 
143
  _**Using the model on input images with other spatial resolution**_ :
144
- The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained with fixed scale conditions. All patches used for training are derived from aerial images with 0.2 meters spatial resolution. Only flip and rotate augmentations were performed during the training process.
145
  No data augmentation method concerning scale change was used during training. The user should pay attention that generalization issues can occur while applying this model to images that have different spatial resolutions.
146
 
147
  _**Using the model for other remote sensing sensors**_ :
148
- The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained with aerial images of the ([BD ORTHO® product](https://geoservices.ign.fr/bdortho)) that encopass very specific radiometric image processing.
149
  Using the model on other type of aerial images or satellite images may imply the use of transfer learning or domain adaptation techniques.
150
 
151
  _**Using the model on other spatial areas**_ :
152
- The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained on patches reprensenting the French Metropolitan territory.
153
  The user should be aware that applying the model to other type of landscapes may imply a drop in model metrics.
154
 
155
  ---
@@ -166,7 +161,7 @@ Fine-tuning and prediction tasks are detailed in the README file.
166
 
167
  ### Training Data
168
 
169
- 218 400 patches of 512 x 512 pixels were used to train the **FLAIR-INC_RVBIE_resnet34_unet_15cl_norm** model.
170
  The train/validation split was performed patchwise to obtain a 80% / 20% distribution between train and validation.
171
  Annotation was performed at the _zone_ level (~100 patches per _zone_). Spatial independancy between patches is guaranted as patches from the same _zone_ were assigned to the same set (TRAIN or VALIDATION).
172
  The following number of patches were used for train and validation :
@@ -191,13 +186,11 @@ Statistics of the TRAIN+VALIDATION set :
191
  | Red Channel (R) | 105.08 |52.17 |
192
  | Green Channel (G) | 110.87 |45.38 |
193
  | Blue Channel (B) | 101.82 |44.00 |
194
- | Infrared Channel (I) | 106.38 |39.69 |
195
- | Elevation Channel (E) | 53.26 |79.30 |
196
 
197
 
198
  #### Training Hyperparameters
199
 
200
- * Model architecture: Unet (implementation from the [Segmentation Models Pytorch library](https://segmentation-modelspytorch.readthedocs.io/en/latest/docs/api.html#unet))
201
  * Encoder : Resnet-34 pre-trained with ImageNet
202
  * Augmentation :
203
  * VerticalFlip(p=0.5)
@@ -218,17 +211,17 @@ Statistics of the TRAIN+VALIDATION set :
218
 
219
  #### Speeds, Sizes, Times
220
 
221
- The FLAIR-INC_rgbie_15cl_resnet34-unet model was trained on a HPC/AI resources provided by GENCI-IDRIS (Grant 2022-A0131013803).
222
  16 V100 GPUs were used ( 4 nodes, 4 GPUS per node). With this configuration the approximate learning time is 6 minutes per epoch.
223
 
224
- FLAIR-INC_rgbie_15cl_resnet34-unet was obtained for num_epoch=76 with corresponding val_loss=0.56.
225
 
226
 
227
  <div style="position: relative; text-align: center;">
228
  <p style="margin: 0;">TRAIN loss</p>
229
- <img src="figs/train_loss_FLAIR-INC_RGBIE_resnet34_unet_15cl_norm.png" alt="TRAIN loss" style="width: 60%; display: block; margin: 0 auto;"/>
230
  <p style="margin: 0;">VALIDATION loss</p>
231
- <img src="figs/val_loss_FLAIR-INC_RGBIE_resnet34_unet_15cl_norm.png" alt="VALIDATION loss" style="width: 60%; display: block; margin: 0 auto;"/>
232
  </div>
233
 
234
 
@@ -248,7 +241,7 @@ As a result the _Snow_ class is absent from the TEST set.
248
 
249
  #### Metrics
250
 
251
- With the evaluation protocol, the **FLAIR-INC_RVBIE_resnet34_unet_15cl_norm** have been evaluated to **OA= 76.37%** and **mIoU=58.63%**.
252
  The _snow_ class is discarded from the average metrics.
253
 
254
  The following table give the class-wise metrics :
@@ -286,9 +279,9 @@ The following illustration gives the resulting confusion matrix :
286
 
287
  <div style="position: relative; text-align: center;">
288
  <p style="margin: 0;">Normalized Confusion Matrix (precision)</p>
289
- <img src="figs/FLAIR-INC_RVBIE_resnet34_unet_15cl_norm_cm-precision.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
290
  <p style="margin: 0;">Normalized Confusion Matrix (recall)</p>
291
- <img src="figs/FLAIR-INC_RVBIE_resnet34_unet_15cl_norm_cm-recall.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
292
  </div>
293
 
294
 
 
91
  <br>
92
 
93
  <div style="border:1px solid black; padding:25px; background-color:#FDFFF4 ; padding-top:10px; padding-bottom:1px;">
94
+ <h1>FLAIR-INC_rgb_15cl_resnet34-fpn</h1>
95
+ <p>The general characteristics of this specific model <strong>FLAIR-INC_rgb_15cl_resnet34-fpn</strong> are :</p>
96
  <ul style="list-style-type:disc;">
97
  <li>Trained with the FLAIR-INC dataset</li>
98
+ <li>RGB images (true colours)</li>
99
+ <li>FPN with a Resnet-34 encoder</li>
100
  <li>15 class nomenclature : [building, pervious surface, impervious surface, bare soil, water, coniferous, deciduous, brushwood, vineyard, herbaceous, agricultural land, plowed land, swimming pool, snow, greenhouse]</li>
101
  </ul>
102
  </div>
 
119
  Consequently, the model’s prediction would improve if the user images are similar to the original ones.
120
 
121
  _**Radiometry of input images**_ :
122
+ The BD ORTHO input images are distributed in 8-bit encoding format per channel. When traning the model, input normalization was performed (see section **Training Details**).
123
  It is recommended that the user apply the same type of input normalization while inferring the model.
124
 
125
  _**Multi-domain model**_ :
126
  The FLAIR-INC dataset that was used for training is composed of 75 radiometric domains. In the case of aerial images, domain shifts are frequent and are mainly due to : the date of acquisition of the aerial survey (from april to november), the spatial domain (equivalent to a french department administrative division) and downstream radiometric processing.
127
  By construction (sampling 75 domains) the model is robust to these shifts, and can be applied to any images of the ([BD ORTHO® product](https://geoservices.ign.fr/bdortho)).
128
 
 
 
 
 
 
129
  _**Land Cover classes of prediction**_ :
130
  The orginial class nomenclature of the FLAIR Dataset encompasses 19 classes (See the [FLAIR dataset](https://huggingface.co/datasets/IGNF/FLAIR) page for details).
131
  However 3 classes corresponding to uncertain labelisation (Mixed (16), Ligneous (17) and Other (19)) and 1 class with very poor labelling (Clear cut (15)) were desactivated during training.
 
136
  ## Bias, Risks, Limitations and Recommendations
137
 
138
  _**Using the model on input images with other spatial resolution**_ :
139
+ The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained with fixed scale conditions. All patches used for training are derived from aerial images with 0.2 meters spatial resolution. Only flip and rotate augmentations were performed during the training process.
140
  No data augmentation method concerning scale change was used during training. The user should pay attention that generalization issues can occur while applying this model to images that have different spatial resolutions.
141
 
142
  _**Using the model for other remote sensing sensors**_ :
143
+ The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained with aerial images of the ([BD ORTHO® product](https://geoservices.ign.fr/bdortho)) that encopass very specific radiometric image processing.
144
  Using the model on other type of aerial images or satellite images may imply the use of transfer learning or domain adaptation techniques.
145
 
146
  _**Using the model on other spatial areas**_ :
147
+ The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained on patches reprensenting the French Metropolitan territory.
148
  The user should be aware that applying the model to other type of landscapes may imply a drop in model metrics.
149
 
150
  ---
 
161
 
162
  ### Training Data
163
 
164
+ 218 400 patches of 512 x 512 pixels were used to train the **FLAIR-INC_rgb_15cl_resnet34-fpn** model.
165
  The train/validation split was performed patchwise to obtain a 80% / 20% distribution between train and validation.
166
  Annotation was performed at the _zone_ level (~100 patches per _zone_). Spatial independancy between patches is guaranted as patches from the same _zone_ were assigned to the same set (TRAIN or VALIDATION).
167
  The following number of patches were used for train and validation :
 
186
  | Red Channel (R) | 105.08 |52.17 |
187
  | Green Channel (G) | 110.87 |45.38 |
188
  | Blue Channel (B) | 101.82 |44.00 |
 
 
189
 
190
 
191
  #### Training Hyperparameters
192
 
193
+ * Model architecture: FPN (implementation from the [Segmentation Models Pytorch library](https://segmentation-modelspytorch.readthedocs.io/en/latest/docs/api.html#unet))
194
  * Encoder : Resnet-34 pre-trained with ImageNet
195
  * Augmentation :
196
  * VerticalFlip(p=0.5)
 
211
 
212
  #### Speeds, Sizes, Times
213
 
214
+ The FLAIR-INC_rgb_15cl_resnet34-fpn model was trained on a HPC/AI resources provided by GENCI-IDRIS (Grant 2022-A0131013803).
215
  16 V100 GPUs were used ( 4 nodes, 4 GPUS per node). With this configuration the approximate learning time is 6 minutes per epoch.
216
 
217
+ FLAIR-INC_rgb_15cl_resnet34-fpn was obtained for num_epoch=76 with corresponding val_loss=0.56.
218
 
219
 
220
  <div style="position: relative; text-align: center;">
221
  <p style="margin: 0;">TRAIN loss</p>
222
+ <img src="FLAIR-INC_rgb_15cl_resnet34-fpn_train-loss.png" alt="TRAIN loss" style="width: 60%; display: block; margin: 0 auto;"/>
223
  <p style="margin: 0;">VALIDATION loss</p>
224
+ <img src="FLAIR-INC_rgb_15cl_resnet34-fpn_val-loss.png" alt="VALIDATION loss" style="width: 60%; display: block; margin: 0 auto;"/>
225
  </div>
226
 
227
 
 
241
 
242
  #### Metrics
243
 
244
+ With the evaluation protocol, the **FLAIR-INC_rgb_15cl_resnet34-fpn** have been evaluated to **OA= 74.712%** and **mIoU=55.332%**.
245
  The _snow_ class is discarded from the average metrics.
246
 
247
  The following table give the class-wise metrics :
 
279
 
280
  <div style="position: relative; text-align: center;">
281
  <p style="margin: 0;">Normalized Confusion Matrix (precision)</p>
282
+ <img src="FLAIR-INC_rgb_15cl_resnet34-fpn_confmat_norm-precision.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
283
  <p style="margin: 0;">Normalized Confusion Matrix (recall)</p>
284
+ <img src="FLAIR-INC_rgb_15cl_resnet34-fpn_confmat_norm-recall.png" alt="drawing" style="width: 70%; display: block; margin: 0 auto;"/>
285
  </div>
286
 
287