Understanding Clouds from Satellite Images比赛的discussion调研与colab数据集下载配置
?
colab數據集下載配置代碼：
%%time !pip install -U -q kaggle !mkdir -p ~/.kaggle!echo '{"username":"pupil1","key":"ae776d041bf94ae1bfa9a3843797ad6d"}' > ~/.kaggle/kaggle.json!chmod 600 ~/.kaggle/kaggle.json !mkdir -p understanding_cloud_organization !kaggle competitions download -c understanding_cloud_organization !mv *.zip understanding_cloud_organization/ !mv *.csv understanding_cloud_organization/ !cd /content/understanding_cloud_organization/;unzip train_images.zip !cd /content/understanding_cloud_organization;mkdir train_images;mv *.jpg train_images/ !cd /content/understanding_cloud_organization/;unzip train.csv.zip !cd /content/understanding_cloud_organization/;unzip test_images.zip !cd /content/understanding_cloud_organization;mkdir test_images;mv *.jpg test_images?
?
?
?
根據[2]的描述
The remaining area, which has not been covered by two succeeding orbits, is marked black.0
所以圖片中如果出現黑色區域，就是兩顆衛星都沒有掃描到的地方。如下：
?
?
使用pupil1賬號視角,凡是變色的都是看過的,實在極其沒有意義的不予收錄.
?
| 鏈接 | 備注 |
| Train with crops, Predict with full images | 發帖子的人得分不高 |
| How effective is pseudo-labeling? | (看完了)半監督 |
| [LB 0.628] simple segmentation approach threshold is high? | threshold的用法 |
| Overlapping Labels in Train Data? ? Can a pixel be considered as multiple classes? | (看完了) 根據第一個鏈接,一個像素可以屬于多個類別. Each image was labeled by several people (2-4), so the labels can overlap. In addition, there was no restriction that the labels from a single labeler cannot overlap. To create the masks for this competition, we simply used the union of all labels for each class. So naturally there will be some overlap. |
| AdamAccumulate | (看完了)提到了AdamAccumulate的版本兼容性問題 |
| Hints for late joiners? | (看完了)提到使用steel比賽的方案 |
| Bounding Boxes instead of Segmentation | (看完了)評論中提到: 舉辦方不鼓勵對象檢測的方式,但是帖子的作者認為線性的模型比非線性的模型跟容易泛化,所以堅持使用Bounding Boxes(對象檢測)的方式 |
| use linknet | unet> linknet > fpn |
| Correct Dice Metric | (看完了)討論誤差函數機制 |
| Instance Segmentation->Request for list of past competition | 參考資料 |
| Information: Bad image list ? Corrupt and Mislabeled Images ? Information: Bad image list | 一些損壞的數據 |
| Question about the black area in the image | 有很好的可視化 |
| ResNet34 implementation of Unet works but ResNet 50 and 101 fails? | (看完了)改變模型如果爆內存就減少batch_size |
| Flowers are easy to pick ? | 介紹了一些樹算法 |
| Single model performance | 最佳單模 |
| A best description of Generating mask from encoded pixel | 涉及encoded pixel |
| Adding TTA to the model before optimisation could help ? Augmentations Strategies for this Competition. TTA? | 使用時間強化 |
| Augmentations thred ? Augmentations released version 0.4.0 | 圖像增強的討論 |
| Questions about the origin of the data | 討論快照功能 |
| More Tricks to Train w/ Bigger Batches (pytorch) ? Some tricks to train faster (pytorch) ? A trick to use bigger batches for training: gradient accumulation | 討論訓練技巧 |
| Simple Descriptions of Cloud Types / Labeling Process | 討論肉眼區分類別 |
| Fast data loading [Experiments] | ?快速讀取數據 |
| Deeper, Stronger, Better? | 發現resnet18有效 resnext50_32x4d和efficientnet-b5無效? |
| Beware of Pandas value_counts method for validation split | 指出幾個代碼的pandas使用有誤 |
| Efficient Net B4-B7 | 評論區提到修補小batch_size的辦法是使用?gradient accumulation |
| Improving code quality with utility scripts ? Utility scripts for Keras users ? Using High-level frameworks is not learner friendly | 代碼推銷 |
| Object Detection vs Instance Segmentation | 很多概念 |
| Hybrid convolutional and bidirectional LSTM or RNN | 使用RNN網絡 |
| EfficientNets are now available in pytorch segmentation model repo. | 沒看懂這個是干嘛的,房之后再看 |
| New method to tackle severe label noise | 處理label噪音的一篇論文 |
| FPN or Unet: Which one is better? | 提到了FPN以及Unet |
| Some thoughts on this competition | kernel grandmaster的一些想法 |
| what is the label to be taken for overlapping masks? for example, in the image 0011165.jpg, Fish and Flower masks overlap each other for some region. | mask重合 |
| Must read material | 一些資料 |
| Ideas for merging ensemble's predictions ? How to effectively ensemble models with Keras | 討論模型融合 |
| Instance Segmentation->How to predict classes | 討論UNET的輸出怎么改成多分類 |
| What does it mean to use a pretrained resnet encoder with UNET? | 討論UNET使用預訓練的resnet編碼器是什么意思? |
| Regular image segmentation approach | 提到進行語義分割任務的都有兩個數據集 |
| Discussing post processing | 討論后處理 |
| Weakly supervised segmentation | 弱監督分割 |
| Must-see Kernels and topics - Understanding Clouds from Satellite Images | 對于資料的自行總結 |
| RLE Decode in C++ | 提到了RLE技術 |
| Hints from a late joiner's persepctive | 提到了后處理 |
| Impact of using classier for removing the masks | 考慮去掉mask編碼 |
| A Late Joiner's Understanding and Notes | 需要細看 |
| LPT: See what's going on with that commit ? | 介紹了一個有用的訓練的可視化工具 |
| Knock Knock can send you email notification (or slack notification) | 一個工具用來提醒你訓練結束的時候發信息到你郵件通知你 |
?
一些統計數據來自[1]:
Useful Stats::
no. of empty mask = 7055
no. of non-empty mask = 7737
no. of non-empty mask for?Fish?= 1864
no. of non-empty mask for?Flower?= 1509
no. of non-empty mask for?Gravel?= 1982
no. of non-empty mask for?Sugar?= 2382
?
Reference:
[1]Public TestSet Distribution via LB probing
[2]https://www.kaggle.com/c/understanding_cloud_organization/data
總結
以上是生活随笔為你收集整理的Understanding Clouds from Satellite Images比赛的discussion调研与colab数据集下载配置的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： kaggle的图像数据集直接下载到goo
- 下一篇： 对mask进行rle编码然后进行解码-详