This is not new, but it’s a passion project, something I can sink my teeth into. I would like to be able to recognise fish (or other sea animals) by uploading a photo to an app or live feed fish underwater on your phone (in a case). The app will come later, for now, I will develop a dataset and train a model. I am looking for 95% accuracy on the validation test set for this project.
The first thing to do is to locate datasets and organise them. This will also give me an idea of the fish I will be classifying.
Datasets I have found are;
- From QUT – https://wiki.qut.edu.au/display/cyphy/Fish+Dataset
- From Kaggle competition – https://www.kaggle.com/c/the-nature-conservancy-fisheries-monitoring/data
- Fish Recognition ground truth Data – http://groups.inf.ed.ac.uk/f4k/GROUNDTRUTH/RECOG/
- Google Images 😉 – https://google.com
That should do for now.
I downloaded and extracted all the datasets, each with their own classification methods. Let’s take a look at these datasets.
- 4415 Images / 468 Species – Cropped image set 4415.
- 3778 Images / 6 Species (+ Unknown & No Fish) – Unlabeled test images 13153.
- 27370 Images / 23 Species – Image masks for all images 27370.
So some are cropped and ready, others are in a boat or harder to recognise images, and others have image masks. I need to create the classes, or species of fish and put them in named folders. Let’s see what we already have classified and make folders for everything ready to .zip.
Removing all of the unwanted images gives me a total of 30,207 images in 28 Classes. Some will have more than others (the last five classes contain fewer images), but it is a good start.
I will make a Data Set with 8 categories and 200 images per category for starters. All the spare images can go into an unfiltered folder for using to make a bigger dataset later. FOr not the 200 images will be augmented to get me 1000 for each category.
|Dascyllus reticulatus||Head-band / Reticulated Humbug|
|Plectroglyphidodon dickii||Blackbar Devil / Dick’s Damsel|
|Chromis chrysura||Robust Puller / Stout-body Chromis|
|Amphiprion clarkii||Yellowtail Clownfish|
|Chaetodon lunulatus||Oval butterflyfish|
|Myripristis kuntee||Epaulette Squirrelfish / Kuntee Soldierfish|
|Neoniphon sammara||Armed Squirrel-fish / Blood-spot Squirrel-fish|
|Coryphaena Hippurus||Dolphin Fish / Mahi Mahi|
BUILD SOME MODELS
I built three models. The code divides so the same format can be used for datasets, being a .zip of (main_folder / classes / 1.jpg etc).
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 100, 100, 16) 448 _________________________________________________________________ max_pooling2d (MaxPooling2D) (None, 50, 50, 16) 0 _________________________________________________________________ conv2d_1 (Conv2D) (None, 50, 50, 32) 4640 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 25, 25, 32) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 25, 25, 64) 18496 _________________________________________________________________ max_pooling2d_2 (MaxPooling2 (None, 12, 12, 64) 0 _________________________________________________________________ flatten (Flatten) (None, 9216) 0 _________________________________________________________________ dropout (Dropout) (None, 9216) 0 _________________________________________________________________ dense (Dense) (None, 512) 4719104 _________________________________________________________________ dropout_1 (Dropout) (None, 512) 0 _________________________________________________________________ dense_1 (Dense) (None, 9) 4617 ================================================================= Total params: 4,747,305 Trainable params: 4,747,305 Non-trainable params: 0 _________________________________________________________________
This used the same Model as above with the addition of image augmentation.
This model uses the above model with a MobileNET V2 layer that is not trainable.
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= keras_layer (KerasLayer) (None, 1280) 2257984 _________________________________________________________________ dense (Dense) (None, 8) 10248 ================================================================= Total params: 2,268,232 Trainable params: 10,248 Non-trainable params: 2,257,984 _________________________________________________________________
NOTE: I have made a fourth model using ResNet that outperformed all the above models. I also configured my GPU to speed up training over 200 epochs. I think it was around 97% training and 94% validation. You can check it out, It is on GitHub. I have started working on the saved model and exporting to TensorFlow lite.
FOR FUTURE MODELS
I am sure a convolutional neural network is a good start, but let’s take a look at some papers and examples of fish classification using deep learning.
- RCNN – Region-Based CNN. This will help with images that have extra elements.
- Fish Classification with CNN – https://arxiv.org/ftp/arxiv/papers/1805/1805.10106.pdf
- Contour Mapping Fish – http://www.timeseriesclassification.com/description.php?Dataset=Fish
- Image Location – The location of the image as a feature will help classify possible species, but it should have too much weight as incorrect locations could throw things out.
- Fish Classification from Video – https://www.researchgate.net/publication/317558591_Automatic_fish_species_classification_in_underwater_videos_Exploiting_pretrained_deep_neural_network_models_to_compensate_for_limited_labelled_data
Now I have an idea of what approaches to take for future model training. The next instalment of this project will take the above into account.