A simple way to get images for your dataset
Whenever we begin a machine learning project, the first thing that we need is a dataset. Dataset will be the pillar of your training model. You can build the dataset either automatically or manually. Here I am going to share about the manual process.
What is the dataset?
Dataset is the collection of specific data for your ML project needs. The type of data depends on the kind of AI you need to train. Basically, you have two datasets:
Whenever you are training a custom model the important thing is images. Yes, of course, the images play a main role in deep learning. The accuracy of your model will be based on the training images. So, before you train a custom model, you need to plan how to get images? Here, I’m going to share my ideas on the easy way to get images for a dataset.
Get images from google
Yes, we can get images from Google. Using the Download All Images browser extension we can easily get images in a few minutes. You can check out here for more details about this extension! It is available on major browsers. Unfortunately, this extension is not available on the Safari browser.
Once you download images using this extension, you will see the downloaded images in a folder with random filenames. We can rename the files or remove the .png file using the below Python script.
Remove PNGs from the downloaded image folder.
import glob import pathlib for file in glob.glob('frames/*.png'): path = pathlib.Path(file) path.unlink()
Rename your image files
import os os.getcwd() collection = "images/cat" for i, filename in enumerate(os.listdir(collection)): print(filename) os.rename(collection + "/" + filename, collection + "/cat" + str(i) + ".jpg")
Get images from videos
Here we have another way to prepare images for the Dataset. We can easily extract images from video files. Detecto gives a simple solution to get images from the video. Refer Detecto for more information.
pip3 install detecto
Using the following code we can extract images from video files.
from detecto.utils import split_video #split_video('video file path', 'image save path', frame size) split_video('images/cat.mp4', 'images/cat/', step_size=10)
Using your set of images
You can take pictures of objects which you will use to train your model. The important note is to make sure your images are not beyond 800×600. This will help your dataset train much quicker.
from PIL import Image import os import argparse def rescale_images(directory, size): for img in os.listdir(directory): im = Image.open(directory+img) im_resized = im.resize(size, Image.ANTIALIAS) im_resized.save(directory+img) if __name__ == '__main__': parser = argparse.ArgumentParser(description="Rescale images") parser.add_argument('-d', '--directory', type=str, required=True, help='Directory containing the images') parser.add_argument('-s', '--size', type=int, nargs=2, required=True, metavar=('width', 'height'), help='Image size') args = parser.parse_args() rescale_images(args.directory, args.size) ##run this command ##python convert_image_resolution.py -d images/ -s 800 600
I have prepared a video and explained about the above process. Please check out the below video blog.
As an ML noob, I need to figure out the best way to prepare the dataset for training a model. I hope this will be useful. My ultimate idea is to create a Python package for this process. 🙂
See you in my next article 🙂