#
How to create a good dataset?
In order to train an AI model for the task of object detection and classification, it is crucial to create a high-quality dataset. This guide will walk you through the process of creating a robust dataset that will enable artificial intelligence to accurately identify and categorize objects in images and videos.
#
1. Define your goals
Before you start collecting data, clearly define your goals. Determine the types of objects you want to detect and/or classify. Consider the intended use and potential challenges.
#
2. Select data sources
Identify and collect data sources. These can be publicly available datasets such as Kaggle and Open Images Dataset, online image repositories such as Unsplash, our marketplace, or simply your own images taken with a camera or webcam. Make sure that the data is relevant to your task and consider ethical/privacy issues. Anonymize sensitive information in images, obtain the necessary permissions to use the data, and be transparent about the origins of the dataset. Comply with privacy and consent regulations.
#
3. Add detection labels and/or assign classification categories
In the image editor, draw bounding boxes to indicate the location of objects and/or assign classification categories to define the object depicted in the image. Use the Automatic mode
to draw one type of detection label continuously and/or use the Edit many
in dataset details to select multiple images and assign them to a category in a bulk. To make your work easier, use keyboard and mouse shortcuts or the sharing feature, which allows you to work together with other users on the same dataset.
To learn more about detection labels and classification categories, read the Detection and Classification articles.
#
4. Ensure diversity
Creating a diverse dataset is crucial for reliable model performance. To achieve this, consider a wide range of object variations, such as different sizes, angles, lighting, and background conditions. In addition, similar objects should be placed in various positions around the target object to improve the model's generalization capabilities. It is important to change the position of the target object frequently during data collection to prevent the model from associating the object only with certain positions. To further diversify the dataset, consider covering up to 20% of an object, adjusting zoom levels, and varying object sizes, keeping in mind that smaller objects may require a closer camera distance.
For classification, each image in the dataset should contain only one object.
Keep in mind: Be careful when including an excessive number of similar images, as this may lead to overfitting, where the model becomes too specific to the training set and struggles to generalize effectively to new data.
#
Examples
Suppose you want to create three different datasets for object detection. One with different types of knots, another with compasses and chocolate coins, and the third with playing card symbols. Each of these datasets is different and requires taking various pictures and considering a variety of issues, which we will explain below.
Knots
For this dataset, you should:
- use different colors of rope,
- create knots in different sizes,
- place the knots in different positions on many different backgrounds such as desk, floor, wall, etc.,
- hold knots in hands in different positions on different backgrounds, you can cover up to 20% of the knot,
- place different objects in different positions around the knots, these objects can be watch, tape, scissors, or other things you have at hand,
- use different lighting, you can increase or decrease the brightness, and use different colors, such as white or yellow.
Compasses and chocolate coins
For this dataset, you should:
- place the compasses and chocolate coins in different positions on many different backgrounds,
- hold the compasses and chocolate coins in hands in different positions on different backgrounds,
- place different objects in different positions around the compasses and chocolate coins, these objects should be round in shape to resemble the target objects,
- zoom in and out of the camera from the compasses and chocolate coins,
- use different lighting, but be careful with the brightness as chocolate coins reflect light.
Playing card symbols
For this dataset, you should:
- place the cards in different positions on many different backgrounds,
- hold the cards in hands in different positions on different backgrounds,
- zoom in and out of the camera from the cards, but be careful because zooming out too much may cause the model not to recognize the symbols,
- use different lighting, but be careful with the brightness as cards reflect light.
#
5. Extend your data
If you do not have enough images, or if you need more images of different sizes, angles, brightness and contrast, you can use the Dataset manipulator
to extend your data by generating multiple images in the cloud based on your input dataset.
Note that using the manipulator is not mandatory. If the dataset already contains poor quality images, the manipulator may exacerbate these issues.
To learn more, read the Dataset manipulator article.
#
6. Data quality assurance
Thoroughly review and clean up your dataset. Remove duplicate and blurred images, and handle missing or inaccurate annotations. Regularly perform quality checks to maintain dataset integrity.
#
Conclusion
Creating a high-quality dataset for object detection and/or classification is the basis for developing a successful AI model. By following the steps outlined in this guide, you can ensure that your dataset is diverse, well annotated, and provides the basis for accurate and reliable object detection and classification systems.