Data annotation is the process of adding labels to raw data, such as drawing bounding boxes around cars in photos, tagging names in text or ranking chatbot answers, so a machine learning model can learn from examples. It matters because supervised learning depends on labeled data, and outsourcing lets companies label large volumes faster with trained annotators and quality checks.
Key Takeaways
- Data annotation turns unlabeled text, images, audio, video and 3D sensor data into labeled training examples for supervised machine learning.
- Common task types include image classification, bounding boxes, semantic segmentation, named entity recognition, text classification, video object tracking and 3D point cloud labeling.
- Label quality is usually measured with checks such as inter-annotator agreement (for example Cohen’s kappa) and expert review.
- Outsourcing speeds up large projects, but buyers should check data security, quality processes and how annotators are paid and protected.
- Biased or careless labels produce biased models, so annotation guidelines and diverse data matter as much as volume.
The technique of marking data available in various formats such as text, video, or photos is known as data annotation. Labeled data sets are essential for supervised machine learning so that the machine can interpret the input patterns.
Data must also be carefully annotated using the appropriate tools and methodologies to train the computer vision-based machine learning model in data annotation. A variety of data annotation methods can be used to produce such data sets for such purposes.
Many machine learning and artificial intelligence applications use annotated data through data annotation and text annotation. At the same time, annotation is one of the most time-consuming and labor-intensive parts of a machine learning program. McKinsey has identified data labeling as one of the significant limitations on AI deployment in enterprises, because supervised models need large volumes of human-labeled examples.
Much of the data organizations produce is unstructured. Unstructured data, such as emails, photos, recordings and videos, has no predefined format or fields, so a model cannot learn from it until it is organized and labeled. You must feed information to an algorithm for it to process and give outputs and inferences while creating an AI model.
For supervised learning, the algorithm needs examples in which the correct answer has already been marked. Adding those labels is called data annotation (or data labeling); when the data is text, it is often called text annotation.
How the Data Annotation Process Works
An AI model could use data annotation to determine whether the data it receives is audio, video, text, images, or a mix of forms. The model would then classify the data and carry out its responsibilities based on the functions and parameters.
Data annotation is unavoidable because AI and machine learning models are trained, and often retrained, on examples to improve their accuracy. The technique is most critical in supervised learning, where the model learns the relationship between inputs and the labels humans have attached to them; more high-quality labeled examples generally help the model generalize better.
For example, self-driving car systems are trained on annotated camera images, video frames and 3D sensor (lidar) point clouds, so the algorithms can make driving judgments in real time, relying on data collected from varied tech components such as computer vision, sensors, NLP (Natural Language Processing), and more.
Without the technique, a model would have no way of knowing whether an oncoming obstacle is another car, a pedestrian, an animal, or a barricade. This only leads to an unfavorable outcome and the AI model’s failure.
Accurate annotation gives a model a sound basis for training. Whether the model powers chatbots, speech recognition, automation or other operations, better labels generally lead to better results, although no model is foolproof and every model still needs testing on new data.
What Is Data Annotation Outsourcing, and Why Is It Important?
You may train your AI and machine learning models faster by outsourcing data annotation and text annotation to a specialist provider instead of building a large in-house labeling team.
Many firms run dedicated data annotation teams that handle this repetitive, high-volume work, so a company’s own engineers can concentrate on building and improving the product.
With established workflows, quality assurance and progress reporting, data annotation and text annotation providers can support a wide range of annotation needs. Typical services include video labeling, image labeling, text labeling, conversational AI data and content moderation.
What Are the Main Types of Data Annotation?
Data annotation types depend on the kind of data and the task the model must learn. Amazon SageMaker Ground Truth, a widely used labeling service, groups its built-in task types into four families: images, text, video and 3D point clouds.
| Data type | Common annotation tasks | Typical use |
|---|---|---|
| Images | Single-label and multi-label classification, bounding boxes, semantic segmentation, label verification | Product recognition, medical imaging, quality inspection |
| Text | Named entity recognition, single-label and multi-label text classification | Search, document processing, sentiment analysis |
| Video | Video classification, frame-by-frame object detection, object tracking | Driver assistance, security footage, sports analytics |
| 3D point clouds | Object detection, object tracking, semantic segmentation | Self-driving cars and robotics using lidar |
| Chatbot output | Writing example answers and ranking model responses by preference | Reinforcement learning from human feedback (RLHF) |
In reinforcement learning from human feedback (RLHF), human annotators write prompt-and-response examples for fine-tuning and then rank pairs of model answers. A reward model learns to predict which answer people prefer. OpenAI used this approach for InstructGPT, and it became widely known with ChatGPT in late 2022.
Why Data Annotation Matters: The ImageNet Example
ImageNet shows how much modern AI depends on human labeling. In 2006, computer scientist Fei-Fei Li began work on expanding image-recognition training data, and in 2007 the labeling was outsourced to Amazon Mechanical Turk. Between July 2008 and April 2010, about 49,000 workers from 167 countries filtered and labeled more than 160 million candidate images.
The finished ImageNet database contains more than 14 million hand-annotated images in more than 20,000 categories, and at least one million images have bounding boxes. In the 2012 ImageNet challenge, the AlexNet model achieved a top-5 error of 15.3%, more than 10.8 percentage points lower than the runner-up, a result widely seen as a turning point for deep learning.
How Does a Data Annotation Project Work, Step by Step?
- Define the task. Decide what the model must predict and which label types are needed, such as classes, boxes or entities.
- Write annotation guidelines. Give clear definitions, examples and edge cases so different annotators label the same item the same way.
- Prepare and secure the data. Remove or mask personal data where possible and agree who may access the files.
- Run a pilot batch. Label a small sample, compare annotators and fix unclear rules before scaling up.
- Label at scale with review. Use a second reviewer or expert checks on a share of the work, and route disagreements to a senior annotator.
- Measure quality. Track agreement between annotators and accuracy against a set of gold-standard items.
- Iterate with the model. Retrain, look at the model’s errors and send confusing examples back for relabeling.
How Is Annotation Quality Measured?
Annotation quality is often measured with inter-annotator agreement. Cohen’s kappa is a statistic for agreement between two raters on categorical labels that corrects for agreement expected by chance. A kappa of 1 means complete agreement and 0 means no agreement beyond chance.
| Cohen’s kappa | Landis and Koch interpretation |
|---|---|
| Below 0 | No agreement |
| 0 to 0.20 | Slight |
| 0.21 to 0.40 | Fair |
| 0.41 to 0.60 | Moderate |
| 0.61 to 0.80 | Substantial |
| 0.81 to 1 | Almost perfect |
These bands are a rule of thumb, not a universal standard, so teams usually combine agreement scores with expert review and gold-standard test items.
How Can Biased Labels Harm an AI Model?
Biased labels or unbalanced data produce biased models. A 2018 study by Joy Buolamwini and Timnit Gebru found that two facial-analysis datasets were 79.6% and 86.2% lighter-skinned subjects. Human annotators also make errors, and fields such as medical imaging and legal analysis need annotators with domain expertise.
In-House vs Outsourced Data Annotation
| Factor | In-house team | Outsourced provider |
|---|---|---|
| Speed to scale | Slower: hiring and training needed | Faster: existing trained workforce |
| Domain knowledge | Strong for company-specific data | Varies; specialist providers exist for medical, legal and similar data |
| Data control | Data stays inside the company | Requires contracts, access controls and security checks |
| Cost structure | Fixed salaries and tools | Pay per task, hour or project; prices vary by provider |
| Management effort | Company runs guidelines and QA | Shared, but the buyer should still audit quality |
How to Choose a Data Annotation Outsourcing Provider
- Security: ask how data is stored, who can see it and whether personal data is handled under the privacy laws that apply to you.
- Quality process: ask for their review workflow, agreement metrics and a paid pilot before a long contract.
- Tools and formats: confirm they support your label types and export formats.
- Domain expertise: for medical, legal or financial data, confirm annotators have relevant training.
- Worker conditions: ask how annotators are paid and supported, especially for content moderation work.
What Are the Labor Concerns in Data Annotation Outsourcing?
Labor conditions are a real risk in annotation supply chains. A January 2023 Time investigation reported that Kenyan workers employed through Sama, a San Francisco-based training-data company founded in 2008, were paid less than $2 per hour to label toxic content for OpenAI’s ChatGPT safety systems. An Oxford Internet Institute study found that Remotasks, a crowdsourcing platform set up in 2017 by Scale AI, met fair-work standards in only one of ten criteria.
Scale AI itself, founded in 2016 and based in San Francisco, offers data annotation, RLHF and model evaluation services; Meta took a 49% non-voting stake in the company in June 2025. Amazon’s SageMaker Ground Truth labeling service is, as of October 2026, no longer open to new customers, though existing customers can keep using it. The size of the global annotation market is estimated very differently by different research firms, so any single market figure should be treated with caution.
For related reading, see why Python is used for AI and machine learning, how data science and business analytics shape businesses, outsourcing content moderation services and factors to consider while outsourcing.
Frequently Asked Questions
What is data annotation in simple terms?
Data annotation is labeling raw data, such as tagging objects in photos, names in text or words in audio, so a machine learning model can learn from correct examples.
Is data annotation the same as data labeling?
Data annotation and data labeling are generally used as the same thing: adding tags or labels to data for machine learning. Text annotation is the same work applied to text.
Why is data annotation important for AI?
Data annotation is important because supervised machine learning learns from labeled examples. Without accurate labels, a model cannot tell, for instance, whether an object ahead is a car, a pedestrian or a barrier.
What are the main types of data annotation?
The main types are image annotation (classification, bounding boxes, segmentation), text annotation (named entity recognition, classification), video annotation (object detection and tracking), 3D point cloud annotation and human preference ranking for chatbots.
Should a company outsource data annotation?
Outsourcing suits large or fast-growing labeling projects that need many trained annotators. Sensitive data or very specialized domains may be better handled in-house or by vetted specialist providers under strict contracts.
How is data annotation quality checked?
Quality is checked with clear guidelines, review by a second annotator or expert, gold-standard test items and agreement statistics such as Cohen’s kappa.
