Collecting and Cleaning Data

One-sentence summary

Collecting data means recording information from the world in an organised way; cleaning data means finding and fixing the missing, wrong or repeated parts of that record.

Why does it matter?

In the previous lesson we learned what data is. But before a computer or an artificial intelligence can produce useful results, it first needs good data. A system that works with bad data gives bad results. We have a short way of saying this: garbage in, garbage out.

Imagine you want to find your class's favourite fruit. Some friends filled in the form twice, one wrote their height instead of a fruit, and one did not answer at all. If you count the results as they are, you reach the wrong conclusion. Learning to collect data first and then clean it is the first and most important step of every data and AI project.

Remember: AI is not magic. It is a tool that finds patterns in data. If your data is broken, the pattern it finds will be broken too.

How do we collect data?

There are many ways to collect data. Let's look at three common ones.

1. Measurement

We measure something with a ruler, scale or thermometer and write it down. For example, you could measure the room temperature every morning for a week and record it in a table. The key here is to take every measurement in the same way: same time, same place, same unit.

2. Forms and surveys

We ask people questions and record their answers. A survey about the class's favourite fruit is an example. It helps to give clear options; offering a list instead of saying "write a fruit" reduces the cleaning work later.

3. Sensors

In the robotics module we used sensors. A distance sensor, light sensor or temperature sensor collects data on its own. Sensors are fast, but when they fail or point the wrong way they can also produce nonsense values.

Everyday example: Tracking water

If you write down how many glasses of water you drink each day for a week, that is a data-collection activity:

Monday: 6
Tuesday: 7
Wednesday: 5
Thursday: 8
Friday: 6

Because you use the same unit (glasses) every day, this data is organised and can be compared later.

What problems appear in data?

Collected data is never perfect from the start. Let's look at the three most common problems.

1. Missing data

A cell is left empty. Maybe someone did not answer the survey, or maybe the sensor could not take a reading at that moment. We cannot simply ignore missing data; we either mark it or keep that row separate.

2. Wrong data

The value makes no sense. If a person's height is recorded as 3 metres, or someone is said to have drunk 200 glasses of water in one day, that is an error. We usually call this an outlier.

3. Repeated data

The same record is entered twice. If a friend filled in the form twice, their vote is counted twice and spoils the result.

Everyday example: A height list

Suppose a class's heights are collected in centimetres:

[142, 150, 3, 148, 150, -5, 145]

Here 3 is far too small (probably typed wrong), and -5 is impossible (height cannot be negative). Is the second 150 a real duplicate or two different students? Only the person who collected the data knows. This is where cleaning begins.

How do we clean data?

Cleaning is not about deleting data; it is about making it trustworthy. We do three main things:

• Correct: If we know the right value for a wrong entry, we fix it.
• Mark: Instead of deleting a missing or suspicious value, we make it visible (for example we write None), because knowing how many are missing is also information.
• Filter out: We remove values that are clearly impossible (like a negative height).

Important rule: we do not delete a value just because "we don't like it." We only filter out what is clearly wrong. If we throw away a real but surprising value, we distort the data.

Simple Python: filtering out wrong heights

The program below keeps the sensible values in a height list and puts the impossible ones in a separate list. We use only standard Python.

# Height measurements in centimetres
heights = [142, 150, 3, 148, 150, -5, 145]

clean = []
wrong = []

for height in heights:
    # We choose a sensible range for a human height
    if 100 <= height <= 210:
        clean.append(height)
    else:
        wrong.append(height)

print("Clean data:", clean)
print("Filtered-out wrong values:", wrong)

average = sum(clean) / len(clean)
print("Average of the clean data:", round(average, 1))

This program is not an AI. It only decides based on a rule we chose (sensible means between 100 and 210). Because we chose the rule, we are also responsible for it. If we pick the wrong range, we might throw away correct data by mistake, so we set the rule carefully.

Mini practice

The survey data below shows a class's favourite colour. But it has problems inside it:

["blue", "red", "BLUE", "", "green", "red", "bleu"]

Your task:

1. Mark the empty answer ("") as missing data.
2. Fix the difference in capital and small letters (BLUE and blue are the same colour).
3. Fix the spelling mistake (bleu is really blue).
4. After cleaning, count how many votes each colour received.

Hint: In Python, text.lower() turns a word into small letters. You can use a dictionary (dict) to count. The result you should get: blue 3, red 2, green 1, missing 1.

Common mistakes

Calculating before cleaning

Taking an average or a count before cleaning the data is the most common mistake. Clean first, then calculate.

Throwing away surprising but real data

Do not assume an unexpected value is an "error" straight away. Maybe it really happened. Only filter out what is impossible, and investigate what is merely unusual.

Silently deleting missing data

Deleting an empty cell and ignoring it hides how many people did not answer. Marking it is more honest.

Writing the same thing differently

To a computer, blue, BLUE and Blue are three different things. Choosing one single format while collecting makes your job easier.

Safety note

When collecting data, the most important rule is to protect personal information.

• Do not put personal details such as your friends' names, phone numbers, addresses or student numbers into a project. In surveys, use nicknames like "Student 1, Student 2" instead of names.
• Do not upload the data you collect to online AI tools or public sites. These tools may store the data and others may see it.
• When collecting other people's data, ask permission and say clearly what you are collecting it for.
• Use these kinds of tools with an adult's guidance and follow the platforms' age rules.

Remember: data is information about real people. Handling it with care is your responsibility.

Lesson summary

• Data is collected in an organised way through methods like measurement, forms/surveys and sensors.
• Collected data often contains missing, wrong and repeated values.
• Cleaning means correcting, marking, and filtering out impossible values.
• A simple rule (such as a sensible range) can filter out wrong values, but the human who chooses the rule is responsible for the result.
• Never putting personal data into open tools and treating data with respect is a basic safety rule.

Check questions

1. What are the three ways of collecting data mentioned in this lesson?
2. What does "garbage in, garbage out" mean?
3. Which data problem does the value -5 in a height list show?
4. Why do we prefer to mark a missing value instead of deleting it?
5. Which safety rule should you think about before uploading survey data to an online AI tool?

Answers

1. Measurement (with tools like a ruler or scale), forms/surveys (by asking people questions) and sensors (like a distance, light or temperature sensor).
2. If the data is broken or wrong, the calculation and the AI result made from it come out broken too; good results need good data first.
3. Wrong (illogical / impossible) data; a height cannot be negative, so this is an outlier.
4. Because how many answers are missing is also information; deleting it hides that fact and can make the result look wrong.
5. The rule of protecting personal information: the data should contain no names or addresses, such data should not be uploaded to open tools, and an adult should be consulted.

Source and verification note

For “Collecting and Cleaning Data”, verification focuses on whether the relationship between How do we collect data? and 2. Forms and surveys remains consistent across examples. Datasets in this module are small and educational; real personal data should not be used. An AI result should be evaluated not only for accuracy but also for data balance, error distribution and explainability.

Next lesson

Tablo ve Grafik Okuma (Reading Tables and Charts): We will turn our cleaned data into a table and a chart and learn to read what it means.