AdvertisementYour brand could be hereYour brand hereOn every Darslab page · students, parents and teachersinfo@darslab.comAsk for pricesPricesinfo@darslab.com
☰ Year plan · Computing

Ask smart questions

Week 19 · 2.3 Creating databases · 3 lessons
3

Too much data

Computing 9: Topic 2.3, pp. 217–256
GoalDescribe big data using the five Vs and give applications, then plan, test and evaluate a small two-table database and query on paper.
New words
big data · very large and complex collections of data that ordinary methods cannot easily processvolume / velocity · how much data there is, and how fast it is producedvariety / veracity · how many different kinds of data there are, and how far the data can be trustedvalue · how useful the data is once it has been made sense of
Explanation

Big data is data in such huge amounts, and so complex, that a spreadsheet or an ordinary database cannot handle it well. It is described by five Vs. Volume is the amount, velocity is the speed at which new data arrives, and variety is the mixture of numbers, text, pictures and sound. Veracity means that the data must be accurate and trustworthy, so wrong values must be found and removed before use. Value means the data is only worth keeping if it is turned into something useful. Weather forecasting, road traffic control, farming and banking all use big data. When the data is about people, they must give permission, and only the facts that are really needed should be kept.

VMeaningQuestion to ask
Volumehow much datais it too big for a spreadsheet?
Velocityhow fast it arrivesdoes it keep arriving every second?
Varietymany kindsnumbers, pictures, text, sound?
Veracitytrustworthyare there wrong readings?
Valueusefulnesswhat decision can it help?
Worked examples
StepWorking
Readings per hour10 000 × 60 = 600 000
Readings per day600 000 × 24 = 14 400 000
Volume and velocity. A country has 10 000 weather stations and each sends one reading every minute. In one day that is 10 000 × 60 × 24 = 14 400 000 readings. The data is large (volume) and never stops coming (velocity), so a spreadsheet would soon be full.
Tap-card records(numbers)Road camerapicturesPassengermessages (text)Bus study data
Variety. A city studies its bus service. The data comes from several sources and in several forms: numbers from the tap cards, pictures from road cameras and typed messages from passengers. A single table with fixed fields cannot hold all of these, which is why big-data tools are needed.
B8fx=AVERAGE(B2:B7)
AB
1HourTemp (°C)
2112
3213
43−999
5414
6515
7614
8Average of all six−155.17
9Average of five13.6
Veracity. One weather sensor broke and sent −999 °C. If the readings are averaged as they are, the answer is −155.17 °C, which is nonsense. When the impossible reading is removed first, the average of the other five is 13.6 °C. Big data must be cleaned and checked before it is trusted.
Class activity

Mini project on paper (no computer). A school wants a lost-property database. Plan: draw two tables, Items (an Item_ID, what it is, colour, where it was found) and Owners (an Owner_ID, name, class). Choose the primary key of each and add a foreign key to link them. Make: write six invented records. Test: write one query as a grid with at least two criteria, including one OR or NOT, and work out which records it finds. Swap with a partner, who checks your results. Evaluate: does the database meet its purpose, which is to return an item to its owner? Write one thing that works well and one improvement, such as a missing field or a better key.

Practice
1
Match each situation to one of the five Vs: 1) a power company stores a reading from each of 5 million meters every few seconds; 2) a farm mixes soil numbers, drone pictures and spoken notes; 3) a check removes impossible heights from a list of tree sizes; 4) a council uses traffic records to decide where to build a new crossing.
SituationWhich V?
Millions of meter readings every few seconds?
Numbers, drone pictures and spoken notes?
Removing impossible tree heights?
Using traffic records to choose a crossing site?
2
A small club has 40 members. Does it need big data tools for its membership list? Explain.
3
A bus company wants to keep a record of every journey made with each passenger’s card. Give two things it must do to treat the passengers’ data responsibly.
4
A count of yellow cars on a road uses 100 readings, but 12 are wrong because a camera could not see at night. Which V is at risk, and what should be done before the data is used?