Project · 40 min

Project: the report of a week

The goal

The bike share service wants a report for the week from 15 June to 21 June. You get the raw file of the trips, as the system exported it, and the file of the stations. You write the program that makes the report. The program uses each main tool of this course one time.

This is the order of the work. It is the order of almost every data job.

Load the files.
Clean the rows, and count how many you keep.
Join the rows with the second table, and add the values that you need.
Summarize: count, group, rank, and describe.
Write the result to a file.

The files

week_trips.csv has 24 rows. The file has the problems that you know from lesson 8: spaces, different capital letters, values that are not numbers, and a row that repeats. Each trip has the moment when it started, in the ISO format.
stations.csv has the zone of each station. One start station of the trips is not in this file.
bikedata.py has load_rows. main.py already loads the two files into raw and stations. Every value in them is text.

The rules for clean rows

clean_trips makes one clean row from each raw row that it keeps. A clean row has these keys.

Key	Value
`trip_id` and `started_at`	The same text as in the raw row.
`start` and `end`	The name with single spaces, and with a capital letter at the start of each word.
`minutes`	A whole number.
`rider`	The text with no spaces around it, in lower-case letters.

The function leaves out a raw row in two cases:

The value of minutes is not a whole number of 1 or more.
A row with the same trip_id is already in the result.

The report

The dictionary report has six keys.

Key	Value
`rows_read`	The number of rows in `raw`.
`rows_kept`	The number of rows in `trips`.
`summary`	The result of `summarize(trips)`.
`top_stations`	The result of `top_stations(trips, 3)`.
`by_zone`	The result of `count_by(trips, "zone")`.
`by_day`	The result of `count_by(trips, "day")`.

How to work

Write one function, and try it with a print line under it. For example: print(clean_trips(raw)[:2]).
The first five tasks ask for a function. The checker calls each function with a few small rows, and not with the file. A message of a check shows the rows that the checker used.
You can write more functions than the tasks ask for. Small functions with one job are easier to test.
When a task is hard, open the lesson that the hint names.

After the tasks

Your program now runs on each file that has the same columns. Try to extend the report:

Add the share of the trips of members, in percent.
Add the hour of the day with the most trip starts.
Add the list of the start stations that are not in stations.csv.

Tasks

7 to do

Task 1, not checked yet. Write clean_trips(rows). It returns a new list with the clean rows of rows. The section "The rules for clean rows" in the text has the rules.
Task 2, not checked yet. Write add_details(trips, zone_of). It returns a list with one row for each trip. Each row has all the keys of the trip and two more. zone is the zone of the start station from the dictionary zone_of, or "unknown". day is the name of the weekday of started_at.
Task 3, not checked yet. Write summarize(trips). It returns a dictionary with the keys trips (the number of trips), minutes (the total minutes), mean (the mean of the minutes, with 1 digit after the decimal point) and median. For an empty list, it returns {"trips": 0}.
Task 4, not checked yet. Write count_by(trips, column). It returns a dictionary with the number of trips for each value of that column.
Task 5, not checked yet. Write top_stations(trips, n). It returns a list of pairs for the n start stations with the most trips. Each pair has the name of the station and its number of trips. The largest number is first. Stations with the same number are in alphabetical order.
Task 6, not checked yet. Use your functions on the files. Make a dictionary zone_of from stations. Make a list trips: the clean rows of raw, with their details. Then make the dictionary report. The section "The report" in the text has its six keys.
Task 7, not checked yet. Write report to the file report.json, with an indent of 2 spaces. Then print the line Kept 20 of 24 rows. Take the two numbers from report.

Task 1 of 7

Write clean_trips(rows). It returns a new list with the clean rows of rows. The section "The rules for clean rows" in the text has the rules.

All tasks