Problem 1: The data set AirPassengers built into R lists total numbers of international airline passengers, 1949 to 1960. Explain the variables present in this dataset. Using the variables in this dataset and the formal definition of tidy data that we learned in lecture, is this data set tidy? Explain why or why not.
AirPassengers## Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
## 1949 112 118 132 129 121 135 148 148 136 119 104 118
## 1950 115 126 141 135 125 149 170 170 158 133 114 140
## 1951 145 150 178 163 172 178 199 199 184 162 146 166
## 1952 171 180 193 181 183 218 230 242 209 191 172 194
## 1953 196 196 236 235 229 243 264 272 237 211 180 201
## 1954 204 188 235 227 234 264 302 293 259 229 203 229
## 1955 242 233 267 269 270 315 364 347 312 274 237 278
## 1956 284 277 317 313 318 374 413 405 355 306 271 306
## 1957 315 301 356 348 355 422 465 467 404 347 305 336
## 1958 340 318 362 348 363 435 491 505 404 359 310 337
## 1959 360 342 406 396 420 472 548 559 463 407 362 405
## 1960 417 391 419 461 472 535 622 606 508 461 390 432The dataset contains the variables for number of passengers, years, and months. The dataset is not tidy. There should be one column for year, one for month, and one for the number of passengers. Instead, the data are arranged such that years vary along the rows and months along the columns.
Problem 2: The function data() lists all datasets that are available in R by default. Look through the list and identify a dataset that is tidy. Explain the variables present in this dataset and why the dataset is tidy.
I pick the dataset OrchardSprays:
head(OrchardSprays)## decrease rowpos colpos treatment
## 1 57 1 1 D
## 2 95 2 1 E
## 3 8 3 1 B
## 4 69 4 1 H
## 5 92 5 1 G
## 6 90 6 1 FThis dataset contains variables for row of the design (rowpos), column of the design (colpos), treatment level (treatment), and response (decrease). It contains one row per observation, one column of measured values (decrease), and three columns describing the conditions under which that value was measured (rowpos, colpos, treatment).
Problem 3a: Save a tidy version of the dataset Titanic using the code below. Filter for those did not survive (Survived == "No") and save the results in a new dataframe. What do you notice about the results?
titanic <- as.data.frame(Titanic) # this converts the dataset `Titanic` into a tidy dataframe; doesn't always work but it will in this case
titanic_filtered <- titanic %>%
filter(Survived == "No")
titanic_filtered## Class Sex Age Survived Freq
## 1 1st Male Child No 0
## 2 2nd Male Child No 0
## 3 3rd Male Child No 35
## 4 Crew Male Child No 0
## 5 1st Female Child No 0
## 6 2nd Female Child No 0
## 7 3rd Female Child No 17
## 8 Crew Female Child No 0
## 9 1st Male Adult No 118
## 10 2nd Male Adult No 154
## 11 3rd Male Adult No 387
## 12 Crew Male Adult No 670
## 13 1st Female Adult No 4
## 14 2nd Female Adult No 13
## 15 3rd Female Adult No 89
## 16 Crew Female Adult No 3More male adult passengers died than any other group of passengers.
Problem 3b: Using the new dataframe you just created, group by Class, use summarize() to sum the counts, then use mutate() to calculate the relative percentage of deaths in a new column for each group. What can you say about these results?
titanic_filtered %>%
group_by(Class) %>%
summarize(group_freq = sum(Freq)) %>%
mutate(percent = group_freq/sum(group_freq))## # A tibble: 4 x 3
## Class group_freq percent
## <fct> <dbl> <dbl>
## 1 1st 122 0.0819
## 2 2nd 167 0.112
## 3 3rd 528 0.354
## 4 Crew 673 0.452Crew members and 3rd class passengers suffered the most loss, making up >80% of deaths combined.
Problem 4 (if time): This package contains information about all flights that departed from NYC (e.g. EWR, JFK and LGA) to destinations in the United States, Puerto Rico, and the American Virgin Islands) in 2013: 336,776 flights in total. To help understand what causes delays, it also includes a number of other useful datasets. In particular, the data table flights lists on-time departure and arrival information for 336,776 individual flights:
install.packages("nycflights13") # this dataset is not pre-installed, so you have to install it with this line of code## Installing package into '/stor/home/rmc3665/R/x86_64-pc-linux-gnu-library/3.4'
## (as 'lib' is unspecified)library(nycflights13) # and then load it with this line of code
flights## # A tibble: 336,776 x 19
## year month day dep_time sched_dep_time dep_delay arr_time
## <int> <int> <int> <int> <int> <dbl> <int>
## 1 2013 1 1 517 515 2 830
## 2 2013 1 1 533 529 4 850
## 3 2013 1 1 542 540 2 923
## 4 2013 1 1 544 545 -1 1004
## 5 2013 1 1 554 600 -6 812
## 6 2013 1 1 554 558 -4 740
## 7 2013 1 1 555 600 -5 913
## 8 2013 1 1 557 600 -3 709
## 9 2013 1 1 557 600 -3 838
## 10 2013 1 1 558 600 -2 753
## # … with 336,766 more rows, and 12 more variables: sched_arr_time <int>,
## # arr_delay <dbl>, carrier <chr>, flight <int>, tailnum <chr>,
## # origin <chr>, dest <chr>, air_time <dbl>, distance <dbl>, hour <dbl>,
## # minute <dbl>, time_hour <dttm>We would like to collect some information about arrival delays of United Airlines (UA) flights. Do the following: pick all UA departures with non-zero arrival delay and calculate the mean arrival delay for each of the corresponding flight numbers. Which flight had the longest mean arrival delay and how long was that delay?
flights_filtered <- filter(flights, carrier == "UA" & arr_delay != 0)
flights_grouped <- group_by(flights_filtered, flight)
flights_summary <- summarize(flights_grouped, mean_delay = mean(arr_delay))
filter(flights_summary, mean_delay == max(mean_delay))## # A tibble: 1 x 2
## flight mean_delay
## <int> <dbl>
## 1 1510 283# more efficiently (and neatly, I think):
flights %>%
filter(carrier == "UA" & arr_delay != 0) %>%
group_by(flight) %>%
summarize(mean_delay = mean(arr_delay)) %>%
arrange(desc(mean_delay))## # A tibble: 1,283 x 2
## flight mean_delay
## <int> <dbl>
## 1 1510 283
## 2 125 113
## 3 640 111
## 4 1084 86
## 5 348 85.5
## 6 18 76
## 7 546 76
## 8 1105 67.4
## 9 626 57.1
## 10 1145 57
## # … with 1,273 more rowsFlight 1510 had the longest delay, with an average arrival delay of 283 minutes.