class: center, middle, title-slide .title[ # From Data to Visualization 1 ] .author[ ### Claus O. Wilke ] .date[ ### last updated: 2022-06-26 ] --- ## Topics covered -- 1. Visualizing amounts -- 2. Visualizing distributions -- 3. Visualizing associations and trends --- class: middle center ## Visualizing amounts --- ## We often encounter datasets containing simple amounts --- ## We often encounter datasets containing simple amounts Example: Highest grossing movies Dec. 2017 <br> .center[ <table> <thead> <tr> <th style="text-align:right;"> rank </th> <th style="text-align:left;"> title </th> <th style="text-align:right;"> amount </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:left;"> Star Wars </td> <td style="text-align:right;"> 71.57 </td> </tr> <tr> <td style="text-align:right;"> 2 </td> <td style="text-align:left;"> Jumanji </td> <td style="text-align:right;"> 36.17 </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:left;"> Pitch Perfect 3 </td> <td style="text-align:right;"> 19.93 </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:left;"> Greatest Showman </td> <td style="text-align:right;"> 8.81 </td> </tr> <tr> <td style="text-align:right;"> 5 </td> <td style="text-align:left;"> Ferdinand </td> <td style="text-align:right;"> 7.32 </td> </tr> </tbody> </table> ] .tiny-font.absolute-bottom-right[ Data source: Box Office Mojo ] --- ## We can visualize amounts with bar plots <br> .center.move-up-1em[ <!-- --> ] --- ## Bars can also run horizontally <br> .center.move-up-1em[ <!-- --> ] --- ## Pay attention to the order of the bars <br> .center.move-up-1em[ <!-- --> ] --- ## Pay attention to the order of the bars <br> .center.move-up-1em[  ] --- ## We can use dots instead of bars <br> .center.move-up-1em[ <!-- --> ] --- ## Dots are preferable if we want to truncate the axes .center.move-up-1em[ <!-- --> ] --- ## Dots are preferable if we want to truncate the axes .center.move-up-1em[ <!-- --> ] .absolute-bottom-right[ bar lengths do<br>not accurately<br>represent the<br>data values ] --- ## Dots are preferable if we want to truncate the axes .center.move-up-1em[ <!-- --> ] .absolute-bottom-right[ key features<br>of the data<br>are obscured ] --- ## Dots are preferable if we want to truncate the axes .center.move-up-1em[  ] --- ## We use grouped bars for higher-dimensional datasets -- <br> .center.move-up-1em[ <!-- --> ] .absolute-bottom-right[ Data source: United States Census Bureau, 2016 ] --- ## We are free to choose by which variable to group <br> .center[ <!-- --> ] .absolute-bottom-right[ Data source: United States Census Bureau, 2016 ] --- ## We can also use multiple plot panels (facets) .center[ <!-- --> ] .absolute-bottom-right[ Data source: United States Census Bureau, 2016 ] --- class: middle center ## Visualizing distributions --- ## Passengers on the Titanic .center.small-font[ <table> <thead> <tr> <th style="text-align:right;"> age </th> <th style="text-align:left;"> sex </th> <th style="text-align:left;"> class </th> <th style="text-align:left;"> survived </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 0.17 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 0.33 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> died </td> </tr> <tr> <td style="text-align:right;"> 0.80 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 0.83 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 0.83 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 0.92 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 1st </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 1.00 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 1.00 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 1.00 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 1.00 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> </tbody> </table> <table> <thead> <tr> <th style="text-align:right;"> age </th> <th style="text-align:left;"> sex </th> <th style="text-align:left;"> class </th> <th style="text-align:left;"> survived </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 1.0 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 1.5 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> died </td> </tr> <tr> <td style="text-align:right;"> 1.5 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> died </td> </tr> <tr> <td style="text-align:right;"> 2.0 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 1st </td> <td style="text-align:left;"> died </td> </tr> <tr> <td style="text-align:right;"> 2.0 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 2.0 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> died </td> </tr> <tr> <td style="text-align:right;"> 2.0 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> died </td> </tr> <tr> <td style="text-align:right;"> 2.0 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 2.0 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 2.0 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> </tbody> </table> <table> <thead> <tr> <th style="text-align:right;"> age </th> <th style="text-align:left;"> sex </th> <th style="text-align:left;"> class </th> <th style="text-align:left;"> survived </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:left;"> male </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 2nd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:left;"> female </td> <td style="text-align:left;"> 3rd </td> <td style="text-align:left;"> survived </td> </tr> </tbody> </table> ] --- ## Histogram: Define bins and count cases .pull-left.small-font[ <table> <thead> <tr> <th style="text-align:left;"> age range </th> <th style="text-align:right;"> count </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> 0–5 </td> <td style="text-align:right;"> 36 </td> </tr> <tr> <td style="text-align:left;"> 6–10 </td> <td style="text-align:right;"> 19 </td> </tr> <tr> <td style="text-align:left;"> 11–15 </td> <td style="text-align:right;"> 18 </td> </tr> <tr> <td style="text-align:left;"> 16–20 </td> <td style="text-align:right;"> 99 </td> </tr> <tr> <td style="text-align:left;"> 21–25 </td> <td style="text-align:right;"> 139 </td> </tr> <tr> <td style="text-align:left;"> 26–30 </td> <td style="text-align:right;"> 121 </td> </tr> <tr> <td style="text-align:left;"> 31–35 </td> <td style="text-align:right;"> 76 </td> </tr> <tr> <td style="text-align:left;"> 36–40 </td> <td style="text-align:right;"> 74 </td> </tr> </tbody> </table> <table> <thead> <tr> <th style="text-align:left;"> age range </th> <th style="text-align:right;"> count </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> 41–45 </td> <td style="text-align:right;"> 54 </td> </tr> <tr> <td style="text-align:left;"> 46–50 </td> <td style="text-align:right;"> 50 </td> </tr> <tr> <td style="text-align:left;"> 51–55 </td> <td style="text-align:right;"> 26 </td> </tr> <tr> <td style="text-align:left;"> 56–60 </td> <td style="text-align:right;"> 22 </td> </tr> <tr> <td style="text-align:left;"> 61–65 </td> <td style="text-align:right;"> 16 </td> </tr> <tr> <td style="text-align:left;"> 66–70 </td> <td style="text-align:right;"> 3 </td> </tr> <tr> <td style="text-align:left;"> 71–75 </td> <td style="text-align:right;"> 3 </td> </tr> <tr> <td style="text-align:left;"> 76–80 </td> <td style="text-align:right;"> 0 </td> </tr> </tbody> </table> ] -- .pull-right[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- # Histograms depend on the chosen bin width .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Alternative to histogram: Kernel density estimate (KDE) .pull-left[ <!-- --> ] -- .pull-right[ <!-- --> ] -- Histograms show raw counts, KDEs show proportions. (Total area = 1) ??? Figures redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## KDEs also depend on parameter settings .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Careful: Are bars stacked or overlapping? .pull-left[ <!-- --> ] -- .pull-right[ <!-- --> ] -- Stacked or overlapping histograms are rarely a good choice. ??? Figures redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Alternatively: Age pyramid .center[ <!-- --> ] ??? Figures redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## What if we want to show more than two distributions? .pull-left.small-font[ Mean temperatures in Lincoln, NE, in January 2016: .center[ |date | mean temp| |:----------|---------:| |2016-01-01 | 24| |2016-01-02 | 23| |2016-01-03 | 23| |2016-01-04 | 17| |2016-01-05 | 29| |2016-01-06 | 33| |2016-01-07 | 30| |2016-01-08 | 25| |2016-01-09 | 9| |2016-01-10 | 11| |2016-01-11 | 28| |2016-01-12 | 24| |2016-01-13 | 33| |2016-01-14 | 40| |2016-01-15 | 29| |2016-01-16 | 19| |2016-01-17 | 5| |2016-01-18 | 11| |2016-01-19 | 22| |2016-01-20 | 28| |2016-01-21 | 25| |2016-01-22 | 22| |2016-01-23 | 28| |2016-01-24 | 30| |2016-01-25 | 26| |2016-01-26 | 29| |2016-01-27 | 33| |2016-01-28 | 41| |2016-01-29 | 41| |2016-01-30 | 39| |2016-01-31 | 35| ]] -- .pull-right[ <!-- --> ] --- ## What if we want to show more than two distributions? .pull-left.small-font[ Mean temperatures in Lincoln, NE, in January 2016: .center[ |date | mean temp| |:----------|---------:| |2016-01-01 | 24| |2016-01-02 | 23| |2016-01-03 | 23| |2016-01-04 | 17| |2016-01-05 | 29| |2016-01-06 | 33| |2016-01-07 | 30| |2016-01-08 | 25| |2016-01-09 | 9| |2016-01-10 | 11| |2016-01-11 | 28| |2016-01-12 | 24| |2016-01-13 | 33| |2016-01-14 | 40| |2016-01-15 | 29| |2016-01-16 | 19| |2016-01-17 | 5| |2016-01-18 | 11| |2016-01-19 | 22| |2016-01-20 | 28| |2016-01-21 | 25| |2016-01-22 | 22| |2016-01-23 | 28| |2016-01-24 | 30| |2016-01-25 | 26| |2016-01-26 | 29| |2016-01-27 | 33| |2016-01-28 | 41| |2016-01-29 | 41| |2016-01-30 | 39| |2016-01-31 | 35| ]] .pull-right[ <!-- --> How can we compare distributions across months? ] --- ## A bad idea: Many overlapping density plots .center[ <!-- --> ] --- ## Another bad idea: Stacked density plots .center[ <!-- --> ] --- ## Somewhat better: Small multiples .center[ <!-- --> ] --- ## Instead: Show values along y, conditions along x .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) -- A boxplot is a crude way of visualizing a distribution. --- ## How to read a boxplot .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## If you like density plots, consider violins .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) -- A violin plot is a density plot rotated 90 degrees and then mirrored. --- ## How to read a violin plot .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## For small datasets, you can also use a strip chart Advantage: Can see raw data points instead of abstract representation. .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) -- Horizontal jittering may be necessary to avoid overlapping points. --- ## For small datasets, you can also use a strip chart Advantage: Can see raw data points instead of abstract representation. .center[ <!-- --> ] Horizontal jittering may be necessary to avoid overlapping points. ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## For small datasets, you can also use a strip chart Advantage: Can see raw data points instead of abstract representation. .center[ <!-- --> ] Horizontal jittering may be necessary to avoid overlapping points. ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## And the final option, a ridgeline plot .center[ <!-- --> ] ??? Figure redrawn from [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) -- Notice the single fill color. More colors would be distracting. --- class: middle center ## Visualizing associations and trends --- ## We visualize associations with scatter plots .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## We visualize associations with scatter plots .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Regression lines emphasize the overall trend .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Regression lines emphasize the overall trend .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Regression lines emphasize the overall trend .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Regression lines emphasize the overall trend .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Regression lines emphasize the overall trend .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Regression lines emphasize the overall trend .center[ <!-- --> ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Detrending: Removing the underlying trend .center[ <!-- --> ] -- .small-font[ Did housing prices in California decline substantially from 1990 to 1998? ] -- .small-font[ Did housing prices in West Virginia recover by 2017? ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Detrending: Removing the underlying trend .center[ <!-- --> ] .small-font[ Did housing prices in California decline substantially from 1990 to 1998? ] .small-font[ Did housing prices in West Virginia recover by 2017? ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Detrending: Removing the underlying trend .center[ <!-- --> ] .small-font[ Did housing prices in California decline substantially from 1990 to 1998? — yes ] .small-font[ Did housing prices in West Virginia recover by 2017? — no ] ??? Figure redrawn after [Claus O. Wilke. Fundamentals of Data Visualization. O'Reilly, 2019.](https://clauswilke.com/dataviz) --- ## Further reading Relevant chapters from Fundamentals of Data Visualization: - [Chapter 6: Visualizing amounts](https://clauswilke.com/dataviz/visualizing-amounts.html) - [Chapter 7: Visualizing distributions: Histograms and density plots](https://clauswilke.com/dataviz/histograms-density-plots.html) - [Chapter 9: Visualizing many distributions at once](https://clauswilke.com/dataviz/boxplots-violins.html) - [Chapter 12: Visualizing associations among two or more quantitative variables](https://clauswilke.com/dataviz/visualizing-associations.html) - [Chapter 14: Visualizing trends](https://clauswilke.com/dataviz/visualizing-trends.html)