Week 4: Working with data.frames and tibbles

PPOL 6805: GIS for Spatial Data Science

Christy Hsu
th1010@georgetown.edu

2026-09-18

R Coding Workshop: 2nd Meeting

Outline

  • Recap
  • R Operators
  • Import files
  • Subsetting a Data Frame
    • Base R advanced:
      • tapply()
      • sample()
    • The Tidyverse approach: dplyr package

Recap

  1. Name your Variables smartly and annotate your code with comments
  • Name your variables as nouns1
  • lowercase, concatenate with underscores _
  • Concise and Meaningful
  • rm() command
  1. print() , class() and mapview()
  2. R Data Types and Data Structures

R Operators

Try them out:

Arithmetic Operators

  • + - * / ^

Comparison Operators

  • > <
  • ==
  • !=

Other Binary Operators

  • &
  • |
  • !
  • %in%

\(\sqrt{x^2 + y^2}\)

x <- 3
y <- 4
print((x^2 + y^2)^(1/2))
[1] 5

\(mile = kilometer * 0.62137\)

print((3000 * 0.62137))
[1] 1864.11
print((3000 * 0.62137) * 1.609344)
[1] 2999.994
# | code-fold: true
# install.packages('measurements')
library(measurements)
conv_unit(3000, 'km', 'mi')
[1] 1864.114

R Binary Operators

Binary operators in R: Be aware of the left hand side and the right hand side of binary operators.

Vector Recycling

vector1 <- 1:5
print(vector1)
[1] 1 2 3 4 5
print(vector1 + 1)
[1] 2 3 4 5 6
print(vector1 * 1.609344)
[1] 1.609344 3.218688 4.828032 6.437376 8.046720

This helps us make sense of the following evaluation:

print(vector1 > 3)
[1] FALSE FALSE FALSE  TRUE  TRUE
# vector1 <- 1:5
print(vector1 != 3)
[1]  TRUE  TRUE FALSE  TRUE  TRUE
'capitol' == 'CAPITOL'
[1] FALSE
# ascii or utf8
print('>' < '0')
[1] TRUE
print('a' < 'A')
[1] TRUE
us_states <- state.abb
midwest_states <- c(
  "IL", "IN", "MI", "OH", "WI","IA", "KS", "MN", "MO", "NE", "ND", "SD"
)
us_states %in% midwest_states
 [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[13]  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE FALSE
[25]  TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE FALSE
[37] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[49]  TRUE FALSE
us_states <- state.abb
midwest_states <- c(
  "IL", "IN", "MI", "OH",
  "WI", "IA", "KS", "MN",
  "MO", "NE", "ND", "SD"
)
print(is.vector(midwest_states))
[1] TRUE
!is.vector(midwest_states)
[1] FALSE
us_states %in% midwest_states
 [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[13]  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE FALSE
[25]  TRUE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE FALSE
[37] FALSE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE
[49]  TRUE FALSE

what happened here?

double_vector <- seq(21.9, 25.3, 0.1)
int_vector <- 20L:26L
double_vector %in% int_vector
 [1] FALSE  TRUE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE
[13] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE FALSE FALSE
[25] FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE FALSE FALSE FALSE

Import and Export Files

Subsetting data.frame Continued

We want to extract or access certain rows or columns of our dataframe.

utah_df <- data.frame(
    national_park = c(
    "Bryce Canyon", "Canyonlands", "Arches", "Zion", "Capitol Reef"
    ),
    lat = c(
        37.640621053549125, 38.478777627059635, 38.6167568289248,
        37.200271934321734, 38.291603924096385
        ),
    lon = c(
        -112.16957627116382, -109.8251716515892, -109.61982474559946, 
        -112.98700616100083, -111.2619347149233
        )
)
utah_df
  national_park      lat       lon
1  Bryce Canyon 37.64062 -112.1696
2   Canyonlands 38.47878 -109.8252
3        Arches 38.61676 -109.6198
4          Zion 37.20027 -112.9870
5  Capitol Reef 38.29160 -111.2619

tapply(): Advanced way for data subsetting

Demo Dataset Description

Variable Documentation 

# install.packages('bayesrules')
# import bayesrules package and 
library(bayesrules)

# bookban_df <- bayesrules::book_banning
bookban_df <- book_banning
# print(head(bookban_df))
summary(bookban_df)
    title              book_id       author               date           
 Length:931         143    : 17   Length:931         Min.   :2000-01-01  
 Class :character   868    : 12   Class :character   1st Qu.:2002-11-16  
 Mode  :character   1849   : 10   Mode  :character   Median :2006-02-10  
                    1025   :  9                      Mean   :2005-12-16  
                    1083   :  8                      3rd Qu.:2009-03-18  
                    399    :  7                      Max.   :2010-09-09  
                    (Other):868                      NA's   :11          
      year      removed explicit antifamily occult  language lgbtq   violent
 Min.   :2000   0:714   0:632    0:891      0:895   0:694    0:842   0:797  
 1st Qu.:2002   1:217   1:299    1: 40      1: 36   1:237    1: 89   1:134  
 Median :2006                                                               
 Mean   :2005                                                               
 3rd Qu.:2009                                                               
 Max.   :2010                                                               
 NA's   :11                                                                 
    state           political_value_index median_income    hs_grad_rate   
 Length:931         Min.   :-20.2000      Min.   :-8466   Min.   :-6.662  
 Class :character   1st Qu.: -1.8000      1st Qu.: 1274   1st Qu.: 1.038  
 Mode  :character   Median :  2.0000      Median : 4218   Median : 2.338  
                    Mean   :  0.0304      Mean   : 4530   Mean   : 2.833  
                    3rd Qu.:  4.0000      3rd Qu.: 9908   3rd Qu.: 5.538  
                    Max.   : 13.4000      Max.   :19936   Max.   : 8.738  
                                                                          
 college_grad_rate
 Min.   :-9.2237  
 1st Qu.:-2.0237  
 Median : 0.2763  
 Mean   : 0.6043  
 3rd Qu.: 2.5763  
 Max.   : 9.1763
  • ask your data questions:
tapply(
    bookban_df$removed == '1',
    bookban_df$state,
    mean
    )
        AK         AL         AR         AZ         CA         CO         CT 
0.11111111 0.50000000 0.00000000 0.41666667 0.32000000 0.12345679 0.16666667 
        DE         FL         GA         IA         ID         IL         IN 
0.50000000 0.30769231 0.30769231 0.25000000 0.85714286 0.30769231 0.30000000 
        KS         KY         LA         MA         MD         ME         MI 
0.76923077 0.15384615 0.41666667 0.37500000 0.00000000 0.00000000 0.19444444 
        MN         MO         MS         MT         NC         ND         NE 
0.05555556 0.72727273 1.00000000 0.00000000 0.20000000 0.33333333 0.00000000 
        NH         NJ         NM         NY         OH         OK         OR 
0.28571429 0.18181818 0.66666667 0.44000000 0.34482759 0.34782609 0.04237288 
        PA         RI         SC         SD         TN         UT         VA 
0.07432432 0.33333333 0.40000000 0.33333333 0.38461538 0.00000000 0.52941176 
        VT         WA         WI         WV         WY 
0.00000000 0.33333333 0.30000000 0.00000000 0.25000000

The data science question that is asked here: What fraction of book challenges were successful in each state?

  • The first argument: specify the variable we are interested in which is the books that their remove request were approved
  • The 2nd argument: how we want to group our data, often for the purpose of comparison, and this case we naturally want to compare between states.
  • The 3rd argument: takes a function, which is the operation that we want to apply to our variable of interest, here it is the mean() function for acquiring the fraction of approved request.

R factor

As we have observed, the bookban_df$removed, removed column is of factor data type, taking on values ‘0’ and ‘1’. And by making the comparison (add == 1), we get a logical vector of TRUE(1) and FALSE(0) which can be treated as 1 and 0 correspondingly, which mean() can operate on.

What are we trying to ask for the following two code blocks?

tapply(
    bookban_df$political_value_ind,
    bookban_df$lgbtq,
    mean)
        0         1 
 0.102019 -0.647191 
tapply(bookban_df$title, bookban_df$state, length)
 AK  AL  AR  AZ  CA  CO  CT  DE  FL  GA  IA  ID  IL  IN  KS  KY  LA  MA  MD  ME 
  9  10   5  12  50  81   6   2  26  13  16  14  39  20  13  13  12   8   5   3 
 MI  MN  MO  MS  MT  NC  ND  NE  NH  NJ  NM  NY  OH  OK  OR  PA  RI  SC  SD  TN 
 36  18  11   1   8  20   6   2   7  11   3  25  29  23 118 148   3  15   3  13 
 UT  VA  VT  WA  WI  WV  WY 
  1  34  13   9  10   3   4

Tidyverse

  • tidyverse is a selection of R packages, by running the following import code, we can have most of the packages that we will use to do data analysis
  • Among them, dplyr and ggplot2 will be used for almost all of the upcoming assignments
library(tidyverse)

|>: meet the pipeline operator

bookban_df |> select(date) |> pull(date) |> class()
[1] "Date"
bookban_df['date']$date |> class()
[1] "Date"

The Tidyverse way of initializing data frames

utah_df <- data.frame(
    national_park = c(
    "Bryce Canyon", "Canyonlands", "Arches", "Zion", "Capitol Reef"
    ),
    lat = c(
        37.640621053549125, 38.478777627059635, 38.6167568289248,
        37.200271934321734, 38.291603924096385
        ),
    lon = c(
        -112.16957627116382, -109.8251716515892, -109.61982474559946, 
        -112.98700616100083, -111.2619347149233
        )
)
utah_df
  national_park      lat       lon
1  Bryce Canyon 37.64062 -112.1696
2   Canyonlands 38.47878 -109.8252
3        Arches 38.61676 -109.6198
4          Zion 37.20027 -112.9870
5  Capitol Reef 38.29160 -111.2619
utah_tribble <- tibble::tribble(
    ~national_park, ~lat, ~lon,
    "Bryce Canyon", 37.640621053549125, -112.16957627116382,
    "Canyonlands", 38.478777627059635, -109.8251716515892,
    "Arches", 38.6167568289248, -109.61982474559946,
    "Zion", 37.200271934321734, -112.98700616100083,
    "Capitol Reef", 38.291603924096385, -111.2619347149233
)
utah_tribble
# A tibble: 5 × 3
  national_park   lat   lon
  <chr>         <dbl> <dbl>
1 Bryce Canyon   37.6 -112.
2 Canyonlands    38.5 -110.
3 Arches         38.6 -110.
4 Zion           37.2 -113.
5 Capitol Reef   38.3 -111.

Why tibble object?

  • Try uncomment the first line of the code!
# print(bookban_df)

as_tibble(bookban_df)
# A tibble: 931 × 17
   title      book_id author date        year removed explicit antifamily occult
   <chr>      <fct>   <chr>  <date>     <dbl> <fct>   <fct>    <fct>      <fct> 
 1 House of … 927     Allen… 2005-04-01  2005 0       1        0          1     
 2 It's Not … 1024    Harri… 2008-02-06  2008 1       0        0          0     
 3 King Stork 1087    Pyle,… 2008-10-02  2008 0       0        0          0     
 4 How They … 936     Levit… 2008-10-05  2008 0       0        0          0     
 5 Ghost in … 764     Masam… 2008-10-02  2008 0       0        0          0     
 6 King Stork 1087    Pyle,… 2003-09-13  2003 0       0        0          0     
 7 Queer      1489    Gage,… 2003-09-13  2003 0       0        0          0     
 8 Witness    2023    Hesse… 2003-09-13  2003 0       0        0          0     
 9 It's Perf… 1025    Harri… 2001-12-22  2001 0       1        0          0     
10 Brimstone… 318     Koert… 2006-01-30  2006 1       0        1          0     
# ℹ 921 more rows
# ℹ 8 more variables: language <fct>, lgbtq <fct>, violent <fct>, state <chr>,
#   political_value_index <dbl>, median_income <dbl>, hs_grad_rate <dbl>,
#   college_grad_rate <dbl>
  • tibble object print the data type of each column by default

dplyr

select(): subset by column name

  • tip: uncomment the following code block in Positron to acquire a copiable vector of all the column names of your dataset
# bookban_df |> colnames() |> View()
  • or
varnames <- c("title", "book_id", "author", "date", "year", "removed", "explicit", 
"antifamily", "occult", "language", "lgbtq", "violent", "state", 
"political_value_index", "median_income", "hs_grad_rate", "college_grad_rate"
)

Sometimes, we are not interested in all the given variables

state_level_vars <- c( 
"political_value_index", "median_income", "hs_grad_rate", "college_grad_rate"
)
bookban_df |> select(-any_of(state_level_vars)) |> tail(4)
                        title book_id                               author
928       When Dad Killed Mom    1964                       Lester, Julius
929         Geology Book, The     755                  Morris, Dr. John D.
930     And Tango Makes Three     143 Parnell, Peter and Justin Richardson
931 Darkest Night of the Year     505                         Koontz, Dean
          date year removed explicit antifamily occult language lgbtq violent
928 2002-04-21 2002       1        0          0      0        0     0       1
929 2010-05-05 2010       0        0          0      0        0     0       0
930 2009-08-22 2009       0        0          0      0        0     1       0
931 2007-12-03 2007       0        1          0      0        0     0       0
    state
928    WY
929    WY
930    WY
931    WY
  • Being mindful of the data type after subsetting
bookban_df |> select(title) |> class()
[1] "data.frame"
bookban_df[, 'title'] |> class()
[1] "character"

dplyr

filter(): keep rows that match our criteria

We might have a particular interest in Wisconsin

bookban_df |> filter(state == 'WI')
                                                                   title
1                                               Lords of Discipline, The
2                                                                 Damage
3                                                        Athletic Shorts
4                                                            Teens & Sex
5  Doing It Right:  Making Smart, Safe, and Satisfying Choices About Sex
6                                                          Carry Me Down
7                                                   Grapes of Wrath, The
8                                                                   TTYL
9                                                  It's Perfectly Normal
10                                                 Harry Potter (series)
   book_id          author       date year removed explicit antifamily occult
1     1177     Conroy, Pat 2002-11-16 2002       1        1          0      0
2      487    Jenkis, A.M. 2009-11-18 2009       0        1          0      0
3      171 Crutcher, Chris 2006-10-10 2006       1        0          0      0
4     1766  Marcovitz, Hal 2010-03-20 2010       0        1          0      0
5      560 Pardes, Bronwen 2010-03-20 2010       0        1          0      0
6      361    Hyland, M.J. 2007-07-22 2007       0        0          0      0
7      745 Steinbeck, John 2003-07-14 2003       1        0          0      0
8     1849 Myracle, Lauren 2009-02-25 2009       0        1          0      0
9     1025   Harris, Robie 2001-09-22 2001       0        0          0      0
10     868   Rowling, J.K. 2000-09-30 2000       0        0          0      1
   language lgbtq violent state political_value_index median_income
1         1     0       1    WI                   2.4        4234.5
2         0     0       0    WI                   2.4        4234.5
3         1     1       0    WI                   2.4        4234.5
4         0     0       0    WI                   2.4        4234.5
5         0     0       0    WI                   2.4        4234.5
6         0     0       1    WI                   2.4        4234.5
7         1     0       0    WI                   2.4        4234.5
8         0     0       0    WI                   2.4        4234.5
9         0     0       0    WI                   2.4        4234.5
10        0     0       0    WI                   2.4        4234.5
   hs_grad_rate college_grad_rate
1      5.538042          -1.62373
2      5.538042          -1.62373
3      5.538042          -1.62373
4      5.538042          -1.62373
5      5.538042          -1.62373
6      5.538042          -1.62373
7      5.538042          -1.62373
8      5.538042          -1.62373
9      5.538042          -1.62373
10     5.538042          -1.62373
# bookban_df[bookban_df$state != "WI", ]

Data Cleaning

bookban_df |> filter(year > 2004) |> dim()
[1] 515  17
bookban_df[bookban_df$year > 2004, ] |> dim()
[1] 526  17

What happened?

  • we have missing values in the **year* column

count(): result in a new dataframe with the distribution of the values that a given column takes on

bookban_df |> count(year)
   year   n
1  2000  71
2  2001 111
3  2002  52
4  2003  88
5  2004  83
6  2005  52
7  2006  54
8  2007  38
9  2008 122
10 2009 197
11 2010  52
12   NA  11