```{r, echo=FALSE, include=FALSE}
# Download images
imgs <- c('data-structures-overview.png')

for (i in imgs) {
  if(!file.exists(i)){
  download.file(url = paste0('https://raw.githubusercontent.com/anyone-can-cook/rclass1/master/lectures/intro_to_r/', i), 
                destfile = i,
                mode = 'wb')
  }
}
```

# Introduction

## Libraries we will use

**Load the packages** we will use by running this code chunk:

```{r, message=FALSE}
library(tidyverse)
library(haven)
library(labelled)
library(lubridate)
```

**If package not yet installed**, then must install before you load. Install in "console" rather than .Rmd file:

-   Generic syntax: `install.packages("package_name")`
-   Install "tidyverse": `install.packages("tidyverse")`

**Note**: When we load package, name of package is not in quotes; but when we install package, name of package is in quotes:

-   `install.packages("tidyverse")`
-   `library(tidyverse)`

## Dataset we will use

Recall from Into the tidyverse lecture, we used the data set `wwlist`. This data set represents "prospects" purchased by Western Washington U.

- Essentially, universities identify/target "prospect" by buying student lists from College Board/ACT (e.g., $.50+ per prospect).  
- Data universities purchase on students includes their contact info (e.g., address, email), academics (e.g., GPA, AP coursework), socioeconomic and demographic characteristics.  
- Universities choose which prospective students' information to purchase by filtering on criteria like their zip code, GPA, test score range, etc. 

```{r}
rm(list = ls()) # remove all objects

load(url("https://github.com/anyone-can-cook/rclass1/raw/master/data/prospect_list/wwlist_merged.RData"))
```

# Attributes and augmented vectors

## Review data structures: Vectors

\medskip

**Two types of vectors:**

1.  **Atomic vectors**
2.  **Lists**

\medskip

![Overview of data structures (Grolemund and Wickham, 2018)](data-structures-overview.png){width="60%"}

### Atomic vectors

\medskip An **atomic vector** is a collection of values

-   Each value in an atomic vector is an **element**
-   All elements within vector must have the same **data type**

```{r}
(a <- c(1,2,3)) # parentheses () assign and print object in one step
length(a) # length = number of elements
typeof(a) # numeric atomic vector, type=double
str(a) # investigate structure of object
```

Can assign **names** to vector elements, creating a **named atomic vector**

```{r}
(b <- c(v1=1,v2=2,v3=3))
length(b) 
typeof(b) 
str(b) 
```

### Lists

\medskip

-   Like atomic vectors, **lists** are objects that contain **elements**
-   However, **data type** can differ across elements within a list
    -   E.g., an element of a list can be another list

```{r}
list_a <- list(1,2,"apple")
typeof(list_a)
length(list_a)
str(list_a)

list_b <- list(1, c("apple", "orange"), list(1, 2))
length(list_b)
str(list_b)
```

Like atomic vectors, elements within a list can be named, thereby creating a **named list**

```{r}
# not named
str(list_b) 

# named
list_c <- list(v1=1, v2=c("apple", "orange"), v3=list(1, 2, 3))
str(list_c) 
```

### Data frames

\medskip A **data frame** is a list with the following characteristics:

-   All the elements must be **vectors** with the same **length**
-   Data frames are **augmented lists** because they have additional **attributes**

```{r}
# a regular list
(list_d <- list(col_a = c(1,2,3), col_b = c(4,5,6), col_c = c(7,8,9)))
typeof(list_d)
attributes(list_d)
```

```{r}
# a data frame
(df_a <- data.frame(col_a = c(1,2,3), col_b = c(4,5,6), col_c = c(7,8,9)))
typeof(df_a)
attributes(df_a)
```

## Converting between types

Functions for converting between types:

-   `as.logical()`: Convert to `logical`
-   `as.numeric()`: Convert to `numeric`
-   `as.integer()`: Convert to `integer`
-   `as.character()`: Convert to `character`
-   `as.list()`: Convert to `list`
-   `as.data.frame()`: Convert to `data.frame`

_Before converting between vectors types, it is important to know that the most complex type always wins._ 

```{r}
typeof(c(TRUE, 3L))
#> [1] "integer"
typeof(c(2L, 4.5))
#> [1] "double"
typeof(c(2.5, "j"))
#> [1] "character"
```

_Credit:_ [R for Data Science, section on Coercion](https://r4ds.had.co.nz/vectors.html#coercion)

<br>

**Convert to `logical` using `as.logical()`**

Character vector coerced to logical vector:

```{r}
# Only "TRUE"/"FALSE", "True"/"False", "T"/"F", "true"/"false" are able to be coerced to logical type
as.logical(c("TRUE", "FALSE", "True", "False", "true", "false", "T", "F", "t", "f", ""))
```

Numeric vector coerced to logical vector:

```{r}
# 0 is treated as FALSE, while all other numeric values are treated as TRUE
as.logical(c(0, 0.0, 1, -1, 20, 5.5))
```

<br>

**Convert to `numeric` using `as.numeric()`**

Logical vector coerced to numeric vector:

```{r}
# FALSE is mapped to 0 and TRUE is mapped to 1
as.numeric(c(FALSE, TRUE))
```

Character vector coerced to numeric vector:

```{r, warning = FALSE}
# Strings containing numeric values can be coerced to numeric (leading 0's are dropped) 
# All other characters become NA
as.numeric(c("0", "007", "2.5", "abc", "."))
```

<br>

**Convert to `integer` using `as.integer()`**

Logical vector coerced to integer vector:

```{r}
# FALSE is mapped to 0 and TRUE is mapped to 1
as.integer(c(FALSE, TRUE))
```

Character vector coerced to integer vector:

```{r, warning = FALSE}
# Strings containing numeric values can be coerced to integer (leading 0's are dropped, decimals are truncated) 
# All other characters become NA
as.integer(c("0", "007", "2.5", "abc", "."))
```

Numeric vector coerced to integer vector:

```{r, warning = FALSE}
# All decimal places are truncated
as.integer(c(0, 2.1, 10.5, 8.8, -1.8))
```

<br>

**Convert to `character` using `as.character()`**

Logical vector coerced to character vector:

```{r}
as.character(c(FALSE, TRUE))
```

Numeric vector coerced to character vector:

```{r, warning = FALSE}
as.character(c(-5, 0, 2.5))
```

Integer vector coerced to character vector:

```{r, warning = FALSE}
as.character(c(-2L, 0L, 10L))
```

<br>

**Convert to `list` using `as.list()`**

Atomic vectors coerced to list:

```{r}
# Logical vector
as.list(c(TRUE, FALSE))

# Character vector
as.list(c("a", "b", "c"))

# Numeric vector
as.list(1:3)
```

<br>

**Convert to `data.frame` using `as.data.frame()`**

Lists coerced to dataframe:

```{r}
# Create a list
l <- list(A = c("x", "y", "z"), B = c(1, 2, 3))
str(l)

# Convert to class `data.frame`
df <- as.data.frame(l, stringsAsFactors = F)
str(df)
```

<details>

<summary>**Example**: Practical example of converting type</summary>

When working with data, it may be helpful to label values for certain variables. Data files often come with a code book that defines how values are coded. Let's look at an example of labeling values and how converting data type may come into play.

We'll look at the `FIPS` variable from the [Integrated Postsecondary Education Data System (IPEDS)](https://nces.ed.gov/ipeds/use-the-data) data. The [state FIPS code](https://en.wikipedia.org/wiki/Federal_Information_Processing_Standard_state_code#FIPS_state_codes) is a numeric code that identifies a state. For example, `1` is the FIPS code for `Alabama`, `2` is the FIPS code for `Alaska`, etc. We'll want to label each numeric value in the `FIPS` column with the corresponding state name.

```{r}
# Library for labeling variables and values in a dataframe
library(labelled)

# Read in IPEDS data and codebook
ipeds_df <- read.csv('https://raw.githubusercontent.com/cyouh95/recruiting-chapter/master/data/ipeds_hd2017.csv', header = TRUE, na.strings=c('', 'NA'), stringsAsFactors = F)
ipeds_values <- read.csv('https://raw.githubusercontent.com/cyouh95/recruiting-chapter/master/data/ipeds_hd2017_values.csv', header = TRUE, na.strings=c('', 'NA'), stringsAsFactors = F)

# The codebook defines how variables are coded, such as STABBR, FIPS, and other variables
head(ipeds_values)

# Filter codebook for just the values for the FIPS variable
fips_values <- ipeds_values %>% filter(varname == 'FIPS') %>% select(varname, codevalue, valuelabel)
head(fips_values)
```

<br> When we read in the data from the CSV files, R automatically tries to determine the data type of each variable. As seen below, the `FIPS` column from the `ipeds_df` that we want to label is of type `integer`, while the `codevalue` column from the codebook is of type `character` (since not all values are numeric):

```{r}
# Type of `FIPS` column
str(ipeds_df$FIPS)

# Type of `codevalue` column
str(fips_values$codevalue)
```

<br> This discrepancy becomes a problem when we try to label the value using the `labelled` library:

```{r, eval=F}
# Error: `x` and `labels` must be same type
val_label(ipeds_df$FIPS, fips_values[1, 'codevalue']) <- fips_values[1, 'valuelabel']
```

<br> To resolve this, we can use `as.integer()` to convert the `codevalue` from `character` type to `integer` before trying to label the value:

```{r}
# This now works
val_label(ipeds_df$FIPS, as.integer(fips_values[1, 'codevalue'])) <- fips_values[1, 'valuelabel']

# Check value labels
val_labels(ipeds_df$FIPS)

# We can use as.integer() to convert the entire vector (ie. codevalue column) to integer
fips_values$codevalue <- as.integer(fips_values$codevalue)

# Type of `codevalue` column
str(fips_values$codevalue)

# Use loop to label the rest of the values
for (i in 1:nrow(fips_values)) {
  val_label(ipeds_df$FIPS, fips_values[i, 'codevalue']) <- fips_values[i, 'valuelabel']
}

# Check value labels
val_labels(ipeds_df$FIPS)
```

</details>


## Atomic vectors versus augmented vectors

**Atomic vectors** \[our focus so far\]

-   I think of atomic vectors as "just the data"
-   Atomic vectors are the building blocks for augmented vectors

\medskip

**Augmented vectors**

-   **Augmented vectors** are atomic vectors with additional **attributes** attached

**Attributes**

-   **Attributes** are additional "metadata" that can be attached to any object (e.g., vector or list)

Example: Variables of a dataset

-   A data frame is a list
-   Each element in the list is a variable, which consists of:
    -   Atomic vector ("just the data")
    -   Any attributes we want to attach to each element/variable
-   Variable **name**, an attribute of the data frame object

Other examples of attributes in R

-   Value **labels**: Character labels (e.g., "Charter School") attached to numeric values
-   Object **class**: Specifies how object is treated by object oriented programming language

**Main takeaway**:

-   **Augmented vectors** are **atomic vectors** (just the data) with additional **attributes** attached

## Attributes and functions to identify/modify attributes

Description of attributes from Grolemund and Wickham 20.6

-   "Any vector can contain arbitrary additional **metadata** through its **attributes**"
-   "You can think of **attributes** as named list of vectors that can be attached to any object"

Functions to identify and modify attributes

-   `attributes()` function to describe all attributes of an object
-   `attr()` to see individual attribute of an object or set/change an individual attribute of an object

### Describe all attributes of an object

```{r, eval=FALSE}
# pull up help file for the attributes() function
?attributes
```

Attributes of a **named atomic vector**:

```{r}
# create named atomic vector
(vector1 <- c(a = 1, b = 2, c = 3, d = 4))
attributes(vector1)

attributes(vector1) %>% str() # a named list of vectors!

# remove all attributes from the object
attributes(vector1) <- NULL
vector1
attributes(vector1)
```

### Attributes of a variable in a data frame

\medskip

**Accessing variable using `[[]]` subset operator**

-   Recall `object_name[["element_name"]]` accesses contents of the element
-   If object is a data frame, `df_name[["var_name"]]` accesses contents of variable
    -   For simple vars like `firstgen`, syntax yields an atomic vector ("just the data")
-   Shorthand syntax for `df_name[["var_name"]]` is `df_name$var_name`

```{r}
str(wwlist[["firstgen"]])
attributes(wwlist[["firstgen"]])

str(wwlist$firstgen) # same same
attributes(wwlist$firstgen)
```

**Accessing variable using `[]` subset operator**

-   `object_name["element_name"]` creates object of same type as `object_name`
-   If object is a data frame, `df_name["var_name"]` returns a data frame containing just the `var_name` column

```{r, results="hide"}
str(wwlist["firstgen"])
attributes(wwlist["firstgen"])
```

### Attributes of lists and data frames

\medskip

Attributes of a **named list**:

```{r}
list2 <- list(col_a = c(1,2,3), col_b = c(4,5,6))
str(list2)
attributes(list2)
```

Note that the `names` attribute is an attribute of the list, not an attribute of the elements within the list (which are atomic vectors)

```{r}
list2[['col_a']] # the element named 'col_a'
str(list2[['col_a']]) # structure of the element named 'col_a'
attributes(list2[['col_a']]) # attributes of element named 'col_a'
```

Attributes of a **data frame**:

```{r}
list3 <- data.frame(col_a = c(1,2,3), col_b = c(4,5,6))
str(list3)
attributes(list3)
```

Note: attributes `names`, `class` and `row.names` are attributes of the data frame

-   they are not attributes of the elements (variables) within the data frame, which are atomic vectors (i.e., just the data)

```{r}
str(list3[['col_a']]) # structure of the element named 'col_a'
attributes(list3[['col_a']]) # attributes of element named 'col_a'
```

### attr() function to get or set specific attributes of an object

```{r, eval=FALSE, include=FALSE}
?attr
```

Syntax

-   Get: `attr(x, which, exact = FALSE)`
-   Set: `attr(x, which) <- value`

Arguments

-   `x`: an object whose attributes are to be accessed
-   `which`: a non-empty character string specifying which attribute is to be accessed
-   `exact` (logical): should `which` be matched exactly? default is `exact = FALSE`
-   `value`: an object, new value of attribute, or `NULL` to remove attribute

\medskip

**Using `attr()` to get specific attribute of an object**

```{r}
vector1 <- c(a = 1, b= 2, c= 3, d = 4)
attributes(vector1)
attr(x=vector1, which = "names", exact = FALSE)
attr(vector1, "names")
attr(vector1, "name") # we don't provide exact name of attribute
attr(vector1, "name", exact = TRUE) # don't provide exact name of attribute
```

**Using `attr()` to set specific attribute of an object** (output omitted)

```{r, results = "hide"}
(vector1 <- c(a = 1, b= 2, c= 3, d = 4))
attributes(vector1) # see all attributes

attr(x=vector1, which = "greeting") <- "Hi!" # create new attribute
attr(x=vector1, which = "greeting") # see attribute

attr(vector1, "farewell") <- "Bye!" # create attribute

attr(x=vector1, which = "names") # see names attribute
attr(x=vector1, which = "names") <- NULL # delete names attribute

attributes(vector1) # see all attributes
```

### attr() function on data frames

\medskip

**Using `wwlist`, create data frame with three variables**

```{r, results = "hide"}
wwlist_small <- wwlist[1:25, ] %>% select(hs_state,firstgen,med_inc_zip)
str(wwlist_small)
attributes(wwlist_small)
attributes(wwlist_small) %>% str()
```

**Get/set attribute of a data frame**

```{r, results = "hide"}
#get/examine names attribute
attr(x=wwlist_small, which = "names") 
str(attr(x=wwlist_small, which = "names")) # names attribute is character atomic vector, length=3
#add new attribute to data frame
attr(x=wwlist_small, which = "new_attribute") <- "contents of new attribute"
attributes(wwlist_small)
```

**Get/set attribute of a variable in data frame**

```{r, results = "hide"}
str(wwlist_small$med_inc_zip)
attributes(wwlist_small$med_inc_zip)
#create attribute for variable med_inc_zip
attr(wwlist_small$med_inc_zip, "inc attribute") <- "inc attribute contents"
#investigate attribute for variable med_inc_zip
attributes(wwlist_small$med_inc_zip)
str(wwlist_small$med_inc_zip)
attr(wwlist_small$med_inc_zip, "inc attribute")
```

### Why add attributes to data frame or variables of data frame?

Pedagogical reasons

-   Important to know how you can apply `attributes()` and `attr()` to data frames and to variables within data frames

\medskip

Example practical application: interactive dashboards

-   When creating "dashboard" you might want to add "tooltips"
    -   "Tooltip" is a message that appears when cursor is positioned over an icon
    -   The text in the tooltip is the contents of an attribute
-   Example dashboard: [LINK](https://jkcf.shinyapps.io/dashboard/)

### Student exercise

1.  Using `wwlist`, create data frame of 30 observations with three variables: `state`, `zip5`, `pop_total_zip`

2.  Return all attributes of this new data frame using `attributes()`. Then, get the `names` attribute of the data frame using `attr()`.

3.  Add a new attribute to the data frame called `attribute_data` whose content is `"new attribute of data"`. Then, return all attributes of the data frame as well as get the value of the newly created `attribute_data`.

4.  Return the attributes of the variable `pop_total_zip` in the data frame.

5.  Add a new attribute to the variable `pop_total_zip` called `attribute_variable` whose content is `"new attribute of variable"`. Then, return all attributes of the variable as well as get the value of the newly created `attribute_variable`.

<details>

<summary><strong>Solutions to student exercise</strong></summary>

```{r, results = "hide"}
# Part 1
wwlist_exercise <- wwlist[1:30, ] %>% select(state, zip5, pop_total_zip)

# Part 2
attributes(wwlist_exercise)
attr(x=wwlist_exercise, which = "names") 

# Part 3
attr(x=wwlist_exercise, which = "attribute_data") <- "new attribute of data"

attributes(wwlist_exercise)
attr(wwlist_exercise, which ="attribute_data")

# Part 4
attributes(wwlist_exercise$pop_total_zip)

# Part 5
attr(wwlist_exercise$pop_total_zip, "attribute_variable") <- "new attribute of variable"

attributes(wwlist_exercise$pop_total_zip)
attr(wwlist_exercise$pop_total_zip, "attribute_variable")
```

</details>

<br>

# Object class

\medskip

Every object in R has a **class**

-   Class is an **attribute** of an object
-   Object class controls how functions work and defines the rules for how objects can be treated by object oriented programming language
    -   E.g., which functions you can apply to object of a particular class
    -   E.g., what the function does to one object class, what it does to another object class

You can use the `class()` function to identify object class:

```{r}
(vector2 <- c(a = 1, b= 2, c= 3, d = 4))
typeof(vector2)
class(vector2)
```

When I encounter a new object I often investigate object by applying `typeof()`, `class()`, and `attributes()` functions:

```{r}
typeof(vector2)
class(vector2)
attributes(vector2)
```

## Why is object class important?


\medskip

Specific functions usually work with only particular **classes** of objects

-   "Date" functions usually only work on objects with a date class
-   "String" functions usually only work on objects with a character class
-   Functions that do mathematical computation usually work on objects with a numeric class

### Class and object-oriented programming

\medskip

R is an object-oriented programming language

\medskip

Definition of object oriented programming from this [LINK](https://www.webopedia.com/TERM/O/object_oriented_programming_OOP.html)

\medskip

> "Object-oriented programming (OOP) refers to a type of computer programming in which programmers define not only the data type of a data structure, but also the types of operations (functions) that can be applied to the data structure."

\medskip

Object **class** is fundamental to object oriented programming because:

-   Object class determines which functions can be applied to the object
-   Object class also determines what those functions do to the object
    -   E.g., a specific function might do one thing to objects of **class** A and another thing to objects of **class** B
    -   What a function does to objects of different class is determined by whoever wrote the function

\medskip

Many different object classes exist in R

-   You can also create your own classes
    -   Example: the `labelled` class is an object class created by Hadley Wickham when he created the `haven` package
-   In this course we will work with classes that have been created by others

### Functions care about object **class**, not object **type**

\medskip

**Example**: `sum()` applies to **numeric**, **logical**, or **complex** class objects

```{r, eval=FALSE, include=FALSE}
?sum
```

Apply `sum()` to object with class = **logical**:

```{r}
x <- c(TRUE, FALSE, NA, TRUE)
typeof(x)
class(x)
sum(x, na.rm = TRUE)
```

Apply `sum()` to object with class = **numeric**:

```{r}
typeof(wwlist$med_inc_zip) 
class(wwlist$med_inc_zip) 
wwlist$med_inc_zip[1:5]
sum(wwlist$med_inc_zip[1:5], na.rm = TRUE) 
```

What happens when we try to apply `sum()` to an object with class = **character**?

```{r, eval=FALSE}
typeof(wwlist$hs_city)
class(wwlist$hs_city)
wwlist$hs_city[1:5]
sum(wwlist$hs_city[1:5], na.rm = TRUE) 
```

**Example**: `year()` from `lubridate` package applies to date-time objects

```{r, include=FALSE, results = "hide"}
library(lubridate)
```

```{r, eval=FALSE, include=FALSE}
?year
```

Apply `year()` to object with class = **Date**:

```{r}
wwlist$receive_date[1:5]
typeof(wwlist$receive_date)
class(wwlist$receive_date) 
year(wwlist$receive_date[1:5])
```

What happens when we try to apply `year()` to an object with class = **numeric**?

```{r, eval=FALSE}
typeof(wwlist$med_inc_zip) 
class(wwlist$med_inc_zip) 
year(wwlist$med_inc_zip[1:10]) 
```

**Example**: `tolower()` applies to **character** class objects

-   Syntax: `tolower(x)`
-   `x` is "a character vector, or an object that can be coerced to character by `as.character()`"

Most string functions are intended to apply to objects with a **character** class

-   **type** = character
-   **class** = character

```{r, eval=FALSE, include=FALSE}
?tolower
```

Apply `tolower()` to object with class = **character**:

```{r}
str(wwlist$hs_city)
typeof(wwlist$hs_city)
class(wwlist$hs_city)

wwlist$hs_city[1:6]
tolower(wwlist$hs_city[1:6])
```


# Class == factor

**Recoding variable `ethn_code` from data frame `wwlist`**

Let's first recode the `ethn_code` variable:

```{r, results= "hide"}
wwlist <- wwlist %>%  
  mutate(ethn_code = 
    recode(ethn_code,
      "american indian or alaska native" = "nativeam",
      "asian or native hawaiian or other pacific islander" = "api",
      "black or african american" = "black",
      "cuban" = "latinx",
      "mexican/mexican american" = "latinx",
      "not reported" = "not_reported",
      "other-2 or more" = "multirace",
      "other spanish/hispanic" = "latinx",
      "puerto rican" = "latinx",
      "white" = "white"          
    )
  )

str(wwlist$ethn_code)
wwlist %>% count(ethn_code)
```

## Factors

**Factors** are an object *class* used to display categorical data (e.g., marital status)

-   A factor is an **augmented vector** built by attaching a **levels** attribute to an (atomic) integer vectors

Usually, we would prefer a categorical variable (e.g., race, school type) to be a factor variable rather than a character variable

-   So far in the course I have made all categorical variables character variables because we have not introduced factors yet

**Create factor version of character variable `ethn_code` using base R `factor()` function**:

```{r}
str(wwlist$ethn_code)
class(wwlist$ethn_code)

# create factor var; tidyverse approach
wwlist <- wwlist %>% mutate(ethn_code_fac = factor(ethn_code)) 
#wwlist$ethn_code_fac <- factor(wwlist$ethn_code) # base r approach

str(wwlist$ethn_code)
str(wwlist$ethn_code_fac)
```

**Character variable `ethn_code`**:

```{r}
typeof(wwlist$ethn_code)
class(wwlist$ethn_code)
attributes(wwlist$ethn_code)
str(wwlist$ethn_code)
```

**Factor variable `ethn_code_fac`**:

```{r}
typeof(wwlist$ethn_code_fac)
class(wwlist$ethn_code_fac)
attributes(wwlist$ethn_code_fac)
str(wwlist$ethn_code_fac)
```

## Working with factor variables

Main things to note about variable `ethn_code_fac`

-   **type** = integer
-   **class** = factor, because the variable has a **levels** attribute
-   Underlying data are integers, but the values of the **levels** attribute is what's displayed:

```{r}
# Print first few obs of ethn_code_fac
wwlist$ethn_code_fac[1:5]

# Print count for each category in ethn_code_fac
wwlist %>% count(ethn_code_fac)
```

Apply `as.integer()` to display underlying integer values of factor the variable

```{r, eval=FALSE, include=FALSE}
?as.integer
```

Investigate `as.integer()` function:

```{r}
typeof(wwlist$ethn_code_fac)
class(wwlist$ethn_code_fac)

typeof(as.integer(wwlist$ethn_code_fac))
class(as.integer(wwlist$ethn_code_fac))
```

Display underlying integer values of variable `ethn_code_fac`:

```{r}
wwlist %>% count(as.integer(ethn_code_fac))
```

\medskip

Refer to categories of a factor (e.g., when filtering obs) using values of **levels** attribute rather than underlying values of variable

-   Values of **levels** attribute for `ethn_code_fac` (output omitted)

```{r, results='hide'}
attributes(wwlist$ethn_code_fac)
```

\medskip

**Example**: Count the number of prospects in `wwlist` who identify as "white"

```{r}
# referring to variable value; this doesn't work
wwlist %>% filter(ethn_code_fac==7) %>% count() 

#referring to value of level attribute; this works
wwlist %>% filter(ethn_code_fac=="white") %>% count()
```

**Example**: Count the number of prospects in `wwlist` who identify as "white"

-   To refer to underlying integer values, apply `as.integer()` function to factor variable

```{r}
attributes(wwlist$ethn_code_fac)
wwlist %>% filter(as.integer(ethn_code_fac)==7) %>% count
```

## How to identify the variable values associated with factor levels

Create a factor version of the character variable `psat_range`

```{r, results="hide", warning = FALSE}
wwlist %>% count(psat_range)
wwlist <- wwlist %>% mutate(psat_range_fac = factor(psat_range))
wwlist %>% count(psat_range_fac)
attributes(wwlist$psat_range_fac)
```

Investigate values associated with factor levels using `levels()` and `nlevels()`

```{r, results="hide"}
levels(wwlist$psat_range_fac) #starts at 1
nlevels(wwlist$psat_range_fac) #7 levels total
levels(wwlist$psat_range_fac)[1:3] #prints levels 1-3
```

Once values associated with factor levels are known:

-   Can filter based on underlying integer values using `as.integer()`

```{r}
wwlist %>% filter(as.integer(psat_range_fac)==4) %>% count()
```

-   Or filter based on value of factor **levels**

```{r}
wwlist %>% filter(psat_range=="1270-1520") %>% count()
```


### Student exercise working with factors

1.  After running the code below, use `typeof()`, `class()`, `str()`, and `attributes()` functions to check the new variable `receive_year`\
2.  Create a factor variable from the input variable `receive_year` and name it `receive_year_fac`\
3.  Run the same functions (`typeof()`, `class()`, etc.) from the first question using the new variable you created\
4.  Get a count of `receive_year_fac`. (**hint:** you could also run this in the console to see values associated with each factor)

Run this code to create a year variable from the input variable `receive_date`:

```{r, results="hide", message=FALSE}
# wwlist %>% glimpse()

library(lubridate) # load library if you haven't already
wwlist <- wwlist %>%
  mutate(receive_year = year(receive_date)) # create year variable with lubridate

# Check variable
wwlist %>% 
  count(receive_year)

wwlist %>%
  group_by(receive_year) %>% 
  count(receive_date)

```

<details>

<summary><strong>Solutions to student exercise working with factors `%>%`</strong></summary>

1.  After running the code below, use `typeof()`, `class()`, `str()`, and `attributes()` functions to check the new variable `receive_year`

```{r}
typeof(wwlist$receive_year)
class(wwlist$receive_year)
str(wwlist$receive_year)
attributes(wwlist$receive_year) 
```

2.  Create a factor variable from the input variable `receive_year` and name it `receive_year_fac`\

```{r}
# create factor var; tidyverse approach
wwlist <- wwlist %>%
  mutate(receive_year_fac = factor(receive_year))  

```

3.  Run the same functions (`typeof()`, `class()`, etc.) from the first question using the new variable you created\

```{r}
typeof(wwlist$receive_year_fac)
class(wwlist$receive_year_fac)
str(wwlist$receive_year_fac)
attributes(wwlist$receive_year_fac)   
```

4.  Get a count of `receive_year_fac`. (**hint:** you could also run this in the console to see values associated with each factor)

```{r}
wwlist %>%
  count(receive_year_fac)
```

</details>

<br>

# Class == labelled

## Data we will use to introduce `labelled` class

High school longitudinal surveys from National Center for Education Statistics (NCES)

-   Follow U.S. students from high school through college, labor market

\medskip

We will be working with [High School Longitudinal Study of 2009 (HSLS:09)](https://nces.ed.gov/surveys/hsls09/index.asp)

-   Follows 9th graders from 2009
-   Data collection waves
    -   Base Year (2009)
    -   First Follow-up (2012)
    -   2013 Update (2013)
    -   High School Transcripts (2013-2014)
    -   Second Follow-up (2016)

## Using `haven` package to read SAS/SPSS/Stata datasets into R

[`haven`](https://haven.tidyverse.org/), which is part of **tidyverse**, "enables R to read and write various data formats" from the following statistical packages:

-   SAS
-   SPSS
-   Stata

\medskip

If you haven't installed `haven` run the following code in your console.

-   Generic syntax: `install.packages("package_name")`
-   Install "haven": `install.packages("haven")`

Make sure to load `haven` (e.g., `library(haven)`)

When using `haven` to read data, resulting R objects have these characteristics:

-   Data frames are **tibbles**, Tidyverse's preferred **class** of data frames
-   Transform variables with "value labels" into the `labelled()` class
    -   `labelled` is an object class, just like `factor` is an object class
    -   `labelled` is an object **class** created by folks who created `haven` package
    -   `labelled` and `factor` classes are both viable alternatives for categorical variables
    -   Helpful description of `labelled` class [HERE](https://haven.tidyverse.org/articles/semantics.html)
-   Dates and times converted to R date/time classes
-   Character vectors not converted to factors

Use `read_dta()` function from `haven` package to import Stata dataset into R

```{r, results="hide"}
hsls <- read_dta(file="https://github.com/ozanj/rclass/raw/master/data/hsls/hsls_stu_small.dta")
```

**Must** run this code chunk; permanently changes uppercase variable names to lowercase

```{r, results="hide"}
names(hsls)
names(hsls) <- tolower(names(hsls)) # convert names to lowercase
names(hsls) # names now lowercase

str(hsls) # ugh
```

**Investigate variable `s3classes` from data frame `hsls`**

-   Identifies whether respondent taking postsecondary classes as of 11/1/2013

```{r, results="hide"}
typeof(hsls$s3classes)
class(hsls$s3classes)
str(hsls$s3classes)
```

**Investigate attributes of `s3classes`**

```{r, results="hide"}
attributes(hsls$s3classes) # all attributes

#specific attributes: using syntax: attr(x, which, exact = FALSE)
attr(x=hsls$s3classes, which = "label") # label attribute
attr(x=hsls$s3classes, which = "labels") # labels attribute
```

## What is object class = `labelled`?

\medskip

**Variable labels** are labels attached to a specific variable (e.g., marital status) **Value labels** \[in Stata\] are labels attached to specific values of a variable, e.g.:

-   Var value `1` attached to value label "married", `2`="single", `3`="divorced"

\medskip

`labelled` is object class for importing vars with **value labels** from SAS/SPSS/Stata

-   `labelled` object class created by `haven` package
-   Characteristics of variables in R data frame with `class==labelled`:
    -   Data `type` can be numeric(double) or character
    -   To see `value labels` associated with each value:
        -   `attr(df_name$var_name,"labels")`
        -   E.g., `attr(hsls$s3classes,"labels")`

Investigate the attributes of `hsls$s3classes`

```{r, results="hide"}
typeof(hsls$s3classes)
class(hsls$s3classes)
str(hsls$s3classes)
attributes(hsls$s3classes)
```

Use `attr(object_name,"attribute_name")` to refer to each attribute

```{r, results="hide"}
attr(hsls$s3classes,"label")
attr(hsls$s3classes,"format.stata")
attr(hsls$s3classes,"class")
attr(hsls$s3classes,"labels")
```

## `labelled` package

Purpose of the `labelled` package is to work with data imported from SPSS/Stata/SAS using the `haven` package

-   `labelled` package contains functions to work with objects that have `labelled` class
-   From package documentation:
    -   "purpose of the `labelled` package is to provide functions to manipulate *metadata* as variable labels, value labels and defined missing values using the `labelled` class and the `label` attribute introduced in `haven` package."
-   More info on the `labelled` package: [LINK](https://cran.r-project.org/web/packages/labelled/vignettes/intro_labelled.html)

Functions in `labelled` package

-   [Full list](https://www.rdocumentation.org/packages/labelled/versions/1.1.0)

```{r, eval=FALSE, include=FALSE}
?labelled
```

## Get variable and value labels

**Get variable labels** using `var_label()`

```{r}
hsls %>% select(s3classes) %>% var_label()
```

\medskip

**Get value labels** using `val_labels()`

```{r}
hsls %>% select(s3classes) %>% val_labels()
```

## Working with `labelled` class data

\medskip

Create frequency tables with `labelled` class variables using `count()`

-   when running individual code chunks, default setting is to show variable **values** not **value labels**

```{r}
hsls %>% count(s3classes)
```

\medskip

when running individual code chunks, to make frequency table show **value labels** add `%>% as_factor()` to pipe

-   `as_factor()` is function from `haven` that converts an object to a factor

```{r}
hsls %>% count(s3classes) %>% as_factor()
```

To isolate values of `labelled` class variables in `filter()` function:

-   Refer to variable **value**, not the **value label**

**Task**

-   How many observations in var `s3classes` associated with "Unit non-response"
-   How many observations in var `s3classes` associated with "Yes"

General steps to follow:

1.  Investigate object
2.  Use `filter()` to isolate desired observations

Investigate object

```{r, results="hide"}
class(hsls$s3classes)
hsls %>% select(s3classes) %>% var_label() #show variable label
hsls %>% select(s3classes) %>% val_labels() #show value label

hsls %>% count(s3classes) # freq table, values
hsls %>% count(s3classes) %>% as_factor() # freq table, value labels
```

Filter specific values

```{r, results="hide"}
hsls %>% filter(s3classes==-8) %>% count() # -8 = unit non-response
hsls %>% filter(s3classes==1) %>% count() # 1 = yes
```

## Set variable and value labels

\medskip

**Set variable labels** using `var_label()` or `set_variable_labels()`

```{r, eval=F}
# Set one variable label
var_label(df_name$var_name) <- 'variable label'

# Set multiple variable labels
df_name <- df_name %>%
  set_variable_labels(
    var_name_1 = 'variable label 1',
    var_name_2 = 'variable label 2',
    var_name_3 = 'variable label 3'
  )
```

\medskip

**Set value labels** using `val_label()` or `set_value_labels()`

```{r, eval=F}
# Set one value label
val_label(df_name$var_name, 'variable_value') <- 'value_label'

# Set multiple value labels
df_name <- df_name %>%
  set_value_labels(
    var_name_1 = c('value_label_1' = 'variable_value_1',
                   'value_label_2' = 'variable_value_2',
    var_name_2 = c('value_label_3' = 'variable_value_3',
                   'value_label_4' = 'variable_value_4')
  )
```

### Create example data frame

```{r}
df <- tribble(
  ~id, ~edu, ~sch,
  #--|--|----
  1, 2, 2,
  2, 1, 1,
  3, 3, 2,
  4, 4, 2,
  5, 1, 2
)
df
str(df)
```

### Set variable labels

Use `set_variable_labels()` or `var_label()` to manually set variable labels

```{r}
str(df$sch)
var_label(df$sch)

# Using set_variable_labels()
df <- df %>%
  set_variable_labels(
    id = "Unique identification number",
    edu = "Education level"
  )

# Using var_label()
var_label(df$sch) <- 'Type of school attending'

str(df$sch)
var_label(df$sch)
```

### Set value labels

Use `set_value_labels()` or `val_label()` to manually set value labels

```{r}
val_labels(df$sch)

# Using set_value_labels()
df <- df %>%
  set_value_labels(
    edu = c('High School' = 1,
            'AA degree' = 2,
            'BA degree' = 3,
            'MA or higher' = 4),
    sch = c('Private' = 1))

# Using val_label()
val_label(df$sch, 2) <- 'Public'

str(df$sch)
val_labels(df$sch)
```

### View the set variable and value labels

```{r}
# View variable and value labels using attributes()
attributes(df$sch)

# View variable label
var_label(df$sch)
attr(df$sch, 'label')

# View value labels
val_labels(df$sch)
attr(df$sch, 'labels')
```

### `labelled` student exercise

1.  Get variable and value labels of the variable `s3hs` in the `hsls` data frame
2.  Get a count of the variable `s3hs` showing the values and the value labels (**hint**: use `as_factor()`)
3.  Get a count of the rows whose value for `s3hs` is associated with "Missing" (**hint**: use `filter()`)
4.  Get a count of the rows whose value for `s3hs` is associated with "Missing" or "Unit non-response"
5.  Add variable label for `pop_asian_zip` & `pop_asian_state` in data frame `wwlist`
6.  Add value labels for `ethn_code` in data frame `wwlist`

<details>

<summary><strong>`labelled` student exercise solutions</strong></summary>

1.  Get variable and value labels of the variable `s3hs` in the `hsls` data frame

```{r}
hsls %>% 
  select(s3hs) %>% 
  var_label() 

hsls %>% 
  select(s3hs) %>% 
  val_labels()
```

2.  Get a count of the variable `s3hs` showing the values and the value labels (**hint**: use `as_factor()`)

```{r}
hsls %>% 
  count(s3hs) 

hsls %>% 
  count(s3hs) %>% 
  as_factor() 

```

3.  Get a count of the rows whose value for `s3hs` is associated with "Missing" (**hint**: use `filter()`)

```{r}
hsls %>%
  filter(s3hs== -9) %>% 
  count()
```

4.  Get a count of the rows whose value for `s3hs` is associated with "Missing" or "Unit non-response"

```{r}
hsls %>%
  filter(s3hs== -9 | s3hs== -8) %>% 
  count()
```

5.  Add variable label for `pop_asian_zip` & `pop_asian_state` in data frame `wwlist`

```{r}
# variable labels
wwlist %>% select(pop_asian_zip, pop_asian_state) %>% var_label()

# set variable labels
wwlist <- wwlist %>% 
  set_variable_labels(
    pop_asian_zip = "total asian population in zip",
    pop_asian_state ="total asian population in state"
  )

# attribute of variable
attributes(wwlist$pop_asian_zip)
attributes(wwlist$pop_asian_state)
```

6.  Add value labels for `ethn_code` in data frame `wwlist`

```{r, results="hide"}
# count
wwlist %>% count(ethn_code)

# value labels
wwlist %>% select(ethn_code) %>% val_labels

# set value labels to ethn_code variable
wwlist <- wwlist %>% 
  set_value_labels(
    ethn_code = c("asian or native hawaiian or other pacific islander" = "api",
                  "black or african american" = "black",
                  "cuban or mexican/mexican american or other spanish/hispanic or puerto rican" = "latinx",
                  "other-2 or more" = "multirace",
                  "american indian or alaska native" = "nativeam",
                  "not reported" = "not_reported",
                  "white" = "white"
    )
  )
```

</details>

<br>

# Comparing class labelled to class factor

|                               | `class==labelled`    | `class==factor`  |
|-------------------------------|----------------------|------------------|
| data type                     | numeric or character | integer          |
| name of value label attribute | labels               | levels           |
| refer to data using           | variable values      | levels attribute |

\bigskip

So should you work with `class==labelled` or `class==factor`?

-   No right or wrong answer; this is a subjective decision
-   Personally, I prefer `factor` class
    -   Easier to run analysis (see section below)
-   `labelled` class
    -   Feels more suited to working with survey data variables, where there are usually several different values that represent different kinds of "missing" values. 
-   You can use both! We recommend using labelled variables, particularly for exploratory data analysis, and factor variables for analysis when using categorical variables. 

### Why factor variables are essential for analysis

<br> 

**When running analyses using categorical variables (e.g., marital status: single, married, divorced, etc.), it is best practice to use `class == factor` over `class == labelled`.** 

To demonstrate a quick example, let's load data from ELS. We will use this data set later in the graphing and ggplot lecture, so don't worry about understanding it now. 

```{r echo=FALSE}
load(file = url('https://github.com/anyone-can-cook/educ152/raw/main/data/els/output_data/els_stu.RData'))
```

Say we are interested in the relationship between having an internship or co-op job (X) (in 2005-2006) on subsequent earnings (Y) (in 2011) after controlling for standardized reading test score (as a measure for high school achievement), sex, and race/ethnicity.  

-   We have two linear models below, the first using variables that are class labelled and the second using variables that are class factor. 

<br> 

**Linear model `lm()` model using categorical variables of class `labelled`**
```{r echo=FALSE}
#class(df_els_stu_allobs$f2intern0506) 
#class(df_els_stu_allobs$f3ern2011)
#attributes(df_els_stu_allobs$bytxrstd)
attributes(df_els_stu_allobs$f1sex)
attributes(df_els_stu_allobs$f1race_v2)
```

```{r}
intern_mod1 <- lm(formula = f3ern2011 ~ f2intern0506 + bytxrstd + f1sex + f1race_v2, data = df_els_stu_allobs %>% filter(f2enroll0506==1))

summary(intern_mod1)
```

<br>

**Linear model `lm()` model using categorical variables of class `factor`**
```{r echo=FALSE}
#class(df_els_stu_allobs_fac$f2intern0506) 
#class(df_els_stu_allobs_fac$f3ern2011) #labelled + double
#attributes(df_els_stu_allobs_fac$f2enroll0506)
#attributes(df_els_stu_allobs_fac$bytxrstd) #labelled + double
attributes(df_els_stu_allobs_fac$f1sex)
attributes(df_els_stu_allobs_fac$f1race_v2)
```

```{r}
intern_mod2 <- lm(formula = f3ern2011 ~ f2intern0506 + bytxrstd + f1sex + f1race_v2, data = df_els_stu_allobs_fac %>% filter(f2enroll0506=="yes"))

summary(intern_mod2)
```

**Why does this matter?** 

Notice that for the linear model `intern_mod1`, the coefficients for categorical variables sex and race are grouped together, whereas in the second linear model `intern_mod2`, using class factor variables, coefficients are calculated separately by "category" (e.g., female, male, black, white, Asian, etc.). 


## Converting `class==labelled` to `class==factor`

The `as_factor()` function from `haven` package converts variables with `class==labelled` to `class==factor`

-   Can be used for descriptive statistics

```{r, results="hide"}
hsls %>% select(s3classes) %>% count(s3classes)
hsls %>% select(s3classes) %>% count(s3classes) %>% as_factor()
```

-   Can create object with some or all `labelled` vars converted to `factor`

```{r}
hsls_f <- as_factor(hsls, only_labelled = TRUE)
```

Let's examine this object

```{r, results="hide"}
glimpse(hsls_f)
hsls_f %>% select(s3classes,s3clglvl) %>% str()
typeof(hsls_f$s3classes)
class(hsls_f$s3classes)
attributes(hsls_f$s3classes)

hsls_f %>% select(s3classes) %>% var_label()
hsls_f %>% select(s3classes) %>% val_labels()
```

## Working with `class==factor` data

Showing factor levels associated with a factor variable

```{r}
hsls_f %>% count(s3classes)
```

Showing variable values associated with a factor variable

```{r}
hsls_f %>% count(as.integer(s3classes))
```

When sub-setting observations (e.g., filtering), refer to the `level` attribute, not the underlying variable integer value.

```{r}
hsls_f %>% filter(s3classes=="Yes") %>% count(s3classes)
```

# Appendix: Creating factor variables

## Create factors \[from string variables\]

To create a factor variable from string variable:

1.  Create a character vector containing underlying data
2.  Create a vector containing valid levels
3.  Attach levels to the data using the `factor()` function

```{r}
# Underlying data: months my fam is born
x1 <- c("Jan", "Aug", "Apr", "Mar")
# Create vector with valid levels
month_levels <- c("Jan", "Feb", "Mar", "Apr", "May", "Jun", 
  "Jul", "Aug", "Sep", "Oct", "Nov", "Dec")
# Attach levels to data
x2 <- factor(x1, levels = month_levels)
```

Note how attributes differ:

```{r}
str(x1)
str(x2)
```

Sorting also differs:

```{r}
sort(x1)
sort(x2)
```

Let's create a character version of variable `hs_state` and then turn it into a factor:

```{r, eval=FALSE}
#wwlist %>%
#  count(hs_state)
# Subset obs to West Coast states 
wwlist_temp <- wwlist %>%
  filter(hs_state %in% c("CA", "OR", "WA"))

# Create character version of high school state for West Coast states only
wwlist_temp$hs_state_char <- as.character(wwlist_temp$hs_state)

# Investigate character variable
str(wwlist_temp$hs_state_char)
table(wwlist_temp$hs_state_char)

# Create new variable that assigns levels
wwlist_temp$hs_state_fac <- factor(wwlist_temp$hs_state_char, levels = c("CA","OR","WA"))
str(wwlist_temp$hs_state_fac)
attributes(wwlist_temp$hs_state_fac)

#wwlist_temp %>%
#  count(hs_state_fac)
rm(wwlist_temp)

```

How the `levels` argument works when underlying data is character:

-   Matches value of underlying data to value of the level attribute
-   Converts underlying data to integer, with level attribute attached

\medskip See [Chapter 15 of Wickham](https://r4ds.had.co.nz/factors.html) for more on factors (e.g., modifying factor order, modifying factor levels)

## Creating factors \[from integer vectors\]

Factors are just integer vectors with level attributes attached to them. So, to create a factor:

1.  Create a vector for the underlying data
2.  Create a vector that has level attributes
3.  Attach levels to the data using the `factor()` function

```{r}
a1 <- c(1,1,1,0,1,1,0) # A vector of data
a2 <- c("zero","one") # A vector of labels

# Attach labels to values
a3 <- factor(a1, labels = a2)
a3
str(a3)

```

Note: By default, `factor()` function attached "zero" to the lowest value of vector `a1` because "zero" was the first element of vector `a2`

Let's turn an integer variable into a factor variable in the `wwlist` data frame

Create integer version of `receive_year`:

```{r}
#typeof(wwlist_temp$receive_year)
wwlist$receive_year_int <- as.integer(wwlist$receive_year)
str(wwlist$receive_year_int)
typeof(wwlist$receive_year_int)

```

Assign levels to values of integer variable:

```{r, eval=FALSE}
wwlist$receive_year_fac <- factor(wwlist$receive_year_int, 
      labels=c("Twenty-sixteen","Twenty-seventeen","Twenty-eighteen"))
str(wwlist$receive_year_fac)
str(wwlist$receive_year)

#Check variable
wwlist %>%
  count(receive_year_fac)

wwlist %>%
  count(receive_year)
```
