```{r, echo=FALSE, include=FALSE}
# Download images
imgs <- c('fp1.JPG', 'fp2.JPG', 'data-structures-overview.png', 'pane_layout_23.jpeg')

for (i in imgs) {
  if(!file.exists(i)){
  download.file(url = paste0('https://raw.githubusercontent.com/anyone-can-cook/rclass1/master/lectures/intro_to_r/', i), 
                destfile = i,
                mode = 'wb')
  }
}
```

# What is R? Why R?

For detailed info visit [R-project.org](https://www.r-project.org/about.html)

The Inter-University Consortium for Political and Social Research [(ICPSR)](https://www.icpsr.umich.edu/icpsrweb/content/shared/ICPSR/faqs/what-is-r.html) says:

> R is "an alternative to traditional statistical packages such as SPSS, SAS, and Stata such that it is an extensible, open-source language and computing environment for Windows, Macintosh, UNIX, and Linux platforms. Such software allows for the user to freely distribute, study, change, and improve the software under the [Free Software Foundation's GNU General Public License](https://www.gnu.org/home.en.html)."

I don't find this definition particularly helpful. I think of *R* as:

-   An "open source" programming language and software that provide collections of interrelated "functions"
-   "open source" means that *R* is free and created by the user community. The user community can modify basic things about *R* and add new capabilities to what R can do
-   a "function" is usually something that takes in some "input," processes this input in some way, and creates some "output"
    -   e.g., the `max()` function takes as input a collection of numbers (e.g., 3,5,6) and returns as output the number with the maximum value
    -   e.g., the `lm()` function takes in as inputs a dataset and a statistical model you specify within the function, and returns as output the results of the regression model

## Base R vs. R packages

**Base R**

-   When you install R, you automatically install the ["Base R"](https://stat.ethz.ch/R-manual/R-devel/library/base/html/00Index.html) set of functions
-   Example of a few of the functions in in Base R:
    -   `as.character()` function
    -   `print()` function
    -   `setwd()` function

**R packages**

-   an R "package" (or "library") is a collection of (related) functions developed by the R community
-   Examples of R packages:
    -   `tidyverse` package for manipulating and visualizing data
    -   `igraph` package for network analyses
    -   `leaflet` package for mapping
    -   `rvest` package for webscraping
-   **All** R packages are free!
-   Often a package we install may be a collection of packages (e.g., `tidyverse`) and/or may depend on other packages, which will be automatically installed for you

**Installing and Loading R packages**

You only need to install a package once. To install an R package use `install.package()` function.

```{r warning=FALSE, message=FALSE}
#install.packages("tidyverse")
```

You need to load a package everytime you plan to use it. To load a package use the `library()` function.

```{r}
library(tidyverse)
```

## RStudio

"[RStudio](https://www.rstudio.com/products/rstudio/features/) is an integrated development environment (IDE) for R. It includes a console, syntax-highlighting editor that supports direct code execution, as well as tools for plotting, history, debugging and workspace management."

[Pane Layout from RStudio User Guide](https://docs.posit.co/ide/user/ide/guide/ui/ui-panes.html) ![](pane_layout_23.jpeg)

## Markdown

**What is Markdown?**

-   Markdown is a set of rules that define how the layout and presentation of text and images appear in a document. One of the most popular markup languages is HTML. For example, when you go to a website, there are certain sections like a header, navigation bar, specific headings, color schemes, font styles, etc.\
-   The [Markdown language](https://daringfireball.net/projects/markdown/) was created by John Gruber in 2004.

**How is Markdown different than a WYSIWYG (WIZ--ee--wig) editor?**

-   What you see is what you get (WYSIWYG) editing software enables you to point and click to make changes to the format, text, and images in a document and view the changes immediately (e.g., Microsoft Word, Google Docs).

-   In a Markdown file, you use Markdown syntax to format text and images.

**Example Markdown syntax:**

## Level two heading

### Level three heading

-   Bullet point

**Bold text**

*italics*

**What is markdown used for?**

-   HTML or PDF documents
-   Websites
-   Note taking (Obsidian)
-   Books
-   Presentations and more

See [The Markdown Guide](https://www.markdownguide.org/getting-started/) for more on Markdown.

## Quarto (.qmd)

**What is Quarto?**

From [How Quarto works in R](https://quarto.org/docs/get-started/hello/images/rstudio-qmd-how-it-works.png)![](rstudio-qmd-how-it-works.png)

-   From [What can I use Quarto for?](https://quarto.org/docs/faq/rmarkdown.html#what-can-i-use-quarto-for)

    -   "Quarto is an open-source scientific and technical publishing system built on Pandoc. You can weave together narrative text and code to produce elegantly formatted output as documents, web pages, blog posts, books and more."

-   Think of a Quarto file with the extension .qmd as a document written in Markdown syntax that embeds R code (Markdown + R code).

-   The integration of Quarto in R is relatively new and builds off of **R Markdown**.

    -   From [RStudio](https://rmarkdown.rstudio.com/articles_intro.html): "R Markdown is a file format for making dynamic documents with R. An R Markdown document is written in markdown (an easy-to-write plain text format) and contains chunks of embedded R code."

-   Similar to Quarto, for the last ten years, R users have relied on R Markdown to produce static and dynamic documents. However, "Quarto is a multi-language, next generation version of R Markdown from Posit, with many new new features and capabilities. Like R Markdown, Quarto uses knitr to execute R code, and is therefore able to render most existing Rmd files without modification." From [Quarto](https://quarto.org/).

**Key Takeaways**:

-   Quarto does not depend on R; therefore, anyone using Python, Javascript, Julia, etc., can create documents with Quarto (.qmd) and collaborate on projects much more easily.

-   The Quarto visual editor in R Studio provides a WYSIWYG editing interface, making document creation seamless.

-   Quarto works with R Markdown and has additional features, making it easier to use.

**What is Quarto used for?**

-   HTML, PDF, MS Word documents\
-   Presentations in Revealjs, Powerpoint, and Beamer
-   Markdown (GitHub)
-   Wikis

**How we will be using Quarto in this class**

-   All lectures created using Quarto (.qmd) file extension
-   You will use Quarto to complete homework assignments

**After this class you might:**

-   never use Microsoft Word again!
-   Use Quarto to create: papers for class; presentations; journal manuscripts; your dissertation; etc.

**Further Reading:**

-   Markdown Guide, Basic Syntax: https://www.markdownguide.org/basic-syntax/
-   A Brief History of R Markdown: https://slides.yihui.org/2021-Brazilian-R-Day.html#1
-   R Markdown the Definitive Guide: https://bookdown.org/yihui/rmarkdown/
-   RStudio User Guide Quarto visual editor: https://docs.posit.co/ide/user/ide/guide/documents/visual-editor.html

```{=html}
<!--
## Why learn R? R can do a lot of stuff!

How we have used R+RStudio+RMarkdown in our research team

-   Stuff traditional statistical software (e.g., SPSS, Stata) can do
    -   Data manipulation, creating analysis datasets
    -   [Descriptive statistics and statistical models](https://ozanj.github.io/soc_of_ed_presentation/#/slide-15)
    -   Graphs
-   Stuff traditional statistical software cannot do
    -   [Static policy reports](https://emraresearch.org/sites/default/files/2019-03/joyce_report.pdf)
    -   Static presentations
        -   All lectures for this class written in RMarkdown
    -   [Interactive presentations](https://ozanj.github.io/joyce_report/#/title)
    -   Interactive map
        -   [Third Way map](https://ozanj.github.io/joyce_report/#/4)
        -   [Karina's recruitment redlining map](https://ksalazar3.github.io/defense/#/24/4)
    -   [Interactive dashboards](https://jkcf.shinyapps.io/dashboard/)
    -   Interactive graphs

Some of the other stuff R can create/do:

-   [Websites](https://bookdown.org/yihui/rmarkdown/websites.html); [journals](https://bookdown.org/yihui/rmarkdown/journals.html); [books](https://bookdown.org/); [web-scraping](https://www.analyticsvidhya.com/blog/2017/03/beginners-guide-on-web-scraping-in-r-using-rvest-with-hands-on-knowledge/); network analysis; machine learning/artificial intelligence


# What is this course about?

### What is data management?

- Data management is everything we do to create analysis datasets that are ready to analyze, e.g.:
  - collect data
  - read/import data into statistical programming language
  - clean data
  - integrate data from multiple sources (e.g, join/merge, append)
  - change organizational structure of data so it is suitable for analysis
  - create "analysis variables" from "input variables"
  - Make sure that you have created analysis variables correctly

### Why I don't call this class "R for data science"
 
 
Learn to walk before you can run!

- "data science" implies doing fancy, sexy things like mapping, network analysis, web-scraping, etc.
- But if you don't know how to clean data, these sexy analyses and visualizations will be useless
- "80% of data science is data cleaning"
- The skills you learn in this data management oriented class will be usefull for fancy data science stuff down the road!

### Who is this class for?

This class is for anyone who wants to work with data, that is people who want to be:

- researchers working with survey data and doing traditional statistical analyses
- researchers who want to do "data science" oreinted research 
- analysts working at think tanks/non-profits or as institutional researchers
- journalists who create interactive data visualizations

-->
```
# Executing R commands

## R as a calculator

```{r}
5
5+2
10*3
```

## Shortcuts for Executing Commands in R

Three ways to execute commands in R

1.  Type/copy commands directly into the "console"
2.  \`code chunks' in RMarkdown (.Rmd & .QMD files)
    -   Can execute one command at a time, one chunk at a time, or "knit" the entire\
        -   **Cmd/Ctrl + Alt + i**: insert a new code chunk
    -   **Cmd/Ctrl + Enter**: execute highlighted line(s) within chunk
    -   **Cmd/Ctrl + Shift + k**: "knit" entire document
3.  R scripts (.R files)
    -   This is just a text file full of R commands
    -   **Cmd/Ctrl + Enter**: execute highlighted line(s)
    -   **Cmd/Ctrl + Shift + Enter** (without highlighting any lines): run entire script

```{r}
5+2
10*3
```

# R objects and data structures

## Preview of lecture on objects

-   This section of the lecture provides a conceptual and practical introduction to "objects" in R
-   **Important**: goal is to begin to develop familiarity with concepts that we will introduce in more detail in later weeks
    -   I don't expect you to understand or retain all this information perfectly
    -   So just focus on understanding as much as you can and ask any questions that come to mind

## Assignment

**Assignment** refers to creating an "object" and assigning values to it

-   The object may be a variable, a dataset, a bit of text that reads "la la la"
-   `<-` is the assignment operator
    -   in other languages `=` is the assignment operator
-   general syntax:
    -   `object_name <- object_values`
    -   good practice to put a space before and after assignment operator

```{r}
# Create an object and assign value
a <- 5
a

b <- "yay!"
b
```

## Objects

R is an "object-oriented" programming language (like Python, JavaScript). So, what is an **object**?

-   formal computer science definitions are confusing because they require knowledge of concepts we haven't introduced yet

-   More intuitively, I think objects as anything I assign values to

    -   For example, below, `a` and `b` are the names of objects I assigned values to

```{r}
a <- 5
a
b <- "yay!"
b
```

-   Ben Skinner (R maven) says "Objects are like boxes in which we can put things: data, functions, and even other objects."

Many commercial statistical software packages (e.g., SPSS, Stata) operate on datasets, which consist of rows of observations and columns of variables

-   Usually, these packages can open only one dataset at a time
-   By contrast, in R everything is an object and there is no limit to the number of objects R can hold (except memory)

```{=html}
<!--

### Objects

Understanding what an "object" is in R can be confusing because almost everything is an object

- There is no limit to the number of objects R can hold (except memory)
- Objects can be categorized by "__type__" (which we will discuss today) and by "__class__" (which we will discuss more in later weeks)

\medskip 

Object "type" refers to how the object is stored internally by R

- Some object "types" require more memory than others
- For example, an "integer" storage type takes up less money than a "character" storage type
  - therefore, it makes sense to store the value `5` as an integer rather than a character

\medskip 

"Class" is an "attribute" of an object -- built on top of object type -- that defines what things you can do to an object

- For example, a date is an object with a numeric _type_ and a date _class_
- R "functions" do different things to different types/classes of objects
  - e.g., date functions are meant to process objects with type=numeric and class=date; these functions don't work on objects with type=character (e.g., "yay!")

-->
```
## (Atomic) Vectors

**The fundamental data structure in R is the "vector"**

-   A vector is a collection of values
-   The individual values within a vector are called "elements"
-   Values in a vector can be numeric, character (e.g., "Apple"), or some other *type*

**Formal classification of vectors in R**

Here, I introduce the classification of vectors by Grolemund and Wickham

There are two broad types of **vectors**

1.  **Atomic vectors**. An object that contains elements. Six "types" of atomic vectors:
    -   **logical**, **integer**, **double**, **character**, **complex**, and **raw**.
        -   **Integer** and **double** vectors are collectively known as **numeric** vectors.
2.  **Lists**. Like atomic vectors, lists are objects that contain elements
    -   elements within a list may be atomic vectors
    -   elements within a list may also be other lists; that is lists can contain other lists

**Table 1. The six types of atomic vectors in R: double, integer, logical, character, complex, and raw**

```{r}
#| echo: false
#| message: false

library(kableExtra); library(tidyverse)

Type <- c("double (or numeric)","integer", "logical", "character", "complex", "raw")
Example <- c('-0.5, 120.9, 5.0','-1L, 3L, 5L', 'TRUE, FALSE', '"California", "New York"', '-5+11i, 3+4i, 0+3i', '01, ff')
Comment <- c('Floating point numbers with double precision', '"Long" integers', 'Boolean', 'Text', 'Real+imaginary numbers', 'Raw bytes (as hexadecimal)')


dt <- tibble(Type, Example, Comment)


dt %>%
  kbl() %>%
  kable_styling(bootstrap_options = c("striped", "hover", "condensed"))
```

[Atomic Vectors table adapted from (DiSCDown, Introduction to Programming with R, Chapter 4 Vectors)](https://discdown.org/rprogramming/vectors.html)

<br>

**Visual representation of the Grolemund and Wickham classification**

![Overview of data structures (Grolemund and Wickham, 2018, chapter 20)](data-structures-overview.png){width="60%"}

One difference between atomic vectors and lists: **homogeneous** vs. **heterogeneous** elements

-   atomic vectors are **homogeneous**: all elements within atomic vector must be of the same type
-   lists can be **heterogeneous**: e.g., one element can be an integer and another element can be character

<br>

**Intuitive approach to vectors used by Dr. Ben Skinner**:

-   data **type**: logical, numeric (integer and double), character, etc.
-   data **structure**: vector, list, matrix, etc.

I find Skinner's classification more intuitive conceptually. However, it isn't completely consistent with how R and R functions think about objects. <br>

<br> **Let's practice creating simple vectors**

Below we use the combine function `c()` to create a numeric vector that contains three elements

-   Help file says that `c()` "combines values into a vector or list"

```{r}
#?c # to see help file for the c() "combine" function
x <- c(4, 7, 9) # create object called x, which is a vector with three elements 
# (each an integer)
x # print object x
```

Vector where the elements are characters

```{r}
animals <- c("lions", "tigers", "bears", "oh my") # create object called animals
animals
```

<br> **Student task**

Either in the R console or within the R markdown file, do the following:

1.  Create a vector called `v1` with three elements, where all the elements are numbers. Then print the values.
2.  Create a vector called `v2` with four elements, where all the elements are characters (i.e., enclosed in single '' or double "" quotes). Then print the values.
3.  Create a vector called `v3` with five elements, where some elements are numeric and some elements are characters. Then print the values.

<details>

<summary><strong>Solutions</strong></summary>

```{r}
v1 <- c(1, 2, 3) 
# create a vector called v1 with three elements
# all the elements are numbers
v1 # print value
```

```{r}
v2 <- c("a", "b", "c", "d") 
# create a vector called v2 with four elements
# all the elements are characters
v2 # print value
```

```{r}
v3 <- c(1, 2, 3, "a", "b") 
# create a vector called v3 with five element
# some elements are numeric and some elements are characters
v3 # print value
```

</details>

<br>

```{=html}
<!--
When printing object `v3` numbers were printed with quotations around them (e.g., `"1"`), because they are stored as "characters" 

- Why? A vector must be only one "type" (numeric, character, or logical). You can’t mix modes in the same vector.
-->
```
```{=html}
<!--
## Atomic vectors

__Developing an intuitive understanding of vector types__

**Grolemund and Wickham classification**:

1.  **Atomic vectors**. six "types": logical, integer, double, character, complex, raw.
2.  **Lists**

Problem with this classification:

-   Not conceptually intuitive
-   Technically, lists are a type of vector, but people often think of atomic vectors and lists as fundamentally different things

**Classification used by my R maven Ben Skinner**:

-   data **type**: logical, numeric (integer and double), character, etc.
-   data **structure**: vector, list, matrix, etc.

I find Skinner's classification more intuitive conceptually. However, it isn't completely consistent with how R and R functions think about objects. 

-->
```
### length() function to get the number of elements

**"Length" of an atomic vector is the number of elements**

For remainder of lecture, I'll use the term **vector** to refer to atomic vectors

Use `length()` function to examine vector length

```{r}
x <- c(4, 7, 9)
x
length(x)

animals <- c("lions", "tigers", "bears", "oh my")
animals
length(animals)
```

A single number (or a single string/character) is a vector with `length==1`

```{r}
z <- 5
length(z)
length("Tommy")
```

### typeof() function to get the data type of a vector

The "type" of an atomic vector refers to the elements within the vector.

While there are six "types" of atomic vectors, we'll focus on the following types:

-   numeric:
    -   "integer" (e.g., 5)
    -   "double" (e.g., 5.5)
-   character (e.g., "ozan")
-   logical (e.g., `TRUE`, `FALSE`)

Use `typeof()` function to examine vector type

```{r}
x
typeof(x)

p <- c(1.5, 1.6)
p
typeof(p)

animals
typeof(animals)
```

**Data type of a vector, numeric**

Numeric vectors can be "integer" (e.g., 5) or "double" (e.g., 5.5)

```{r}
typeof(1.5)
```

R stores numbers as doubles by default.

```{r}
x
typeof(x)
```

To make an integer, place an `L` after the number:

```{r}
typeof(5)
typeof(5L)
```

**Data type of a vector, character**

In contrast to "numeric" data types which are used to store numbers, the "character" data type is used to store **strings** of text.

-   Strings may contain any combination of numbers, letters, symbols, etc.
-   Character vectors are sometimes referred to as string vectors

When creating a vector where elements have `type==character` (or when referring to the value of a string), place single \`\` or double "" quotes around text

-   the text within quotes is the "string"

```{r}
c1 <- c("cat",'cash','candy cane')
c1
typeof(c1)
length(c1)
```

Numeric values can also be stored as strings

```{r}
c2 <- c("1","2","3")
c2
typeof(c2)
```

**Data type of a vector, logical**

Logical vectors can take three possible values: `TRUE`, `FALSE`, `NA`

-   `TRUE`, `FALSE`, `NA` are special keywords; they are different from the character strings `"TRUE"`, `"FALSE"`, `"NA"`
-   Don't worry about `NA` for now

```{r}
typeof(TRUE)
typeof("TRUE")

typeof(c(TRUE,FALSE,NA))
typeof(c(TRUE,FALSE,NA,"FALSE"))

log <- c(TRUE,TRUE,FALSE,NA,FALSE)
typeof(log)
length(log)
```

We'll learn more about logical vectors later

**All elements in (atomic) vector must have same data type.**

Atomic vectors are **homogenous**;

-   An atomic vector has one data type
-   all elements within an atomic vector must have the same data "type"

If a vector contains elements of different type, the vector type will be type of the most "complex" element

Atomic vector types from simplest to most complex:

-   logical \< integer \< double \< character

```{r}
typeof(c(TRUE,TRUE,NA))

# recall L after an integer forces type to be integer 
# rather than double
typeof(c(TRUE,TRUE,NA,1L)) 
typeof(c(TRUE,TRUE,NA,1.5))
typeof(c(TRUE,TRUE,NA,1.5,"howdy!"))
```

### Named vectors

All vectors can be "named" (i.e., name individual elements within vector)

Example of creating an unnamed vector

-   the `str()` function "compactly display\[s\] the internal structure of an R object" \[from help file\]; very useful for describing objects

```{r}
#?str
x <- c(1,2,3,"hi!")
x
str(x)
```

Example of creating a named vector

```{r}
y <- c(a=1,b=2,3,c="hi!")
y
str(y)
```

## Sequences

(Loose) definition: a sequence is a set of numbers in ascending or descending order

A vector containing a "sequence" of numbers (e.g., 1, 2, 3) can be created using the colon operator `:` with the notation `start:end`

```{r}
-5:5
5:-5
s<- 1:10 #same as this: s<- c(1:10)
s
length(s)
```

Creating sequences using `seq()` function - basic syntax \[with default values\]:

```{r, eval=FALSE}
seq(from = 1, to = 1, by = 1)
```

```{r}
seq(10,15)
seq(from=10,to=15,by=1)
seq(from=100,to=150,by=10)
```

## Vectorized math

Most mathematical operations operate on each element of the vector

-   e.g., add a single value to a vector and that value will be added to each element of the vector

```{r}
1:3
1:3+.5
(1:3)*2
```

Mathematical operations involving two vectors with the same length behave differently

-   e.g., for addition: add element 1 of vector 1 to element 1 of vector 2, add element 2 of vector 1 to element 2 of vector 2, etc.

```{r}
c(1,1,1)+c(1,0,2)
c(1,1,1)*c(1,0,2)
```

## Lists

**Lists**

What is a **list**?

-   Like (atomic) vectors, a list is an object that contains **elements**
-   Unlike vectors, data types can differ across elements within a list
-   An element within a list can be another list
    -   this characteristic makes lists more complicated than vectors
    -   suitable for representing hierarchical data

Lists are more complicated than vectors; today we'll just provide a basic introduction

**Create lists using `list()` function**

Create a vector (for comparison purposes)

```{r}
a <- c(1,2,3)
typeof(a)
length(a)
```

Create a list

```{r}
b <- list(1,2,3)
typeof(b)
length(b)
b # print list is awkward
```

**Investigate structure of lists using `str()` function**

When investigating lists, `str()` is better than printing the list

```{r}
b <- list(1,2,3)
typeof(b)
length(b)
str(b) # 3 elements, each element is a numeric vector w/ length=1
```

Each element of a list can be a vector of different length (i.e., different number of elements)

```{r}
c <- list(c(3,4),c(-5,1,3))
typeof(c)
length(c)
str(c) # 2 elements; element 1=vector w/ length=2; element 2=vector w/length=3
```

**Elements within lists can have different data types**

Lists are **heterogeneous**

-   data types can differ across elements within a list

```{r}
b <- list(1,2,"apple")
typeof(b)
length(b)
str(b)
```

Vectors are **homogeneous**

```{r}
a <- c(1,2,"apple")
typeof(a)
str(a)
```

**Lists can contain other lists**

```{r}
x1 <- list(c(1,2), list("apple", "orange"), list(1, 2, 3))
str(x1)
```

The **first element** of list is a numeric vector with length=2

```{r}
x1[[1]]
```

The **second element** is a list with length=2

-   first element is character vector with length=1
-   second element is character vector with length=1

```{r}
x1[[2]]
```

The **third element** is a list with length=3

-   first element is numeric vector with length=1
-   second element is numeric vector with length=1
-   third element is numeric vector with length=1

```{r}
x1[[3]]
```

**You can name each element in the list**

```{r}
x2 <- list(a=c(1,2), b=list("apple", "orange"), c=list(1, 2, 3))
str(x2)
```

`names()` function shows names of elements in the list

```{r}
names(x2) # has names
names(x1) # no names
```

**Access individual elements in a "named" list**

Syntax: `list_name$element_name`

```{r}
x2 <- list(a=1, b=list("apple", "orange"), c=list(1, 2, 3))
x2$a
typeof(x2$a)
length(x2$a)

typeof(x2$b)
length(x2$b)

typeof(x2$c)
length(x2$c)
```

Note: We'll spend more time practicing "accessing elements of a list" in upcoming weeks

**Compare structure of list to structure of element within a list**

```{r}
str(x2)

str(x2$c)
```

**A DATASET IS JUST A LIST!!!!!**

A data frame is a list with the following characteristics:

-   Data type can differ across elements (like all lists)
-   Each element (column) is a variable
-   Each element in a data frame must have the same length
    -   The length of an element is the number of observations (rows)
    -   Thus, each variable in a data frame has same number of observations
-   Each element is named
    -   these element names are the variable names
-   Typically, each **element**(variable) in a data frame is a **vector**
    -   Elements can also be lists. Happens when the variable has a complicated data structure
        -   e.g., a variable that identifies the "\@" mentions in a tweet

```{r, include=FALSE}
library(tidyverse)
df <- select(mtcars,mpg,cyl,hp) # ignore for now
```

```{r}
names(df)
head(df, n=4) # print first few rows
```

Additionally, data frames have "attributes"; we'll discuss those in upcoming weeks

**A data frame is a named list**

```{r}
head(df, n= 5)
typeof(df)
names(df)
length(df) # length=number of variables
str(df)
```

Like any named list, we can examine the elements

-   Individual elements of a data frame are the variables
-   These variables are vectors with length equal to the number of rows/observations

```{r}
typeof(df$mpg)
length(df$mpg) # length=number of rows/obs
str(df$mpg)
```

## Main takeaways about atomic vectors and lists

Basic data stuctures

1.  **(Atomic) vectors**: **logical**, **integer**, **double**, **character**.
    -   each element in vector must have same data type
2.  **Lists**:
    -   Data type can differ across elements

**Takeaways**

-   These concepts are difficult; ok to feel confused
-   I will reinforce these concepts throughout the course
-   Good practice: run simple diagnostics on any new object
    -   `length()` : how many **elements** in the object
    -   `typeof()` : what **type** of data is the object
    -   `str()` : hierarchical structure of the object
-   These data structures (vectors, lists) and data types (e.g., character, numeric, logical) are the basic building blocks of all object oriented programming languages
-   Application to statistical analysis
    -   Datasets are just lists
    -   The individual elements -- columns/variables -- within a dataset are just vectors
-   These structures and data types are foundational for all "data science" applications
    -   e.g., mapping, webscraping, network analysis, etc.

# Using R functions

## What are functions

\*\*Functions\*\* are pre-written bits of code that accomplish some task.

Functions generally follow three sequential steps:

1.  take in an **input** object(s)
2.  **process** the input.
3.  **return** (A) a new object or (B) a visualizatoin (e.g., plot)

For example, `sum()` function calculates sum of elements in a vector

1.  **input**. Takes in a vector of elements (numeric or logical)
2.  **processing**. Calculates the sum of elements
3.  **return**. Returns numeric vector of length=1; value is sum of input vector

```{r}
sum(c(1,2,3))
typeof(sum(c(1,2,3))) # type of object created by sum()
length(sum(c(1,2,3))) # length of object created by sum()

#sum(c(TRUE,TRUE,FALSE))
#typeof(sum(c(TRUE,TRUE,FALSE))); length(sum(c(TRUE,TRUE,FALSE)))
```

## Function syntax

Components of a function

-   function name (e.g., `sum()`, `length()`, `seq()`)
-   function arguments
    -   Inputs that the function takes, which determine what function does

        -   can be vectors, data frames, logical statements, etc.

    -   In "function call" you specify values to assign to these function arguments

        -   e.g., `sum(c(1,2,3))`

    -   Separate arguments with a comma `,`

        -   e.g., `seq(10,15)` Example: the sequence function, `seq()`

```{r}
seq(10,15)
```

## Function syntax: More on function arguments

Usually, function arguments have names

-   e.g., the `seq()` function includes the arguments `from`, `to`, `by`
-   when you call the function, you need to assign values to these arguments; but you usually don't have to specify the name of the argument

```{r}
seq(from=10, to=20, by=2)
seq(10,20,2)
```

Many function arguments have "default values", set by whoever wrote the function

-   if you don't specify a value for that argument, the default value is inserted
-   e.g., partial list of default values for `seq()`: `seq(from=1, to=1, by=1)`

```{r}
seq()
seq(to=10)
seq(10) # R assigned value of 10 to "to" rather than "from" or "by"
```

## Function arguments, the `na.rm` argument

When R performs a calculation and an input has value `NA`, output value is `NA`

```{r}
5+4+NA
```

R functions that perform calculations often have argument named `na.rm`

-   `na.rm` argument asks whether to remove `NA` values prior to calculation

-   For most functions, default value is `na.rm = FALSE`

    -   This means "do not remove `NAs`" prior to calculation
    -   e.g., default values for `sum()` function: `sum(..., na.rm = FALSE)`

    ```{r}
    sum(c(1,2,3,NA), na.rm = FALSE) # default value
    sum(c(1,2,3,NA))
    ```

-   if you specify, `na.rm = TRUE`, `NA` values removed prior to calculation

```{r}
sum(c(1,2,3,NA), na.rm = TRUE)
```

## Help files for functions

To see help file on a function, type `?function_name` without parentheses

```{r, eval=FALSE}
?sum
?seq
```

**Contents of help files**

-   **Description**. What the function does
-   **Usage**. Syntax, including default values for arguments
-   **Arguments**. Description of function arguments
-   **Details**. Details and idiosyncracies of about how the function works.
-   **Value**. What (object) the function "returns"
    -   e.g., `sum()` returns vector of length 1 whose value is sum of input vector
-   **References**. Additional reading
-   **See Also**. Related functions
-   **Examples**. Examples of function in action
-   Bottom of help file identifies the package the function comes from

\*\*Practice!\*\*

-   when you encounter a new function, spend two minutes reading the help file
-   over time, help files will feel less cryptic and will start to feel helpful

## Function arguments, the dot-dot-dot (`...`) argument

On help file for many functions, you will see an argument called `...`, referred to as the "dot-dot-dot" argument

```{r, eval=FALSE}
?sum
?seq
```

"Dot-dot-dot" arguments have several uses. What you should know for now:

-   `...` refers to arguments that are "un-named"; but user can specify values
    -   e.g., default syntax for `sum()`: `sum(..., na.rm = FALSE)`
        -   argument `na.rm` is "named" (name is `na.rm`); argument `...` un-named
-   `...` used to allow a function to take an arbitrary number of arguments:

```{r}
#Here, sum function takes 1 un-named argument, specifically c(10,5,NA)
sum(c(10,5,NA),na.rm=TRUE)

#Here the sum function takes 3 un-named arguments
sum(10,5,NA,na.rm=TRUE)

#Here the sum function takes 5 un-named arguments
sum(10,5,10,20,NA,na.rm=TRUE)
```

# Appendix

## Directories and filepaths (skim)

**Directories and filepaths**

-   Goal: Give you a very brief overview of "directories" (i.e., folders) and "filepaths" (tells you where folder is located) in R
-   For the most part, this course won't require extensive knowledge of working with filepaths. But the second course in this sequence will.

**(Current) Working directory**

-   The folder/directory in which you are currently working
-   This is where R looks for files
-   Files located in your current working directory can be accessed without specifying a filepath because R automatically looks in this folder

Function `getwd()` shows current working directory

```{r}
getwd()
```

Command `list.files()` lists all files located in working directory

```{r}
getwd()
list.files()
```

**Working directory, "Code chunks" vs. "console" and "R scripts"**

When you run **code chunks** in RMarkdown files (.Rmd), the working directory is set to the filepath where the .Rmd file is stored

```{r}
getwd()
list.files()
```

When you run code from the **R Console** or an **R Script**, the working directory is your R Project directory (we'll cover this in the next section).

Command `getwd()` shows current working directory

```{r}
getwd()
```

**Absolute vs. relative filepath**

**Absolute file path**: The absolute file path is the complete list of directories needed to locate a file or folder.\
`setwd("/Users/pm/Desktop/rclass1/lectures/intro_to_r")`

**Relative file path**: The relative file path is the path relative to your current location/directory. Assuming your current working directory is in the "intro_to_r" folder and you want to change your directory to the data folder, your relative file path would look something like this:\
`setwd("../../data")`

```         
        File path shortcuts (Mac)
```

| **Key** | **Description**                                                    |
|--------------------------|----------------------------------------------|
| \~      | tilde is a shortcut for user's home directory (mine is my name pm) |
| ../     | moves up a level                                                   |
| ../../  | moves up two level                                                 |

```{r, eval=FALSE, include=FALSE}
getwd()
setwd('~')
getwd()
```

Use relative path to move working directory up one level

```{r}
getwd()
setwd('../')
getwd()
```

**Exercise**

1.  Let's create a folder on our desktop and name it red\
2.  Inside the red folder, create two subfolders named orange and yellow\
3.  Inside the yellow folder create another subfolder named green

Make sure to name these folders in lowercase.

You should have 1 folder on your desktop called red. Inside the red folder you have two folders called orange and yellow. Inside the yellow folder you have a folder called green.

Here is a visual of how it should look...

**File path visual**

![](fp1.JPG){width="200px"} ![](fp2.JPG){width="200px"}

**Exercise continued**

Let's say we want to get to the green folder using the absolute file path.\
1. View your current working directory `getwd()`\
2. Set your working directory to the green folder using the absolute file path\
3. Now set your working directory to the orange folder using the relative file path (hint: use `../`)

<details>

<summary><strong>Solutions</strong></summary>

\[Solution for Mac users\]

```{r, eval=FALSE}
getwd()
setwd("~/Desktop/red/yellow/green")
getwd() 
setwd("../../orange")
getwd()
```

</details>

<br>

## Create "R project"

**What is an R project?**

-   Helps you keep all files for a project in one place
-   When you open an R project, the file-path of your current working directory is automatically set to the file-path of your R-project

**How to create an "R project"**

1.  In RStudio, click on "File" \>\> "New Project" \>\> "Existing Directory" \>\> "New Project"
2.  "Browse" to find folder where you want in which you want to save the ".Rproj" folder you just saved

-   let's call this folder "my_rclass_folder", but it can be any folder we want
-   the name of the ".Rproj" file will be my_rclass_folder.Rproj

1.  Then click on Create Project

Using R Project

-   save files/folders associated with this project within the "my_rclass_folder"
-   When working you want to work on the project, click on the .Rproj folder "my_rclass_folder.Rproj"
-   whenever you run a command from your **R Console** or an **R Script**, the "working directory" will be set to the filepath where "my_rclass_folder.Rproj" is saved
-   As you will see in the future (EDUC260B), this will make it easy to collaborate with colleagues
    -   you will be able to run the exact same scripts on different computers!

**Next, you follow these steps**

-   You can add any additional sub-folders you want to the "rclass1" folder
    -   e.g., "syllabus", "resources"
-   You can add any additional files you want to the sub-directory folders you unzipped
    -   e.g., in "rclass1/lectures/intro_to_r" you might add an additional document of notes you took

## R Markdown

**What is R Markdown**

-   R Markdown documents embed R code, output associated with R code, and text into one document
-   An R Markdown document is a "'Living' document that updates every time you compile \["knit"\] it"
-   R Markdown documents have the extension .Rmd
    -   Can think of them as text files with the extension .Rmd rather than .txt
-   At top of .Rmd file you specify the "output" style, which dictates what kind of formatted document will be created
    -   e.g., `html_document` or `pdf_document`
-   When you compile \["knit"\] a .Rmd file, the resulting formatted document can be an HTML document, a PDF document, an MS Word document, or many other types

*This slide borrows from Darin Christensen*

**How people use R Markdown**

R Markdown creates many types of static and dynamic/interactive documents

-   Example of [static policy report](https://emraresearch.org/sites/default/files/2019-03/joyce_report.pdf)
-   Example of [dynamic/interactive presentation](https://ozanj.github.io/joyce_report/#/title)

How I use R Markdown

-   Journal manuscripts; reports; presentations; for taking notes when I am learning new methods or reading an empirical paper

How we will be using R Markdown files in this class:

-   Homework you submit will be .Rmd files, where "output" style will be `html_document` or `pdf_document`
-   Lectures we write are .Rmd files, where the output style will `html_document`(can also be `beamer_presentation` or `word_document`)
    -   `beamer_presentation` is essentially a PDF document, where each page is a slide

**Creating R Markdown documents**

**Do this with a partner**

Approach for creating a RMarkdown document.

1.  Point-and-click from within RStudio
    -   Click on *File* \>\> *New File* \>\> *R Markdown* \>\> *Document* \>\> choose *HTML* \>\> click *OK*
        -   Optional: add title (this is not the file name, just what appears at the top of document)
        -   Optional: add author name
    -   Save the .Rmd file; *File* \>\> *Save As*
        -   Any file name
        -   Recommend you save it in same folder you saved this lecture
    -   "Knit" the entire .Rmd file
        -   Point-and-click OR shortcut: **Cmd/Ctrl + Shift + k**

```{r eval=FALSE, echo=FALSE}
# 2. From blank text file
#     - create blank text file
#         - can give it any name, but change extension from .txt to .Rmd
#     - Open this blank .Rmd file in Rstudio
#     - copy this text into file [LINK HERE][PATRICIA CREATE LINK TO sample_simple_rmarkdown.txt FILE WHICH IS STORED IN LECTURE 2 DIRECTORY]
#     - "knit" the entire .Rmd file
#         - point-and-click OR shortcut: __Cmd/Ctrl + Shift + k__

#Don't worry about this. Ozan's notes
```

**Components of a .Rmd file**

An R Markdown (.Rmd) file consists of several parts

1.  **YAML header**
    -   YAML stands for "yet another markup language"
    -   Controls settings that apply to the whole document (e.g., "output" should be `html_document` or `pdf_document`, whether to include table of contents, etc.)
    -   YAML header goes at the very top of the document
    -   Starts with a line of three horizontal dashes `---`; ends with a line of three horizontal dashes `---`
2.  **Text** in body of .Rmd file
    -   e.g., headings; description of results, etc.
3.  **R code chunks** in body of .Rmd file

```{r, eval=FALSE}
a <- c(2,4,6)
a
a-1
```

4.  **R output** associated with code chunks

```{r, echo=FALSE}
a <- c(2,4,6)
a
a-1
```

## Comment: Running R code chunks vs. "knit" entire .Rmd file

Two ways to execute R commands in .Rmd file:

1.  "Knit" entire .Rmd file
    -   shortcut: **Cmd/Ctrl + Shift + k**
2.  "Run" code chunk or selected lines within code chunk
    -   Run selected line(s): **Cmd/Ctrl + Enter**
    -   Run current chunk: **Cmd/Ctrl + Shift + Enter**

Comment on default settings for RStudio:

-   When you knit entire .Rmd file, "objects" created within .Rmd file will not be available after file compiles
-   When you run code chunk (or selected lines in chunk), objects created by lines you run will be in your "environment" until you remove them or quit R session

**Output types of .Rmd file**

Common/important output types:

-   **html_document**: R Markdown originally designed to create HTML documents
    -   Most features/code in .Rmd files were written for html_document
    -   Many of these features are available in other output types
    -   When learning R Markdown, best to start by learning html_document
-   **pdf_document**: Requires installation of `tinytex` R package or LaTeX (MiKTeX/MacTeX)
    -   How it works:
        -   You write .Rmd code
        -   When you compile, this .Rmd code is transformed into LaTeX code
        -   LaTeX "engine" creates the formatted .pdf file
    -   Can include some of the same features available for *html_document*
    -   Can insert LaTeX commands in .Rmd file with *pdf_document* output
-   **beamer_presentation**: Requires installation of LaTeX
    -   "beamer" is the name for presentations written in LaTeX
    -   Essentially creates PDF of presentation slides
    -   Lectures for this class created with *beamer_presentation* output
    -   Note: YAML header includes `beamer_header.tex` file, which creates some formatting rules and additional commands

**Learning more about R Markdown**

Resources

-   Cheat sheets and quick reference:
    -   [Cheat Sheet](https://www.rstudio.com/wp-content/uploads/2015/02/rmarkdown-cheatsheet.pdf)
    -   [Quick Reference](https://www.rstudio.com/wp-content/uploads/2015/03/rmarkdown-reference.pdf) \[I prefer the quick reference\]
-   Chapters/books
    -   [Chapter 27](http://r4ds.had.co.nz/r-markdown.html) of "R for Data Science" book
    -   [R Markdown: The Definative Guide](https://bookdown.org/yihui/rmarkdown/) book \[I prefer this book\]

How you will learn R Markdown

-   Lectures written as .Rmd file
    -   During class run "code chunks" and try to "knit" entire .Rmd file
-   I'll assign **small** amount of reading on R Markdown
    -   Prior to next week:
        -   Spend 15 minutes familiarizing yourself with [Quick Reference](https://www.rstudio.com/wp-content/uploads/2015/03/rmarkdown-reference.pdf)
        -   Read section [3.1 of R Markdown: The Definative Guide](https://bookdown.org/yihui/rmarkdown/html-document.html), about creating html_document
-   Homework must be written in .Rmd file
    -   You will submit .Rmd file AND output of compiled file
    -   For next week, you will submit homework as html_document output

````{=html}
<!--
## Directory structure for this class

In order to be able to "knit" entire lectures \[rather than just run specific code chunks\] make sure that you have the following directory structure:

-   rclass1
    -   lectures
        -   intro_to_r/
        -   ...
        -   join_data/
        -   beamer_header.tex

What is beamer_header.tex?

-   A text file that contains \LaTeX code
-   This code creates formatting rules that are applied to all lecture slides
-   If you go YAML header you will see:

```{r, eval=FALSE}
    includes:
      in_header: ../beamer_header.tex
```

-   This runs beamer_header.tex; assumes that beamer_header.tex is located one level up from your current directory
-   If you don't have beamer_header.tex saved to appropriate place, you can download it here [LINK](https://github.com/anyone-can-cook/rclass1/blob/master/lectures/beamer_header.tex)
-   Note: we may revise beamer_header.tex as we work out formatting bugs
-->
````

## Matrices \[optional\]

**Matrices**

A **matrix** is a collection of elements arranged in a two-dimensional rectangular layout

-   A matrix is another "data structure," in addition to vectors and lists
-   Create a matrix named `m` with 2 rows and 3 columns

```{r}
m <- matrix( 
  c(2, 4, 3, 1, 5, 7), # the data elements 
  nrow=2,              # number of rows 
  ncol=3,              # number of columns 
  byrow = TRUE         # fill matrix by rows 
)

m # print matrix m
```

**Investigate matrix `m`**

```{r}
typeof(m) # type = "double"
str(m) # type = numeric; has two rows and three columns
class(m) # class = matrix; more on class later
```

<br>

**Like atomic vectors, matrices are homogenous data structures**

```{r}
m2 <- matrix( 
  c(2, 4, 3, "a", "b", "c"), # the data elements 
  nrow=2,              # number of rows 
  ncol=3,              # number of columns 
  byrow = TRUE         # fill matrix by rows 
)

m2
```

**Investigate matrix `m`**

```{r}
typeof(m2) # type = "character"
str(m2) # type = character; has two rows and three columns
```

<br>

**Why are data frames based on lists rather than matrices?**

-   a matrix is a homogenous data structure, so you couldn't have both a numeric variable and a character variable
-   a list is a heterogeneous data structure, allowing for variables of different underlying data types

**How are matrices used in R**

-   the underlying code for statistical models uses matrices (e.g., covariance matrix, matrix of regression coefficients)
-   For most data manipulation tasks, matrices used much less frequently than vectors and lists
    -   In this course, we won't use matrices much at all
-   Particular applications (e.g., social network analysis) make frequent use of matrices
-   Create an "adjacency matrix," which defines whether actors are connected to one another in a social network

```{r}
a <- matrix( 
  c(NA,NA,NA,NA,
    1,NA,NA,NA,
    1,0,NA,NA,
    0,1,0,NA), # the data elements 
  nrow=4,              # number of rows 
  ncol=4,              # number of columns 
  byrow = TRUE         # fill matrix by rows 
)
a_names <- c("a","b","c","d") # create vector of actor names
rownames(a) <- a_names # assign names to rows
colnames(a) <- a_names # assign names to columns
a
```

````{=html}
<!-- Code not working, unable to open the connection to the github data file

__Practical example of matrices: Network data of recruiting visits from colleges to high schools__

For this example, don't worry about understanding the code

-   We will explain this code over coming weeks

Object `g_2mode_privu` contains:

-   off-campus recruiting visits from private colleges/universities to private high schools
-   load `g_2mode_privu`

```{r, include = FALSE, results="hide", eval=FALSE}
#install.packages("igraph") # install igraph package
library(igraph) # load igraph package
```

```{r, results = "hide", eval=FALSE}
# load igraph object of visits by private colleges/universities to private high schools
load(url("https://github.com/cyouh95/recruiting-chapter/raw/master/data/g_2mode_privu.rda"))
```

<br>

__Investigate object `g_2mode_privu`__

```{r eval=FALSE}
typeof(g_2mode_privu) # igraph package stores network data as type = list
class(g_2mode_privu) # class = igraph
```

The "vertices" consist of colleges and high schools

```{r, eval=FALSE}
vcount(g_2mode_privu) # 1356 "nodes" or "vertices"
table(V(g_2mode_privu)$type) # 1330 private high schools; 26 private colleges/universities
```

An "edge" occurs when two vertices are connected to one another

-   in `g_2mode_privu`, an edge occurs when a particular college visits a particular high school

```{r, eval=FALSE}
ecount(g_2mode_privu)
```

<br>

An igraph object (type = list; class = igraph) can be decomposed into:

-   a dataframe that contains characteristics (referred to as "attributes" in network literature) of the vertices
-   a matrix that identifies whether two vertices are connected by an edge

Create a data frame of vertex attributes

```{r, results="hide", eval=FALSE}
df_attr <- as_data_frame(x = g_2mode_privu, what = "vertices")

typeof(df_attr) # type = list
length(df_attr) # length = number of variables
class(df_attr) # data.frame
str(df_attr)
```

```{r, include = FALSE, results="hide", eval=FALSE}
#install.packages("tidyverse")
library(tidyverse) # load tidyverse package
```

__Examine data frame of vertex attributes__

```{r, results = "hide", eval=FALSE}
#df_attr %>% glimpse()
# vertices = universities
df_attr %>% filter(type == TRUE) %>% select(unitid,univ_name_ipeds,state_code_ipeds, classification_ipeds)

# vertices = high schools
df_attr %>% filter(type == FALSE) %>% select(ppin,name_pss,city_pss, state_code_pss)
```

<br>

__Examine individual variables within data frame__

-   takeaway: invidividual variables within data frame are vectors!

```{r, results = "hide", eval=FALSE}
typeof(df_attr$name_pss) # vector of type = character
str(df_attr$name_pss) # length of vector is number of obs

typeof(df_attr$total_enrolled_pss) # vector of type = integer
str(df_attr$total_enrolled_pss)
```

__Create adjacency matrix of which colleges visited which high schools__

```{r, eval=FALSE}
adj_mat <- as_adjacency_matrix(graph = g_2mode_privu, type = "both", 
                               sparse = FALSE)
```

__Investigate adjacency matrix__

```{r, eval=FALSE}
typeof(adj_mat) # a numeric "double" object
class(adj_mat) # matrix class
#str(adj_mat) # 1,356 rows by 1,356 columns
```

__Print a few cells, visits by colleges (columns) to high schools (rows)__

-   colleges: "`139658`" = Emory University; "`147767`" = Northwestern University; "`152080`" = University of Notre Dame

```{r, eval=FALSE}
#syntax: object_name[<rows to print>,<columns to print>]
adj_mat[c("00000226","00000237","00000714"),c("139658","147767","152080")]
```

__Plot the 2-mode network__

```{r, echo = FALSE, eval=FALSE}
plot(
  x = g_2mode_privu, 
  vertex.label = if_else(V(g_2mode_privu)$type, V(g_2mode_privu)$univ_abbrev_ipeds, ""),
  vertex.color = if_else(V(g_2mode_privu)$type, "lightblue", "salmon"),
  vertex.shape = if_else(V(g_2mode_privu)$type, "circle", "circle"),
  vertex.size = if_else(V(g_2mode_privu)$type, 7, 2),
  edge.lty = 1,
  layout = layout_with_kk,
  margin = -.6
)
```
-->
````
