```{r, echo=FALSE, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>", highlight = TRUE)
  #comment = "#>" makes it so results from a code chunk start with "#>"; default is "##"
```

# Introduction

Load packages:

```{r, message=FALSE}
library(tidyverse)
library(stringr)  # package for manipulating strings (part of tidyverse)
library(lubridate)  # package for working with dates and times
#library(rvest)  # package for reading and manipulating HTML
```

**If package not yet installed**, then must install before you load. Install in "console" rather than .Rmd file:

-   Generic syntax: `install.packages("package_name")`
-   Install "tidyverse": `install.packages("tidyverse")`

**Note**: When we load package, name of package is not in quotes; but when we install package, name of package is in quotes:

-   `install.packages("tidyverse")`
-   `library(tidyverse)`

Resources used to create this lecture:

-   https://r4ds.had.co.nz/strings.html
-   https://www.tutorialspoint.com/r/r_strings.htm
-   https://swcarpentry.github.io/r-novice-inflammation/13-supp-data-structures/
-   https://www.statmethods.net/input/datatypes.html
-   https://www.stat.berkeley.edu/\~s133/dates.html

## Dataset we will use

We will use `rtweet` to pull Twitter data from the PAC-12 universities. We will use the university admissions Twitter handle if there is one, or the main Twitter handle for the university if there isn't one:

-   We wrote a short tutorial on using `rtweet` in the Fall 2020 version of this class:
    -   [LINK](https://anyone-can-cook.github.io/rclass1/in_class/rtweet/intro_to_rtweet.html) to html file
    -   [LINK](https://anyone-can-cook.github.io/rclass1/in_class/rtweet/intro_to_rtweet.Rmd) to .Rmd file

```{r}
# library(rtweet)
# 
# p12 <- c("uaadmissions", "FutureSunDevils", "caladmissions", "UCLAAdmission",
#          "futurebuffs", "uoregon", "BeaverVIP", "USCAdmission",
#          "engagestanford", "UtahAdmissions", "UW", "WSUPullman")
# p12_full_df <- search_tweets(paste0("from:", p12, collapse = " OR "), n = 500)
#
# saveRDS(p12_full_df, "p12_dataset.RDS")

# Load previously pulled Twitter data
# p12_full_df <- readRDS("p12_dataset.RDS")
p12_full_df <- readRDS(url("https://github.com/anyone-can-cook/rclass1/raw/master/data/twitter/p12_dataset.RDS", "rb"))
#glimpse(p12_full_df)

p12_df <- p12_full_df %>% select("user_id", "created_at", "screen_name", "text", "location")
head(p12_df)
```

# Review of Data structures and types

What is an **object**?

-   Everything in R is an object
-   We can classify objects based on their **class** and **type**
    -   `class()`: What kind of object is it (high-level)?
        -   The class of the object determines what kind of functions we can apply to it
    -   `typeof()`: What is the object's data type (low-level)?
-   Objects may be combined to form **data structures**

[![](https://d33wubrfki0l68.cloudfront.net/1d1b4e1cf0dc5f6e80f621b0225354b0addb9578/6ee1c/diagrams/data-structures-overview.png){width="400px"}](https://r4ds.had.co.nz/vectors.html)

*Credit: [R for Data Science](https://r4ds.had.co.nz/vectors.html)*

<br> Basic **data types**:

-   Logical (`TRUE`, `FALSE`)
-   Numeric (e.g., `5`, `2.5`)
-   Integer (e.g., `1L`, `4L`, where `L` tells R to store as `integer` type)
-   Character (e.g., `"R is fun"`)

Basic **data structures**:

_Review from Intro to R and Attributes and Class lectures:_ 

-   [Atomic vectors](https://anyone-can-cook.github.io/rclass1-redesign/lectures/updated_lectures/intro_to_r/intro_to_r.html#atomic-vectors)
-   [Lists](https://anyone-can-cook.github.io/rclass1-redesign/lectures/updated_lectures/intro_to_r/intro_to_r.html#lists)
    -   [Dataframes](https://anyone-can-cook.github.io/rclass1-redesign/lectures/updated_lectures/attributes_and_class/attributes_and_class.html#data-frames)



# String basics

What are **strings**?

-   String is a type of data in R
-   You can create strings using either single quotes (`'`) or double quotes (`"`)
    -   Internally, R stores strings using double quotes
-   The `class()` and `typeof()` a string is `character`

<br> **Example**: Creating string using single quotes

Notice how R stores strings using double quotes internally:

```{r}
my_string <- 'This is a string'
my_string
```

<br> **Example**: Creating string using double quotes

```{r}
my_string <- "Strings can also contain numbers: 123"
my_string
```

<br> **Example**: Checking class and type of strings

```{r}
class(my_string)
typeof(my_string)
```

<br> **Note**: To include quotes as part of the string, we can either use the other type of quotes to surround the string (i.e., `'` or `"`) or escape the quote using a backslash (`\`). *We won't be going in-depth into escaping characters for this class, but see appendix for more details if you are interested.*

```{r}
# Include quote by using the other type of quotes to surround the string 
my_string <- "There's no issues with this string."
my_string

# Include quote of the same type by escaping it with a backslash
my_string <- 'There\'s no issues with this string.'
my_string
```

```{r, eval=F}
# This would not work
my_string <- 'There's an issue with this string.'
my_string
```

<br>

# `stringr` package

> "A consistent, simple and easy to use set of wrappers around the fantastic `stringi` package. All function and argument names (and positions) are consistent, all functions deal with `NA`'s and zero length vectors in the same way, and the output from one function is easy to feed into the input of another."

*Credit: `stringr`[R documentation](https://www.rdocumentation.org/packages/stringr/versions/1.4.0)*

The `stringr` package:

-   The `stringr` package is based off the `stringi` package and is part of **Tidyverse**
-   `stringr` contains functions to work with strings
-   For many functions in the `stringr` package, there are equivalent "base R" functions
-   But `stringr` functions all follow the same rules, while rules often differ across different "base R" string functions, so we will focus exclusively on `stringr` functions
-   Most `stringr` functions start with `str_` (e.g., `str_length`)

## `str_length()`

<br> **The `str_length()` function**:

```{r, eval = FALSE}
?str_length

# SYNTAX
str_length(string)
```

-   Function: Find string length
-   Arguments:
    -   `string`: Character vector (or vector coercible to character)
-   Note that `str_length()` calculates the length of a string, whereas the `length()` function (which is not part of `stringr` package) calculates the number of elements in an object

<br>


**Example**: Using `str_length()` on string

```{r}
str_length("cats")
```

Compare to `length()`, which treats the string as a single object:

```{r}
length("cats")
```


**Example**: Using `str_length()` on a character vector

```{r}
str_length(c("cats", "in", "hat"))
```

Compare to `length()`, which finds the number of elements in the vector:

```{r}
length(c("cats", "in", "hat"))
```


<details>

<summary>**Example**: Using `str_length()` on other vectors coercible to character</summary>

Logical vectors can be coerced to character vectors:

```{r}
str_length(c(TRUE, FALSE))
```

Numeric vectors can be coerced to character vectors:

```{r}
str_length(c(1, 2.5, 3000))
```

Integer vectors can be coerced to character vectors:

```{r}
str_length(c(2L, 100L))
```

</details>

<details>

<summary>**Example**: Using `str_length()` on dataframe column</summary>

Recall that the columns in a dataframe are just vectors, so we can use `str_length()` as long as the vector is coercible to character type. Let's look at the `screen_name` column from the `p12_df`:

```{r}
# `p12_df` is a dataframe object
str(p12_df)

# `screen_name` column is a character vector
str(p12_df$screen_name)
```

<br> **\[Base R method\]** Use `str_length()` to calculate the length of each `screen_name`:

```{r}
# Let's focus on just the unique screen names
unique(p12_df$screen_name)

str_length(unique(p12_df$screen_name))
```

<br> **\[Tidyverse method\]** Use `str_length()` to calculate the length of each `screen_name`:

```{r}
# Let's focus on just the unique screen names
p12_df %>% select(screen_name) %>% unique()

#p12_df %>% select(screen_name) %>% unique() %>% str_length()
```

Notice that the above line does not work as expected because we passed in a dataframe to `str_length()` and it is trying to coerce that to character:

```{r}
class(p12_df %>% select(screen_name) %>% unique())
```

An alternative way is to add a column to the dataframe that contains the result of applying `str_length()` to the `screen_name` vector:

```{r}
p12_df %>% select(screen_name) %>% unique() %>% 
  mutate(screen_name_len = str_length(screen_name))
```

</details>

## `str_c()`

<br> **The `str_c()` function**:

```{r, eval = FALSE}
?str_c

# SYNTAX AND DEFAULT VALUES
str_c(..., sep = "", collapse = NULL)
```

-   Function: Concatenate strings between vectors (element-wise)
-   Arguments:
    -   The input is one or more character vectors (or vectors coercible to character)
        -   Zero length arguments are removed
        -   Scalar inputs (vectors of length 1) are recycled to the common length of vector inputs 
    -   `sep`: String to insert between input vectors
    -   `collapse`: Optional string used to combine input vectors into single string

````{=html}
<!--

ADDED 11/3/2020. JUST MESSING AROUND TO SEE HOW IT TREATS ELEMENTS OF A VECTOR VS. SEPARATE VECTORS
```{r}
str_c("a","b", sep = "", collapse = NULL)
str_c("a","b", sep = "~", collapse = NULL)

str_c("a","b", sep = "", collapse = ";")
str_c("a","b", sep = "~", collapse = ";")

str_c(c("a"),c("b"), sep = "~", collapse = ";")
str_c(c("a"),c("b"), sep = "", collapse = ";")
```

```{r}
s1 <- c("s1e1","s1e2","s1e3")
s2 <- c("s2e1","s2e2","s2e3")
s3 <- c("s3e1","s3e2","s3e3","s3e4","s3e5","s3e6")

str_c(s1, sep = "", collapse = NULL)
str_c(s1, sep = "", collapse = NULL) %>% length()
str_c(s1, sep = "", collapse = NULL) %>% str_length()


str_c(s1, sep = "~", collapse = NULL)
str_c(s1, sep = "~", collapse = ";")

```

ADDED 11/3/2020. JUST MESSING AROUND TO SEE HOW IT TREATS ELEMENTS OF A VECTOR VS. SEPARATE VECTORS
```{r}
s1 <- c("a","b","c")
s2 <- c("d","e","f")
s3 <- c("a","b","c","d","e","f")

str_c("a","b", sep = "", collapse = NULL)

str_c(s1, sep = "", collapse = NULL)
str_c(s2, sep = "", collapse = NULL)
str_c(s3, sep = "", collapse = NULL)
str_c(s1,s2, sep = "", collapse = NULL)

# add a separator
str_c(s1, sep = "~", collapse = NULL)
str_c(s2, sep = "~", collapse = NULL)
str_c(s3, sep = "~", collapse = NULL)
str_c(s1,s2, sep = "~", collapse = NULL)

```

-->
````

<br> **Example**: Using `str_c()` on one vector

Since we only provided one input vector, it has nothing to concatenate with, so `str_c()` will just return the same vector:

```{r}
str_c(c("a", "b", "c"))
```

Note that specifying the `sep` argument will also not have any effect because we only have one input vector, and `sep` is the separator between multiple vectors:

```{r}
str_c(c("a", "b", "c"), sep = "~")

# Check length: Output is the original vector of 3 elements
str_c(c("a", "b", "c")) %>% length()
```

As seen above, `str_c()` returns a vector by default (because the default value for the `collapse` argument is `NULL`). But we can specify a string for `collapse` in order to collapse the elements of the output vector into a single string:

```{r}
str_c(c("a", "b", "c"), collapse = "|")

# Check length: Output vector of length 3 is collapsed into a single string
str_c(c("a", "b", "c"), collapse = "|") %>% length()

# Check str_length: This gives the length of the collapsed string, which is 5 characters long
str_c(c("a", "b", "c"), collapse = "|") %>% str_length()
```

<br> **Example**: Using `str_c()` on more than one vector

When we provide multiple input vectors, we can see that the vectors get concatenated element-wise (i.e., 1st element from each vector are concatenated, 2nd element from each vector are concatenated, etc):

```{r}
str_c(c("a", "b", "c"), c("x", "y", "z"), c("!", "?", ";"))
```

The default separator for each element-wise concatenation is an empty string (`""`), but we can customize that by specifying the `sep` argument:

```{r}
str_c(c("a", "b", "c"), c("x", "y", "z"), c("!", "?", ";"), sep = "~")

# Check length: Output vector is same length as input vectors
str_c(c("a", "b", "c"), c("x", "y", "z"), c("!", "?", ";"), sep = "~") %>% length()
```

Again, we can specify the `collapse` argument in order to collapse the elements of the output vector into a single string:

```{r}
str_c(c("a", "b", "c"), c("x", "y", "z"), c("!", "?", ";"), collapse = "|")

# Check length: Output vector of length 3 is collapsed into a single string
str_c(c("a", "b", "c"), c("x", "y", "z"), c("!", "?", ";"), collapse = "|") %>% length()

# Specifying both `sep` and `collapse`
str_c(c("a", "b", "c"), c("x", "y", "z"), c("!", "?", ";"), sep = "~", collapse = "|")
```

<br> **Example**: Using `str_c()` on "strings"

What do we mean by "strings"?

-   Informally, We can think of a "string" as being a character vector with `length()` equal to 1 (i.e., one element).
-   Another way to think of it, a "string" is anything you put in between quotes".
-   Loosely, we can also think of individual elements within a character vector as strings

Below, passing 3 strings into `str_c()` is like passing in 3 vectors of size 1 each.

-   Remember that vectors are concatenated element-wise, so these strings will be joined like this:

```{r}
str_c("a", "b", "c")

# Again, we can think of strings as being character vectors of size 1
str_c(c("a"), c("b"), c("c"))
```

We can use `sep` to specify how the elements are separated:

```{r}
str_c("a", "b", "c", sep = "~")
```

Since we only have 1 element in each vector, the output from `str_c()` is a vector of length 1. Thus, `collapse` will not be useful here since it works to collapse multiple elements in the output vector into a single string:

```{r}
str_c("a", "b", "c", collapse = "|")
```

<br>

<details>

<summary>**Example**: Using `str_c()` on types other than character</summary>

When we provide a non-character vector (such as a numeric or logical vector), it will get coerced into a character vector:

```{r}
str_c(c("a", "b", "c"), c(1, 2, 3), c(TRUE, FALSE, FALSE))

# Specifying both `sep` and `collapse`
str_c(c("a", "b", "c"), c(1, 2, 3), c(TRUE, FALSE, FALSE), sep = "~", collapse = "|")
```

Note that we can also use any other single element input (other than string) that can be coerced to character:

```{r}
str_c(TRUE, 1.5, 2L, "X") 
```

</details>

<details>

<summary>**Example**: Using `str_c()` on vectors of different lengths</summary>

<br> 

When trying to join vectors of different length, you will run into an error. 

-   This is because new recycling rules are applied to the `stringr()` package to be consistent with other tidyverse recycling rules (see Wickham blog 12/05/2022)[https://www.tidyverse.org/blog/2022/12/stringr-1-5-0/#recycling-rules]

-   In practice this makes sense when working with data frames and tidyverse because columns (variables) in a data frame must be the same length. 


```{r, warning = FALSE, eval=FALSE}
str_c("#", c("a", "b", "c", "d"), c(1, 2, 3), c(TRUE, FALSE))

# Specifying both `sep` and `collapse`
str_c("#", c("a", "b", "c", "d"), c(1, 2, 3), c(TRUE, FALSE), sep = "~", collapse = "|")
```

</details>

<details>

<summary>**Example**: Using `str_c()` on dataframe columns</summary>

<br> 

Let's combine the `user_id` and `screen_name` columns from `p12_df`. We'll focus on unique Twitter handles:

```{r}
p12_unique_df <- p12_df %>% select(user_id, screen_name) %>% unique()
p12_unique_df
```

<br> **\[Base R method\]** Use `str_c()` to combine `user_id` and `screen_name`:

```{r}
str_c(p12_unique_df$user_id, "=", p12_unique_df$screen_name, sep = " ", collapse = ", ")
str_c(p12_unique_df$user_id, "=", p12_unique_df$screen_name, sep = " ") # without collapsing to one element
```

<br> **\[Tidyverse method\]** Use `str_c()` to combine `user_id` and `screen_name`:

```{r}
p12_unique_df %>% mutate(twitter_handle = str_c(user_id,screen_name))

p12_unique_df %>% mutate(twitter_handle = str_c("User #", user_id, " is @", screen_name))
```

</details>

## `str_sub()`

<br> **The `str_sub()` function**:

```{r, eval = FALSE}
?str_sub

# SYNTAX AND DEFAULT VALUES
str_sub(string, start = 1L, end = -1L)
str_sub(string, start = 1L, end = -1L, omit_na = FALSE) <- value
```

-   Function: Subset strings
-   Arguments:
    -   `string`: Character vector (or vector coercible to character)
    -   `start`: Position of first character to be included in substring (default: `1`)
    -   `end`: Position of last character to be included in substring (default: `-1`)
        -   Negative index means counting backwards from the end of the string
        -   If an element in the vector is shorter than the specified `end`, it will just include all the available characters that it does have
    -   `omit_na`: If `TRUE`, missing values in any of the arguments provided will result in an unchanged input
-   When `str_sub()` is used in the assignment form, you can replace the subsetted part of the string with a `value` of your choice
    -   If an element in the vector is too short to meet the subset specification, the replacement `value` will be concatenated to the end of that element
    -   Note that this modifies your input vector directly, so you must have the vector saved to a variable (see example below)

<br>

<details>

<summary>**Example**: Using `str_sub()` to subset strings</summary>

<br> 

If no `start` and `end` positions are specified, `str_sub()` will by default return the entire (original) string:

```{r}
str_sub(string = c("abcdefg", 123, TRUE))
```

Note that if an element is shorter than the specified `end` (i.e., `123` in the example below), it will just include all the available characters that it does have:

```{r}
str_sub(string = c("abcdefg", 123, TRUE), start = 2, end = 4)
```

Remember we can also use negative index to count the position starting from the back:

```{r}
str_sub(c("abcdefg", 123, TRUE), start = 2, end = -2)
```

</details>

<details>

<summary>**Example**: Using `str_sub()` to replace strings</summary>

<br> 

If no `start` and `end` positions are specified, `str_sub()` will by default return the original string, so the entire string would be replaced:

```{r}
v <- c("A", "AB", "ABC", "ABCD", "ABCDE")
str_sub(v, start = 1,end =-1)

str_sub(v, start = 1,end =-1) <- "*"
v
```

If an element in the vector is too short to meet the subset specification, the replacement `value` will be concatenated to the end of that element:

```{r}
v <- c("A", "AB", "ABC", "ABCD", "ABCDE")
v
str_sub(v, start = 2, end = 3)

str_sub(v, start = 2, end = 3) <- "*"
v
```

Note that because the replacement form of `str_sub()` modifies the input vector directly, we need to save it in a variable first. Directly passing in the vector to `str_sub()` would give us an error:

```{r, eval = FALSE}
# Does not work
str_sub(c("A", "AB", "ABC", "ABCD", "ABCDE")) <- "*"
```

</details>

<details>

<summary>**Example**: Using `str_sub()` on dataframe column</summary>

<br> 

We can use `as.character()` to turn the `created_at` value to a string, then use `str_sub()` to extract out various date/time components from the string:

```{r}

#str(p12_df %>% select(created_at))
typeof(p12_df$created_at)
class(p12_df$created_at)
head(p12_df$created_at)

p12_datetime_df <- p12_df %>% select(created_at) %>%
  mutate(
      dt_chr = as.character(created_at), #convert to character
      date_chr = str_sub(dt_chr, 1, 10), #subset values in position 1 and 10 to grab the date
      yr_chr = str_sub(dt_chr, 1, 4), #subset values in position 1 and 4 to grab the year
      mth_chr = str_sub(dt_chr, 6, 7),
      day_chr = str_sub(dt_chr, 9, 10),
      hr_chr = str_sub(dt_chr, -8, -7),
      min_chr = str_sub(dt_chr, -5, -4),
      sec_chr = str_sub(dt_chr, -2, -1)
    )
p12_datetime_df
```

</details>

## Other `stringr` functions

Other useful `stringr` functions:

-   `str_to_upper()`: Turn strings to uppercase
-   `str_to_lower()`: Turn strings to lowercase
-   `str_sort()`: Sort a character vector
-   `str_trim()`: Trim whitespace from strings (including `\n`, `\t`, etc.)
-   `str_pad()`: Pad strings with specified character

<br>

<details>

<summary>**Example**: Using `str_to_upper()` to turn strings to uppercase</summary>

<br> 

Turn column names of `p12_df` to uppercase:

```{r}
# Column names are originally lowercase
names(p12_df)

# Turn column names to uppercase
names(p12_df) <- str_to_upper(names(p12_df))
names(p12_df)
```

</details>

<details>

<summary>**Example**: Using `str_to_lower()` to turn strings to lowercase</summary>

<br> 

Turn column names of `p12_df` to lowercase:

```{r}
# Column names are originally uppercase
names(p12_df)

# Turn column names to lowercase
names(p12_df) <- str_to_lower(names(p12_df))
names(p12_df)
```

</details>

<details>

<summary>**Example**: Using `str_sort()` to sort character vector</summary>

<br> 

Sort the vector of `p12_df` column names:

```{r}
# Before sort
names(p12_df)

# Sort alphabetically (default)
str_sort(names(p12_df))

# Sort reverse alphabetically
str_sort(names(p12_df), decreasing = TRUE)
```

</details>

<details>

<summary>**Example**: Using `str_trim()` to trim whitespace from string</summary>

<br> 

```{r}
# Trim whitespace from both left and right sides (default)
str_trim(c("\nABC ", " XYZ\t"))

# Trim whitespace from left side
str_trim(c("\nABC ", " XYZ\t"), side = "left")

# Trim whitespace from right side
str_trim(c("\nABC ", " XYZ\t"), side = "right")
```

</details>

<details>

<summary>**Example**: Using `str_pad()` to pad string with character</summary>

<br> 

Let's say we have a vector of zip codes that has lost all leading 0's. We can use `str_pad()` to add that back in:

```{r}
# Pad the left side of strings with "0" until width of 5 is reached
str_pad(c(95035, 90024, 5009, 5030), width = 5, side = "left", pad = "0")
```

</details>

# Dates and times

> "Date-time data can be frustrating to work with in R. R commands for date-times are generally unintuitive and change depending on the type of date-time object being used. Moreover, the methods we use with date-times must be robust to time zones, leap days, daylight savings times, and other time related quirks, and R lacks these capabilities in some situations. Lubridate makes it easier to do the things R does with date-times and possible to do the things R does not."

*Credit: `lubridate`[documentation](https://lubridate.tidyverse.org/)*

**How are dates and times stored in R?** (From [Dates and Times in R](https://www.stat.berkeley.edu/~s133/dates.html))

-   The `Date` class is used for storing dates
    -   "Internally, `Date` objects are stored as the number of days since January 1, 1970, using negative numbers for earlier dates. The `as.numeric()` function can be used to convert a `Date` object to its internal form."
-   POSIX classes can be used for storing date plus times
    -   "The `POSIXct` class stores date/time values as the number of seconds since January 1, 1970"
    -   "The `POSIXlt` class stores date/time values as a list of components (hour, min, sec, mon, etc.) making it easy to extract these parts"
-   There is no native R class for storing only time

**Why use date/time objects?**

-   Using date/time objects makes it easier to fetch or modify various date/time components (e.g., year, month, day, day of the week)
    -   Compared to if the date/time is just stored in a string, these components are not as readily accessible and need to be parsed
-   You can perform certain arithmetics with date/time objects (e.g., find the "difference" between date/time points)

## Creating date/time objects by parsing input

Functions that create date/time objects **by parsing character or numeric input**:

-   Create `Date` object: `ymd()`, `ydm()`, `mdy()`, `myd()`, `dmy()`, and `dym()`
    -   `y` stands for year, `m` stands for month, `d` stands for day
    -   Select the function that represents the order in which your date input is formatted, and the function will be able to parse your input and create a `Date` object
-   Create `POSIXct` object: `ymd_h()`, `ymd_hm()`, `ymd_hms()`, etc.
    -   `h` stands for hour, `m` stands for minute, `s` stands for second
    -   For any of the previous 6 date functions, you can append `h`, `hm`, or `hms` if you want to provide additional time information in order to create a `POSIXct` object
    -   To force a `POSIXct` object without providing any time information, you can just provide a timezone (using `tz`) to one of the date functions and it will assume midnight as the time
    -   You can use `Sys.timezone()` to get the timezone for your location

<br>

<details>

<summary>**Example**: Creating `Date` object from character or numeric input</summary>

The `lubridate` functions are flexible and can parse dates in various formats:

```{r}
d <- mdy("1/1/2024")
d

d <- mdy("1-1-2024")
d

d <- mdy("Jan. 1, 2024")
d

d <- ymd(20240101)
d
```

<br> Investigate the `Date` object:

```{r}
class(d)
typeof(d)

# Number of days since January 1, 1970
as.numeric(d)
```

</details>

<details>

<summary>**Example**: Creating `Date` objects from dataframe column</summary>

Using the `p12_datetime_df` we created earlier, we can create `Date` objects from the `date_chr` column:

```{r}
# Use `ymd()` to parse the string stored in the `date_chr` column
p12_datetime_df %>% select(created_at, dt_chr, date_chr) %>%
  mutate(date_ymd = ymd(date_chr))
```

</details>

<details>

<summary>**Example**: Creating `POSIXct` object from character or numeric input</summary>

The `lubridate` functions are flexible and can parse AM/PM in various formats:

```{r}
dt <- mdy_h("12/31/2023 11pm")
dt

dt <- mdy_hm("12/31/2023 11:59 pm")
dt

dt <- mdy_hms("12/31/2023 11:59:59 PM")
dt

dt <- ymd_hms(20231231235959)
dt
```

<br> Investigate the `POSIXct` object:

```{r}
class(dt)
typeof(dt)

# Number of seconds since January 1, 1970
as.numeric(dt)
```

<br> We can also create a `POSIXct` object from a date function by providing a timezone. The time would default to midnight:

```{r}
dt <- mdy("1/1/2024", tz = "UTC")
dt

# Number of seconds since January 1, 1970
as.numeric(dt)  # Note that this is indeed 1 sec after the previous example
```

</details>

<details>

<summary>**Example**: Creating `POSIXct` objects from dataframe column</summary>

Using the `p12_datetime_df` we created earlier, we can recreate the `created_at` column (class `POSIXct`) from the `dt_chr` column (class `character`):

```{r}
# Use `ymd_hms()` to parse the string stored in the `dt_chr` column
p12_datetime_df %>% select(created_at, dt_chr) %>%
  mutate(datetime_ymd_hms = ymd_hms(dt_chr))
```

</details>

## Creating date/time objects from individual components

Functions that create date/time objects **from various date/time components**:

-   Create `Date` object: `make_date()`
    -   Syntax and default values: `make_date(year = 1970L, month = 1L, day = 1L)`
    -   All inputs are coerced to integer
-   Create `POSIXct` object: `make_datetime()`
    -   Syntax and default values: `make_datetime(year = 1970L, month = 1L, day = 1L, hour = 0L, min = 0L, sec = 0, tz = "UTC")`
    - Input values should be numeric 

<br>

<details>

<summary>**Example**: Creating `Date` object from individual components</summary>

There are various ways to pass in the inputs to create the same `Date` object:

```{r}
d <- make_date(2024, 1, 1)
d

# Characters can be coerced to integers
d <- make_date("2024", "01", "01")
d

# Remember that the default values for month and day would be 1L
d <- make_date(2024)
d
```

</details>

<details>

<summary>**Example**: Creating `Date` objects from dataframe columns</summary>

Using the `p12_datetime_df` we created earlier, we can create `Date` objects from the various date component columns:

```{r}
# Use `make_date()` to create a `Date` object from the `yr_chr`, `mth_chr`, `day_chr` fields
p12_datetime_df %>% select(created_at, dt_chr, yr_chr, mth_chr, day_chr) %>%
  mutate(date_make_date = make_date(year = yr_chr, month = mth_chr, day = day_chr))
```

</details>

<details>

<summary>**Example**: Creating `POSIXct` object from individual components</summary>

```{r}
# Inputs should be numeric
d <- make_datetime(2023, 12, 31, 23, 59, 59)
d
```

</details>

<details>

<summary>**Example**: Creating `POSIXct` objects from dataframe columns</summary>

Using the `p12_datetime_df` we created earlier, we can recreate the `created_at` column (class `POSIXct`) from the various date and time component columns (class `character`):

```{r}
# Use `make_datetime()` to create a `POSIXct` object from the `yr_chr`, `mth_chr`, `day_chr`, `hr_chr`, `min_chr`, `sec_chr` fields
# Convert inputs to integers first
p12_datetime_df %>%
  mutate(datetime_make_datetime = make_datetime(
    as.integer(yr_chr), as.integer(mth_chr), as.integer(day_chr), 
    as.integer(hr_chr), as.integer(min_chr), as.integer(sec_chr)
  )) %>%
  select(datetime_make_datetime, yr_chr, mth_chr, day_chr, hr_chr, min_chr, sec_chr)
```

</details>

## Date/time object components

Storing data using date/time objects makes it easier to **get and set** the various date/time components.

-   Basic accessor functions:
    -   `date()`: Date component
    -   `year()`: Year
    -   `month()`: Month
    -   `day()`: Day
    -   `hour()`: Hour
    -   `minute()`: Minute
    -   `second()`: Second
    -   `week()`: Week of the year
    -   `wday()`: Day of the week (`1` for Sunday to `7` for Saturday)
    -   `am()`: Is it in the am? (returns `TRUE` or `FALSE`)
    -   `pm()`: Is it in the pm? (returns `TRUE` or `FALSE`)
-   To **get** a date/time component, you can simply pass a date/time object to the function
    -   Syntax: `accessor_function(<date/time_object>)`
-   To **set** a date/time component, you can assign into the accessor function to change the component
    -   Syntax: `accessor_function(<date/time_object>) <- "new_component"`
    -   Note that `am()` and `pm()` can't be set. Modify the time components instead.

<br>

<details>

<summary>**Example**: Getting date/time components</summary>

```{r}
# Create datetime for New Year's Eve
dt <- make_datetime(2023, 12, 31, 23, 59, 59)
dt
dt %>% class()

# Get date
date(dt)

# Get hour
hour(dt)

# Is it pm?
pm(dt)

# Day of the week (1 = Sunday)
wday(dt)

year(dt)
```

</details>

<details>

<summary>**Example**: Setting date/time components</summary>

```{r}
# Create datetime for New Year's Eve
dt <- make_datetime(2023, 12, 31, 23, 59, 59)
dt

# Get week of year
week(dt)

# Set week of year (move back 1 week)
week(dt) <- week(dt) - 1

# Date now moved from New Year's Eve to Christmas Eve
dt

# Set day to Christmas Day
day(dt) <- 25

# Date now moved from Christmas Eve to Christmas Day
dt
```

</details>

<details>

<summary>**Example**: Getting date/time components from dataframe column</summary>

Using the `p12_datetime_df` we created earlier, we can isolate the various date/time components from the `POSIXct` object in the `created_at` column:

```{r}
# The extracted date/time components will be of numeric type
p12_datetime_df %>% select(created_at) %>%
  mutate(
    yr_num = year(created_at),
    mth_num = month(created_at),
    day_num = day(created_at),
    hr_num = hour(created_at),
    min_num = minute(created_at),
    sec_num = second(created_at),
    ampm = ifelse(am(created_at), 'AM', 'PM')  # am()/pm() returns TRUE/FALSE
  )
```

</details>

## Time spans

![](https://raw.githubusercontent.com/anyone-can-cook/rclass2/master/assets/images/time_spans.png)

3 ways to represent time spans (From [lubridate cheatsheet](https://rawgit.com/rstudio/cheatsheets/master/lubridate.pdf))

-   **Intervals** represent specific intervals of the timeline, bounded by start and end date-times
    -   Example: People with birthdays between the **interval** October 23 to November 22 are Scorpios
-   **Periods** track changes in clock times, which ignore time line irregularities
    -   Example: Daylight savings time ends at the beginning of November and we gain an hour - this extra hour is *ignored* when determining the **period** between October 23 to November 22
-   **Durations** track the passage of physical time, which deviates from clock time when irregularities occur
    -   Example: Daylight savings time ends at the beginning of November and we gain an hour - this extra hour is *added* when determining the **duration** between October 23 to November 22

**Time spans using `lubridate`**

Using the `lubridate` package for time spans:

-   **Interval**
    -   Create an interval using `interval()` or `%--%`
        -   Syntax: `interval(<date/time_object1>, <date/time_object2>)` or `<date/time_object1> %--% <date/time_object2>`
-   **Periods**
    -   "Periods are time spans but don’t have a fixed length in seconds, instead they work with '*human*' times, like days and months." (From [R for Data Science](https://r4ds.had.co.nz/dates-and-times.html#periods))
    -   Create periods using functions whose name is the time unit pluralized (e.g., `years()`, `months()`, `weeks()`, `days()`, `hours()`, `minutes()`, `seconds()`)
        -   Example: `days(1)` creates a period of 1 day - it does not matter if this day happened to have an extra hour due to daylight savings ending, since periods do not have a physical length

            ```{r}
            days(1)
            ```
    -   You can add and subtract periods
    -   You can also use `as.period()` to get period of an interval
-   **Durations**
    -   Durations keep track of the physical amount of time elapsed, so it is "stored as seconds, the only time unit with a consistent length" (From [lubridate cheatsheet](https://rawgit.com/rstudio/cheatsheets/master/lubridate.pdf))
    -   Create durations using functions whose name is the time unit prefixed with a `d` (e.g., `dyears()`, `dweeks()`, `ddays()`, `dhours()`, `dminutes()`, `dseconds()`)
        -   Example: `ddays(1)` creates a duration of `86400s`, using the standard conversion of `60` seconds in an minute, `60` minutes in an hour, and `24` hours in a day:

            ```{r}
            ddays(1)
            ```

            Notice that the output says this is equivalent to *approximately* `1` day, since it acknowledges that not all days have `24` hours. In the case of daylight savings, one particular day may have `25` hours, so the duration of that day should be represented as:

            ```{r}
            ddays(1) + dhours(1)
            ```
    -   You can add and subract durations
    -   You can also use `as.duration()` to get duration of an interval

<br>

<details>

<summary>**Example**: Working with interval</summary>

```{r}
# Use `Sys.timezone()` to get timezone for your location (time is midnight by default)
scorpio_start <- ymd("2023-10-23", tz = Sys.timezone())
scorpio_end <- ymd("2023-11-22", tz = Sys.timezone())

scorpio_start
# These datetime objects have class `POSIXct`
class(scorpio_start)

# Create interval for the datetimes
scorpio_interval <- scorpio_start %--% scorpio_end  # or `interval(scorpio_start, scorpio_end)`
scorpio_interval <- interval(scorpio_start, scorpio_end)
scorpio_interval

# The object has class `Interval`
class(scorpio_interval)
as.numeric(scorpio_interval)
```

</details>

<details>

<summary>**Example**: Working with period</summary>

If we use `as.period()` to get the period of `scorpio_interval`, we see that it is a period of `30` days. We do not worry about the extra `1` hour gained due to daylight savings ending:

```{r}
# Period is 30 days
scorpio_period <- as.period(scorpio_interval)
scorpio_period

# The object has class `Period`
class(scorpio_period)
```

<br> Because periods work with "human" times like days, it is more intuitive. For example, if we add a period of `30` days to the `scorpio_start` datetime object, we get the expected end datetime that is `30` days later:

```{r}
# Start datetime for Scorpio birthdays (time is midnight)
scorpio_start

# After adding 30 day period, we get the expected end datetime (time is midnight)
scorpio_start + days(30)
```

</details>

<details>

<summary>**Example**: Working with duration</summary>

If we use `as.duration()` to get the duration of `scorpio_interval`, we see that it is a duration of `2595600` seconds. It takes into account the extra `1` hour gained due to daylight savings ending:

```{r}
# Duration is 2595600 seconds, which is equivalent to 30 24-hr days + 1 additional hour
scorpio_duration <- as.duration(scorpio_interval)
scorpio_duration

# The object has class `Duration`
class(scorpio_duration)

# Using the standard 60s/min, 60min/hr, 24hr/day conversion,
# confirm duration is slightly more than 30 "standard" (ie. 24-hr) days
2595600 / (60 * 60 * 24)

# Specifically, it is 30 days + 1 hour, if we define a day to have 24 hours
seconds_to_period(scorpio_duration)
```

<br> Because durations work with physical time, when we add a duration of `30` days to the `scorpio_start` datetime object, we do not get the end datetime we'd expect:

```{r}
# Start datetime for Scorpio birthdays (time is midnight)
scorpio_start

# After adding 30 day duration, we do not get the expected end datetime
# `ddays(30)` adds the number of seconds in 30 standard 24-hr days, but one of the days has 25 hours
scorpio_start + ddays(30)

# We need to add the additional 1 hour of physical time that elapsed during this time span
scorpio_start + ddays(30) + dhours(1)
```

</details>

<br>

# Appendix

## Special Characters

> "A sequence in a string that starts with a `\` is called an **escape sequence** and allows us to include special characters in our strings."

*Credit: [Escape sequences](https://campus.datacamp.com/courses/string-manipulation-with-stringr-in-r/string-basics?ex=4) from DataCamp*

<br> **Special characters** are characters that will not be interpreted literally.

Common **special characters**:

-   `\n`: newline
-   `\t`: tab
-   `\`: used for escaping purposes
    -   `\'`: literal single quote
    -   `\"`: literal double quote
    -   `\\`: literal backslash

These characters followed by a backslash `\` take on a new meaning. The `n` by itself is just an `n`. When you add a backslash to the `\n` you are escaping it and making it a special character where `\n` now represents a newline.

<br> **The `writeLines()` function**:

```{r, eval = FALSE}
?writeLines

# SYNTAX AND DEFAULT VALUES
writeLines(text, con = stdout(), sep = "\n", useBytes = FALSE)
```

-   "`writeLines()` displays quotes and backslashes as they would be read, rather than as R stores them." (From [writeLines](https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/writeLines) documentation)
-   When we include **escape sequences** in the string, it is helpful to use `writeLines()` to see how the escaped string looks
-   `writeLines()` will also output the string without showing the outer pair of double quotes that R uses to store it, so we only see the content of the string

<br>

<details>

<summary>**Example**: Escaping single quotes</summary>

```{r}
my_string <- 'Escaping single quote \' within single quotes'
my_string
```

Alternatively, we could've just created the string using double quotes:

```{r}
my_string <- "Single quote ' within double quotes does not need escaping"
my_string
```

Using `writeLines()` shows us only the content of the string without the outer pair of double quotes that R uses to store strings:

```{r}
writeLines(my_string)
```

</details>

<details>

<summary>**Example**: Escaping double quotes</summary>

```{r}
my_string <- "Escaping double quote \" within double quotes"
my_string
```

Alternatively, we could've just created the string using single quotes:

```{r}
my_string <- 'Double quote " within single quotes does not need escaping'
my_string
```

Notice how the backslash still showed up in the above output to escape our double quote from the outer pair of double quotes that R uses to store the string. This is no longer an issue if we use `writeLines()` to only show the string content:

```{r}
writeLines(my_string)
```

</details>

<details>

<summary>**Example**: Escaping double quotes within double quotes</summary>

```{r}
my_string <- "I called my mom and she said \"Echale ganas!\""
my_string
```

Using `writeLines()` shows us only the content of the string without the backslashes:

```{r}
writeLines(my_string)
```

</details>

<details>

<summary>**Example**: Escaping backslashes</summary>

To include a literal backslash in the string, we need to escape the backslash with another backslash:

```{r}
my_string <- "The executable is located in C:\\Program Files\\Git\\bin"
my_string
```

Use `writeLines()` to see the escaped string:

```{r}
writeLines(my_string)
```

</details>

<details>

<summary>**Example**: Other special characters</summary>

```{r}
my_string <- "A\tB\nC\tD"
my_string
```

Use `writeLines()` to see the escaped string:

```{r}
writeLines(my_string)
```

</details>

**Escape special characters using Twitter data**

Let's take a look at some tweets from our PAC-12 universities.

-   Let's start by grabbing observations 1-3 from the `text` column.

```{r}
#Twitter example of \n newline special characters
p12_df$text[1:3]
```

-   Using `writeLines()` we can see the contents of the strings as they would be read, rather than as R stores them.

```{r}
writeLines(p12_df$text[1:3])
```

<details>

<summary>**Example**: Escaping double quotes using Twitter data</summary>

-   Using Twitter data you may encounter a lot of strings with double quotes.

    -   In the example below, our string includes special characters `\"` and `\n` to escape the double quotes and the newline character.

    ```{r}
    #Twitter example of \" double quotes special characters
    p12_df$text[24]
    ```

-   Using `writeLines()` we can see the contents of the strings as they would be read, rather than as R stores them.

    -   We no longer see the escaped characters `\"` or `\n`

    ```{r}
    writeLines(p12_df$text[24])
    ```

</details>
