Useful R Stuff https://abraver.github.io/usefulRstuff/index.html quarto-1.3.361 Fri, 17 May 2024 05:00:00 GMT Tables for Word in flextable Aaron Braver https://abraver.github.io/usefulRstuff/posts/tables_for_word_flextable/index.html
Setup

On those occasions where I have a coauthor working in Word rather than LaTeX, it’s sometimes useful to be able to output nicely formatted, editable tables directly from R. I like the tables from gt, but they don’t render certain style and formatting options correctly in Word or RTF output (see this issue).1

So, in this post I’ll show some tables made with flextable, which has better Word support.

There are a few options if you don’t need editable output, or output with fancy formatting.

Editable but not pretty

The option I use the most, for sending tables to Word or Excel is clipr::write_clip(df), which writes df to the clipboard for easy pasting. There’s also clipr::write_last_clip() which can be called right after you create a dataframe and copies it to the clipboard.

Pretty but not editable

Another option is to save a gt as an image or a pdf:

head(mtcars, 3) |> 
  gt() |> 
  gtsave("table.png") # or table.pdf, or whatever

(Saving as an image requires that webshot2 be installed.)

flextable

Piping a dataframe to flextable() produces a pretty decent initial result:

palmerpenguins::penguins |> 
  select(species, island, bill_length_mm, sex, year) |> 
  slice_sample(n = 5) |>  # select 5 random rows
  flextable()

species

island

bill_length_mm

sex

year

Gentoo

Biscoe

49.1

male

2,009

Adelie

Dream

39.5

female

2,007

Chinstrap

Dream

46.8

female

2,009

Gentoo

Biscoe

46.8

female

2,009

Adelie

Biscoe

40.6

male

2,008

While we’re at it, let’s change the formatting of that year column. colformat_num targets numeric columns; the j argument narrows it down to specific columns if needed. colformat_num passes ... to format(), so you can use most format() arguments—here I’ve just used big.mark.

penguin_flextable_1 <- palmerpenguins::penguins |> 
  select(species, island, bill_length_mm, sex, year) |> 
  slice_sample(n = 5) |> 
  flextable() |> 
  colformat_num(j = "year", big.mark = "")

penguin_flextable_1

species

island

bill_length_mm

sex

year

Chinstrap

Dream

49.0

male

2008

Gentoo

Biscoe

52.2

male

2009

Adelie

Dream

42.2

female

2007

Gentoo

Biscoe

50.8

male

2009

Adelie

Dream

42.3

male

2007

You can also format columns with a function using set_formatter(). We could, for example, make the species column lowercase and the island column uppercase:

penguin_flextable_1 |> 
  set_formatter(species = tolower, island = toupper)

species

island

bill_length_mm

sex

year

chinstrap

DREAM

49.0

male

2008

gentoo

BISCOE

52.2

male

2009

adelie

DREAM

42.2

female

2007

gentoo

BISCOE

50.8

male

2009

adelie

DREAM

42.3

male

2007

You can also pass a anonymous function if you need to specify arguments:

penguin_flextable_1 |> 
  set_formatter(sex = \(x) ifelse(x == "male", "m", "f"))

species

island

bill_length_mm

sex

year

Chinstrap

Dream

49.0

m

2008

Gentoo

Biscoe

52.2

m

2009

Adelie

Dream

42.2

f

2007

Gentoo

Biscoe

50.8

m

2009

Adelie

Dream

42.3

m

2007

A particularly good use of this is using one of the labeller functions from scales, e.g. if you want to format currency:

data.frame(A = c(12.1, 21.56, 3.64), B = c(44.0, 54.44, 6)) |> 
  flextable() |> 
  set_formatter(A = scales::label_dollar())

A

B

$12.10

44.00

$21.56

54.44

$3.64

6.00

Multiple heading levels/grouping

You can also make fairly nice tables with a grouping structure in the header using separate_header(). This function will interpret column names with a separator (e.g. “_” as groups and sub-groups.

Consider the following (unremarkable) dataframe which summarizes penguin bill length by species and sex:

penguin_data <- palmerpenguins::penguins |> 
  select(species, sex, bill_length_mm, year) |> 
  drop_na(sex) |> 
  group_by(species, sex, year) |> 
  summarize(bill.length = mean(bill_length_mm), .groups = "drop")

penguin_data |> 
  slice_sample(n = 5)
species sex year bill.length
Adelie female 2008 36.51600
Gentoo female 2009 46.26000
Chinstrap female 2008 46.00000
Gentoo male 2008 48.53913
Chinstrap male 2009 51.10000

Let’s pipe that into flextable, repeat the year column formatting from before, and also round the bill.length column to 2 digits.

penguin_flextable_2 <- penguin_data |> 
  slice_sample(n = 5) |> 
  flextable() |> 
  colformat_num(j = "year", big.mark = "") |> 
  colformat_double(digits = 2)

penguin_flextable_2

species

sex

year

bill.length

Chinstrap

male

2008

51.40

Gentoo

female

2009

46.26

Chinstrap

female

2009

47.01

Gentoo

male

2007

49.00

Chinstrap

female

2008

46.00

We can make this a lot easier to read if we have species and island as header rows. We’ll start by pivot_wider-ing these two columns. Note that pivot_wider defaults to combining species and island with an underscore (this becomes important momentarily):

penguin_flextable_3 <- penguin_data |> 
  pivot_wider(names_from = c("species", "sex"), values_from = "bill.length") |> 
  flextable() |> 
  colformat_num(j = "year", big.mark = "") |> # reapply our formatting from before
  colformat_double(digits = 2)

penguin_flextable_3

year

Adelie_female

Adelie_male

Chinstrap_female

Chinstrap_male

Gentoo_female

Gentoo_male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

separate_header() parses the column names using the underscore, and makes two header levels:

penguin_flextable_4 <- penguin_flextable_3 |> 
  separate_header()

penguin_flextable_4

year

Adelie

Chinstrap

Gentoo

female

male

female

male

female

male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

You can change separator with `separate_header(…, split = “…regex…”). This is useful if your variable names already use underscore as a separator.

Note that since in the combined column names, species comes before sex (e.g. Adelie_male), it’s been used as the top header row. If you switch the order of the names_from argument in pivot_wider, you get sex as the top row. This gives a slightly unexpected result: why are female/male repeated across each category?

penguin_data |> 
  pivot_wider(names_from = c("sex", "species"), values_from = "bill.length") |> 
  flextable() |> 
  separate_header() |> 
  colformat_num(j = "year", big.mark = "") |> # reapply our formatting from before
  colformat_double(digits = 2)

year

female

male

female

male

female

male

Adelie

Adelie

Chinstrap

Chinstrap

Gentoo

Gentoo

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

separate_header() doesn’t rearrange columns to put relevant groups next to each other. If you want a spanning column, you’ll need to reorder the columns manually with relocate():

penguin_data |> 
  pivot_wider(names_from = c("sex", "species"), values_from = "bill.length") |>
  relocate(starts_with("female")) |> 
  flextable() |> 
  separate_header() |> 
  colformat_num(j = "year", big.mark = "") |> # reapply our formatting from before
  colformat_double(digits = 2)

female

year

male

Adelie

Chinstrap

Gentoo

Adelie

Chinstrap

Gentoo

37.92

46.57

45.06

2007

39.95

50.88

49.00

36.52

46.00

45.30

2008

40.60

51.40

48.54

37.41

47.01

46.26

2009

40.56

51.10

50.88

With savvy use of pivot_wider(names_from = ...) combined with relocate, you can the ordering of header rows and spanning columns you’re after.

Change labels

We can edit text/values with labelizor(). The example below changes all instances of female/male to f/m:

penguin_flextable_4 |> 
  labelizor(labels = c("female" = "f", "male" = "m"))

year

Adelie

Chinstrap

Gentoo

f

m

f

m

f

m

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

Adding the j argument selects columns to restrict the changes to:

penguin_flextable_4 |> 
  labelizor(j = c("Adelie_female", "Chinstrap_female"), labels = c("female" = "f", "male" = "m"))

year

Adelie

Chinstrap

Gentoo

f

male

f

male

female

male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

Note that for the j argument, the rownames are based on the underlying df, not what you seen in the flextable. That’s why I have to refer to Adelie_female, which was one of the columns that resulted from out pivot_wider()ing. You can do a lot with j (and in other functions that support it, i for rows), including using logical expressions to filter columns. See here for more details. The labels argument takes name-value pairs where the name is the value to replace and the value is the replacement value.

Conditional formatting

You can target cells with specific values for various kinds of formatting. For example, color() sets the text color. The color argument takes either a vector of colors or a function which returns a vector of colors—here I’ve just used an anonymous function to color code based on whether a value is below or above 40:

penguin_flextable_4 |> 
  color(color = \(x) ifelse(x < 40, "red", "black"))

year

Adelie

Chinstrap

Gentoo

female

male

female

male

female

male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

You can get more fancy and use a scale from the scales package:

penguin_flextable_4 |> 
  color(color = scales::col_numeric(palette = "viridis", domain = c(30, 52)))
Warning: Some values were outside the color scale and will be treated as NA

year

Adelie

Chinstrap

Gentoo

female

male

female

male

female

male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

 
penguin_flextable_4 |> 
  color(color = scales::col_bin(palette = "viridis", domain = c(30, 60), bins = 3))
Warning: Some values were outside the color scale and will be treated as NA

year

Adelie

Chinstrap

Gentoo

female

male

female

male

female

male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

You can also select rows or columns with the i and j arguments, using formula syntax. To target rows, provide i with a formula to filter through the rows. Here we color in red rows for years greater than 2007:

penguin_flextable_4 |> 
  color(i = ~ year > 2007, color = "red" )

year

Adelie

Chinstrap

Gentoo

female

male

female

male

female

male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

You can restrict this to specific columns with the j argument:

penguin_flextable_4 |> 
  color(i = ~ year > 2007, j = c("year", "Adelie_female"), color = "red" )

year

Adelie

Chinstrap

Gentoo

female

male

female

male

female

male

2007

37.92

39.95

46.57

50.88

45.06

49.00

2008

36.52

40.60

46.00

51.40

45.30

48.54

2009

37.41

40.56

47.01

51.10

46.26

50.88

Getting flextables into Word

There are three main ways to get a flextable into Word: open a temporary preview file that you can copy from with print(preview = "docx"), save to a Word file with save_as_docx(), or insert into an existing Word document with the officer package.

Temporary Word file

This is probably the easiest method. The following code opens a new temporary document in Word (assuming it’s installed on your computer) that contains the flextable object.

penguin_flextable_4 |> 
  print(preview = "docx")

That’s it! You can now copy and paste that table wherever you need. NB: this creates a temporary file which will be deleted when your R session closes, so save any output you want in a separate document.

Save to a new Word file

save_as_docx will do as the name suggests—insert the flextable into a new .docx file whose location you specify:

penguin_flextable_4 |> 
  save_as_docx(path = "penguin_table.docx") # I find it necessary to explicitly name the path argument

Import into an existing Word file.

The package officer lets you read a Word document into R, manipulate it, and save it again with any additions or changes.

Read in an existing Word document with officer::read_docx(). Here I’ll read the penguin_table.docx document we created above.:

penguin_doc <- officer::read_docx("penguin_table.docx")

Now add your flextable with flextable::body_add_flextable():

penguin_doc <- flextable::body_add_flextable(penguin_doc, penguin_flextable_4)

And finally, save this Word object as a Word document:

penguin_doc |> 
  print(target = "penguin_table.docx")

Conclusion

That’s it! flextable has a bunch more functionality, but these are the things I find myself using regularly. Check out the flextable page or the flextable book (free online).

Footnotes

  1. If you’re reading this some time in the future, it might be worth checking that issue to see if it’s been resolved.↩︎

]]> tables https://abraver.github.io/usefulRstuff/posts/tables_for_word_flextable/index.html Fri, 17 May 2024 05:00:00 GMT Sort numbered strings (or factor levels, or filenames) correctly even without zero-padding Aaron Braver https://abraver.github.io/usefulRstuff/posts/naturalsort/index.html
Setup
library(tidyverse)
library(naturalsort)

It’s quite common to see lists of items (data, files, etc) that are numbered, such as the hypothetical list of files below:

filenames <- c("file2.csv", "file1.csv",  "file3.csv",
               "file11.csv", "file10.csv", "file20.csv")
print(filenames)
[1] "file2.csv"  "file1.csv"  "file3.csv"  "file11.csv" "file10.csv"
[6] "file20.csv"

If you want to sort these by number, you run into a problem, since the filenames are strings: 1 is followed by 10, which is followed by 2, since 10 precedes 2 “alphabetically”:

filenames |> sort()
[1] "file1.csv"  "file10.csv" "file11.csv" "file2.csv"  "file20.csv"
[6] "file3.csv"

One solution is to rename your items such that they are zero-padded. A kludge with stringr’s str_replace()andstr_pad()can get the job done. Because of the leading zeros,sort() will get the result you expect:

padded <- str_replace(filenames, "[0-9]+", \(x) str_pad(x, 2, pad="0"))
padded |> sort()
[1] "file01.csv" "file02.csv" "file03.csv" "file10.csv" "file11.csv"
[6] "file20.csv"

Rather than renaming your items, naturalsort::naturalsort() orders your items in “human natural” order:

filenames |> naturalsort::naturalsort()
[1] "file1.csv"  "file2.csv"  "file3.csv"  "file10.csv" "file11.csv"
[6] "file20.csv"

The naturalsort package also comes with the command naturalfactor(), which can reorder a factor in the same way, or turn an unordered list of strings into a factor:

my_factor <- factor(c("level_1", "level_10", "level_2"))
naturalsort::naturalfactor(my_factor)
[1] level_1  level_10 level_2 
Levels: level_1 < level_2 < level_10
c("level1", "level10", "level2") |> naturalfactor()
[1] level1  level10 level2 
Levels: level1 < level2 < level10
]]> data cleaning sorting https://abraver.github.io/usefulRstuff/posts/naturalsort/index.html Fri, 12 Jan 2024 06:00:00 GMT View the structure of deeply nested objects or complex function calls with lobstr and listviewer Aaron Braver https://abraver.github.io/usefulRstuff/posts/lobstr/index.html

When working with deeply nested structures (either objects or function calls), sometimes it’s helpful to visualize what you’re working with. The lobstr package provides functions for viewing these structures in an easy-to-read way—I find it more intuitive to parse than str().

If you’re using RStudio you can, of course, use the viewer with View(), but that works best for purely rectangular data with no hierarchical structure.

Tree structure for (nested) dataframes

lobstr::tree()

lobstr::tree() provides a tree structure for data structures, and is particularly useful when you have nested structures.

We’ll start with the a subset of the mpg data provided in ggplot2. We nest by cyl, and then call tree(), resulting in a handy representation of the structure (scroll to see full output).

library(tidyverse)
library(lobstr)

mpg_subset <-  mpg |> 
  select(class, cyl, manufacturer, hwy, cty)

mpg_subset |>
  nest(.by = cyl) |> 
  lobstr::tree()
S3<tbl_df/tbl/data.frame>
├─cyl<int [4]>: 4, 6, 8, 5
└─data: <list>
  ├─S3<tbl_df/tbl/data.frame>
  │ ├─class<chr [81]>: "compact", "compact", "compact", "compact", "compact", "compact", "compact", "compact", "midsize", "midsize", ...
  │ ├─manufacturer<chr [81]>: "audi", "audi", "audi", "audi", "audi", "audi", "audi", "audi", "chevrolet", "chevrolet", ...
  │ ├─hwy<int [81]>: 29, 29, 31, 30, 26, 25, 28, 27, 27, 30, ...
  │ └─cty<int [81]>: 18, 21, 20, 21, 18, 16, 20, 19, 19, 22, ...
  ├─S3<tbl_df/tbl/data.frame>
  │ ├─class<chr [79]>: "compact", "compact", "compact", "compact", "compact", "compact", "compact", "midsize", "midsize", "midsize", ...
  │ ├─manufacturer<chr [79]>: "audi", "audi", "audi", "audi", "audi", "audi", "audi", "audi", "audi", "chevrolet", ...
  │ ├─hwy<int [79]>: 26, 26, 27, 25, 25, 25, 25, 24, 25, 26, ...
  │ └─cty<int [79]>: 16, 18, 18, 15, 17, 17, 15, 15, 17, 18, ...
  ├─S3<tbl_df/tbl/data.frame>
  │ ├─class<chr [70]>: "midsize", "suv", "suv", "suv", "suv", "suv", "2seater", "2seater", "2seater", "2seater", ...
  │ ├─manufacturer<chr [70]>: "audi", "chevrolet", "chevrolet", "chevrolet", "chevrolet", "chevrolet", "chevrolet", "chevrolet", "chevrolet", "chevrolet", ...
  │ ├─hwy<int [70]>: 23, 20, 15, 20, 17, 17, 26, 23, 26, 25, ...
  │ └─cty<int [70]>: 16, 14, 11, 14, 13, 12, 16, 15, 16, 15, ...
  └─S3<tbl_df/tbl/data.frame>
    ├─class<chr [4]>: "compact", "compact", "subcompact", "subcompact"
    ├─manufacturer<chr [4]>: "volkswagen", "volkswagen", "volkswagen", "volkswagen"
    ├─hwy<int [4]>: 29, 29, 28, 29
    └─cty<int [4]>: 21, 21, 20, 20

This works for multiply-nested objects as well—here we nest the mpg dataset by cyl and by class:

mpg_subset |> 
  nest(.by = c(cyl, class)) |> 
  nest(.by = class) |> 
  head(2) |> 
  tree()
S3<tbl_df/tbl/data.frame>
├─class<chr [2]>: "compact", "midsize"
└─data: <list>
  ├─S3<tbl_df/tbl/data.frame>
  │ ├─cyl<int [3]>: 4, 6, 5
  │ └─data: <list>
  │   ├─S3<tbl_df/tbl/data.frame>
  │   │ ├─manufacturer<chr [32]>: "audi", "audi", "audi", "audi", "audi", "audi", "audi", "audi", "nissan", "nissan", ...
  │   │ ├─hwy<int [32]>: 29, 29, 31, 30, 26, 25, 28, 27, 29, 27, ...
  │   │ └─cty<int [32]>: 18, 21, 20, 21, 18, 16, 20, 19, 21, 19, ...
  │   ├─S3<tbl_df/tbl/data.frame>
  │   │ ├─manufacturer<chr [13]>: "audi", "audi", "audi", "audi", "audi", "audi", "audi", "toyota", "toyota", "toyota", ...
  │   │ ├─hwy<int [13]>: 26, 26, 27, 25, 25, 25, 25, 26, 26, 27, ...
  │   │ └─cty<int [13]>: 16, 18, 18, 15, 17, 17, 15, 18, 18, 18, ...
  │   └─S3<tbl_df/tbl/data.frame>
  │     ├─manufacturer<chr [2]>: "volkswagen", "volkswagen"
  │     ├─hwy<int [2]>: 29, 29
  │     └─cty<int [2]>: 21, 21
  └─S3<tbl_df/tbl/data.frame>
    ├─cyl<int [3]>: 6, 8, 4
    └─data: <list>
      ├─S3<tbl_df/tbl/data.frame>
      │ ├─manufacturer<chr [23]>: "audi", "audi", "chevrolet", "chevrolet", "chevrolet", "hyundai", "hyundai", "hyundai", "nissan", "nissan", ...
      │ ├─hwy<int [23]>: 24, 25, 26, 29, 26, 26, 26, 28, 27, 26, ...
      │ └─cty<int [23]>: 15, 17, 18, 18, 17, 18, 18, 19, 19, 19, ...
      ├─S3<tbl_df/tbl/data.frame>
      │ ├─manufacturer<chr [2]>: "audi", "pontiac"
      │ ├─hwy<int [2]>: 23, 25
      │ └─cty<int [2]>: 16, 16
      └─S3<tbl_df/tbl/data.frame>
        ├─manufacturer<chr [16]>: "chevrolet", "chevrolet", "hyundai", "hyundai", "hyundai", "hyundai", "nissan", "nissan", "toyota", "toyota", ...
        ├─hwy<int [16]>: 27, 30, 26, 27, 30, 31, 31, 32, 29, 27, ...
        └─cty<int [16]>: 19, 22, 18, 18, 21, 21, 23, 23, 21, 21, ...

If the output is a bit overwhelming with all of the values being printed, you can specify the val_printer argument—here we give it an empty anonymous function so that no values are printed. This helps to see the structure a little bit more clearly:

mpg_subset |> 
  nest(.by = c(cyl, class)) |> 
  nest(.by = class) |> 
  head(2) |> 
  tree(val_printer = \(x) "" )
S3<tbl_df/tbl/data.frame>
├─class<chr [2]>: 
└─data: <list>
  ├─S3<tbl_df/tbl/data.frame>
  │ ├─cyl<int [3]>: 
  │ └─data: <list>
  │   ├─S3<tbl_df/tbl/data.frame>
  │   │ ├─manufacturer<chr [32]>: 
  │   │ ├─hwy<int [32]>: 
  │   │ └─cty<int [32]>: 
  │   ├─S3<tbl_df/tbl/data.frame>
  │   │ ├─manufacturer<chr [13]>: 
  │   │ ├─hwy<int [13]>: 
  │   │ └─cty<int [13]>: 
  │   └─S3<tbl_df/tbl/data.frame>
  │     ├─manufacturer<chr [2]>: 
  │     ├─hwy<int [2]>: 
  │     └─cty<int [2]>: 
  └─S3<tbl_df/tbl/data.frame>
    ├─cyl<int [3]>: 
    └─data: <list>
      ├─S3<tbl_df/tbl/data.frame>
      │ ├─manufacturer<chr [23]>: 
      │ ├─hwy<int [23]>: 
      │ └─cty<int [23]>: 
      ├─S3<tbl_df/tbl/data.frame>
      │ ├─manufacturer<chr [2]>: 
      │ ├─hwy<int [2]>: 
      │ └─cty<int [2]>: 
      └─S3<tbl_df/tbl/data.frame>
        ├─manufacturer<chr [16]>: 
        ├─hwy<int [16]>: 
        └─cty<int [16]>:

tree is also handy for list-columns that have complex objects like models:

mpg_subset |> 
  nest(.by = class) |> 
  head(2) |> # Get a subset of the data so output is manageable
  mutate(data = map(data, \(x) lm(cty ~ manufacturer, data = x))) |> 
  tree()
S3<tbl_df/tbl/data.frame>
├─class<chr [2]>: "compact", "midsize"
└─data: <list>
  ├─S3<lm>
  │ ├─coefficients<dbl [5]>: 17.9333333333333, 2.06666666666666, 1.81666666666667, 4.31666666666667, 2.85238095238095
  │ ├─residuals<dbl [47]>: 0.066666666666665, 3.06666666666667, 2.06666666666666, 3.06666666666667, -1.93333333333334, 0.0666666666666665, 0.0666666666666665, 0.0666666666666665, -1.93333333333333, 2.06666666666667, ...
  │ ├─effects<dbl [47]>: -137.988281957008, -0.184506241605778, -0.803194840135016, 8.56434361157096, 7.67570324864425, -0.0407824196725389, -0.0407824196725389, -0.0407824196725389, -2.04078241967254, 1.95921758032746, ...
  │ ├─rank: 5
  │ ├─fitted.values<dbl [47]>: 17.9333333333333, 17.9333333333333, 17.9333333333333, 17.9333333333333, 17.9333333333333, 17.9333333333333, 17.9333333333333, 17.9333333333333, 17.9333333333333, 17.9333333333333, ...
  │ ├─assign<int [5]>: 0, 1, 1, 1, 1
  │ ├─qr: S3<qr>
  │ │ ├─qr<dbl [235]>: -6.85565460040104, 0.145864991497895, 0.145864991497895, 0.145864991497895, 0.145864991497895, 0.145864991497895, 0.145864991497895, 0.145864991497895, 0.145864991497895, 0.145864991497895, ...
  │ │ ├─qraux<dbl [5]>: 1.14586499149789, 1.02683653004132, 1.03957282449068, 1.08213154946517, 1.13553443533542
  │ │ ├─pivot<int [5]>: 1, 2, 3, 4, 5
  │ │ ├─tol: 1e-07
  │ │ └─rank: 5
  │ ├─df.residual: 42
  │ ├─contrasts: <list>
  │ │ └─manufacturer: "contr.treatment"
  │ ├─xlevels: <list>
  │ │ └─manufacturer<chr [5]>: "audi", "nissan", "subaru", "toyota", "volkswagen"
  │ ├─call: <language> lm(formula = cty ~ manufacturer, data = x)
  │ ├─terms: S3<terms/formula> cty ~ manufacturer
  │ └─model: S3<data.frame>
  │   ├─cty<int [47]>: 18, 21, 20, 21, 16, 18, 18, 18, 16, 20, ...
  │   └─manufacturer<chr [47]>: "audi", "audi", "audi", "audi", "audi", "audi", "audi", "audi", "audi", "audi", ...
  └─S3<lm>
    ├─coefficients<dbl [7]>: 16, 2.80000000000001, 3.00000000000001, 4.00000000000001, 1.00000000000001, 3.85714285714287, 2.57142857142858
    ├─residuals<dbl [41]>: -1.00000000000003, 1.00000000000001, 1.05208052500074e-14, 0.200000000000008, 3.2, -0.800000000000004, -0.8, -1.8, -1, -1, ...
    ├─effects<dbl [41]>: -120.097622892338, 0.10476454436544, 0.736955526660781, 3.98035930389303, -3.23748103044015, 4.17435145669764, 3.72635402044873, -1.27026155514514, -0.923635942390468, -0.923635942390468, ...
    ├─rank: 7
    ├─fitted.values<dbl [41]>: 16, 16, 16, 18.8, 18.8, 18.8, 18.8, 18.8, 19, 19, ...
    ├─assign<int [7]>: 0, 1, 1, 1, 1, 1, 1
    ├─qr: S3<qr>
    │ ├─qr<dbl [287]>: -6.40312423743285, 0.156173761888606, 0.156173761888606, 0.156173761888606, 0.156173761888606, 0.156173761888606, 0.156173761888606, 0.156173761888606, 0.156173761888606, 0.156173761888606, ...
    │ ├─qraux<dbl [7]>: 1.15617376188861, 1.0503406378574, 1.06742880959931, 1.02124277862812, 1.02296110820256, 1.03939209519442, 1.09021806425183
    │ ├─pivot<int [7]>: 1, 2, 3, 4, 5, 6, 7
    │ ├─tol: 1e-07
    │ └─rank: 7
    ├─df.residual: 34
    ├─contrasts: <list>
    │ └─manufacturer: "contr.treatment"
    ├─xlevels: <list>
    │ └─manufacturer<chr [7]>: "audi", "chevrolet", "hyundai", "nissan", "pontiac", "toyota", "volkswagen"
    ├─call: <language> lm(formula = cty ~ manufacturer, data = x)
    ├─terms: S3<terms/formula> cty ~ manufacturer
    └─model: S3<data.frame>
      ├─cty<int [41]>: 15, 17, 16, 19, 22, 18, 18, 17, 18, 18, ...
      └─manufacturer<chr [41]>: "audi", "audi", "audi", "chevrolet", "chevrolet", "chevrolet", "chevrolet", "chevrolet", "hyundai", "hyundai", ...

listviewer

If you’re looking for a more GUI-type visualization of nested dfs, consider listviewer::reactjson():

mpg_subset |> 
  nest(.by = c(cyl, class)) |> 
  nest(.by = class, .key = "class_data") |> 
  listviewer::reactjson(collapsed = 4) # collapse after 4 levels deep


reactjson gives essentially the same view as tree: each column of the df is its own node. You can, though, get a more row-wise view which I sometimes find more intuitive by first converting to JSON—here each level of class is its own node:

mpg_subset |> 
  nest(.by = c(cyl, class)) |> 
  nest(.by = class, .key = "class_data") |> 
  jsonlite::toJSON() |> 
  listviewer::reactjson(collapsed = 4) # collapse after 4 levels deep


The listviewer package also provides an alternative JSON viewer, jsonedit():

mpg_subset |> 
  nest(.by = c(cyl, class)) |> 
  nest(.by = class, .key = "class_data") |> 
  jsonlite::toJSON() |> # convert df to JSON
  listviewer::jsonedit()


(A quick note: toJSON() only converts a limited selection of objects; it can’t, for example, convert an lm model to visualize like we did with the midwest_data data above.)

Abstract syntax trees for complex function calls

lobstr also provides the function ast() which gives a visual representation of complex function calls, letting you see the order in which the functions are being evaluated. Here we see that our unnamed function calls + on x and y.

ast(function(x = 1, y = 2) { x + y } )
█─`function` 
├─█─x = 1 
│ └─y = 2 
├─█─`{` 
│ └─█─`+` 
│   ├─x 
│   └─y 
└─<inline srcref>

This is particularly useful when using pipes, where the linear order of the code is actually the reverse of what it would be without piping. Here, for example, we can see that mutate is applying to a grouped df, which itself was filtered.

mpg |> 
  filter(manufacturer == "honda") |> 
  group_by(cyl) |> 
  mutate(mean_hwy = mean(hwy)) |> 
  ast()
█─mutate 
├─█─group_by 
│ ├─█─filter 
│ │ ├─mpg 
│ │ └─█─`==` 
│ │   ├─manufacturer 
│ │   └─"honda" 
│ └─cyl 
└─mean_hwy = █─mean 
             └─hwy
]]> misc https://abraver.github.io/usefulRstuff/posts/lobstr/index.html Mon, 10 Jul 2023 05:00:00 GMT Composing multiple plots with patchwork Aaron Braver https://abraver.github.io/usefulRstuff/posts/patchwork/index.html
Setup

patchwork

It’s often useful to compose multiple plots together into a single image. There are ways of doing this in base R, e.g., by using par(mfrow=...), but i’ve always found this unintuitive and finicky. patchwork makes composing ggplots simple, while allowing for a high degree of customization. (The patchwork documentation is very good, and this post will borrow liberally from it, but it’s worth browsing for more details.) (For those looking for an alternative to patchwork, check out cowplot::plot_grid().)

Let’s start with 4 simple plots, and look at various ways of composing them. (These plots are nicely arranged in a grid by Quarto for the purposes of this page if your browser window is big enough, but note that they are each a separate image.)

Code
p1 <- ggplot(palmerpenguins::penguins) +
  geom_point(aes(bill_length_mm, bill_depth_mm,
                 color = species)) +
  ggtitle("Plot 1")

p2 <- ggplot(palmerpenguins::penguins) +
  geom_boxplot(aes(y = bill_length_mm, species,
                 color = species)) +
  ggtitle("Plot 2")

p3 <- ggplot(palmerpenguins::penguins) +
  geom_point(aes(bill_length_mm, body_mass_g,
                 color = species)) +
  ggtitle("Plot 3")


p4 <- ggplot(palmerpenguins::penguins) +
  geom_bar(aes(bill_length_mm, fill = species)) +
  ggtitle("Plot 4")

p1; p2; p3; p4

The simplest operation provided by patchwork is the + operator. Adding two plots together with + glues them together into a single image, side by side. You can do this with as many plots as you like–it will try to give a nice grid if possible. Note that all plots below are combined into a single image.

library(patchwork)
p1 + p2 + p3 + p4

You can specify the number of rows or columns in your composed plot with plot_layout(nrow=..., ncol=...). Setting byrow to false fills by column, rather than by row (so the left column below has plots 1, 2, and 3). (Click image to embiggen.)

p1 + p2 + p3 + p4 +
  plot_layout(nrow = 3, byrow = FALSE)

See the section below on more complicated layouts and ?plot_layout for more options.

Packing and stacking (the | and / operators)

Composing two plots with the | operator works a lot like the + operator above, except it ensures that the plots will end up side-by-side (“packed”) rather than one on top of the other. The / operator does the opposite—it ensures that plots are stacked.”

p1 | p2

p1 / p2

Parentheses can help to disambiguate possible layouts. (All of the operators work essentially like their arithmetic counterparts with respect to parentheses.) Without parentheses, as in the first tab below, the / operator takes precedence over the | operator:

p1 / p2 | p3

p1 / (p2 | p3)

Annotating the composed plot

plot_annotation() will annotate the combined, composed plot. The title and subtitle, arguments work as you would expect. The tag_levels argument lets you specify one of c('a', 'A', '1', 'i', 'I') for various sub-plot numbering:

(p1 | (p2 / p3)) + 
  plot_annotation(title = 'Lots of data about penguins',
    subtitle = 'There\'s never too much info about penguins',
    tag_levels = "A") &
  theme(plot.title = element_text(hjust = .5, color = "blue"),
        plot.tag = element_text(size = 20))

(The & operator applies the theme specification to all subplots within the composed plot (and overall things like the main title/subtitle); the * operator applies the theme specifications to all subplots in the current nesting. level.)

If you want a theme element to apply only to the overall plot, and not the subplots, use the theme argument to plot_annotation() rather than & theme(...).

(p1 | (p2 / p3)) + 
  plot_annotation(title = 'Lots of data about penguins',
    subtitle = 'There\'s never too much info about penguins', tag_levels = "A",
    theme = theme(plot.title = element_text(hjust = .5, color = "blue"),
        plot.tag = element_text(size = 20)))

More complicated layouts

We saw above that we can modify the layout with plot_layout(ncol = ...). You can also specify relative widths and heights for the rows and columns. For example, you can provide widths with a vector of n columns to specify each.

Some plot theme changes for visibility in the following examples
p1 <- p1 + theme(legend.position = "none", plot.background = element_rect(color = "black", linewidth = 2, fill = "white"))
p2 <- p2 + theme(legend.position = "none", plot.background = element_rect(color = "blue", linewidth = 2, fill = "white"))
p3 <- p3 + theme(legend.position = "none", plot.background = element_rect(color = "green", linewidth = 2, fill = "white"))
p4 <- p4 + theme(legend.position = "none", plot.background = element_rect(color = "red", linewidth = 2, fill = "white"))
p1 + p2 + p3 + p4 + 
  plot_layout(widths = c(3, 1), heights = c(2, 1))

You can also specify quite complicated layouts with plot_layout(design = ...), in two formats. The simplest is to use a textual representation, as below. Letters represent plots (in the order in which they appear), with #s representing empty spaces. You must specify a perfect rectangular shape.

layout <- "
##BBBB
AACCDD
##CCDD
"
p1 + p2 + p3 + p4 + 
  plot_layout(design = layout)

Using wrap_plots, you can specify which plot is represented by which letter, rather than letting that be handled automatically:

layout <- '
A#B
#C#
D#E
'
wrap_plots(D = p1, C = p2, B = p3, design = layout)

The alternative way to specify a layout is with one or more area()s, which take arguments for the top, left, bottom, and right bounds of a grid.1 The grid is specified in rows and columns, with row 1 column 1 being the top left, and numbers increasing as you go down and right. You can have plots overlap with this type of specification.

layout <- c(
  area(t = 2, l = 1, b = 5, r = 4), # This will be p1
  area(t = 1, l = 3, b = 3, r = 5) # this will be p2
)
p1 + p2 + 
  plot_layout(design = layout)

Insets

It’s also possible to specify insets using inset_element(). Here, the position arguments should be relative to 1, with (0, 0) being bottom left and (1, 1) being top right. The align_to argument defaults to panel, but accepts values plot or full as well.

p1 + inset_element(p2, top = 1, right = 1, bottom = .5,
                   left = .5, align_to = "full")

Controlling legends

Reset plots to their original theme
p1 <- p1 + theme(legend.position = "right", plot.background = element_blank())
p2 <- p2 + theme(legend.position = "right", plot.background = element_blank())
p3 <- p3 + theme(legend.position = "right", plot.background = element_blank())
p4 <- p4 + theme(legend.position = "right", plot.background = element_blank())

plot_layout(guides = "collect") will collect all of the legends and put them in the same location. Legends that are identical between multiple subplots are collapsed, as with plots 1 and 3 below:

p1 + p2 + p3 + p4 + plot_layout(guides = "collect")

p1 + p2 + p3 + p4 + plot_layout(guides = "keep")

You can influence where the collected legends show up by including guide_area() in your layout specification:

guide_area() + p1 + p2 + p3 + plot_layout(guides = "collect")

Nesting

Some plot theme changes for visibility in the following examples
p1 <- p1 + theme(legend.position = "none", plot.background = element_rect(color = "black", linewidth = 2, fill = "white"))
p2 <- p2 + theme(legend.position = "none", plot.background = element_rect(color = "blue", linewidth = 2, fill = "white"))
p3 <- p3 + theme(legend.position = "none", plot.background = element_rect(color = "green", linewidth = 2, fill = "white"))
p4 <- p4 + theme(legend.position = "none", plot.background = element_rect(color = "red", linewidth = 2, fill = "white"))

Plots can be nested at multiple levels. Compare the following plots: in the first we use wrap_plots() to combine p1 and p2 into a single unit (which is then added to plot 3); in the second we don’t do any nesting.

wrap_plots(p1 + p2) + p3

(p1 + p2) + p3 # Parens are vacuous here

If you don’t use wrap_plots(), the nesting behavior with + is a little unintuitive. With a single plot on the left of + and a composite plot on the right, you get the expected nesting, with the composite plot taking up half of the whole plot:

composite <- p1 + p2
p3 + composite

So far, so good. But, what if we change the order of arguments around +? All of a sudden, the plot is divided into 3 equal panels!

composite + p3

So, + will nest a composite on the right, but will not nest a composite on the left. You can force the reverse of this by using the - operator, which keeps its arguments at the same nesting level—but I personally find the wrap_plots() syntax more intuitive.

composite - p3

p3 - composite

Alignment across multiple slides

Some plot theme changes for visibility in the following examples
p1 <- p1 + theme(legend.position = "top", plot.background = element_blank())
p2 <- p2 + theme(legend.position = "right", plot.background = element_blank())
p3 <- p3 + theme(legend.position = "bottom", plot.background = element_blank())
p4 <- p4 + theme(legend.position = "left", plot.background = element_blank())

When including plots in a slideshow, you often get an annoying jerking sensation as you move from one slide to the next:

You can generate aligned plots align_patches. Notice here how the axis labels are aligned with each other across multiple slides.

plots_aligned <- align_patches(p1, p2, p3, p4)
for (p in plots_aligned) {
  plot(p)
}

See the relevant documentation for more details.

Footnotes

  1. Somewhat annoyingly, the argument order here is tlbr, rather than trbl as in css, which is how my brain works.↩︎

]]> visualization https://abraver.github.io/usefulRstuff/posts/patchwork/index.html Fri, 07 Jul 2023 05:00:00 GMT Spotlight subsets of data in your plots with gghighlight Aaron Braver https://abraver.github.io/usefulRstuff/posts/gghighlight/index.html
Setup

The problem

We often want to make a subset of data stand out from the rest, but doing so manually is a bit tedious. You could, for example, add a logical column to the data highlight, and set a plot aesthetic to make 1s in a bright color and 0s in grey.

But gghighlight automates this process for ggplot2 plots. Let’s look at the life expectancy dataset from gapminder. Even when color-coding by continent, it’s very difficult to pick out individual countries. We can use gghighlight, though, to emphasize countries with the top 10 highest populations:

Code
library(gghighlight)

base_plot <- gapminder::gapminder |> 
  ggplot(aes(x = year, y = lifeExp, color = continent)) +
  geom_line(aes(group = country))

pretty_plot <- base_plot +
  scale_x_continuous(expand = expansion(mult = c(0,0.25))) +
  geom_line(aes(group = country)) +
  gghighlight(max(pop), max_highlight = 10,
              unhighlighted_params = list(alpha = 0.25,
                                          color = NULL
              ),
              line_label_type = "ggrepel_text",
              label_params = list(
                                  hjust = 0,
                                  direction = "y",
                                  nudge_x = 10,
                                  segment.size = 0.2
              ))

base_plot
pretty_plot

Vanilla ggplot: Too busy to glean any insights

Countries with the top 10 highest populations gghighlighted

gghighlight basics

Let’s start with the busy plot from above, which we’ve saved as base_plot. The gghighlight() function can be added to any ggplot, and requires only a predicate to select the highlighted portion of the data (passed directly to dplyr::filter(), so anything you can do there is fair game). Here we highlight countries with a population greater than 50 million.1

base_plot +
  gghighlight(pop > 50000000)

Since there are so many un-highlighted lines, it’s quite hard to see what’s happening. We can change the aesthetics of the un-highlighted lines with unhighlighted_params.

base_plot +
  gghighlight(pop > 50000000,
              unhighlighted_params = list(alpha = 0.25))

More complicated filtering criteria

Since the first argument(s) of gghighlight get passed directly to filter, you can include multiple filtering criteria, like population over 50 million and only countries in Asia:

base_plot +
  gghighlight(pop > 50000000, continent == "Asia",
              unhighlighted_params = list(alpha = 0.25))

(Note that in this plot, where we have relatively few highlighted lines, gghighlight adds labels for us.)

We can also provide a non-logical predicate to filter by, for example the top ten countries by population. The predicate is used to order the results, and the max_highlight argument determines the top n rows/groups based on that criterion:

base_plot +
  gghighlight(max(pop), max_highlight = 10,
              unhighlighted_params = list(alpha = 0.25))

Labels

The labels in the previous plot make things pretty crowded. gghighlight uses the ggrepel::geom_label_repel geom by default, and arguments can be passed to that function with the label_params argument.2 Here we’ll modify max.overlaps, which removes some labels when there are too many overlapping objects:

base_plot +
  gghighlight(max(pop), max_highlight = 10,
              unhighlighted_params = list(alpha = 0.25),
              label_params = list(
                max.overlaps = 5
              ))

What if we want to have all of the labels off on one side, rather than on top of the plotted data? We can once again avail ourselves of label_params. I also prefer the labels to be text only—no border—which is accomplished with line_label_type = "ggrepel_text".

base_plot +
  gghighlight(max(pop), max_highlight = 10,
    unhighlighted_params = list(alpha = 0.25),
    line_label_type = "ggrepel_text",
    label_params = list(
      hjust = 0, # align label text
      direction = "y", # allow shifting labels on y axis to make room
      nudge_x = 10, # push everything further to the right
      segment.size = 0.2 # make pointing lines thinner
    )
  )

One final tweak is to expand the plot rightward, so there is more room for labels. This is done by modifying the scale of the plot, as below. (I have not found a good way to make the labels appear in the margin)

base_plot +
 # Modify scale:
  scale_x_continuous(expand = expansion(mult = c(0,0.25))) +
  gghighlight(max(pop), max_highlight = 10,
    unhighlighted_params = list(alpha = 0.25),
    line_label_type = "ggrepel_text",
    label_params = list(
      hjust = 0, # align label text
      direction = "y", # allow shifting labels on y axis to make room
      nudge_x = 10, # push everything further to the right
      segment.size = 0.2 # make pointing lines thinner
    )
  )

You’ll notice that in the plot above, the labels are drawn inside the plotting area, and not in the margin. I have not found a good way to make the labels in the margin only. The closest built-in option to gghighlight is to set line_label_type = "sec_axis" instead of "ggrepel_text", but this does not prevent the labels from overlapping each other (since it’s not using ggrepel anymore). This is ok in a plot with only a few—or very spaced out—labels, but doesn’t work well for our current plot. You can see in the second plot below that it looks pretty good when we select, say, only the top 3 countries by population.

base_plot +
  gghighlight(max(pop), max_highlight = 10,
    unhighlighted_params = list(alpha = 0.25),
    line_label_type = "sec_axis"
    )
base_plot +
  gghighlight(max(pop), max_highlight = 3, # ***
    unhighlighted_params = list(alpha = 0.25),
    line_label_type = "sec_axis"
    )

Other geoms

gghighlight works on geoms other than line, and in fact looks pretty snazzy in barplots/histograms plotted by group, where there is a lot of overlap. In the highlighted plot we facet by continent, but also show the overall distribution in gray.

gapminder::gapminder |> 
  ggplot(aes(x = lifeExp, fill = continent)) +
  geom_histogram() +
  facet_wrap(~continent) +
  gghighlight()

gapminder::gapminder |> 
  ggplot(aes(x = lifeExp, fill = continent)) +
  geom_histogram()

And with geom_point, for example, we can call out the top ten countries by population in the data from 2002:

gapminder::gapminder |> 
  filter(year == 2002) |> 
  ggplot(aes(x = lifeExp, y = pop, color = country)) +
  geom_point(group = 1) +
  gghighlight(max(pop), max_highlight = 10,
              label_key = country,
              label_params = list(
                max.overlaps = 1
              ))
Warning: ggrepel: 6 unlabeled data points (too many overlaps). Consider
increasing max.overlaps

Note above that we’re warned that there were “too many overlaps,” and therefore some labels are omitted. In fact, if there are too many overlapping data points, all labels will be omitted You can remedy this by either having fewer highlighted points and/or changing max.overlaps.

gapminder::gapminder |> 
  filter(year == 2002) |> 
  ggplot(aes(x = lifeExp, y = pop, color = country)) +
  geom_point(group = 1) +
  gghighlight(max(pop), max_highlight = 5,
              label_key = country,
              label_params = list(
                max.overlaps = 50
              ))

Footnotes

  1. NB: unless you set use_direct_label = FALSE, the legend will not be included in a highlighted plot. Also, if you want the colors to match an unhighlighted version of the plot, use keep_scales = TRUE.↩︎

  2. See the geom_label_repel documentation for more options.↩︎

]]> visualization https://abraver.github.io/usefulRstuff/posts/gghighlight/index.html Thu, 06 Jul 2023 05:00:00 GMT khroma and colorblindr: Tools for colorblind-friendly plots Aaron Braver https://abraver.github.io/usefulRstuff/posts/khroma/index.html
Load packages

khroma - colorblind-friendly palettes

khroma contains a number of color schemes that are colorblind-friendly, divided into diverging, qualitative, and sequential types:

khroma::info() |>
  group_by(type) |>
  summarize(palettes = paste(palette, collapse = ", "))
type palettes
diverging broc, cork, vik, lisbon, tofino, berlin, roma, bam, vanimo, oleron, bukavu, fes, sunset, BuRd, PRGn
qualitative bright, highcontrast, vibrant, muted, mediumcontrast, pale, dark, light, okabeito, okabeitoblack, stratigraphy, soil, land
sequential devon, lajolla, bamako, davos, bilbao, nuuk, oslo, grayC, hawaii, lapaz, tokyo, buda, acton, turku, imola, batlow, batlowW, batlowK, brocO, corkO, vikO, romaO, bamO, YlOrBr, iridescent, discreterainbow, smoothrainbow

The color function takes a palette name and returns a function, which in turn returns a palette of specified size:

library(khroma)

bright_fun <- color("bright")
bright_fun(5)
     blue       red     green    yellow      cyan 
"#4477AA" "#EE6677" "#228833" "#CCBB44" "#66CCEE" 
attr(,"missing")
[1] NA

And you can preview these palettes with plot_scheme():

plot_scheme(bright_fun(7), names = TRUE,
            colours = TRUE) # NB: `colours` must be spelled British-ly

Further, you can preview the palette as viewed by several types of colorblindness:

plot_scheme_colorblind(bright_fun(7))


Each of the schemes have built-in ggplot scales in the form scale_color_{palettename} and scale_fill_{palettename}:

plot <- midwest |> 
  filter(percbelowpoverty < 30) |> 
  ggplot(aes(x = percollege, y = percbelowpoverty, color = state)) +
    geom_point(size = 3, alpha =.5) +
    scale_color_bright()

plot
Percent of population with a college degree vs. percent of population below the poverty line, by state. Data from ggplot2::midwest

Percent of population with a college degree vs. percent of population below the poverty line, by state. Data from `ggplot2::midwest`

(NB: you can use the sequential and diverging type palettes for discrete/categorical data if you add the argument discrete = TRUE)

colorblindr - Preview your plots with colorblind simulation

colorblindr::cvd_grid will simulate how your plots will look to people with various types of colorblindness:

# remotes::install_github("clauswilke/colorblindr")

plot <- msleep |> 
  filter(!is.na(vore)) |> 
  ggplot(aes(x = sleep_rem, fill = vore)) +
  geom_density(alpha = 0.75) +
  scale_fill_light()

plot

colorblindr::cvd_grid(plot)
Density of hours of REM sleep, for carnivores, omnivores, herbivores, and insectivores. Data from the mammal sleep dataset (ggplot::msleep).

The same plot in various simulations of colorblindness

]]> visualization colors https://abraver.github.io/usefulRstuff/posts/khroma/index.html Tue, 04 Jul 2023 05:00:00 GMT Paginate a faceted graph and save as a single multi-page PDF Aaron Braver https://abraver.github.io/usefulRstuff/posts/paginate_faceted_graphs/index.html

When you have a plot with just a few facets, it’s easy to fit them all on a single page. But what if you have a whole bunch of facets that you want to visualize? For example, the USArrests dataset has data from all 50 US states.

If we just use the usual facet_wrap or facet_grid from ggplot2, you end up with one massive, smooshed-together image:

library(tidyverse)

plot_1 <- USArrests |>  # dataset on US arrests
  rownames_to_column("State") |>   # This is just so we can see the state names
  ggplot(aes(x = UrbanPop, y = Murder)) +
  geom_point()  +
  facet_wrap(~ State,
             nrow = 5, ncol = 10)

plot_1

The ggforce library, among its many useful functions, includes facet_wrap_paginate and facet_grid_paginate. These work like facet_wrap and facet_grid, but take an additional page argument. Here we specify that we want 4 rows and 5 columns per page, so for 50 states there will be 2 full pages plus one page with the remaining 10 states:

library(ggforce)

plot_2 <- plot_1  +
  facet_wrap_paginate(~ State,
             nrow = 4, ncol = 5, page = 2)

plot_2

Note that we had to specify which page we wanted to print. We can then easily put this into a loop to generate all the necessary pages. The helpful n_pages() will count the pages needed, which we can then use in a loop. This gives us 3 separate plots.

plot_pages <- plot_2 |> 
  n_pages()

for (i in 1:plot_pages) {
  print(
    plot_1 + facet_wrap_paginate(~ State,
             nrow = 4, ncol = 5, page = i)
  )  
}


That’s better, but for inclusion in a longer document or for sharing the plot, you may want all three pages in a single PDF. This can be accomplished by calling pdf() before the loop and dev.off() after.1

pdf('many_plots.pdf', width = 11, height = 8.5) #start building pdf
  for (i in 1:plot_pages) {
    print( # don't forget this
      plot_1 +
      facet_wrap_paginate(~State, nrow = 4, ncol = 5, page = i)
    )
  }
dev.off() # end building pdf

The resulting PDF is a single file with 3 pages, as we expect.

(This post was inspired by this conversation on StackOverflow.)

Footnotes

  1. In the past I’ve had issues with getting pdf() to work, especially if I use any fancy fonts in the ggplot theme. If you run into issues, you can replace pdf() with cairo_pdf(..., onefile = TRUE), which seems to work even with exotic fonts.↩︎

]]> visualization https://abraver.github.io/usefulRstuff/posts/paginate_faceted_graphs/index.html Sat, 01 Jul 2023 05:00:00 GMT Upgrading R without losing all your packages Aaron Braver https://abraver.github.io/usefulRstuff/posts/upgrading_R/index.html

If you upgrade R to a new major or minor release (e.g. 4.0 or 4.1, but not 4.1.1), your library of installed packages doesn’t come along for the ride.

Before you upgrade

Before you upgrade, you should make a list of all of your installed packages—this is easy with sessioninfo::package_info("installed"). If you’ve got packages installed from multiple sources (like CRAN and also Github), it’s best to save these as separate lists to facilitate automated reinstallation of these grouped by source:

library(tidyverse)

# Figure out what's installed
installed_packages <- sessioninfo::package_info("installed") |> 
  mutate("where_from" = dplyr::case_when(
    str_detect(source, "CRAN") ~ "CRAN",
    str_detect(source, "Github") ~ "Github",
    .default = "other" # catch any packages from other sources;
                       # not automatically installed below
    )
  ) |> 
  nest(.by = where_from)

# Where to write the lists of packages.  Change as needed
file_location <- fs::path("~/Desktop/tmp/")

# Write these lists to disk
installed_packages %>% 
  pwalk(function(where_from, data) {
    write_csv(data,
              file = fs::path(file_location,
                              paste0(where_from, "_packages"),
                              ext = "csv"))
  })

After you upgrade

Then, after you upgrade R, you can re-install the packages from these lists:

# Install manually some things we'll need since they won't be installed anymore
install.packages(c("remotes", "dplyr", "stringr"))

file_location <- "~/Desktop/tmp/" # Change as needed

# Install CRAN packages
CRAN_packages <- read.csv(paste0(file_location, "CRAN_packages", ".csv"))
remotes::install_cran(CRAN_packages$package)

# Install Github packages
github_packages <- read.csv(paste0(file_location, "Github_packages", ".csv")) |>
  dplyr::mutate(github_source = stringr::str_extract(source,
      "(?<=Github \\().*(?=@)")) # Get the username and repo name
remotes::install_github(github_packages$github_source)

NB: If you have locally installed packages, or packages from another source, these aren’t automatically reinstalled with the code above, but are listed in other_packages.csv.

]]> upgrading package management https://abraver.github.io/usefulRstuff/posts/upgrading_R/index.html Fri, 30 Jun 2023 05:00:00 GMT Interactive plots with ggiraph Aaron Braver https://abraver.github.io/usefulRstuff/posts/ggiraph/index.html

ggiraph

ggiraph logo

ggiraph allows for the creation of interactive plots in documents that will output as HTML or in Shiny. In static HTML pages, ggiraph allows for tooltips, mouseover highlighting, and JavaScript on click. In Shiny documents there is additional interactivity available—but since this document is static HTML, I won’t cover these here.

(For more details, see the ggiraph overview, and for even more details, see the ggiraph book; some examples here come from those sources.)

(You may also be interested in the plotly package as another option for adding interactivity to plots.)

Simple example

As a simple example, here is a plot of the palmerpenguins dataset, showing bill_depth_mm vs. bill_length_mm; species are shown by color, and hovering over a point shows which island the sample was collected on (and highlights all other points from that island):

library(ggiraph)
library(ggplot2)
library(palmerpenguins) # Load penguins dataset

plot_1 <- penguins |> 
  ggplot(
  aes(
    x = bill_depth_mm, y = bill_length_mm,
    tooltip = paste("Island:", island),
    data_id = island, color = species
    )) +
  geom_point_interactive(
     hover_nearest = TRUE
  )

girafe(ggobj = plot_1)

The data_id argument is how items will be grouped for tooltips and hover effects—here, we want all the dots from the same island to behave together.

Note, too, the ggobj argument to girafe(). The actual first argument to girafe() is not a ggplot object; this means the ggobj argument needs to be called explicitly, and also makes piping into girafe a bit annoying (hence saving the plot to plot_1 and referring to that within the girafe() call.

Slightly more complicated example

In this slightly more complicated example, the weight of chicks is shown over time. Hovering over a line highlights all chicks in that diet (and dims those in other diets)1, and shows the chick number in a tooltip.

The styling of moused-over elements is controlled with the css argument to opts_hover (which is called inside girafe()). I like making non-moused-over elements gray (in opts_hover_inv) and leaving the moused-over elements with their original color by specifiying `css=““.

plot_2 <- ChickWeight |>
  ggplot(
  aes(
    x = Time, y = weight, color = Diet,
    group = Chick,
    tooltip = paste0("Diet ", Diet, " (Chick ", Chick, ")"),
    data_id = Diet
    )) +
  geom_line_interactive(
     hover_nearest = TRUE
  )

girafe(ggobj = plot_2,
      options = list(
        opts_hover(css = ""), # Don't change CSS for selected group
        opts_hover_inv(css = "opacity: 0.1"), # CSS for other groups
        opts_toolbar(saveaspng = FALSE) # Don't include save button
      ))

Javascript on click

The onclick aesthetic allows for execution of JavaScript on click. This is probably more useful in a Shiny project, but it does work even on static HTML pages. Here we get an alert when we click:

plot_3 <- penguins |> 
  ggplot(aes(x = species, fill = island,
             tooltip = sprintf('Island: %s | Species: %s', island, species),
             data_id = island,
             onclick = sprintf('alert("Island: %s | Species: %s");', island, species)
             )
         ) +
  geom_bar_interactive()

girafe(ggobj = plot_3,
       options = list(
         opts_hover(css = ""), # Don't change CSS for selected group
         opts_hover_inv(css = "fill: gray"), # CSS for non-selected groups
         opts_toolbar(saveaspng = FALSE) # Don't include save button
       )
)

Footnotes

  1. For similar functionality in static graphs (i.e., highlighting one particular group), the gghighlight package.↩︎

]]> visualization interactivity https://abraver.github.io/usefulRstuff/posts/ggiraph/index.html Thu, 29 Jun 2023 05:00:00 GMT Summary statistics for dfs with skimr Aaron Braver https://abraver.github.io/usefulRstuff/posts/skimr/index.html
Setup

skimr::skim() provides handy summary statistics for dfs and related objects, including little sparkline-style histograms right in the output.

library(skimr)

palmerpenguins::penguins |> 
  skim()
── Data Summary ────────────────────────
                           Values                  
Name                       palmerpenguins::penguins
Number of rows             344                     
Number of columns          8                       
_______________________                            
Column type frequency:                             
  factor                   3                       
  numeric                  5                       
________________________                           
Group variables            None                    

── Variable type: factor ───────────────────────────────────────────────────────
  skim_variable n_missing complete_rate ordered n_unique
1 species               0         1     FALSE          3
2 island                0         1     FALSE          3
3 sex                  11         0.968 FALSE          2
  top_counts                 
1 Ade: 152, Gen: 124, Chi: 68
2 Bis: 168, Dre: 124, Tor: 52
3 mal: 168, fem: 165         

── Variable type: numeric ──────────────────────────────────────────────────────
  skim_variable     n_missing complete_rate   mean      sd     p0    p25    p50
1 bill_length_mm            2         0.994   43.9   5.46    32.1   39.2   44.4
2 bill_depth_mm             2         0.994   17.2   1.97    13.1   15.6   17.3
3 flipper_length_mm         2         0.994  201.   14.1    172    190    197  
4 body_mass_g               2         0.994 4202.  802.    2700   3550   4050  
5 year                      0         1     2008.    0.818 2007   2007   2008  
     p75   p100 hist 
1   48.5   59.6 ▃▇▇▆▁
2   18.7   21.5 ▅▅▇▇▂
3  213    231   ▂▇▃▅▂
4 4750   6300   ▃▇▆▃▂
5 2009   2009   ▇▁▇▁▇

It also handles grouped data nicely:

palmerpenguins::penguins |> 
  group_by(species) |> 
  skim()
── Data Summary ────────────────────────
                           Values                      
Name                       group_by(palmerpenguins::...
Number of rows             344                         
Number of columns          8                           
_______________________                                
Column type frequency:                                 
  factor                   2                           
  numeric                  5                           
________________________                               
Group variables            species                     

── Variable type: factor ───────────────────────────────────────────────────────
  skim_variable species   n_missing complete_rate ordered n_unique
1 island        Adelie            0         1     FALSE          3
2 island        Chinstrap         0         1     FALSE          1
3 island        Gentoo            0         1     FALSE          1
4 sex           Adelie            6         0.961 FALSE          2
5 sex           Chinstrap         0         1     FALSE          2
6 sex           Gentoo            5         0.960 FALSE          2
  top_counts               
1 Dre: 56, Tor: 52, Bis: 44
2 Dre: 68, Bis: 0, Tor: 0  
3 Bis: 124, Dre: 0, Tor: 0 
4 fem: 73, mal: 73         
5 fem: 34, mal: 34         
6 mal: 61, fem: 58         

── Variable type: numeric ──────────────────────────────────────────────────────
   skim_variable     species   n_missing complete_rate   mean      sd     p0
 1 bill_length_mm    Adelie            1         0.993   38.8   2.66    32.1
 2 bill_length_mm    Chinstrap         0         1       48.8   3.34    40.9
 3 bill_length_mm    Gentoo            1         0.992   47.5   3.08    40.9
 4 bill_depth_mm     Adelie            1         0.993   18.3   1.22    15.5
 5 bill_depth_mm     Chinstrap         0         1       18.4   1.14    16.4
 6 bill_depth_mm     Gentoo            1         0.992   15.0   0.981   13.1
 7 flipper_length_mm Adelie            1         0.993  190.    6.54   172  
 8 flipper_length_mm Chinstrap         0         1      196.    7.13   178  
 9 flipper_length_mm Gentoo            1         0.992  217.    6.48   203  
10 body_mass_g       Adelie            1         0.993 3701.  459.    2850  
11 body_mass_g       Chinstrap         0         1     3733.  384.    2700  
12 body_mass_g       Gentoo            1         0.992 5076.  504.    3950  
13 year              Adelie            0         1     2008.    0.822 2007  
14 year              Chinstrap         0         1     2008.    0.863 2007  
15 year              Gentoo            0         1     2008.    0.792 2007  
      p25    p50    p75   p100 hist 
 1   36.8   38.8   40.8   46   ▁▆▇▆▁
 2   46.3   49.6   51.1   58   ▂▇▇▅▁
 3   45.3   47.3   49.6   59.6 ▃▇▆▁▁
 4   17.5   18.4   19     21.5 ▂▆▇▃▁
 5   17.5   18.4   19.4   20.8 ▅▇▇▆▂
 6   14.2   15     15.7   17.3 ▅▇▇▆▂
 7  186    190    195    210   ▁▆▇▅▁
 8  191    196    201    212   ▁▅▇▅▂
 9  212    216    221    231   ▂▇▇▆▃
10 3350   3700   4000   4775   ▅▇▇▃▂
11 3488.  3700   3950   4800   ▁▅▇▃▁
12 4700   5000   5500   6300   ▃▇▇▇▂
13 2007   2008   2009   2009   ▇▁▇▁▇
14 2007   2008   2009   2009   ▇▁▆▁▇
15 2007   2008   2009   2009   ▆▁▇▁▇

And finally, since the output of skim() has class data.frame, you can include it in a pipeline, e.g. to filter by one of the summary statistics:

palmerpenguins::penguins |> 
  skim() |> 
  filter(numeric.sd > 1)
── Data Summary ────────────────────────
                           Values                  
Name                       palmerpenguins::penguins
Number of rows             344                     
Number of columns          8                       
_______________________                            
Column type frequency:                             
  numeric                  4                       
________________________                           
Group variables            None                    

── Variable type: numeric ──────────────────────────────────────────────────────
  skim_variable     n_missing complete_rate   mean     sd     p0    p25    p50
1 bill_length_mm            2         0.994   43.9   5.46   32.1   39.2   44.4
2 bill_depth_mm             2         0.994   17.2   1.97   13.1   15.6   17.3
3 flipper_length_mm         2         0.994  201.   14.1   172    190    197  
4 body_mass_g               2         0.994 4202.  802.   2700   3550   4050  
     p75   p100 hist 
1   48.5   59.6 ▃▇▇▆▁
2   18.7   21.5 ▅▅▇▇▂
3  213    231   ▂▇▃▅▂
4 4750   6300   ▃▇▆▃▂
palmerpenguins::penguins |> 
  skim() |> 
  filter(factor.n_unique > 1)
── Data Summary ────────────────────────
                           Values                  
Name                       palmerpenguins::penguins
Number of rows             344                     
Number of columns          8                       
_______________________                            
Column type frequency:                             
  factor                   3                       
________________________                           
Group variables            None                    

── Variable type: factor ───────────────────────────────────────────────────────
  skim_variable n_missing complete_rate ordered n_unique
1 species               0         1     FALSE          3
2 island                0         1     FALSE          3
3 sex                  11         0.968 FALSE          2
  top_counts                 
1 Ade: 152, Gen: 124, Chi: 68
2 Bis: 168, Dre: 124, Tor: 52
3 mal: 168, fem: 165


To refer to the summary statistic columns, be sure to preface the column name with the variable type, e.g. factor. or numeric. as appropriate. For a list of variable types and the default summary statistics for each:

$AsIs
[1] "n_unique"   "min_length" "max_length"

$character
[1] "min"        "max"        "empty"      "n_unique"   "whitespace"

$complex
[1] "mean"

$Date
[1] "min"      "max"      "median"   "n_unique"

$difftime
[1] "min"      "max"      "median"   "n_unique"

$factor
[1] "ordered"    "n_unique"   "top_counts"

$haven_labelled
[1] "mean" "sd"   "p0"   "p25"  "p50"  "p75"  "p100" "hist"

$list
[1] "n_unique"   "min_length" "max_length"

$logical
[1] "mean"  "count"

$numeric
[1] "mean" "sd"   "p0"   "p25"  "p50"  "p75"  "p100" "hist"

$POSIXct
[1] "min"      "max"      "median"   "n_unique"

$Timespan
[1] "min"      "max"      "median"   "n_unique"

$ts
 [1] "start"      "end"        "frequency"  "deltat"     "mean"      
 [6] "sd"         "min"        "max"        "median"     "line_graph"

Finally, if you want to print the summary, but return the original df, use skim_tee().

]]> dataframes summarizing https://abraver.github.io/usefulRstuff/posts/skimr/index.html Mon, 26 Jun 2023 05:00:00 GMT fuzzyjoin: Join dfs with inexact matching Aaron Braver https://abraver.github.io/usefulRstuff/posts/fuzzyjoin/index.html
Setup 
library(tidyverse)

Join dfs by string distance

Sometimes you want to join dfs on fuzzy criteria. That’s where fuzzyjoin comes in. Consider this example lifted partially from the documentation:

Set up a df with some typos. We’ll join this to ggplot2::diamonds by cut, which has levels: Fair, Good, Very Good, Premium, Ideal.

# Create a dataframe with some typos
d <- data.frame(approximate_name = c("Idea", "Premiums", "Premioom",
                                     "VeryGood", "VeryGood", "Faiir"),
                type = 1:6)

A regular inner join finds no matches, due to the typos:

inner_join(diamonds, d, by = c(cut = "approximate_name")) |> 
  nrow()
[1] 0

stringdist_inner_join() however will match strings based on any of the methods available in the stringdist package (e.g. Levenshtein distance, Jaccard distance, etc).

fuzzyjoin::stringdist_inner_join(diamonds, d,
                                 by = c(cut = "approximate_name"),
                                 method = "lv",
                                 max_dist = 3,
                                 distance_col = "distance") |> 
  relocate(approximate_name, .after = "cut") |>
  head(10)
1
Levenshtein distance
2
Move these columns closer together for easier visual comparison
carat cut approximate_name color clarity depth table price x y z type distance
0.23 Ideal Idea E SI2 61.5 55 326 3.95 3.98 2.43 1 1
0.21 Premium Premiums E SI1 59.8 61 326 3.89 3.84 2.31 2 1
0.21 Premium Premioom E SI1 59.8 61 326 3.89 3.84 2.31 3 2
0.29 Premium Premiums I VS2 62.4 58 334 4.20 4.23 2.63 2 1
0.29 Premium Premioom I VS2 62.4 58 334 4.20 4.23 2.63 3 2
0.24 Very Good VeryGood J VVS2 62.8 57 336 3.94 3.96 2.48 4 1
0.24 Very Good VeryGood J VVS2 62.8 57 336 3.94 3.96 2.48 5 1
0.24 Very Good VeryGood I VVS1 62.3 57 336 3.95 3.98 2.47 4 1
0.24 Very Good VeryGood I VVS1 62.3 57 336 3.95 3.98 2.47 5 1
0.26 Very Good VeryGood H SI1 61.9 55 337 4.07 4.11 2.53 4 1

Join dfs by regex matches

Also extremely useful are the regex_*_join functions, which join based on whether there is a regex match between columns. (This example also from the documentation)

d <- data.frame(regex_name = c("^Idea", "mium", "Good"), type = 1:3)

fuzzyjoin::regex_inner_join(diamonds, d, by = c(cut = "regex_name")) |> 
  relocate(regex_name, .after = cut) |> 
  head()
carat cut regex_name color clarity depth table price x y z type
0.23 Ideal ^Idea E SI2 61.5 55 326 3.95 3.98 2.43 1
0.21 Premium mium E SI1 59.8 61 326 3.89 3.84 2.31 2
0.23 Good Good E VS1 56.9 65 327 4.05 4.07 2.31 3
0.29 Premium mium I VS2 62.4 58 334 4.20 4.23 2.63 2
0.31 Good Good J SI2 63.3 58 335 4.34 4.35 2.75 3
0.24 Very Good Good J VVS2 62.8 57 336 3.94 3.96 2.48 3

Join dfs by absolute difference between columns

Say you want to join by two columns so long as they are equal within a certain degree of tolerance. Here we join so long as sepal_lengths$Sepal.Length is within 0.5 of the iris$Sepal.Length:

sepal_lengths <- data.frame(Sepal.Length = c(5, 6, 7), Type = 1:3)

fuzzyjoin::difference_inner_join(iris, sepal_lengths,
                                 max_dist = 0.5,
                                 distance_col = "distance") |> 
  relocate(Sepal.Length.x, .after = Sepal.Length.y) |> 
  head()
Joining by: "Sepal.Length"
Sepal.Width Petal.Length Petal.Width Species Sepal.Length.y Sepal.Length.x Type distance
3.5 1.4 0.2 setosa 5 5.1 1 0.1
3.0 1.4 0.2 setosa 5 4.9 1 0.1
3.2 1.3 0.2 setosa 5 4.7 1 0.3
3.1 1.5 0.2 setosa 5 4.6 1 0.4
3.6 1.4 0.2 setosa 5 5.0 1 0.0
3.9 1.7 0.4 setosa 5 5.4 1 0.4
]]> dataframes joins https://abraver.github.io/usefulRstuff/posts/fuzzyjoin/index.html Mon, 26 Jun 2023 05:00:00 GMT dplyr::glimpse(): Prettier than str() Aaron Braver https://abraver.github.io/usefulRstuff/posts/glimpse/index.html

Instead of str(df) to see the structure of a dataframe, try dplyr::glimpse()1 instead. It gives a horizontal output and shows more data than fits with str():

dplyr::glimpse(palmerpenguins::penguins)
Rows: 344
Columns: 8
$ species           <fct> Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adel…
$ island            <fct> Torgersen, Torgersen, Torgersen, Torgersen, Torgerse…
$ bill_length_mm    <dbl> 39.1, 39.5, 40.3, NA, 36.7, 39.3, 38.9, 39.2, 34.1, …
$ bill_depth_mm     <dbl> 18.7, 17.4, 18.0, NA, 19.3, 20.6, 17.8, 19.6, 18.1, …
$ flipper_length_mm <int> 181, 186, 195, NA, 193, 190, 181, 195, 193, 190, 186…
$ body_mass_g       <int> 3750, 3800, 3250, NA, 3450, 3650, 3625, 4675, 3475, …
$ sex               <fct> male, female, female, NA, female, male, female, male…
$ year              <int> 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007…

You can also force more data to be shown with the width option:

dplyr::glimpse(palmerpenguins::penguins, width = 200)
Rows: 344
Columns: 8
$ species           <fct> Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adelie, Adel…
$ island            <fct> Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgersen, Torgers…
$ bill_length_mm    <dbl> 39.1, 39.5, 40.3, NA, 36.7, 39.3, 38.9, 39.2, 34.1, 42.0, 37.8, 37.8, 41.1, 38.6, 34.6, 36.6, 38.7, 42.5, 34.4, 46.0, 37.8, 37.7, 35.9, 38.2, 38.8, 35.3, 40.6, 40.5, 37.9, …
$ bill_depth_mm     <dbl> 18.7, 17.4, 18.0, NA, 19.3, 20.6, 17.8, 19.6, 18.1, 20.2, 17.1, 17.3, 17.6, 21.2, 21.1, 17.8, 19.0, 20.7, 18.4, 21.5, 18.3, 18.7, 19.2, 18.1, 17.2, 18.9, 18.6, 17.9, 18.6, …
$ flipper_length_mm <int> 181, 186, 195, NA, 193, 190, 181, 195, 193, 190, 186, 180, 182, 191, 198, 185, 195, 197, 184, 194, 174, 180, 189, 185, 180, 187, 183, 187, 172, 180, 178, 178, 188, 184, 195…
$ body_mass_g       <int> 3750, 3800, 3250, NA, 3450, 3650, 3625, 4675, 3475, 4250, 3300, 3700, 3200, 3800, 4400, 3700, 3450, 4500, 3325, 4200, 3400, 3600, 3800, 3950, 3800, 3800, 3550, 3200, 3150, …
$ sex               <fct> male, female, female, NA, female, male, female, male, NA, NA, NA, NA, female, male, male, female, female, male, female, male, female, male, female, male, male, female, male…
$ year              <int> 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007, 2007…

Footnotes

  1. Technically, glimpse is from the pillar package, but it’s re-exported by dplyr, which I usually have loaded anyway.↩︎

]]> dataframes summarizing https://abraver.github.io/usefulRstuff/posts/glimpse/index.html Mon, 26 Jun 2023 05:00:00 GMT janitor: Clean dirty data, plus improved tables and crosstab Aaron Braver https://abraver.github.io/usefulRstuff/posts/janitor/index.html
Setup 
library(janitor)

janitor contains various tools for examining and cleaning dirty data.

Cleaning dirty data

Clean column names

Let’s create a df with some poorly-chosen column names:

test_df <- as.data.frame(matrix(ncol = 6))
names(test_df) <- c("firstName", "ábc@!*", "% successful (2009)",
                    "REPEAT VALUE", "REPEAT VALUE", "")

clean_names() does just as the name implies:

test_df |> 
  clean_names()
first_name abc percent_successful_2009 repeat_value repeat_value_2 x
NA NA NA NA NA NA

The case argument to clean_names() specifies what case you’d like output names to be in. You can specify any case style that’s available in snakecase::to_any_case(), including “screaming_snake” if you want to be perverse:

test_df |> 
  clean_names(case = "screaming_snake")
FIRST_NAME ABC PERCENT_SUCCESSFUL_2009 REPEAT_VALUE REPEAT_VALUE_2 X
NA NA NA NA NA NA

Check if dfs are row-bind-able

Also useful is compare_df_cols which summarizes whether the specified dfs can be row-bound (i.e., have columns of the same names/types):

df1 <- data.frame(A= 1:2, b = c("big", "small"))
df2 <- data.frame(a = 10:12, b = c("medium", "small", "big"), c = 0, stringsAsFactors = TRUE) # here, column b is a factor
df3 <- df1 |> 
  dplyr::mutate(b = as.character(b))

compare_df_cols(df1, df2, df3)
column_name df1 df2 df3
a NA integer NA
A integer NA integer
b character factor character
c NA numeric NA

If you just want a simple TRUE/FALSE value telling you whether the dfs match, you can use compare_df_cols_same():

str(compare_df_cols_same(df1, df2, df3, verbose = FALSE))
 logi FALSE

Examining data and crosstabs

janitor’s version of tables are called tabyls. You can easily generate crosstabs:

palmerpenguins::penguins |> 
  tabyl(species, island)
species Biscoe Dream Torgersen
Adelie 44 56 52
Chinstrap 0 68 0
Gentoo 124 0 0

There are lots of ways to pretty up the output via adorn_* commands, giving things like column or row percentages, optionally with ns in parentheses:

palmerpenguins::penguins |> 
  tabyl(species, island) |> 
  adorn_totals("col")  |> 
  adorn_percentages("row")  |> 
  adorn_pct_formatting(digits = 2) |> 
  adorn_ns()
species Biscoe Dream Torgersen Total
Adelie 28.95% (44) 36.84% (56) 34.21% (52) 100.00% (152)
Chinstrap 0.00% (0) 100.00% (68) 0.00% (0) 100.00% (68)
Gentoo 100.00% (124) 0.00% (0) 0.00% (0) 100.00% (124)

You can also use the adorn_* functions on regular ol’ dfs:

palmerpenguins::penguins |> 
  dplyr::sample_n(10) |>
  dplyr::select(-year) |> 
  adorn_totals("row")
1
Choose 10 random rows so this doesn’t print forever
species island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g sex
Chinstrap Dream 45.7 17.0 195 3650 female
Adelie Torgersen 35.9 16.6 190 3050 female
Chinstrap Dream 45.5 17.0 196 3500 female
Gentoo Biscoe 49.4 15.8 216 4925 male
Adelie Dream 39.7 17.9 193 4250 male
Chinstrap Dream 49.5 19.0 200 3800 male
Gentoo Biscoe 46.6 14.2 210 4850 female
Gentoo Biscoe 50.5 15.9 222 5550 male
Gentoo Biscoe 50.0 15.2 218 5700 male
Gentoo Biscoe 48.2 14.3 210 4600 female
Total - 461.0 162.9 2050 43875 -

Check out the tabyls vignette for more info.

]]> data cleaning https://abraver.github.io/usefulRstuff/posts/janitor/index.html Sun, 25 Jun 2023 05:00:00 GMT diffobj: Compare two dfs (or other objects) Aaron Braver https://abraver.github.io/usefulRstuff/posts/diffobj/index.html

What’s the difference between two dfs?

diffobj::diffPrint() gives a handy visual way to check differences:

library(diffobj)

a <- b <- matrix(1:100, ncol=2)
a <- a[-20,]
b <- b[-45,]
b[c(18, 44)] <- 999

diffPrint(target=a, current=b)
< a                > b              
@@ 17,6 @@         @@ 17,7 @@       
~       [,1] [,2]  ~       [,1] [,2]
  [16,]   16   66    [16,]   16   66
  [17,]   17   67    [17,]   17   67
< [18,]   18   68  > [18,]  999   68
  [19,]   19   69    [19,]   19   69
~                  > [20,]   20   70
  [20,]   21   71    [21,]   21   71
  [21,]   22   72    [22,]   22   72
@@ 42,6 @@         @@ 43,5 @@       
  [41,]   42   92    [42,]   42   92
  [42,]   43   93    [43,]   43   93
< [43,]   44   94  > [44,]  999   94
< [44,]   45   95  ~                
  [45,]   46   96    [45,]   46   96
  [46,]   47   97    [46,]   47   97
(In RStudio, this is shown in a colorful manner in the Viewer)

This also works for other types of objects, such as character vectors:

diffPrint(letters[1:3], c("a", "B", "c"))
< letters[1:3]     > c("a", "B", "..
@@ 1 @@            @@ 1 @@          
< [1] "a" "b" "c"  > [1] "a" "B" "c"

For more details, see the Introduction to diffobjj vignette, or the diffobj github rep.

]]> dataframes https://abraver.github.io/usefulRstuff/posts/diffobj/index.html Sun, 25 Jun 2023 05:00:00 GMT thinkr::peep(): See intermediate stages of piped dfs Aaron Braver https://abraver.github.io/usefulRstuff/posts/thinkr_peep/index.html
Setup 
library(tidyverse)
library(thinkr)

Insert thinkr::peep(head) into a df pipeline to print a few rows at a particular intermediate stage in the pipeline. Here it prints a few lines of the df before it gets summarized:

palmerpenguins::penguins |> 
  peep(head) |>
  group_by(species) |>
  summarize(across(where(is.numeric), \(x) mean(x, na.rm = TRUE)))
1
How convenient that the base R pipe looks like a penguin. 
# A tibble: 6 × 8
  species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
1 Adelie  Torgersen           39.1          18.7               181        3750
2 Adelie  Torgersen           39.5          17.4               186        3800
3 Adelie  Torgersen           40.3          18                 195        3250
4 Adelie  Torgersen           NA            NA                  NA          NA
5 Adelie  Torgersen           36.7          19.3               193        3450
6 Adelie  Torgersen           39.3          20.6               190        3650
# ℹ 2 more variables: sex <fct>, year <int>
species bill_length_mm bill_depth_mm flipper_length_mm body_mass_g year
Adelie 38.79139 18.34636 189.9536 3700.662 2008.013
Chinstrap 48.83382 18.42059 195.8235 3733.088 2007.971
Gentoo 47.50488 14.98211 217.1870 5076.016 2008.081

You can use multiple functions, like head and tail to see two views of the intermediate step; verbose = TRUE prints the functions used for these peeps:

palmerpenguins::penguins |>
  peep(head, tail, verbose = TRUE) |> 
  group_by(species) |> 
  summarize(across(where(is.numeric), \(x) mean(x, na.rm = TRUE)))
head(.)
# A tibble: 6 × 8
  species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
1 Adelie  Torgersen           39.1          18.7               181        3750
2 Adelie  Torgersen           39.5          17.4               186        3800
3 Adelie  Torgersen           40.3          18                 195        3250
4 Adelie  Torgersen           NA            NA                  NA          NA
5 Adelie  Torgersen           36.7          19.3               193        3450
6 Adelie  Torgersen           39.3          20.6               190        3650
# ℹ 2 more variables: sex <fct>, year <int>
tail(.)
# A tibble: 6 × 8
  species   island bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <fct>     <fct>           <dbl>         <dbl>             <int>       <int>
1 Chinstrap Dream            45.7          17                 195        3650
2 Chinstrap Dream            55.8          19.8               207        4000
3 Chinstrap Dream            43.5          18.1               202        3400
4 Chinstrap Dream            49.6          18.2               193        3775
5 Chinstrap Dream            50.8          19                 210        4100
6 Chinstrap Dream            50.2          18.7               198        3775
# ℹ 2 more variables: sex <fct>, year <int>
species bill_length_mm bill_depth_mm flipper_length_mm body_mass_g year
Adelie 38.79139 18.34636 189.9536 3700.662 2008.013
Chinstrap 48.83382 18.42059 195.8235 3733.088 2007.971
Gentoo 47.50488 14.98211 217.1870 5076.016 2008.081

And you can include arguments to functions inside peep() with dot syntax if you’re using the magrittr pipe %>% (at least for the line directly above the peep):1

palmerpenguins::penguins %>% 
  peep(head(n = 7, x = .)) |> 
  group_by(species) |> 
  summarize(across(where(is.numeric), \(x) mean(x, na.rm = TRUE)))
# A tibble: 7 × 8
  species island    bill_length_mm bill_depth_mm flipper_length_mm body_mass_g
  <fct>   <fct>              <dbl>         <dbl>             <int>       <int>
1 Adelie  Torgersen           39.1          18.7               181        3750
2 Adelie  Torgersen           39.5          17.4               186        3800
3 Adelie  Torgersen           40.3          18                 195        3250
4 Adelie  Torgersen           NA            NA                  NA          NA
5 Adelie  Torgersen           36.7          19.3               193        3450
6 Adelie  Torgersen           39.3          20.6               190        3650
7 Adelie  Torgersen           38.9          17.8               181        3625
# ℹ 2 more variables: sex <fct>, year <int>
species bill_length_mm bill_depth_mm flipper_length_mm body_mass_g year
Adelie 38.79139 18.34636 189.9536 3700.662 2008.013
Chinstrap 48.83382 18.42059 195.8235 3733.088 2007.971
Gentoo 47.50488 14.98211 217.1870 5076.016 2008.081

Footnotes

  1. You can’t use the base pipe’s placeholder, _, here, since it does not work in nested functions.↩︎

]]> dataframes piping https://abraver.github.io/usefulRstuff/posts/thinkr_peep/index.html Thu, 01 Jun 2023 05:00:00 GMT