Display combination of Phonetic Characters on a ggplot graph using a different computer font

Question

I'm creating graphic plots that contain IPA symbols (Phonetic symbols) using ggplot on R. The font used by ggplot looks great for most of the characters, but when it's a combination of characters, they look misaligned when displayed in the output. A combination of characters is something like this: t̪ (t + U032A) or ʈ͡ʂ (U0288+U0282+U0361).

This is the data.frame and code I used for the graph:

phones <- c("a", "e", "k", "ə", "ʈ͡ʂ", "n", "t͡ʃ", "n̪", "s", "l̪", "t̪")
freq   <- c(34,   28, 29,  26,    10,   8,     5,   3,   2,   4,   1)
df     <- data.frame(phones, freq)

  library(ggplot2)

    graph <- ggplot(subset(df, freq > 0), aes(x = reorder(phones, -freq), y = freq)) +
      geom_bar(stat = "identity")
    graph

    png(
      filename = "test.png",
      height   = 1500,  width    = 2500,
      res      = 300,   units    = "px")
    graph +     labs(title = "", 
                 subtitle="", 
                 caption  = "",
                 x = "",
                 y = "")
    dev.off(which = dev.cur())

This creates the following graph:

And I'm expecting something like this (characters in red using a special font Doulos SIL):

I think that I can specify the computer font Doulos SIL (Link to font) (or any other font that display combination of characters) before creating the graph in order to display neat characters, but I haven't been able to do so on my code. I'd appreciate any help, thanks!