'Using character vectors from two different data frames as formula in regression with lm function
Currently i have two data frames that would look something like this:
df1 <- as.data.frame(rbind(c("Name1", "Name2"), c("Name2", "Name3"), c("Name4", "Name5"), c("Name4", "Name3")))
df2 <- as.data.frame(cbind(c(153, 157, 167, 163, 165), c(132, 127, 130, 132, 134), c(72, 83, 85, 90, 86), c(240, 238, 245, 247, 250), c(121, 125, 130, 128, 132)))
colnames(df2) <- c("Name1", "Name2", "Name3", "Name4", "Name5")
df1 contains the information on what pairs i want to regress upon each other, while df2 contains the values to be regressed upon. So I am trying to do something like this:
output_all = NULL
for (i in 1:nrow(df1)) {
output <- lm(print(df1[i,1], quote = FALSE) ~ print(df1[i,2], quote = FALSE), data = df2)
b_0 <- leastsqr_output[[1]]
b_1 <- leastsqr_output[[2]]
output_all <- cbind(b_0, b_1)
}
Which produces error:
Error in `contrasts<-`(`*tmp*`, value = contr.funs[1 + isOF[nn]]) :
contrasts can be applied only to factors with 2 or more levels
In addition: Warning message:
In storage.mode(v) <- "double" : NAs introduced by coercion
Now, I believe my issue lies in the lm function, as i produce the same error from:
output <- lm(print(df1[1,1], quote = FALSE) ~ print(df1[1,2], quote = FALSE), data = df2)
I have tried changing the print with other functions such as cat, as I believe the issue may be in how the name is printed.
Is there a way to make this work?
Solution 1:[1]
Use a apply loop and reformulate instead of a for loop.
After the regressions, there is code to get the summaries and the coefficients.
df1 <- as.data.frame(rbind(c("Name1", "Name2"), c("Name2", "Name3"), c("Name4", "Name5"), c("Name4", "Name3")))
df2 <- as.data.frame(cbind(c(153, 157, 167, 163, 165), c(132, 127, 130, 132, 134), c(72, 83, 85, 90, 86), c(240, 238, 245, 247, 250), c(121, 125, 130, 128, 132)))
colnames(df2) <- c("Name1", "Name2", "Name3", "Name4", "Name5")
output_all <- apply(df1, 1, \(x, data) {
fmla <- reformulate(x[2], x[1])
lm(fmla, data = data)
}, data = df2)
smry_list <- lapply(output_all, summary)
coef_list <- lapply(output_all, coef)
output_all[[1]]
#>
#> Call:
#> lm(formula = fmla, data = data[x])
#>
#> Coefficients:
#> (Intercept) Name2
#> 86.1429 0.5714
smry_list[[1]]
#>
#> Call:
#> lm(formula = fmla, data = data[x])
#>
#> Residuals:
#> 1 2 3 4 5
#> -8.571 -1.714 6.571 1.429 2.286
#>
#> Coefficients:
#> Estimate Std. Error t value Pr(>|t|)
#> (Intercept) 86.1429 161.0126 0.535 0.630
#> Name2 0.5714 1.2289 0.465 0.674
#>
#> Residual standard error: 6.503 on 3 degrees of freedom
#> Multiple R-squared: 0.06723, Adjusted R-squared: -0.2437
#> F-statistic: 0.2162 on 1 and 3 DF, p-value: 0.6736
coef_list
#> [[1]]
#> (Intercept) Name2
#> 86.1428571 0.5714286
#>
#> [[2]]
#> (Intercept) Name3
#> 129.63457330 0.01641138
#>
#> [[3]]
#> (Intercept) Name5
#> 121.5614973 0.9625668
#>
#> [[4]]
#> (Intercept) Name3
#> 205.3129103 0.4649891
Created on 2022-05-07 by the reprex package (v2.0.1)
Edit
In order to run the regressions with a for loop, create a empty list with as many members as regressions to run, then use a code equivalent to the apply code above, with reformulate.
The results are nearly equal, with differences in the name of the data set.
output_all2 <- vector("list", length = nrow(df1))
for(i in seq_len(nrow(df1))) {
fmla <- reformulate(df1[i, 2], df1[i, 1])
output_all2[[i]] <- lm(fmla, data = df2)
}
all.equal(output_all, output_all2)
#> [1] "Component 1: Component 10: target, current do not match when deparsed"
#> [2] "Component 2: Component 10: target, current do not match when deparsed"
#> [3] "Component 3: Component 10: target, current do not match when deparsed"
#> [4] "Component 4: Component 10: target, current do not match when deparsed"
output_all[[1]][[10]]
#> lm(formula = fmla, data = data)
output_all2[[1]][[10]]
#> lm(formula = fmla, data = df2)
Created on 2022-05-07 by the reprex package (v2.0.1)
Solution 2:[2]
I have tried changing the print with other functions such as cat, as I believe the issue may be in how the name is printed.
I think you are looking for as.formula combined with paste. Here I try to follow your steps as close as possible with some modification:
output <- vector('list', nrow(df1))
b_0 <- numeric(nrow(df1))
b_1 <- numeric(nrow(df1))
for (i in 1:nrow(df1)) {
output[[i]] <- lm(as.formula(paste(df1[i,1], "~" ,df1[i,2])), data = df2)
b_0[i] <- coef(output[[i]])[1]
b_1[i] <- coef(output[[i]])[2]
}
output_all <- cbind(b_0, b_1)
The results:
output_all
# b_0 b_1
# [1,] 86.14286 0.57142857
# [2,] 129.63457 0.01641138
# [3,] 121.56150 0.96256684
# [4,] 205.31291 0.46498906
Solution 3:[3]
print as well as cat is rather for human readable output, and not very useful here. What you want is paste (try paste(df1[1, 1], '~', df1[1, 2])). However lm expects an object of class "formula" so we need as.formula.
The next mistakes you made is that you didn't use the coefficients for b0 and b1, and that you need to index the output_all list as well, output_all[[i]].
Finally improve your approach by defining the class and size of your output_all for sake for speed, and rather use seq_len(nrow(.)) instead of 1:nrow(.).
output_all <- vector('list', length=nrow(df1))
for (i in seq_len(nrow(df1))) {
output <- lm(as.formula(paste(df1[i, 1], '~', df1[i, 2])), data=df2)
b_0 <- output$coefficients[[1]]
b_1 <- output$coefficients[[2]]
output_all[[i]] <- cbind(b_0, b_1)
}
output_all
# [[1]]
# b_0 b_1
# [1,] 86.14286 0.5714286
#
# [[2]]
# b_0 b_1
# [1,] 129.6346 0.01641138
#
# [[3]]
# b_0 b_1
# [1,] 121.5615 0.9625668
#
# [[4]]
# b_0 b_1
# [1,] 205.3129 0.4649891
Alternative
However, it looks to me, you might look for all combinations of the names, which could be concisely done like this:
r <- t(combn(names(df2), 2, \(x) lm(reformulate(x[2], x[1]), df2)$coef)) |>
`dimnames<-`(list(combn(names(df2), 2, paste, collapse = ' ~ '), paste0('b', 0:1)))
r
# b0 b1
# Name1 ~ Name2 86.142857 0.57142857
# Name1 ~ Name3 103.652079 0.68927790
# Name1 ~ Name4 -68.061224 0.93877551
# Name1 ~ Name5 -2.251337 1.28342246
# Name2 ~ Name3 129.634573 0.01641138
# Name2 ~ Name4 31.408163 0.40816327
# Name2 ~ Name5 105.491979 0.20053476
# Name3 ~ Name4 -128.432653 0.86734694
# Name3 ~ Name5 -76.310160 1.25401070
# Name4 ~ Name5 121.561497 0.96256684
To access one result, subset the matrix, e.g. for the first result:
r[1, ]
# b0 b1
# 86.1428571 0.5714286
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | |
| Solution 2 | Abdur Rohman |
| Solution 3 |