'How to filter very small values in r?
I have a large dataset in which one column is p-values that range from 0.9 to being extremely small like 5e-79. In R I can sort the data in descending order and see the smallest values are there and in scientific notation.
I am just trying to filter the values to get only those smaller than 0.00000005. However, any line I try for filtering does not actually filter out the large p-values.
For example I am trying:
pval <- select(df, pvalue)
pval <- filter(pval, pvalue < 5e-8)
#or:
pval <- select(df, pvalue)
pval <- pval[pval$pvalue < 0.0000005]
When I view the head of my pval output I am getting:
head(pval)
pvalue
1: 0.954
2: 0.787
3: 0.837
4: 0.629
5: 0.422
6: 0.422
The total number of rows does change from df to pvalue (35568235 to 35503455 rows) but I can see the filter isn't removing all the rows I expect it to.
What can cause this and what can I do to fix it? As there's no error message and I haven't found similar enough questions I'm not sure where to start - any help is appreciated.
I've tried running this with some example data which also seems to have the same problem:
pval <- structure(list(pvalue = c("5.00E-72", "1.40E-78", "0.9", "0.5",
"56-e23", "1-e8")), row.names = c(NA, -6L), class = c("data.table",
"data.frame"))
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
