Statistical Significance

Chris,

Unfortunately, you cannot use the p-value and power to calculate the posterior probability of a hypothesis, given a result R.  The p-value is the probability of the observed result or a more extreme one.  But in Bayesian inference, only the probability of the observed result itself is relevant.

The observed value R will be a realization of some random variable Y.  The correct value for a is the likelihood of R under H0, that is, the value of the probability density function (pdf) of Y specified by H0 at the value R. 

The correct value for b is the marginal likelihood of R under H1, that is, the average value of the likelihood of R averaged over all possible probability density functions under H1. Typically, there are an infinite number of these pdfs. For example if H1 is that the hit rate in a Ganzfeld experiment will not be p=0.25, then there is a pdf under H1 for every value p in the interval [0,1] except for 0.25.  To compute the marginal likelihood, we specify a probability distribution g(p) over the possible values of p, and compute the weighted average of the likelihood of R under each such pdf with weights specified by g(p).  Since g(p) is a continuous function, the marginal likelihood is computed by integration.

I think you have misunderstood the calculation. The probabilities I've considered are the probabilities of obtaining a statistically significant result, not the probabilities of observing a particular value of any variable.

The probability a is the probability of obtaining a significant result under the null hypothesis. So it's not the value of the pdf for any particular value of your variable R, but the integral of the pdf over the range of values of R corresponding to significant results.

The probability b is the power. That is the probability of obtaining a significant result under the psi hypothesis. So that too is an integral of the pdf corresponding to the psi hypothesis, also over the range of values of R corresponding to significant results.

In deriving those equations, no assumption is necessary about what the psi hypothesis is, though of course in any particular application the psi hypothesis will need to be specified mathematically in order to calculate b. It could be a hypothesis of the form you specify, namely a kind of superposition of different values of p. But of course, psi may not work in that simple way, so the hypothesis could be something much more complicated.

(Edit: Just to be clear, I am defining "success" as obtaining a statistically significant result. Not a particular result, but any result in the range defined as statistically significant.)