Despite their importance in supporting experimental conclusions, standard
statistical tests are often inadequate for research areas, like the life
sciences, where the typical sample size is small and the test assumptions
difficult to verify. In such conditions, standard tests tend to be overly
conservative, and fail thus to detect significant effects in the data. Here we
define a novel statistical test for the two-sample problem. Several
characteristics make it an attractive alternative to classical two-sample
tests: 1) It is based on Bayesian model selection, and thus takes into account
uncertainty about the model's parameters, mitigating the problem of small
samples size; 2) The null hypothesis is compared with several alternative
hypotheses, making the test suitable in different experimental scenarios; 3)
The test is constructed as a frequentist test, and defines significance with
the conventional bound on Type I errors. We analyze the power of the test and
find that it is higher than the power of other standard options, like the
t-test (up to 25% higher) for a wide range of sample and effect sizes, and is
at most 1% lower when the assumptions of the t-test are perfectly matched. We
discuss and evaluate two variants of the test, that define different prior
distributions over the parameters of the hypotheses.