Bonferroni-Holm (1979) correction for multiple comparisons. This is a
sequentially rejective version of the simple Bonferroni correction for multiple
comparisons and strongly controls the family-wise error rate at level alpha.
It works as follows:
1) All p-values are sorted in order of smallest to largest. m is the
number p-values.
2) If the 1st p-value is greater than or equal to alpha/m, the procedure
is stopped and no p-values are significant. Otherwise, go on.
3) The 1st p-value is declared significant and now the second p-value is
compared to alpha/(m-1). If the 2nd p-value is greater than or equal
to alpha/(m-1), the procedure is stopped and no further p-values are
significant. Otherwise, go on.
4) Et cetera.
As stated by Holm (1979) "Except in trivial non-interesting cases the
sequentially rejective Bonferroni test has strictly larger probability of
rejecting false hypotheses and thus it ought to replace the classical
Bonferroni test at all instants where the latter usually is applied."
function [corrected_p, h]=bonf_holm(pvalues,alpha)
Required Inputs:
pvalues - A vector or matrix of p-values. If pvalues is a matrix, it can
be of any dimensionality (e.g., 2D, 3D, etc...).
Optional Input:
alpha - The desired family-wise alpha level (i.e., the probability of
rejecting one of more null hypotheses when all null hypotheses are
really true). {default: 0.05}
Output:
corrected_p - Bonferroni-Holm adjusted p-values. Any adjusted p-values
less than alpha are significant (i.e., that null hypothesis
is rejected). The adjusted value of the smallest p-value
is p*m. The ith smallest adjusted p-value is the max of
p(i)*(m-i+1) or adj_p(i-1). Note, corrected p-values can
be greater than 1.
h - A binary vector or matrix of the same dimensionality as
pvalues. If the ith element of h is 1, then the ith p-value
of pvalues is significant. If the ith element of h is 0, then
the ith p-value of pvalues is NOT significant.
Example:
>>p=[.56 .22 .001 .04 .01]; %five hypothetical p-values
>>[cor_p, h]=bonf_holm(p,.05)
cor_p =
0.5600 0.4400 0.0050 0.1200 0.0400
h =
0 0 1 0 1
Conclusion: the third and fifth p-values are significant, but not the
remaining three.
Reference:
Holm, S. (1979) A simple sequentially rejective multiple test procedure.
Scandinavian Journal of Statistics. 6, 65-70.
Author:
David M. Groppe
Kutaslab
Dept. of Cognitive Science
University of California, San Diego
March 24, 20100001 % Bonferroni-Holm (1979) correction for multiple comparisons. This is a 0002 % sequentially rejective version of the simple Bonferroni correction for multiple 0003 % comparisons and strongly controls the family-wise error rate at level alpha. 0004 % 0005 % It works as follows: 0006 % 1) All p-values are sorted in order of smallest to largest. m is the 0007 % number p-values. 0008 % 2) If the 1st p-value is greater than or equal to alpha/m, the procedure 0009 % is stopped and no p-values are significant. Otherwise, go on. 0010 % 3) The 1st p-value is declared significant and now the second p-value is 0011 % compared to alpha/(m-1). If the 2nd p-value is greater than or equal 0012 % to alpha/(m-1), the procedure is stopped and no further p-values are 0013 % significant. Otherwise, go on. 0014 % 4) Et cetera. 0015 % 0016 % As stated by Holm (1979) "Except in trivial non-interesting cases the 0017 % sequentially rejective Bonferroni test has strictly larger probability of 0018 % rejecting false hypotheses and thus it ought to replace the classical 0019 % Bonferroni test at all instants where the latter usually is applied." 0020 % 0021 % 0022 % function [corrected_p, h]=bonf_holm(pvalues,alpha) 0023 % 0024 % Required Inputs: 0025 % pvalues - A vector or matrix of p-values. If pvalues is a matrix, it can 0026 % be of any dimensionality (e.g., 2D, 3D, etc...). 0027 % 0028 % Optional Input: 0029 % alpha - The desired family-wise alpha level (i.e., the probability of 0030 % rejecting one of more null hypotheses when all null hypotheses are 0031 % really true). {default: 0.05} 0032 % 0033 % Output: 0034 % corrected_p - Bonferroni-Holm adjusted p-values. Any adjusted p-values 0035 % less than alpha are significant (i.e., that null hypothesis 0036 % is rejected). The adjusted value of the smallest p-value 0037 % is p*m. The ith smallest adjusted p-value is the max of 0038 % p(i)*(m-i+1) or adj_p(i-1). Note, corrected p-values can 0039 % be greater than 1. 0040 % h - A binary vector or matrix of the same dimensionality as 0041 % pvalues. If the ith element of h is 1, then the ith p-value 0042 % of pvalues is significant. If the ith element of h is 0, then 0043 % the ith p-value of pvalues is NOT significant. 0044 % 0045 % Example: 0046 % >>p=[.56 .22 .001 .04 .01]; %five hypothetical p-values 0047 % >>[cor_p, h]=bonf_holm(p,.05) 0048 % cor_p = 0049 % 0.5600 0.4400 0.0050 0.1200 0.0400 0050 % h = 0051 % 0 0 1 0 1 0052 % 0053 % Conclusion: the third and fifth p-values are significant, but not the 0054 % remaining three. 0055 % 0056 % Reference: 0057 % Holm, S. (1979) A simple sequentially rejective multiple test procedure. 0058 % Scandinavian Journal of Statistics. 6, 65-70. 0059 % 0060 % 0061 % Author: 0062 % David M. Groppe 0063 % Kutaslab 0064 % Dept. of Cognitive Science 0065 % University of California, San Diego 0066 % March 24, 2010 0067 0068 0069 function [corrected_p, h]=bonf_holm(pvalues,alpha) 0070 0071 if nargin<1, 0072 error('You need to provide a vector or matrix of p-values.'); 0073 else 0074 if ~isempty(find(pvalues<0,1)), 0075 error('Some p-values are less than 0.'); 0076 elseif ~isempty(find(pvalues>1,1)), 0077 fprintf('WARNING: Some uncorrected p-values are greater than 1.\n'); 0078 end 0079 end 0080 0081 if nargin<2, 0082 alpha=.05; 0083 elseif alpha<=0, 0084 error('Alpha must be greater than 0.'); 0085 elseif alpha>=1, 0086 error('Alpha must be less than 1.'); 0087 end 0088 0089 s=size(pvalues); 0090 if isvector(pvalues), 0091 if size(pvalues,1)>1, 0092 pvalues=pvalues'; 0093 end 0094 [sorted_p sort_ids]=sort(pvalues); 0095 else 0096 [sorted_p sort_ids]=sort(reshape(pvalues,1,prod(s))); 0097 end 0098 [dummy, unsort_ids]=sort(sort_ids); %indices to return sorted_p to pvalues order 0099 0100 m=length(sorted_p); %number of tests 0101 mult_fac=m:-1:1; 0102 cor_p_sorted=sorted_p.*mult_fac; 0103 cor_p_sorted(2:m)=max([cor_p_sorted(1:m-1); cor_p_sorted(2:m)]); %Bonferroni-Holm adjusted p-value 0104 corrected_p=cor_p_sorted(unsort_ids); 0105 corrected_p=reshape(corrected_p,s); 0106 h=corrected_p<alpha; 0107 0108