5.4 Test on the reference standard deviation

Null hypothesis H_0: m0 = m0’ is tested versus alternative hypothesis H_1: m0 neq m0’. Test criterion is ratio of a posteriori estimate of reference standard deviation

         m0' = sqrt( v'P v / r).

and a priori reference standard deviation m0 (input data parameter m0-apr). For given significance level alpha lower and upper bounds of interval (L, U) are computed so, that if hypothesis H_0 is true, probabilities P(m0’/m0 le D) and P(m0’/m0 ge H) are equal to alpha/2. Lower and upper bounds of the interval are computed as

         L = sqrt((Chi^2_{1-alpha/2,r})/r),
         U = sqrt((Chi^2_{ alpha/2 ,r})/r).

Probability

         P(L < m0'/m0 < U) = conf-pr

is by default 95%, this corresponds to 5% confidence level test.

Exceeding the upper limit H of the confidence interval can be caused even by a single gross error (one outlying observation). Method of Least Squares is generally very sensitive to presence of outliers. Safely can be detected only one observation whose elimination leads to maximal decrease of a posteriori estimate of reference standard deviation

(6)      m0''  = sqrt{(v'P v - delta)/(r-1)},
         delta = max(v_i^2/q_vi),

where

(7)      q_vi = 1/p_i - q_Li

is weight coefficient of i-th residual. If the set of observations contains only one gross error, the outlying observation is likely to be detected, but this can not be guaranteed.

In addition, program gama-local computes a posteriori estimate of reference standard deviation separately for horizontal distances and directions and/or angles after formula from

         m0'_t = sqrt(sum{~v^2_it}) / sum{~q_vi}),    t=d,s,

where symbol t denotes observed distances, directions and/or angles.

Example

m0  apriori  :    10.00
m0' empirical:     9.64         [pvv] : 3.43560e+03

During statistical analysis we work

- with empirical standard deviation 9.64
- with confidence level             95 %

Ratio m0' empirical / m0 apriori: 0.964
95 % interval (0.773, 1.227) contains value m0'/m0
m0'/m0 (distances): 0.997   m0'/m0 (directions): 0.943

Maximal decrease of m0''/m0 on elimination of one observation: 0.892

Maximal studentized residual 2.48 exceeds critical value 1.95
on significance level 5 % for observation #35
<distance from="407" to="422" val="346.415" stdev="5.0" />

This document was generated on February 17, 2024 using texi2html 1.82.