In the following we present the methodology in surveysd
by applying the workflow described in vignette("surveysd")
to multiple consecutive years of EU-SILC data for one country. The
methodology contains the following steps, in this order
- Draw bootstrap replicates from EU-SILC data for each year , separately. Since EU-SILC has a rotating panel design the bootstrap replicate of a household is carried forward through the years. That is, the bootstrap replicate of a household in the follow-up years is set equal to the bootstrap replicate of the same household when it first enters EU-SILC.
- Multiply each set of bootstrap replicates by the sampling weights to obtain uncalibrated bootstrap weights and calibrate each of the uncalibrated bootstrap weights using iterative proportional fitting.
- Estimate the point estimate of interest , for each year and each calibrated bootstrap weight to obtain , , . For fixed apply a filter with equal weights for each on , , to obtain . Estimate the variance of using the distribution of .
Bootstrapping
Bootstrapping has long been around and used widely to estimate confidence intervals and standard errors of point estimates.[Efron (1979)} Given a random sample drawn from an unknown distribution the distribution of a point estimate can in many cases not be determined analytically. However when using bootstrapping one can simulate the distribution of .
Let be a bootstrap sample, e.g. drawing observations with replacement from the sample , then one can estimate the standard deviation of using bootstrap samples through
with as the sample mean over all bootstrap samples.
In context of sample surveys with sampling weights one can use bootstrapping to calculate so called bootstrap weights. These are computed via the bootstrap samples , , where for each every unit of the original sample can appear - to -times. With as the frequency of occurrence of observation in bootstrap sample the uncalibrated bootstrap weights are defined as:
with as the calibrated sampling weight of the original sample. Using iterative proportional fitting procedures one can recalibrate the bootstrap weights , to get the adapted or calibrated bootstrap weights , .
Rescaled Bootstrap
Since EU-SILC is a stratified sample without replacement drawn from a finite population the naive bootstrap procedure, as described above, does not take into account the heterogeneous inclusion probabilities of each sample unit. Thus it will not yield satisfactory results. Therefore we will use the so called rescaled bootstrap procedure introduced and investigated by (Rao and Wu 1988). The bootstrap samples are selected without replacement and do incorporate the stratification as well as clustering on multiple stages (see (Chipperfield and Preston 2007),(Preston 2009)).
For simplistic reasons we will only describe the rescaled bootstrap procedure for a two stage stratified sampling design. For more details on a general formulation please see (Preston 2009).
Sampling design
Consider the finite population which is divided into non-overlapping strata , of which each strata contains of clusters. For each strata , , clusters are drawn, containing households. Furthermore in each cluster of each strata simple random sampling is performed to select a set of households , .
Bootstrap procedure
In contrast to the naive bootstrap procedure where for a stage, containing sampling units, the bootstrap replicate is obtained by drawing sampling units with replacement, for the rescaled bootstrap procedure sampling units are drawn without replacement. Given a value , denotes the largest integer smaller than , whereas denotes the smallest integer lager then . (Chipperfield and Preston 2007) have shown that the choice of either or is optimal for bootstrap samples without replacement, although has the desirable property that the resulting uncalibrated bootstrap weights will never be negative.
At the first stage the -th bootstrap replicate, , for each cluster ,, belonging to strata , is defined by
with
where if cluster is selected in the sub-sample of size and 0 otherwise.
The -th bootstrap replicate at the second stage, , for each household , , belonging to cluster in strata is defined by
with
where if household is selected in the sub sample of size and 0 otherwise.
Single PSUs
When dealing with multistage sampling designs the issue of single PSUs, e.g. a single response unit is present at a stage or in a strata, can occur. When applying bootstrapping procedures these single PSUs can lead to a variety of issues. For the methodology proposed in this work we combined single PSUs at each stage with the next smallest strata or cluster, before applying the bootstrap procedure.
Taking bootstrap replicates forward
The bootstrap procedure above is applied on the EU-SILC data for each year , separately. Since EU-SILC is a yearly survey with rotating penal design the -th bootstrap replicate at the second stage, , for a household is taken forward until the household drops out of the sample. That is, for the household , which enters EU-SILC at year and drops out at year , the bootstrap replicates for the years are set to the bootstrap replicate of the year .
Split households
Due to the rotating penal design so called split households can occur. For a household participating in the EU-SILC survey it is possible that one or more residents move to a new so called split household, which is followed up on in the next wave. To take this dynamic into account we extended the procedure of taking forward the bootstrap replicate of a household for consecutive waves of EU-SILC by taking forward the bootstrap replicate to the split household. That means, that also any new individuals in the split household will inherit this bootstrap replicate.
Taking bootstrap replicates forward as well as considering split households ensures that bootstrap replicates are more comparable in structure with the actual design of EU-SILC.
Uncalibrated bootstrap weights
Using the -th bootstrap replicates at the second stage one can calculate the -th uncalibrated bootstrap weights for each household in cluster contained in strata by
where corresponds to the original household weight contained in the sample.
For ease of readability we will drop the subindices regarding strata and cluster for the following sections, meaning that the -th household in cluster contained in strata , , will now be denoted as the -th household, , where is the position of the household in the data. In accordance to this the -th uncalibrated bootstrap replicates for household are thus denoted as and the original household weight as .
Iterative proportional fitting (IPF)
The uncalibrated bootstrap weights computed through the rescaled bootstrap procedure yields population statistics that differ from the known population margins of specified sociodemographic variables for which the base weights have been calibrated. To adjust for this the bootstrap weights can be recalibrated using iterative proportional fitting as described in (Meraner, Gumprecht, and Kowarik 2016).
Let the original weight be calibrated for sociodemographic variables which are divided into the sets and . and correspond to personal, for example gender or age, or household variables, like region or households size, respectively. Each variable in either or can take on or values with and , , or , , as the corresponding population margins. Starting with the iterative proportional fitting procedure is applied on each , separately. The weights are first updated for personal and afterwards updated for household variables. If constraints regarding the populations margins are not met is raised by 1 and the procedure starts from the beginning. For the following denote as starting weight for fixed .
Adjustment and trimming for
The uncalibrated bootstrap weight for the -th observation is iteratively multiplied by a factor so that the projected distribution of the population matches the respective calibration specification , . For each the calibrated weights against are computed as where the summation in the denominator expands over all observations which have the same value as observation for the sociodemographic variable . If any weights fall outside the range they will be recoded to the nearest of the two boundaries. The choice of the boundaries results from expert-based opinions and restricts the variance of which has a positive effect on the sampling error. This procedure represents a common form of weight trimming where very large or small weights are trimmed in order to reduce variance in exchange for a possible increase in bias ((Potter 1990),(Potter 1993)).
Averaging weights within households
Since the sociodemographic variables include person-specific variables, the weights resulting from the iterative multiplication can be unequal for members of the same household. This can lead to inconsistencies between results projected with household and person weights. To avoid such inconsistencies each household member is assigned the mean of the household weights. That is for each person in household with household members, the weights are defined by This can result in losing the population structure performed in the previous subsection.
Adjustment and trimming for
After adjustment for individual variables the weights are updated for the set of household variables according to a household convergence constraint parameter . The parameters represent the allowed deviation from the population margins using the weights compared to , , . The updated weights are computed as with the summation in the denominator ranging over all households which take on the same values for as observation . As described in the previous subsection the new weight are recoded if they exceed the interval and set to the upper or lower bound, depending of falls below or above the interval respectively.
Convergence
For each adjustment and trimming step the factor , , is checked against convergence constraints for households, , or personal variables , where corresponds to either a household or personal variable. To be more precise for variables in the constraints
and for variables in the constraints
are verified, where the sum in the denominator expands over all observations which have the same value for variables or . If these constraints hold true the algorithm reaches convergence, otherwise is raised by 1 and the procedure repeats itself.
The above described calibration procedure is applied on each year of EU-SILC separately, , thus resulting in so called calibrated bootstrap sample weights , for each year and each household .
Variance estimation
Applying the previously described algorithms to EU-SILC data for multiple consecutive years , , yields calibrated bootstrap sample weights for each year . Using the calibrated bootstrap sample weights it is straight forward to compute the standard error of a point estimate for year with as the vector of observations for the variable of interest in the survey and as the corresponding weight vector, with
with where is the estimate of in the year using the -th vector of calibrated bootstrap weights.
As already mentioned the standard error estimation for indicators in EU-SILC yields high quality results for NUTS1 or country level. When estimation indicators on regional or other sub-aggregate levels one is confronted with point estimates yielding high variance.
To overcome this issue we propose to estimate for 3, consecutive years using the calibrated bootstrap weights, thus calculating , . For fixed one can apply a filter with equal filter weights on the time series to create
Doing this for all , , yields , . The standard error of can then be estimated with
with
Applying the filter over the time series of estimated leads to a reduction of variance for since the filter reduces the noise in and thus leading to a more narrow distribution for .
It should also be noted that estimating indicators from a survey with rotating panel design is in general not straight forward because of the high correlation between consecutive years. However with our approach to use bootstrap weights, which are independent from each other, we can bypass the cumbersome calculation of various correlations, and apply them directly to estimate the standard error. (Bauer et al. 2013) showed that using the proposed method on EU-SILC data for Austria the reduction in resulting standard errors corresponds in a theoretical increase in sample size by about 25. Furthermore this study compared this method to the use of small area estimation techniques and on average the use of bootstrap sample weights yielded more stable results.