Relaxation schedule for the soft corner's weight update

The weight update is relaxed, \(W \mapsto (1 - d) W + d f(W)\). Relaxation moves no fixed point, since \(W = f(W)\) implies \(W = (1 - d) W + d f(W)\); the same argument licenses changing \(d\) between iterations, so the factor can be adapted freely.

.gloc_damp

.gloc_damp_start

.gloc_damp_shrink

Format

length-one numerics.

An object of class numeric of length 1.

Details

The update has the same form as the damping in imputeCellEM and imputeCellwise, which ramp \(\lambda\) adaptively rather than holding it fixed. imputeCellGLoc does the same: it starts at .gloc_damp_start (no relaxation at all) and multiplies the factor by .gloc_damp_shrink whenever max |dW| fails to fall from one iteration to the next, never going below .gloc_damp. The factor never rises again, so a spurious trigger costs iterations, never correctness.

What relaxation is and is not for. Until 7.4.0 the peer-inclusion rule was a hard cut, which made the weight map discontinuous, and the floor was set low (0.25) to suppress the resulting limit cycles. That was treating the symptom: a discontinuous self-map need not have a fixed point at all, so no amount of relaxation could make the stopping rule attainable, and the documentation said as much – configurations existed that cycled forever at the floor. Since the threshold became a band (see .gloc_peer_band) the map is continuous, cycling in the peer-inclusion decision is largely gone, and relaxation is back to its ordinary job: keeping the fixed-point iteration inside its contraction radius. The floor is therefore set by how fast the iteration converges, not by how badly it cycles, and it rises from 0.25 to 0.50.

The floor and the band are a package and must not be quoted apart. The floor is worth about 40% of the convergence gain on the mean-structure arm (131, then 181, then 245 of 260 fits, adding the floor and the scatter condition first and the band second), and it is worth that only with the band: under the hard cut a low floor is the better setting, and raising it costs convergence rather than buying it – measured on 260 pooled fits, the hard cut converges 208 times at 0.5 against 245 at 0.25.

The table below is the 36-configuration sweep, re-measured under the band and under the current convergence rule, from a generator stated in full so that it can be checked – 6 seeds, \(\rho \in \{0, 0.5, 0.8\}\), clean and 5% of cells shifted by +8, \(n = 800\), \(p = 4\), design = ~ 1, weights = "soft", maxit = 200, data from MASS::mvrnorm with \(\Sigma = (1 - \rho) I + \rho\):

damping	hard cut	median iters	band	median iters
1.00 (none)	20/36	8	19/36	9
0.75	26/36	13	29/36	11
0.50	30/36	21	36/36	17
0.25	34/36	44	36/36	36
0.10	35/36	113	36/36	92
adaptive, floor 0.25	34/36	14	36/36	12
adaptive, floor 0.50	–	–	36/36	11.5

Under the hard cut the floor bought convergence monotonically all the way down and never reached 36/36. Under the band, 0.50 already reaches 36/36 and everything below it only costs iterations. The floor is set there.

It is not only the \(p = 4\) sweep that prefers 0.50. On the mean-structure arm this function exists for – design = ~ ., \(n = 200\), 6 continuous and 6 categorical variables, so 19 design columns from 200 rows – the smooth part of the map has a contraction factor near 0.92, and relaxing to 0.25 slows that to 0.98 per iteration, which does not fit inside maxit. Across 40 such fits: floor 0.25 converges 35/40 in a median 89 iterations, floor 0.50 converges 40/40 in a median 47. Every floor-0.25 failure there is an exhausted budget, not a cycle.

Every failure in the hard-cut column of the table, at every damping level, is at \(\rho \ge 0.5\); nothing ever fails at \(\rho = 0\), even undamped. That is the shape of the boundary: threshold crossings are driven by correlation, because when the columns are strongly correlated dropping one peer moves the conditional variance a long way. It is also what makes the adaptive schedule pay: a fixed floor spends several times the iterations it needs on uncorrelated data, where no relaxation is called for at all.

The cold-start fallback. The hard-cut map was discontinuous, so by Brouwer it need not have possessed a fixed point at all and the iteration could sit in a limit cycle indefinitely; which cycle it entered depended on the path taken, so reaching the floor along a weakly relaxed path could land in one that starting at the floor avoided.

Correction, recorded rather than deleted. An earlier version of this paragraph said the discontinuous map "had several fixed points" and that the iteration selected among them, and gave that as the reason the relaxed and adaptive schedules disagreed. That explanation is false. It was falsified by tightening eps from 5e-3 to 1e-5, which collapsed the relative scatter difference between the two schedules from 1.28e-2 to 3.3e-6 and the flag disagreement from 4 cells to zero – distinct fixed points do not vanish under a tighter tolerance. The real cause was the convergence rule: it compared a relaxed step against eps and so stopped at a fixed-point residual 1/damp times looser, which made the stopping point depend on the relaxation factor. That is fixed (see eps), and the fallback is retained for the path effect described above, which is about which cycle is entered, not about which fixed point is selected. When the factor is at the floor and neither max |dW| nor, under the categorical EM, the largest posterior change has improved for .gloc_stall_iters iterations, the iteration therefore falls back once to the cold start at the floor, which is precisely the fixed-.gloc_damp run.

The fallback is kept, but it is worth being precise about what it is now worth, because at the old floor it was doing real damage. Measured on the 40 mean-structure fits at floor 0.25 with the band, it fires in every non-converged run, discards between 30 and 98 iterations of progress – in one case a state whose fixed-point residual was within 26% of the tolerance – and costs four configurations: 35/40 with it against 39/40 without. At the 0.50 floor it never fires there at all (40/40 either way, to the iteration). On the \(p = 4\) sweep at floor 0.50 it is still worth exactly one configuration, 36/36 with against 35/36 without. So it is retained: at the current floor it is free where it used to be harmful, and it still rescues a case.