Papers
arxiv:2202.09276

Geometric Regularization from Overparameterization

Published on Sep 25, 2022
Authors:

Abstract

A geometric explanation is proposed for double descent, linking implicit regularization to how the volume of possible weight updates shrinks as intrinsic dimensionality changes near global minima.

The volume of the distribution of weight sets associated with a loss value may be the source of implicit regularization from overparameterization due to the phenomenon of contracting volume with increasing dimensions for geometric figures demonstrated by hyperspheres. We introduce the geometric regularization conjecture and extract to an explanation for the double descent phenomenon by considering a similar property resulting from shrinking intrinsic dimensionality of the distribution of potential weight set updates available along training path, where if that distribution retracts across a volume verses dimensionality curve peak when approaching the global minima we could expect geometric regularization to re-emerge. We illustrate how data fidelity representational complexity may influence model capacity double descent interpolation thresholds. The existence of epoch and model capacity double descent curves originating from different geometric forms may imply universality of closed n-manifolds having dimensionally adjusted n-sphere volumetric correspondence.

Community

As a parting gift to this platform the author offers a 2022 preprint in which was proposed a physics grounded mechanism for the double descent phenomenon that had been surveyed in the NeurIPS literature. He never found domain experts to share a conversation about the culmination of years of volunteer research. The paper was fully human authored without the use of language models. To the author's knowledge it has never been cited in any formal capacity.

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2202.09276
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2202.09276 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2202.09276 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2202.09276 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.