Line→Circle/Divergence Circuit: Another example of equivariant features being combined to create invariant features is very early line-like complex Gabor detectors being combined to create a small circle unit and diverging lines unit. … these more sophisticated equivariant networks make the weights between two neurons equal if they have the same relative relationship under more general transformations: For our purposes, it suffices to know that these equivariant neural networks have the same weights when there is the same relative relationship between neurons. Looking at mixed3b, we found that the equivariant model learned analogues of many large rotationally equivariant families from InceptionV1, such as curve detectors, boundary detectors, divot detectors, and oriented fur detectors: