So, after a lot of fiddling with advanced assignments, I've figured out that click or press with an invert modifier will make it work as it should. Why the game doesn't apply this by default is beyond me. Nor is any of this shown in the advanced assignment modal, suggesting instead that none is fine. Any hint that changing it will show, like for example by greying out the options until another choice like press is made, would be infinitely better. The way it is done now is exceedingly confusing and breaks expectations created by decennia of industry applying a more or less standard way of doing things.