After some research and a lot of trial and error the picture became clear and a solution emerged.
The problem is that 'axes' is separated from the graphs, legends and other 'creatures' which are added later. In order to fully switch between one 'axes' and all what is drawn on it and another set you have to change the visibility to 'off' for each component. While this works fine its a bit cumbersome.
A much simpler solution is to have each axes be part of a different 'panel' and to have both panels have the same dimensions, position and color (just on top of each other). Then all you have to do is to change the first 'panel' visibility to 'off' and turn the others visibility to 'on' (or vise versa). This way you smoothly switch between them.
Works like charm in Matlab 20165b.