Put bluntly: give presumption to the status quo, and put the burden of proof on those who would change it.
This argument is persuasive against radical revolutionaries, but it is vulnerable to a simple counter-argument: times change, and institutions often do not continue to serve the purpose they were originally designed for. Uncritical acceptance of what is can lead us to path dependence, where we repeat the same mistakes over and over again.
How to resolve this dilemma? I'm reading Robust Political Economy by Mark Pennington, and I found his take on this to be fairly compelling. In defending the classical liberal tradition, he says:
Social evolution depends on the battle between competing ideas and can be halted or reversed owing to human error. There is, therefore, no implication that 'whatever is must be efficient.' For much of human history, the prevailing assumption has been that social order can only be maintained by the exercise of deliberate authority. It has been the contribution of the classical liberal tradition to argue that this is not so... Looking for ways to expose institutions to competitive trial and error may be a more robust method...From this perspective, a limited government is best because it allows maximal experimentation. The wisdom of ages will thus be preserved, while path dependence will fail the competitive test. I'm not sure that's the end of the story for Burke, but it's a good advancement in the classical conservatism vs. classical liberalism debate.
Recognising the benefits of competitive processes is not to deny a role for institutional design. Rather, the central concern is to create a framework within which evolutionary processes can be harnessed to beneficial effect. (p. 43)