By default, when the Private computer checkbox is visible, it is checked by default, meaning that OWA will store some things in your browser by default. Sometimes, this may not be desirable, and you may prefer the checkbox is unchecked by default so as to prevent this. There is no setting to change this default, but there's a workaround, allowing you to obtain the same result. The only downside of this workaround is that it needs to be done anew after each update of Exchange.

To show the Private Computer checkbox in the first place, read this article about enabling the option.

To apply the workaround, you need to manually edit the logon.aspx page so it does not check the checkbox by defaul  ...