By default, when the Private computer checkbox is visible, it is checked by default, meaning that OWA will store some things in your browser by default. Sometimes, this may not be desirable, and you may prefer the checkbox is unchecked by default so as to prevent this. There is no setting to change this default, but there's a workaround, allowing you to obtain the same result. The only downside of this workaround is that it needs to be done anew after each update of Exchange.
To show the Private Computer checkbox in the first place, read this article about enabling the option.
To apply the workaround, you need to manually edit the logon.aspx page so it does not check the checkbox by defaul ...
Exchange OWA stores data offline in your browser for a better user experience and performance. In older releases of Exchange, this behaviour could be turned off at the logon page by unchecking a checkbox, but since Exchange 2013, this checkbox is no longer visible on the logon page. But, this option can be turned on again by changing a configuration option of OWA in Exchange.
The option can be enabled through EMS on the CAS server. If you have more than one CAS server, this needs to be done on all of them.