There was a notification that wasn't hidden on page load (as it should have been). This is because the JS code that hides it is in jQuery, and is run on the page load. But the message to hide is inside an Angular directive which is being "rendered" on page load.

So I suspect that when jQuery tries to hide the span, it doesn't exist in the DOM as the Angular directive hasn't generated it yet. Later, when we click on the checkbox, jQuery can find the span and hide it correctly.

This fix is not really pretty but messing with AngularJS from jQuery is not really simple :/