I was testing in R2014a, which is important because graphics changed in R2014b in ways that matter to this situation.
In R2014a (and earlier) when you select save as for PNG, the renderer gets switched to OpenGL from Painters. There is then a conflict in the layering between the rectangle with 'facecolor', 'w', and the contour plot: both of them are in the plane z = 0, so the renderer has to decide which has priority. In Painters the rule is "whichever one was drawn last", but in OpenGL the rules are more complicated. The OpenGL rules tend to prioritize lines over surfaces, which is why the lines show up. But with coplanar surfaces, I do not recall what the rule is.
For R2014a or earlier, you are going to have to push the rectangle into negative Z or pull the contour plot into positive Z. You can use a hgtransform to parent the object, with makehgtform with a Z translation as the Matrix. Or you could replace the rectangle with a patch that had a negative Z. (The purpose of the rectangle() being essentially to provide the white background and the circular edge drawing.)
For R2014b or later, you have another option: you can also use the new SortMethod property, which is discussed in Mike's blog . Watch out for the limitation discussed at the bottom about "some graphics cards" and about PDF.
The more robust method is to ensure that the contours are "above" the rectangle, even if only a tiny bit.