First off, beauty in in the eye of the beholder!
Personally, I don't think beautiful and matlab GUI ever go in the same sentence! :p
Sadly, neither does the word elegant (except for simple toy UIs). The problem is Matlab has no clear way to cleanly arrange and manage UI widgets, so you end up with very clumsy grid alignment or hacking Java display managers (except for the simplest of UIs). And there are no ways to layer complexity in a UI, i.e. tabs, sliding panes, collapsible panels in a robust way that isn't yet more hacks. Add to that Matlab is incapable of properly smoothing text robustly in a UI, using unicode characters, using alpha transparency (critical for "pretty" graphics) in icons or work cross-platform without position changes and you end up with 90s style pixel vomit UIs. On OS X the Java swing set they use has widgets that don't even fit in the position layouts (which they claim they are trying to improve).
The important point is they work (though GUIDE often doesn't)...
Yair Altman has a wonderful blog, which really is the best example of where you can push Matlab UIs (accepting the fact much of this is stuff TMW has semi-documented, yet never supported!): http://undocumentedmatlab.com/
