I think I found a solution.
I copied .matlab7rc.sh from MATLABROOT to my home directory, and edited to add the directory I need in LD_LIBRARY_PATH to the LDPATH_SUFFIX variable.
The copy in the home dir seems to override the one in MATLABROOT and this seems to effect the workers as well as a normal Matlab session.