Overriding subsref and subsasgn - effect on private properties

I'm overriding subsref and subsasgn for a class. I want to influence the behaviour of obj(...), but I can't find a good way to do this without also either breaking obj.name property access, or breaking the privacy attributes for the properties.

Examples in the documentation (see "A Class with Modified Indexing") suggest that I check the type of indexing within my subsref/subsasgn functions and call the builtin subsref/subsasgn if the type is '.'. The problem is that because these are called from a class method, the access protection on properties is overriden - so I can access and update private properties from outside the class, as if the protection was not there.

Here's an example class

classdef test
    properties (Access = private)
        x = 'ought to be private' 
    end

      methods
          function v = subsref(M, S)
              switch S.type
                  case '()'
                      v = 'this is ok';
                  case '{}'
                      error('{} indexing not supported');
                  case '.'
                      v = builtin('subsref', M, S);  % as per documentation
              end
          end
      end
  end

and here's what goes wrong when I use it:

 >> t = test
t = 
 test with no properties.
 Methods
>> t(1)
ans =
this is ok
>> t.x
ans =
ought to be private

The attempt to access t.x should not succeed.

One solution I can think of is to write set.x and get.x methods for every single private property, to reimplement the protection that the Access attribute ought to provide.

[EDIT - added 16 hours after original post] Another possible solution: write code to analyse the subscript argument, consulting meta.property, before calling the builtin subsref/subsasgn. Not that hard, but it's ugly and probably inefficient to reimplement existing functionality.

Does anyone know a better way?

Answer 1

[James Lebak]

Beginning in MATLAB R2021b it is possible to overload parentheses, dot, and brace indexing independently. This gives the ability to overload paren-indexing while preserving the default behavior and performance of dot-indexing for the class.

See the discussion of modular indexing here to get started.

[David Young]

@James Lebak: I'm delighted to hear that this has happened. It means that I'll start using subsref and subsasgn again - I've been avoiding doing so for 10 years!

Answer 2

[David Young]

I have, rather belatedly, found an official answer to my question, here.

Unfortunately, the recommendation is to overload '.' indexing as well as '()' indexing and write customized checks for the property attributes. My own view is that this is very unsatisfactory: messy and inefficient.

It's a shame. I can't see why there aren't simply three methods, say subsref_round, subsref_curly and subsref_dot, which could be overridden independently. It would avoid all that structure building and switching on the type too, and no downside that I can see.

[Nikolaus Koopmann]

i wish subsref would just be called recursively one level at a time...

[James Lebak]

I know this discussion is more than 10 years old, but in case someone comes across this comment, I want to point out that as of R2021b it is possible to overload the three types of indexing independently as David wants.

See the discussion of modular indexing here to get started.

[Steven Lord]

@James Lebak In order to increase the visiblity of this comment, please turn it into its own answer.

[David Young]

@James Lebak: I'm delighted to hear that this has happened. It means that I'll start using subsref and subsasgn again - I've been avoiding doing so for 10 years!

Answer 3

[Daniel Shub]

Apologies for the long answer (that might not be helpful or even an answer). The answer is so long since I am not sure what I am doing is anywhere near optimal and would love some feedback or to see what others are doing. I find the whole subs* access permissions to be extremely difficult and poorly documented by TMW. I don't know enough about other languages and OOP to know if is MATLAB specific or not. My solution is based upon using a metaclass object to determine the permissions (similar to what you hint at in your edit). The key to this solution is I limit my overloaded subs* methods to accessing only public properties and methods. I am comfortable with this since methods call the builtin subs* methods by default and you need to specifically code a call to the overloaded subs* methods. If you want your methods to be able to use the overloaded subs* methods to access private/protected properties and methods, you have two options. The first way is to extend the subs* methods to determine the access permissions of the caller. You can probably do this with dbstack and metaclass to figure out if the calling function has access to private and/or protected properties and methods. The second way is to write privatesubs* and protectedsubs* methods that have access permissions of private and protected, respectively. If a method has permission to access the protectedsubs* methods, then it also should have permission to access all protected properties and methods. Similarly, if a method has permission to access the privatesubs* methods, then it also should have permission to access all private and protected properties and methods. The first solution is easier for developers of subclasses since they only have to concern themselves with the subs* methods. I find the second easier to implement since I do not have to worry about determining the access permissions of the caller. Below is some slightly edited code for my actual implementation of the subsref method.

I start with defining a subsref method for my TopMost class. The TopMost class is not a child class of any other classes, but it is handle compatible (although I don't think that matters).

function varargout = subsref(Obj, S)
  % Overloaded subsref
  %
  % varargout = subsref(Obj, S)
  %
  % This function overloads the builtin subsref for the TopMost class. It only allows access to
  % public properties and methods. Access to private and protected methods is
  % denied even if subsref is called from another method of the class. If you
  % need to access a private or protected method via a subsref type call, you
  % must implement your own method.

  % Validate the number of arguments.
  nRequiredArguments = 2;
  nOptionalArguments = 0;
  error(nargchk(nRequiredArguments, nRequiredArguments+nOptionalArguments, nargin, 'struct'));

  % Overload the subsref method.
  subsrefcheck(Obj, S);
  [varargout{1:nargout}] = builtin('subsref', Obj, S);
end

The only thing this method does is check if the substruct object is "valid" (see the subsrefcheck function further below). The method hands everything off to the builtin subsref. The reason for this method is if you have the class hierarchy SubClass < ParentClass < TopMost, and SubClass overloads the subsref method, then I want the SubClass subsref method to use the ParentClass subsref method, and not the builtin subsref method, as the default. The problem is that MATLAB throws an error if the ParentClass does not have a subsref method (even though the ParentClass could use the builtin subsref method). By adding a subsref method to TopMost, I assure myself that ParentClass has a subsref method.

function varargout = subsref(Obj, S)
  % Overloaded subsref
  %
  % varargout = subsref(Obj, S)
  %
  % This function overloads the ParentClass subsref method (which might be defined by the TopMost class). It only allows
  % access to public properties and methods. Access to private and protected
  % methods is denied even if subsref is called from another method of the
  % class. If you need to access a private or protected method via a subsref
  % type call, you must implement your own method.

  % Validate the number of arguments.
  nRequiredArguments = 2;
  nOptionalArguments = 0;
  error(nargchk(nRequiredArguments, nRequiredArguments+nOptionalArguments, nargin, 'struct'));

  % Validate the arguments.
  subsrefcheck(Obj, S);

  % Overload the subsref method.
  if strcmp(S(1).type, '.') && strcmp(S(1).subs, 'MyProp')
    Value = Obj.MyPropSubsRefGet(S);
    varargout = {Value};
  else
    [varargout{1:nargout}] = subsref@ParentClass(Obj, S); % This might actually jump all the way to subsref@Topmost(Obj, S);
  end
end

Here the overloaded subsref method calls a special "get" method (MyPropSubsRefGet) for the property MyProp and passes all the other cases on to ParentClass.

Below is my subsrefcheck and the functions it depends on. I use these function in many of my classes, so I do not make them a method of my TopMost class, although I could.

function subsrefcheck(Obj, S)
  % subsrefcheck checks if the substruct is valid for subsref
  %
  % subsrefcheck(Obj, S)
  %
  % Checks if the substruct S is valid for use with subsref on the object Obj.
  % The substruct is not valid for the Obj if the substruct is not valid (see
  % validatesubstruct). Further, if S accesses a property, the substruct is
  % not valid if the get access of the property is not public. Finally, if S
  % accesses a method, the substruct is not valid if the access of the method
  % is not public. The function returns nothing if S is valid and throws the
  % approariate error if S is not valid.

  % Validate the number of arguments.
  nRequiredArguments = 2;
  nOptionalArguments = 0;
  error(nargchk(nRequiredArguments, nRequiredArguments+nOptionalArguments, nargin, 'struct'));

  % Validate the arguments.
  validatesubstruct(S);

  % Parse the substruct
  if length(S) >= 1 && strcmp(S(1).type, '.')
    SubsNameString = S(1).subs;
    SubObj = Obj;
  elseif length(S) >= 2 && strcmp(S(1).type, '()') && strcmp(S(2).type, '.')
    SubsNameString = S(2).subs;
    SubObj = Obj(S(1).subs{:});
  else
    return;
  end

  % Check if the property/method is public.
  switch lower(gettype(Obj, SubsNameString))
    case 'field'
    case 'property'
      [GetAccessString, SetAccessString] = getpropertyaccess(SubObj, SubsNameString); %#ok<NASGU>
      if ~strcmp(GetAccessString, 'public')
        throwAsCaller(MException('MATLAB:class:GetProhibited', ...
          ['Getting the ''', SubsNameString, ''' property of the ''', class(Obj), ''' class is not allowed.']));
      end
    case 'method'
      AccessString = getmethodaccess(SubObj, SubsNameString);
      if ~strcmp(AccessString, 'public')
        throwAsCaller(MException( ...
          'MATLAB:class:MethodRestricted', ...
          ['Cannot access method ''', SubsNameString, ''' in class ''', class(Obj), ''.']));
      end
    otherwise
      throwAsCaller(MException('MATLAB:noSuchMethodOrField', ...
        [ 'No appropriate method, property, or field ', SubsNameString, ' for class ', class(Obj), '.']));
  end
end

function validatesubstruct(S)
  % Checks if S is a valid substruct argument
  %
  % validatesubstruct(S)
  %
  % Returns nothing if S is a valid substruct (i.e., could have been generated
  % by the substruct function) and throws the appropriate error if S is not a
  % valid substruct.

  % Validate the number of arguments.
  nRequiredArguments = 1;
  nOptionalArguments = 0;
  error(nargchk(nRequiredArguments, nRequiredArguments+nOptionalArguments, nargin, 'struct'));

  % Validate the arguments.
  assert(isstruct(S), 'MATLAB:subsArgNotStruc', ...
    'Subscript argument to SUBSREF and SUBSASGN must be a structure.');
  assert(length(fieldnames(S)) == 2, 'MATLAB:subsMustHaveTwo', ...
    'Subscript argument to SUBSREF and SUBSASGN must have two fields.');
  assert(isequal(sort(fieldnames(S)), sort({'subs'; 'type'})), ...
    'MATLAB:subsMustHaveTypeSubs', ['Subscript argument to SUBSREF ', ...
    'and SUBSASGN must have two fields whose names are "type" ', ...
    'and "subs".']);
  assert(~isempty(S), 'MATLAB:subsArgEmpty', ...
    'Subscript argument to SUBSREF and SUBSASGN must not be empty.');
  assert(all(cellfun(@(x)(ischar(x) || iscell(x)), {S.subs})), ...
    'MATLAB:subsSubsMustBeCellOrChar', ...
    ['The "subs" field for the subscript argument to SUBSREF ', ...
    'and SUBSASGN must be a cell or character array.']);
  assert(all(cellfun(@(x)ischar(x), {S.type})), ...
    'MATLAB:subsTypeMustBeChar', ...
    ['The "type" field for the subscript argument to SUBSREF ', ...
    'and SUBSASGN ', '\n', 'must be a character array.']);
  assert(all(cellfun(@(x)any(strcmp(x, {'.'; '()'; '{}'})), {S.type})), ...
    'MATLAB:subsTypeMustBeSquigglyOrSmooth', ...
    ['The "type" field for the subscript argument to SUBSREF ', ...
    'and SUBSASGN ', '\n', 'must be a character array ', ...
    'of "." or "{}" or "()".']);
  assert(all(cellfun(@(x, y)(~strcmp(x, '.') || ~iscell(y) ...
    || ~isempty(y)), {S.type}, {S.subs})), ...
    'MATLAB:subsCellIsEmpty', ...
    ['The "subs" field for the subscript argument to SUBSREF ', ...
    'and SUBSASGN must be a non-empty cell or character array.']);

  for iSub = 1:length(S)
    assert(~strcmp(S(iSub).type, '()') || iscell(S(iSub).subs), ...
      'MATLAB:subsSmoothTypeSubsMustBeCell', ...
      'SUBS field must be a cell array for () TYPE.');
    assert(~strcmp(S(iSub).type, '{}') || iscell(S(iSub).subs), ...
      'MATLAB:subsSquigglyTypeSubsMustBeCell', ...
      'SUBS field must be a cell array for {} TYPE.');
    assert(~strcmp(S(iSub).type, '()') || (iSub == length(S) || ...
      strcmp(S(iSub+1).type, '.')) , ...
      'MATLAB:subsDotMustFollow', ...
      'Only a dot field name can follow ()''s.');
  end
end

function AccessString = getmethodaccess(Obj, MethodNameString)
  % Gets the Access attribute of the method
  %
  % AccessString = getmethodaccess(Obj, MethodNameString)
  %
  % Uses the metaclass of the object Obj to determine the method
  % MethodNameString access permission (private, protected, public) and
  % returns the access permission as AccessString.

  % Validate the number of arguments.
  nRequiredArguments = 2;
  nOptionalArguments = 0;
  error(nargchk(nRequiredArguments, nRequiredArguments+nOptionalArguments, nargin, 'struct'));

  % Validate the arguments.
  assert(isobject(Obj), [mfilename, ':ArgumentCheck'], ...
    ['The Obj of class ''', class(Obj), ''' is not an object of a MATLAB class.']);
  validateattributes(MethodNameString, {'char'}, {'row'}, [mfilename, ':ArgumentCheck'], 'MethodNameString');

  MetaClassObj = metaclass(Obj);
  NameList = {MetaClassObj.MethodList(:).Name};
  iMethod = find(strcmp(MethodNameString, NameList), 1, 'first');
  assert(~isempty(iMethod), [mfilename, ':ArgumentCheck'], ...
    ['''', MethodNameString, ''' is not a method of the ''', class(Obj), ''' class.']);
  AccessString = MetaClassObj.MethodList(iMethod).Access;
end

function [GetAccessString, SetAccessString] = getpropertyaccess(Obj, PropertyNameString)
  % Gets the GetAccess and SetAccess attributes of the property
  %
  % [GetAccessString, SetAccessString] = getpropertyaccess(Obj,
  % PropertyNameString)
  %
  % Uses the metaclass of the object Obj to determine the property
  % PropertyNameString get and set access permissions (private, protected,
  % public) and returns the access permissions as GetAccessString,
  % SetAccessString.

  % Validate the number of arguments.
  nRequiredArguments = 2;
  nOptionalArguments = 0;
  error(nargchk(nRequiredArguments, nRequiredArguments+nOptionalArguments, nargin, 'struct'));

  % Validate the arguments.
  assert(isobject(Obj), [mfilename, ':ArgumentCheck'], ...
    ['The Obj of class ''', class(Obj), ''' is not an object of a MATLAB class.']);
  validateattributes(PropertyNameString, {'char'}, {'row'}, [mfilename, ':ArgumentCheck'], 'PropertyNameString');

  MetaClassObj = metaclass(Obj);
  NameList = {MetaClassObj.PropertyList(:).Name};
  iProperty = find(strcmp(PropertyNameString, NameList), 1, 'first');
  assert(~isempty(iProperty), [mfilename, ':ArgumentCheck'], ...
    ['''', PropertyNameString, ''' is not a property of the ''', class(Obj), ''' class.']);
  GetAccessString = MetaClassObj.PropertyList(iProperty).GetAccess;
  SetAccessString = MetaClassObj.PropertyList(iProperty).SetAccess;
end

I created the validate substruct function by trial and error. Basically I tried the builtin subsref function with every possible combination of arguments I could think of and recorded the errors. I really wish the substruct was a class. I might be being to restrictive on my substruct and reducing the power of subsref, but it is not causing me any problems. Also note that in newer versions of MATLAB the getmethodaccess and getpropertyaccess might be able to use isprop and ismethod. I am not sure what the performance implications of that change would be.

[Walter Roberson]

It looks like a lot of work and thought went in to your response, Daniel!

[David Young]

Daniel, thank you. Your answer does help with what I'm trying to do, and indeed goes beyond it - there's lots of interesting techniques there and I've learnt from it. However, it does bother me that you've had to write so much code, and do so much analysis, in order to achieve something that I think should be simple. I suspect your worries about the documentation of the subs* mechanisms extend also to their design, and you're working around some significant flaws. I see no-one else has replied yet - it would have been great to get some response from TMW here, and I'm inclined to try taking this forward through their support desk.

[Daniel Shub]

I am glad it was helpful. I think this is one case where the closed source nature of MATLAB is problematic. I would love to see how they implemented subsref. Also, there are so few classes in MATLAB that are really built in the MATLAB OO structure that there is basically only the pathetically simple examples to go off of.

I sometimes wonder how many people use the MATLAB OO framework.

Answer 4

[Malcolm Lidierth]

David

Publishing private properties from public methods is not unique to MATLAB. It can be source of problems in other OOP languages too - Java for one. You seem to be asking for the MATLAB-supplied generic public subsref to recognise a "superprivate" property and refuse access to it. As you state, you implement that yourself by customising subsref. A single switch block dealing with '.' would do:

switch propname
case {...private properties list...}
throw(....)
otherwise
...
end

On a related issue: Should a single subsref deal with '.', '()' and '{}'. Yes is my vote. The conditional code has to be executed somewhere - whether by MATLAB or in the user-supplied subsref. If the user has control they can control the sequence e.g. deal with '()' first in a switch block if that is the most common/speed-critical case. Likewise, in the switch block above, deal with the public properties first if they are accessed more often - which seems likely.

[Daniel Shub]

It is not that simple though.

First there is the issue of where the switch block goes. You ideally want your code to work with both obj(1).prop and obj.prop.

Then there is the issue of maintaining the list of private and protected properties. Ideally you shouldn't have to edit the subsref method every time you add a new property. This means you have to kludge together a function based on the metaclass object to determine the private properties and methods.

Finally, what happens if you want to allow the class methods to call the overloaded subsref to access a private method/property. You need to determine if the calling function has access to the method/property.

It seems to me if they modularized the code so that you could overload subsref, or subsref_dot, or subsref_dot_prop, that this would make it easier in some cases, while still providing all the current power. This assumes something about the structure of the built in subsref.

[Malcolm Lidierth]

@Daniel

subsref was there long before the current OOP system when life was much tougher and all methods had to be written from scratch so I doubt TMW will be in a hurry to change it.

obj(1).prop vs obj.prop can be problematic.You can of course make recursive calls to subsref on obj(...).prop so most of the code then needs to be written to deal only with a scalar object. More usually, I just surrender, and have a class that is always scalar - using cell arrays rather than object arrays as needed. A useful consequence of that is that the '()' syntax can be subverted completely so e.g. x(1:10) can return data from the first 10 elements of a property within x.

[David Young]

Daniel, thanks for the comments.

However, I'm not proposing a "superprivate" property, and I do understand why public methods can return the values of private properties (mine do it all the time!) - I don't see that as a shortcoming of MATLAB or Java.

Rather, the problem is around the design of subsref (as Daniel notes).

On your proposal for a switch block - I'm still not convinced. If the "{... private properties list ...}" has to be coded explicitly, then it's a serious maintenance problem, because you have to update it whenever you add a private or protected property to the class. If, on the other hand, it's generated dynamically, then that's fiddly additional code, and a likely performance hit.

On the switch on indexing type having to be done somewhere - no, I don't think so. If there were different methods for the different indexing types, the switch would, in effect, be done at compile time, and in addition there would be no need to build and access a structure containing the indexing type. Sleeker and cleaner, I think.

Still, you are right to point out that subsref has been around a long time, and so will stay as it is. Complaints about design, whether justified or not, are usually trumped by the need for backwards compatibility, and so it is here.

[Daniel Shub]

@Malcolm, to me the power of MATLAB is its ability to handle the vector case. Often this comes at more of a cost then I would like. As for subsref being around since the old OOP system, that doesn't seem to me a reason TMW couldn't have changed it for the new OOP (they did change just about everything else).

What I would really like is some good documentation and examples for subsref. Part of me thinks the lack of documentation is because the system is too complicated to document/use.

[David Young]

Correction to my last comment: I meant to thank both Daniel and Malcolm for their comments. I'm grateful for both sets of thoughts. The rest of the comment was largely a response to Malcolm, in fact.

[Malcolm Lidierth]

@Daniel and David

For the switch block, I see this could get cumbersome with many properties. I guess it could be automated if a naming convention were used e.g. if all private properties begin with “p_”. Then

switch propname(1:2)

case ‘p_”

otherwise

end

@Daniel, I guess there is a problem if you are dealing with lengthy vectors of objects. I generally have few objects and want only to vectorise operations on object contents rather than object arrays. More generally, something that initially surprised me is that sub2ind and ind2sub are m-files, not builtins. I have never found a situation from profiling which suggested it would be worth mex-ing these. Maybe that is why TMW provide m-files: would the overhead of calling a mex outweigh the benefits from a typical call if that is e.g. obj(1:2)?

Oddly, there are no MATLAB functions to convert a substruct cell array to subs or ind (a few days ago, I posted some code here that does that on an old question of Daniel’s).

@Daniel, on documentation, I suspect that TMW might not want to give away too much to those developing open-source clones?

@David, I am sure you are right about compiled code being faster. But what overhead would be added in generating p-code from m-code, then interpreting the p-code? I wonder what effect would that have on the speed of a ‘typical” m-file?

You can always find contexts where TMWs approach is less than optimal, but that does not mean that it isn’t optimal for a typical customer.

[David Young]

@Malcolm, Good points, but now I feel we've reached the stage where it's difficult to go further without a TMW developer taking part in the discussion. One of the most interesting questions would be what a typical customer is!

[Malcolm Lidierth]

@David

Or a Python developer perhaps:

"We are all individuals".

"I'm not"

Answer 5

[Malcolm Lidierth]

David

Would it help to have your test class extend a superclass that has the private properties in it?:

classdef testprivate

    properties (Access = private)
        x = 'ought to be private'
    end

    methods(Access=protected)
        function x=getX(obj)
            x=obj.x;
            return
        end
    end
end

classdef test < testprivate 

        methods(Access=protected)

        function x=getX(obj)
            x=getX@testprivate(obj);
            return
        end

        end

        methods(Access=public)

            function x=BreakTheRules(obj)
                x=obj.getX();
                return
            end

        end
end

Then

>> obj=test;
>> obj.x
Getting the 'x' property of the 'testprivate' class is not allowed.
>> obj.getX()
Error using test/getX
Cannot access method 'getX' in class 'test'.

but,

>> obj.BreakTheRules()
ans =
ought to be private

[David Young]

Malcolm, thank you for thinking about this. I think putting the properties in a superclass helps to some extent, but it's still problematic.

It seems to me that your approach gives the correct behaviour for public and private properties (but not protected) for accesses from outside the test class. However, accesses from inside the test class have to go through the BreakTheRules, so it's still quite fiddly, and potentially inefficient.

I will think about whether to incorporate it, or whether I should settle for just making all my properties private, and disabling dot-indexed access.