Innerjoin when a table contains user-defined objects

I have two tables that I wish to innerjoin() according to the topThick and botThick columns, which are the same in both tables. Why does this fail when Tright contains a column with objects of a user-defined class (here, myclass)?

load testdata
Tleft, Tright

Tleft = 16×4 table
    topThick    botThick    LogImageTop    LogImageBot
    ________    ________    ___________    ___________

       400         400         {[5]}          {[5]}   
       600         400         {[5]}          {[5]}   
       800         400         {[5]}          {[5]}   
      1000         400         {[5]}          {[5]}   
       400         600         {[5]}          {[5]}   
       600         600         {[5]}          {[5]}   
       800         600         {[5]}          {[5]}   
      1000         600         {[5]}          {[5]}   
       400         800         {[5]}          {[5]}   
       600         800         {[5]}          {[5]}   
       800         800         {[5]}          {[5]}   
      1000         800         {[5]}          {[5]}   
       400        1000         {[5]}          {[5]}   
       600        1000         {[5]}          {[5]}   
       800        1000         {[5]}          {[5]}   
      1000        1000         {[5]}          {[5]}   

Tright = 16×3 table
    topThick    botThick       Var1    
    ________    ________    ___________

       400         400      1×1 myclass
       600         400      1×1 myclass
       800         400      1×1 myclass
      1000         400      1×1 myclass
       400         600      1×1 myclass
       600         600      1×1 myclass
       800         600      1×1 myclass
      1000         600      1×1 myclass
       400         800      1×1 myclass
       600         800      1×1 myclass
       800         800      1×1 myclass
      1000         800      1×1 myclass
       400        1000      1×1 myclass
       600        1000      1×1 myclass
       800        1000      1×1 myclass
      1000        1000      1×1 myclass

innerjoin(Tleft,  Tright)

Error using tabular/innerjoin (line 34)
Not enough input arguments.

Answer 1

[Paul]

Hi Matt,

I think the class defintion has to change so that the myclass constructor can accept zero arguments.

I believe that will get past the error. You'll have to check if the innerjoin then yields the correct result.

dbtype myclass

1     classdef myclass
2     
3         properties
4                p=[];
5         end
6         methods
7             function obj=myclass(p)
8                  obj.p=p;
9             end
10        end
11    
12    end

try
    myclass()
catch ME
    ME.message
end

ans = 'Not enough input arguments.'

lines = readlines("myclass.m");
[(1:numel(lines)).',lines]

ans = 12×2 string array
    "1"     "classdef myclass"               
    "2"     ""                               
    "3"     "    properties"                 
    "4"     "           p=[];"               
    "5"     "    end"                        
    "6"     "    methods"                    
    "7"     "        function obj=myclass(p)"
    "8"     "             obj.p=p;"          
    "9"     "        end"                    
    "10"    "    end"                        
    "11"    ""                               
    "12"    "end"                            

lines(1) = "classdef newmyclass";
lines(7) = replace(lines(7),"myclass","newmyclass");
lines = [lines(1:7);"if nargin > 0";lines(8);"end";lines(9:end)];
writelines(lines,"newmyclass.m");
dbtype newmyclass.m

1     classdef newmyclass
2     
3         properties
4                p=[];
5         end
6         methods
7             function obj=newmyclass(p)
8     if nargin > 0
9                  obj.p=p;
10    end
11            end
12        end
13    
14    end

newmyclass()

ans = 
  newmyclass with properties:

    p: []

load testdata
Tright.Var1 = repmat(newmyclass(3),height(Tright),1)

Tright = 16×3 table
    topThick    botThick         Var1     
    ________    ________    ______________

       400         400      1×1 newmyclass
       600         400      1×1 newmyclass
       800         400      1×1 newmyclass
      1000         400      1×1 newmyclass
       400         600      1×1 newmyclass
       600         600      1×1 newmyclass
       800         600      1×1 newmyclass
      1000         600      1×1 newmyclass
       400         800      1×1 newmyclass
       600         800      1×1 newmyclass
       800         800      1×1 newmyclass
      1000         800      1×1 newmyclass
       400        1000      1×1 newmyclass
       600        1000      1×1 newmyclass
       800        1000      1×1 newmyclass
      1000        1000      1×1 newmyclass

innerjoin(Tleft,Tright)

ans = 16×5 table
    topThick    botThick    LogImageTop    LogImageBot         Var1     
    ________    ________    ___________    ___________    ______________

       400         400         {[5]}          {[5]}       1×1 newmyclass
       400         600         {[5]}          {[5]}       1×1 newmyclass
       400         800         {[5]}          {[5]}       1×1 newmyclass
       400        1000         {[5]}          {[5]}       1×1 newmyclass
       600         400         {[5]}          {[5]}       1×1 newmyclass
       600         600         {[5]}          {[5]}       1×1 newmyclass
       600         800         {[5]}          {[5]}       1×1 newmyclass
       600        1000         {[5]}          {[5]}       1×1 newmyclass
       800         400         {[5]}          {[5]}       1×1 newmyclass
       800         600         {[5]}          {[5]}       1×1 newmyclass
       800         800         {[5]}          {[5]}       1×1 newmyclass
       800        1000         {[5]}          {[5]}       1×1 newmyclass
      1000         400         {[5]}          {[5]}       1×1 newmyclass
      1000         600         {[5]}          {[5]}       1×1 newmyclass
      1000         800         {[5]}          {[5]}       1×1 newmyclass
      1000        1000         {[5]}          {[5]}       1×1 newmyclass

[Matt J]

Surely that's a bug, though. Innerjoin is not supposed to require the creation of new objects.

[Matt J]

Props, by the way for editing myclass.m in purely programmatic fashion. A lazy person would have just uploaded a new file ;)

[Paul]

It's not a bug in the sense that down in the bowels of innerjoin it is intentionally creating a vector of new myclass objects, and those objects are instantiated with "default" values. As to why the code operates that way and whether or not it could be implemented differently ... I have no idea. I didn't follow the code after that to see how those default objects get updated to their final values.

At No Input Argument Constructor Requirement the doc discusses two cases (are there others?) where a no-argument constructor is required and I think this situation in innerjoin falls under the second of those.

[Matt J]

I've asked Tech Support to verify if it was intentional. My feeling is that they inadvertently applied the same programming logic as used in outerjoin to innerjoin as well.

[Paul]

AFAICT, the underlying function with all of the hot sauce is tabular.joinInnerOuter, which handles both types of joins.

Loosely speaking ...

The first part of that code develops the indices for the elements of the column variables that need to be joined based on if the join is inner or outer.

Once the indices are obtained, then the second step creates a vector of default values for output, then stuffs into that vector for inner, or parts of that vector for outer, the portions of the input variables based on the step 1 indices. I see your point for an inner join that there won't ever be any default values in the output and so could be handled differently (and slightly more efficiently I suppose) in the second step.

At a minimum, perhaps the top level function tabular.innerjoin could do a better job of checking the inputs and verifying that each has nonkey variables with creatable default values and reporting an understandable error message if not.

All of the above based on R2024a.

[Matt J]

Tech support has classified this as a bug, to be fixed in a future release. The workaround until then is to implement a no-argument syntax for the constructor.

Answer 2

[Matt J]

Attached is a possible workaround.

load testdata

tableInner(Tleft,Tright)

ans = 16×5 table
    topThick    botThick    LogImageTop    LogImageBot       Var1    
    ________    ________    ___________    ___________    ___________

       400         400         {[5]}          {[5]}       1×1 myclass
       400         600         {[5]}          {[5]}       1×1 myclass
       400         800         {[5]}          {[5]}       1×1 myclass
       400        1000         {[5]}          {[5]}       1×1 myclass
       600         400         {[5]}          {[5]}       1×1 myclass
       600         600         {[5]}          {[5]}       1×1 myclass
       600         800         {[5]}          {[5]}       1×1 myclass
       600        1000         {[5]}          {[5]}       1×1 myclass
       800         400         {[5]}          {[5]}       1×1 myclass
       800         600         {[5]}          {[5]}       1×1 myclass
       800         800         {[5]}          {[5]}       1×1 myclass
       800        1000         {[5]}          {[5]}       1×1 myclass
      1000         400         {[5]}          {[5]}       1×1 myclass
      1000         600         {[5]}          {[5]}       1×1 myclass
      1000         800         {[5]}          {[5]}       1×1 myclass
      1000        1000         {[5]}          {[5]}       1×1 myclass

[Paul]

Why not modify the class definition to accept zero arguments in the constructor, if you don't mind me asking?

[Matt J]

It's just not always the desirable behavior in a class design. Sometimes you want an error thrown if someone tries to call a constructor with no arguments.

[Paul]

One possbility to avoid the need to create two temporary tables would be to use dbstack in the zero-argument call to the myclass constructor and take appropriate action if innerjoin is in the stack and error otherwise.

[Matt J]

"Appropriate action" means what? Return some sort of empty object?

And why would it be better to rely on innerjoin? Innerjoin creates wasted extra table data as well. It creates default class objects with the no-argument constructor which never get used.

[Paul]

Yes, return an empty object, whatever that might mean for the object at hand. Maybe an object with all properties empty of the appropriate type.

The objects created with the no-arg constructor are used; they ulitmately become a variable in the table returned from innerjoin.

It's like a pre-allocation thing. Create an array of empty objects

out = repmat(myclass(),N,1) % not the exact code, just showing the idea

Then fill in the output with selected values of the input

out(indexout) = in(indexin)  % in is a vector of myclass pulled from the input table

indexout is logical and indexin is numerical. out is then returned as variable in the table returned from innerjoin.

For an inner join, I think that indexout is always

indexout = true(N,1)

in which case we could simply have

out = in(indexin)

which would be more efficent as I said previously.

But, for an outer join indexout could (most likely would) have some false entries and the outer joined table would typically include "default" values in the variables, e.g., NaN for double and, I presume, myClass() for myClass.

As I said previously, at the point in the code of the indexing there is no distinction between outer and inner joins, which isn't to say that there couldn't be.

Also, key variables can be returned with default values for outer joins. Not sure if outer joins or myClass key variables are of interest.

[Matt J]

It's a valid workaround, but I find it questionable to force the user to do gymnastics in their class constructor design just to accomodate a requirement of innerjoin that it never really uses. Also, it is not clear that it would improve performance:

rng('default');

N=3e3;

Z=table(repmat(myclass,N,1));

T1=array2table(randi(10,N,9)); T1(:,end+1)=Z;

T2=array2table(randi(10,N,10));

timeit(@()innerjoin(T1,T2,Keys=["Var1","Var9"]))

ans = 0.3958

timeit(@()tableInner(T1,T2,["Var1","Var9"]))

ans = 0.1135

[Paul]

"... I find it questionable ..."

I agree.

Would be curious to know what response you get from Tech Support.