How do I remove column x from all variables in my workspace?

I do nor understand how to loop through variale in a workspace and perform the same thing to each of them.

I a sure it is simple...

[dpb]

Doable but not terribly simple and leads to the question of why, if one wants to do the same operation on different variables were they not stored in an array or table or other data structure that allows operations by column or variable?

The better way it would seem would be to fix the original creation of these variables to not use totally independent variable names.

Example code that created these and ultimate intent would add much-needed context...

[James Tursa]

@dpb: +1

[John D'Errico]

@dpb + 1 also.

I would suggest you are possibly thinking as if you are using a spreadsheet. It is far more powerful than just a spreadsheet. But if you treat it as such, then you will be limited to what you could have done in a spreadsheet, and worse, you will spend a great deal of time trying to figure out how to shoehorn the language into what you understand about spreadsheets. For example, you may need to deal with numbered variables, as too many new users want to create.

The trick is to learn to program in MATLAB. You need to learn to use vectors, arrays, matrices, cell arrays, etc. Do this because these various forms of arrays can combine many variables into one place. And then they make it easy to do things like operate on all these "things" at once. At the very least, it becomes easy to write a loop to process everything. Allow your computer to do what it does well.

[dpb]

@JD'E +2 for the added context from this side of the fence...

[Nina Schuback]

Ok, point taken, sorry.

So let's say I convert all my variables into a structure, and try to avoid thinking like a spreadsheet.

A.a=[1,1,1,1;2,2,2,2;3,3,3,3];
A.b=[1,1,1,1;2,2,2,2;3,3,3,3];
A.c=[1,1,1,1;2,2,2,2;3,3,3,3];
A.d=[1,1,NaN,1;2,2,2;3,3,3,3];

For this example I would like to delete column 3 in each field, based on the fact that there is a NaN in column 3 in A.d

[Stephen23]

Even better data design would use one simple numeric array, e.g.:

A = cat(3,...
[1,1,1,1;2,2,2,2;3,3,3,3],...
[1,1,1,1;2,2,2,2;3,3,3,3],...
[1,1,1,1;2,2,2,2;3,3,3,3],...
[1,1,NaN,1;2,2,2,2;3,3,3,3]);

Then your code can be much simpler than if you used a structure:

X = any(any(isnan(A),3),1);
A(:,X,:) = [];

[Nina Schuback]

True.

[John D'Errico]

Nope. Still thinking spreadsheet. :) Instead there you have one variable, with many named fields. It becomes almost the same thing. If you want to use a struct, you might do this:

clear A
A(1).data = [1,1,1,1;2,2,2,2;3,3,3,3];
A(2).data = [1,1,1,1;2,2,2,2;3,3,3,3];
A(3).data = [1,1,1,1;2,2,2,2;3,3,3,3];
A(4).data = [1,1,NaN,1;2,2,2,2;3,3,3,3];

A
A = 
  1×4 struct array with fields:
    data

(Note that as you wrote it, A.d had only 3 numbers in the second row. I fixed that.)

So you might choose to have one field of now a struct array. So each struct in this array has a field called data. I could loop over the structs in this array. You can always keep the list of names somewhere for future use. A cell array might be useful, at least if you will have more interesting names than what I've chosen.

 datanames = {'a','b','c','d'};

Anyway, I can loop over the elements of the struct.

for i = 1:numel(A)
  find(any(isnan(A(i).data),1))
end

ans =
  1×0 empty double row vector
ans =
  1×0 empty double row vector
ans =
  1×0 empty double row vector
ans =
     3

Which would tell me that column 3 of one of those arrays held a NaN in it. So at least I was able to easily loop over the multiple variables. (And I could have been better about how I wrote that loop too.) But I'm still not really using MATLAB well as an array manipulation tool. What I want to do is to treat this as a regular array if possible.

I can extract all of those data arrays like this:

A.data
ans =
     1     1     1     1
     2     2     2     2
     3     3     3     3
ans =
     1     1     1     1
     2     2     2     2
     3     3     3     3
ans =
     1     1     1     1
     2     2     2     2
     3     3     3     3
ans =
     1     1   NaN     1
     2     2     2     2
     3     3     3     3

That would be what is called a comma separated list. Still not very usable. Better is to recognize that these variables are all 3x4 arrays. So I might just catenate them all into one 3-d array. I could do that like this:

A3d = cat(3,A.data)
A3d(:,:,1) =
     1     1     1     1
     2     2     2     2
     3     3     3     3
A3d(:,:,2) =
     1     1     1     1
     2     2     2     2
     3     3     3     3
A3d(:,:,3) =
     1     1     1     1
     2     2     2     2
     3     3     3     3
A3d(:,:,4) =
     1     1   NaN     1
     2     2     2     2
     3     3     3     3

Or, when I created my struct, I might just have immediately recognized that in fact, this is just a 3-d array. So create it as such from the very beginning. Perhaps I might have done this instead:

clear A
A(:,:,1) = [1,1,1,1;2,2,2,2;3,3,3,3];
A(:,:,2) = [1,1,1,1;2,2,2,2;3,3,3,3];
A(:,:,3) = [1,1,1,1;2,2,2,2;3,3,3,3];
A(:,:,4) = [1,1,NaN,1;2,2,2,2;3,3,3,3];

After all, there really was no need to create A as a struct in the first place.

So A is one 3-dimensional array. It has 4 pages, just like A3d.

A
A(:,:,1) =
     1     1     1     1
     2     2     2     2
     3     3     3     3
A(:,:,2) =
     1     1     1     1
     2     2     2     2
     3     3     3     3
A(:,:,3) =
     1     1     1     1
     2     2     2     2
     3     3     3     3
A(:,:,4) =
     1     1   NaN     1
     2     2     2     2
     3     3     3     3

What I'm trying to do is to lead you into the idea of keeping your data in as simple and regular a form as possible. Here the simplest form is just a 3-d array. Now, we can operate on the multiple sub-arrays far more easily. For example, are there any NaNs in there?

any(isnan(A),1)
  1×4×4 logical array
ans(:,:,1) =
   0   0   0   0
ans(:,:,2) =
   0   0   0   0
ans(:,:,3) =
   0   0   0   0
ans(:,:,4) =
   0   0   1   0

I can see there is at least one column that contains a NaN. That NaN falls in the 3rd column, on the 4th page of our array. We can find that information easily enough as:

[colind,pagind] = find(squeeze(any(isnan(A),1)))
colind =
     3
pagind =
     4

The idea is to store your data in a form that is now capable of being manipulated as one array. We can now easily enough just delete the third column of all of our data. I'll be nice, and keep all of A around, creating a new variable called B.

B = A;
B(:,colind,:) = []
B(:,:,1) =
     1     1     1
     2     2     2
     3     3     3
B(:,:,2) =
     1     1     1
     2     2     2
     3     3     3
B(:,:,3) =
     1     1     1
     2     2     2
     3     3     3
B(:,:,4) =
     1     1     1
     2     2     2
     3     3     3

No more NaNs in any column.

The point of all this is to work with arrays. Use MATLAB as the matrix and array manipulation engine it is designed to be. Once you start to do that, everything gets far simpler. Operations that previously would have taken far more work are now little more than single lines of code. And, yes, my response was lengthy, but I was trying to lead you into the way I would prefer you think, a way that will make your life simpler in the end.

[Nina Schuback]

Thanks! That actually is super useful!

Answer 1

[Rik]

Option n_by_Walter+1:

A.a=[1,1,1,1;2,2,2,2;3,3,3,3];
A.b=[1,1,1,1;2,2,2,2;3,3,3,3];
A.c=[1,1,1,1;2,2,2,2;3,3,3,3];
A.d=[1,1,NaN,1;2,2,2,2;3,3,3,3];
data=struct2cell(A);
data=cat(3,data{:});

L=any(isnan(data),3);
col_with_NaN=any(L,1);
data(:,col_with_NaN,:)=[];

Storing into a 3D array gives you a lot of flexibility without a lot of call to complex (and sometimes slow) functions like structfun.

Answer 2

[Walter Roberson]

Option 1

out = structfun(@(F) F(:,~any(isnan(F),1)), A, 'uniform', 0);

This would be to remove columns that have at least one nan, considering the each field by itself; this could lead to fields with different number of columns. The output is a new struct with the same field names.

Option 2

col_has_no_nan = ~any(cell2mat( structfun(@(F) {any(isnan(F),2)}, A) ), 1);
out = structfun(@(F) F(:,col_has_no_nan), A, 'uniform', 0);

This checks all columns of the fields to find columns that have at least one nan in any of the fields, and then eliminates that column from all of the fields.