Vectorization help, separating an array into multiple arrays

Hi. I'm having some trouble with vectorization.

Basically I have an excel sheet with data from a weather model. The model outputs data for separate "weather groups" (locations). So first I made some code that finds where each new weather group starts.

if true
  num=xlsread('SampleWxData.xlsx');
CurrentWxGroupStart = 1;
AllWxGroupsStart = 1;
while (find(num(:,1)>num(CurrentWxGroupStart,1),1))
NextWxGroupStart = find(num(:,1)>num(CurrentWxGroupStart,1),1);
AllWxGroupsStart = horzcat(AllWxGroupsStart, NextWxGroupStart);
CurrentWxGroupStart = NextWxGroupStart;
end
NumberOfWxGroups = size(AllWxGroupsStart,2);

end

I then read the excel sheet again to get the raw data

if true
[rad,time,all]=xlsread('SampleWxData.xlsx'); 
end

This part all works. The format of the excel sheet is 3 columns: WxGroup, time, radiation

Now, I want to create a three dimensional array, concatenating each weather groups data. Basically I have a long list of all the data now, and I want to separate it out by the weather group.

I know how to do this with a for loop, but I want to vectorize it.

I know the following can get me the first Weather Group

if true
  AllWxGroups = (all(AllWxGroupsStart(1,1):AllWxGroupsStart(1,2)-1,:));
end

So I first added the ending point to the variable AllWxGroupsStart and tried

if true
  x=1:size(AllWxGroupsStart,2)-1;
AllWxGroups = all(AllWxGroupsStart(1,x):AllWxGroupsStart(1,x+1)-1,:);
end

But that just returned the first Weather Group again. Not sure why not...I thought putting x there would just do the same thing as before except cycle through all values of x, but I guess not.

Let me know if you need any more clarification.

[Kyle Chudler]

Clarification...here is the basic for loop I want to vectorize

if true
for x = 2:NumberOfWxGroups
  AllWxGroups = cat(3, AllWxGroups, all(AllWxGroupsStart(1,x):AllWxGroupsStart(1,x+1)-1,:));
end
end

Answer 1

[Sean de Wolski]

Could you provide some small examples of the input variables - my guess is that vectorizing, although possible, will likely be slower than the for-loop due to the intermediate arrays that will be required.

To speed this up, I would recommend preallocating AllWxGroups to be the biggest possible size it will be. Then instead of using cat(3,...), which will be slow do to a memory copy, index into the iith slice, and then remove the rest at the end.

%Pseudoish code
AllWxGroups = zeros(10,10,10);
for ii = 1:10
   AllwxGroups(:,:,ii) = your_indexing_expression(ii);
end

[Kyle Chudler]

The excel sheet looks something like this

01    12/13/2012 12:30    6472
01    12/13/2012 13:00    6479
.
.
.
09    12/13/2012 12:30    9762
09    12/13/2012 13:00    9755

etc.

Hmm, interesting. I guess it's just been pound into my head so relentlessly by prof's and such that vectorization is the way to go that I never considered it may actually be slower in some cases.

[Sean de Wolski]

My thoughts on for-loops and vectorization

Also, depending on what you're doing, this may only take a few seconds so it might not be worth it to spend hours optimizing it.

Good luck!