removing specified data from variable

I have a 100x2 dataset I am working with. I also have 2 random distributions of data.

I want to modify my original dataset in the following way:

1. randomly generate a number from random distribution 1 and keep this many rows of the data.
2. randomly generate a number from random distribution 2 and remove this many rows of the data

I want to do this for the full length of the dataset.

Can anybody help me define this?

time = [1:1:100];
var = rand(100,1);
data = [time' var]; %dataset

dist1 = 1 + (20-1).*rand(100,1); %random distribution 1
dist2 = 10 + (30-10).*rand(100,1); %random distribution 2

position1 = randi(length(dist1));
card1 = dist1(position);
position2 = randi(length(dist2))l
card2 = dist2(position);

[KSSV]

What do you mean by keep and remove this many rows of the data?

[C.G.]

So if I have 100 rows of data:

Step 1: Pull number from random distribution 1. E.g. 5 - keep rows 1:5 of data

Step 2: Pull number from random distribution 2: E.g. 10 - remove remove rows 6:15 of data

Step 3:Step 1: Pull number from random distribution 1. E.g. 5 - keep rows 16:21 of data

[Davide Masiello]

What do you mean by "Pull number from random distribution"?

[C.G.]

By pull I mean I want matlab to randomly give me a number from another variable.

dist1 is a random distribution of data. Card 1 randomly gives me a number from that distribution. Thats the number of rows I want to remove.

[Davide Masiello]

Instead of creating a set of random numbers and then pulling a number from it randomly, could you not just create a single random number?

[C.G.]

No, because I each time I want to remove/keep a different amount of rows. My example above was not very clear, apologies.

Step 1: Pull number from random distribution 1. E.g. 5 - keep rows 1:5 of data

Step 2: Pull number from random distribution 2: E.g. 10 - remove remove rows 6:15 of data

Step 3:Step 1: Pull number from random distribution 1. E.g. 7 - keep rows 16:22 of data

Step 4: Pull number from random distribution 2: E.g. 20 - remove remove rows 23:52 of data

[Davide Masiello]

Is there a maximum amount of rows that are kept or removed each time?

[C.G.]

It is all dependant on the number that is generated from the random distribution each time. The maximum number that could be kept or removed is the maximum number in dist1 or dist2

Answer 1

[Davide Masiello]

I think the following code is a simpler way of achieving your task, but it does not implement the "pulling a number from a random distribution", because honestly I still do not understand what that would be for.

Instead, at each iteration it generates a random integer (max 20) and that would be the new increment of rows to either keep or remove.

See below the code with printed text describing the action at each iteration.

data    = [(1:100)' rand(100,1)]     % Dataset

data = 100×2

1.0000 0.8919 2.0000 0.9869 3.0000 0.7180 4.0000 0.3318 5.0000 0.7077 6.0000 0.9559 7.0000 0.8753 8.0000 0.0893 9.0000 0.9560 10.0000 0.7366

datanew = [];

distribution1 = randi(100,100,1);   % Array of random integers (to be replaced with gaussian distribution later)
distribution2 = randi(100,100,1);   % Array of random integers (to be replaced with gaussian distribution later)

index   = 0;
iter    = 1;
while index < size(data,1)
    fprintf('This is iteration number %d.\n',iter)
    
    if isequal(mod(iter,2),1)
        increment = min(distribution1(randi(length(distribution1),1,1)),size(data,1)-index);
        fprintf('The random number is %d.\n',increment)
        fprintf('We keep the rows between %d and %d.\n',[index+1,index+increment])
        datanew = [datanew;data(index+1:index+increment,:)];
    else
        increment = min(distribution2(randi(length(distribution2),1,1)),size(data,1)-index);
        fprintf('The random number is %d.\n',increment)
        fprintf('The rows between %d and %d do not get added to the new dataset.\n',[index+1,index+increment])
    end
    iter  = iter+1;
    index = index+increment;
end

This is iteration number 1.

The random number is 3.

We keep the rows between 1 and 3.

This is iteration number 2.

The random number is 52.

The rows between 4 and 55 do not get added to the new dataset.

This is iteration number 3.

The random number is 45.

We keep the rows between 56 and 100.

size(data)

ans = 1×2

100 2

size(datanew)

ans = 1×2

48 2

[C.G.]

Thank you for your help.

The reason i want to pull numbers from distributions, is because I am working with dat that is gaussian distributed. Eventually I want to change it from a random distribution, to pulling numbers from a example gaussian distributon to simulate processes

[Davide Masiello]

But that creates an additional problem, i.e. how do you "pull" the number from the random distribution?

What criteria do you use?

[C.G.]

I think the way I worded 'pull the number' has confused this.

All I want to do is:

I set the distribution as an array in matlab as a single column array. For ease here, just generating a random 100x1 array of numbers.

I then ask matlab to randomly select a number from the array, and then thats the number of rows to keep.

Then I ask it to choose another number from the same array, and thats the number of rows to remove.

[Davide Masiello]

But why do you first generate a random distribution and then randomly take a value from it?

How is this different from just generating a random number.

I.e.

how is this

distribution1 = randi(10,100,1);        % array of 100 random integers from (max val. = 10)
a = distribution1(randi(100,1,1))       % integer randomly pulled from distribution 1

a = 7

different from this

a = randi(10,1,1)                       % random integer between 1 and 10

a = 2

[Davide Masiello]

Ok I see now, sorry I must have skipped that part.

I have modified my answer so that the number of rows to keep/remove is pulled randomly from the vectors which I called distribution1 and distribution2.

These are random vectors, you can replace them with the gaussian distributions at your discretion.

Answer 2

[C.G.]

As I said, I eventually want to choose from a gaussian distribution. This would mean I would get a large amount of values in the middle range, and few occurances of v.small/large numbers. Currently by just generating a random distribution it could give me any number at any time.

I'm currently just tyring to get the code working and getting numbers from the right places and then I can change from there. I apologise if this has been confusing.

Eventually,

I want 2 distributions e.g.

dist1 = 1 + (10-1).*rand(100,1); %small numbers

dist2 = 50 + (100-1).*rand(100,1); %big numbers

then the rows to keep would be selected from the small number distribution, and the rows to get rid of would be selected from the big number distribution