This is actually pretty simple. I think you may be missing how simple it is.
In three dimensions, 3 points determine a plane. True. At least as long as they are not collinear, this is so.
But, as you have written them, the coefficients of the plane you will generate are not set in stone. You can scale them arbitrarily.
That is, if it is true that
a*x + b*y + c*z + d == 0
then it is also true that
a/2*x + b/2*y + c/2*z + d/2 == 0
In fact, we can choose any non-zero constant k, and scale all of the coefficients of that plane, multiplying them all by k, as you wrote it. So first, I'll show you how to compute the basic plane equation coefficients.
Given three points, I'll store them as rows of the array p123. Since I have seen no actual numbers, I'll create them randomly.
So each row of p123 is a point in R^3. LEARN TO USE ARRAYS IN MATLAB!
How can we define a plane? The solution I like to use is to use the normal vector to the plane, and one point that lies in the plane.
We can arbitrarily choose any of those three points as the point in the plane, or I suppose, i could pick the average of all three points. I'll pich the mean.
Now we can compute the normal vector simple using a cross product, or I can use the function null. It creates a vector of length 3 containing the desired coefficients [a,b;c], so I'll just call it abc.
(That line of code assumes you have release R2016b or later.)
A nice feature of the use of null here is it AUTOMATICALLY normalizes the vector of coefficients [a,b,c] such that a^2+b^2+c^2==1.
Now we can test to see if these coefficients satisfy your goals.
That should yield zero, or at least as close as we can come in terms of floating point arithmetic on double precision numbers. I'm satisfied with that result.
Or if you prefer...
as I am with the latter.
Finally, I did say we could compute the normal vector using a cross product.
N = cross(p123(2,:) - p123(1,:),p123(3,:) - p123(1,:)).';
So the same vector resulted. There could have been an arbitrary sign swap, but that would not have been relevant. In fact, we did see a sign swap of the coefficients as I computed it using the cross product.