How can I stretch an image non-uniformly?

I would like to take an image, and stretch it non-uniformly such that the further from the center of the image the less stretched it is. So the areas near the edges would get condensed while the areas far from the areas would get inflated. How could I go about generating a suitable displacement field to use? Please let me know if creating a displacement field to use would be a good approach and/or how I might do this. Any help is much appreciated, thanks! :)

Answer 1

[Image Analyst]

Use imresize to make the image dimensions and aspect ratio whatever you want.

[Eric Machorro]

This doesn't answer the question posed by the original posting.

[Image Analyst]

See Steve's blog:

Spatial transformations Defining and applying custom transforms Steve on Image Processing

For the special case of barrel and pincushion distortion, see undistortFisheyeImage and associated companion functions.

Answer 2

[DGM]

I'm going to start this example using MIMT tools, since generating the displacement maps would otherwise be a chore. We can work back to IPT from there.

Let's start with the example from the webdocs. The center of the displacement map is neutral, and the periphery compresses the image toward the center.

% source image
inpict = imread('peppers.png');
imshow(inpict)

% generate the base displacement maps
s = size(inpict);
x1 = lingrad(s,[0 0; 0 1],[0; 1]*255); % some linear gradients
y1 = lingrad(s,[0 0; 1 0],[0; 1]*255);
radmask = radgrad(s,[1 1]*0.5,0.4,[1; 1; 0]*255,'linear',[0 0.5 1]); % a radial gradient

% generate a linear combination of the directional maps and a radial mask
xmap = replacepixels(0.5,x1,radmask);
ymap = replacepixels(0.5,y1,radmask);

% apply the displacement
mag = [1 1]*100; % displacement magnitude at full scale (px,[x y])
dpict1 = displace(inpict,mag,'xmap',xmap,'ymap',ymap,'edgetype',0);

imshow(dpict1)

In order to change the behavior, simply flip the directional maps and invert the radial mask. Now it's neutral around the periphery (mostly just the corners), and the largest displacement occurs in the center.

% reorient the base maps
x1 = fliplr(x1);
y1 = flipud(y1);
radmask = iminv(radmask);

% regenerate the displacement maps (same as before)
xmap = replacepixels(0.5,x1,radmask);
ymap = replacepixels(0.5,y1,radmask);

% apply the displacement (same as before)
dpict2 = displace(inpict,mag,'xmap',xmap,'ymap',ymap,'edgetype',0);

imshow(dpict2)

That's MIMT displace(), though. Nobody is going to use that. If you want to use the canonical IPT tool for applying displacement maps, use imwarp(). It can do many things beyond what MIMT displace() was ever intended to do, but for these two simple examples, it's fairly easy to apply with a minor change.

% translate the displacement maps into a format compatible with imwarp()
% MIMT displace() uses separate x,y maps which are 
% any standard grayscale or RGB image where neutral displacement is 50% intensity, 
% and displacement at full-swing is defined by an independent magnitude parameter.
% IPT imwarp() uses a 2-page stack of single-channel literal displacement fields.
D = cat(3,(im2double(xmap)-0.5)*mag(1)*2, ...
          (im2double(ymap)-0.5)*mag(2)*2);

% apply the displacement map
dpict3 = imwarp(inpict,D);

imshow(dpict3)

Okay that looks simple enough, but we're still using MIMT to generate all the maps. Let's go back to the first example and start over with base tools.

% source image
inpict = imread('peppers.png');

% generate the base displacement maps
mag = [1 1]*100; % displacement magnitude at full scale (px,[x y])
s = size(inpict);
% the linear gradients (in displacement-scale)
x1 = linspace(-mag(1),mag(1),s(2));
x1 = repmat(x1,[s(1) 1]);
y1 = linspace(-mag(2),mag(2),s(1));
y1 = repmat(y1.',[1 s(2)]);
% the PWL radial gradient (in unit-scale)
center = [0.5 0.5]; % center (normalized)
breakpts = [0 0.2 0.4 1]; % breakpoint positions (normalized to diagonal)
gradvals = [1 1 0 0]; % values at breakpoints (unit-scale)
[xx yy] = meshgrid(1:s(2),1:s(1));
R = hypot(xx-center(2)*s(2),yy-center(1)*s(1)); % radius from center
R = R/sqrt(s(1)^2 + s(2)^2); % normalize to image diagonal
radmask = interp1(breakpts,gradvals,R,'linear','extrap');

% generate a linear combination of the directional maps and a radial mask
xmap = x1.*(1-radmask); % neutral condition is strictly 0, not 50% gray!
ymap = y1.*(1-radmask);

% stack the displacement maps as imwarp() expects
D = cat(3,xmap,ymap);

% apply the displacement
dpict4 = imwarp(inpict,D);

imshow(dpict4)

That was perceptibly inconvenient, but it's equivalent.

[DGM]

Rereading the original question, I figured I'd better nail down this little detail. This probably isn't an ideal way to do it, but it doesn't really matter at this point.

% source image
inpict = imread('peppers.png');

% generate the base displacement maps
mag = [1 1]*100; % displacement magnitude at full scale (px,[x y])
s = size(inpict);
% the linear gradients (in displacement-scale)
% these have been flipped
x1 = linspace(mag(1),-mag(1),s(2));
x1 = repmat(x1,[s(1) 1]);
y1 = linspace(mag(2),-mag(2),s(1));
y1 = repmat(y1.',[1 s(2)]);

% generate the mask so that it's not strictly circular, 
% but is pinned both at the center disk and border.
% that way the entire border is neutral

% the original PWL radial gradient (in unit-scale)
center = [0.5 0.5]; % center (normalized)
breakpts = [0 0.2 0.4 1]; % breakpoint positions (normalized to diagonal)
gradvals = [0 0 1 1]; % values at breakpoints (unit-scale)
[xx yy] = meshgrid(1:s(2),1:s(1));
R = hypot(xx-center(2)*s(2),yy-center(1)*s(1)); % radius from center
R = R/sqrt(s(1)^2 + s(2)^2); % normalize to image diagonal
radmask = interp1(breakpts,gradvals,R,'linear','extrap');
% a rectangular radial gradient (unit-scale)
xg = repmat(linspace(-1,1,s(2)),[s(1) 1]);
yg = repmat(linspace(-1,1,s(1)).',[1 s(2)]);
xygrad = max(abs(xg),abs(yg));
% combine the two
mk = xygrad.^5; 
radmask = xygrad.*mk + radmask.*(1-mk);

% generate a linear combination of the directional maps and the mask
xmap = x1.*(1-radmask);
ymap = y1.*(1-radmask);

% stack the displacement maps as imwarp() expects
D = cat(3,xmap,ymap);

% apply the displacement
dpict5 = imwarp(inpict,D);

imshow(dpict5)

Note the lower fold on the tablecloth near the center of the image. Compare to the previous similar examples. None of the image content is displaced out-of-frame.

[Eric Machorro]

Wow! this is great! So much depth. This goes a long way in explaining how to do something very similar. Thank you so such a thorough response . As an aside - not that this is your job - any further recommendations on details or a primer on how the displacement maps are formed would be appreciated. The documentation is a little terse on that sub-point.

More simply though ... and I think more diredctly to the question that I have an the original poster has is the following:

In the simplest case, I am trying to plot a NxN pixel image.

Normally this is straightforward such as in

N= 3;

IMG = rand(N,N);

imagesc(IMG); colorbar

Here is the ‘size the pixel is 1x1 for all NxN pixels with a colorbar for scale.

But I have a twist:

• the widths of the pixels in the x direction may not be uniform (i.e. the width of the pixel IMG(:,k) is different for different k = 1..N)
• the heights of the pixel in the y direction may not be uniform 9(i.e. the height of the pixel IMG(:,k) is different for different k = 1..N)

for example, what if the pixels on the middle columns (i.e., IMG(2,:) ) were only .25 units wide – how would one plot that?

I’d like to display the NxN image with the proper scaling – that displays the entirety of each voxel. Note that neither the surf(..) nor mesh(..) commands don’t quite fit the need here because they plots only the middle-point value of each pixel.

[DGM]

Re: Displacement map setup

I'm just going to do this for the MIMT example, since the maps are all plain images and can be displayed as plain images. The non-MIMT example using imwarp() is equivalent with the exception of a scaling (since the magnitude is baked into the displacement map) and offset (since the null point is 0 instead of 50% gray).

% source image
inpict = imread('peppers.png');

% generate the base displacement maps
s = size(inpict);
x1 = lingrad(s,[0 0; 0 1],[1; 0]*255); % some linear gradients
y1 = lingrad(s,[0 0; 1 0],[1; 0]*255);
imshow([x1 y1])

It helps to understand that these maps are used to describe the query location. In other words, these maps describe the source location from the perspective of the destination. The null point is at 50% gray, regardless of the image class. Consider x1; the left side of the output image will be sourced from locations to the right (i.e. Xsource > Xdestination).

Applying these two maps by themselves will uniformly zoom in on the center of an image.

mag = [1 s(1)/s(2)]*100; % displacement magnitude at full scale (px,[x y])
dpict = displace(inpict,mag,'xmap',x1,'ymap',y1,'edgetype',0);
imshow(dpict)

In order to get the desired nonuniformity, I chose to just use basic image composition to effect a weighted mean of the uniform displacement maps and a constant null displacement map. The weighting array is a radial mask with a piecewise-linear profile.

% a radial gradient
radmask = radgrad(s,[1 1]*0.5,0.4,[0; 0; 1]*255,'linear',[0 0.5 1]);
imshow(radmask)

Note that the center is uniformly black and the corners are uniformly white.

The composition is as follows:

% generate a linear combination of the directional maps using a radial mask
xmap = replacepixels(0.5,x1,radmask);
ymap = replacepixels(0.5,y1,radmask);
imshow([xmap ymap])

The inputs here are (FG,BG,mask). The result is conceptually FG.*mask + BG.*(1-mask), so white mask regions contribute content from FG, and black mask regions contribute from BG. The foreground is the null value 0.5, so the corners will be flat gray. The background is the base displacement map, which will be unchanged near the center.

As before:

% apply the displacement
dpict = displace(inpict,mag,'xmap',xmap,'ymap',ymap,'edgetype',0);
imshow(dpict)

Regarding trying to work with nonuniformly-spaced data, that's another story. MIMT tools aren't suited to deal with that unless the data can be expressed in a uniformly-gridded format. That's probably not acceptable.

I'm really not sure if imwarp() or other IPT geometric transformation tools can be used for that. They keep coming up with new transformation tools, and the spatial referencing capabilities are something I never use. Consequently, I never wrapped my head around most of it.

For visualization, I mentioned

https://www.mathworks.com/matlabcentral/fileexchange/35601-sanepcolor-varargin

in the other thread. I've been looking for the aforementioned "newer example", but I never did find it.

[DGM]

Wait a minute. Let me back up here.

You said:

• the widths of the pixels in the x direction may not be uniform (i.e. the width of the pixel IMG(:,k) is different for different k = 1..N)
• the heights of the pixel in the y direction may not be uniform (i.e. the height of the pixel IMG(:,k) is different for different k = 1..N)

The first condition describes data on a nonuniform grid; the other describes data which is not on a grid.

Dealing with nonuniformly-gridded data is the problem that something like sanepcolor() attempts to solve.

If the data is not gridded, then we're limited further. In this scenario, there's nothing keeping pixels from overlapping each other or from having gaps between each other. If the pixels are all square, we could probably use scatter() to illustrate it. Otherwise, if N is small, we might be able to build the image using patch() to draw N^2 rectangles.

[Eric Machorro]

THANK YOU for being so diligent and super-prompt in your responses.

I had a typo in earlier –Restating from before

• The widths of the pixels in the x direction may not be uniform (i.e. the width of the pixel IMG(:,k) is different for different k = 1..N)
• the heights of the pixel in the y direction may not be uniform (i.e. the height of the pixel IMG(k,:) is different for different k = 1..N)”

Note the correctionm in bold. Again, apologies for the confusion that I caused earlier.

I have data associated with the centers of each pixel, but the pixels are not uniformly sized: in other words, I have pixels that are bounded by pixel ‘edges’ in the X and Y direction. The immediate (simplified) example that I’m playing with is

% a simple, sample image
    IMG = zeros(10,10);
    IMG(3,8) = 1;
% non-uniform x-spacing, but uniform y-spacing – the pixel edges
    X = [-140 -130 -105 -70 -10 0 10 25 110 130 140];
    Y = [ -140  -112   -84   -56   -28     0    28    56    84   112   140];
% the midpoints of each ‘pixel’ 
    x_mid = [X(1:end-1)+diff(X)/2];
    y_mid = [Y(1:end-1)+diff(Y)/2];
    
sanePColor(x_mid,y_mid,IMG)

The visualization tool you suggested, https://www.mathworks.com/matlabcentral/fileexchange/35601-sanepcolor-varargin, has a few good ideas w/in its code but falls short of the immediate need. When overlayed with the gird-lines stipulated by the X and Y variables above, it yields this:

which shows a mis-alignment of the grid-lines (e.g. pixel edges) with the corresponding pixels center values.

Your suggestion to use patch() is intriguing, but it isnt clear how to handle the color scale - different patches with difference values of IMG would have to have different colors reflecting the corresponding value of IMG. And then all of these colors would need to be drawn from the same palette that was then summarized in a single colorbar for the whole figure.

Again, thank you for your effort and time in looking at this. You're being super helpful.

Answer 3

[Eric Machorro]

Epilogue - of sorts:

I was trying to avoid this as a solution, but for small-ish plots, this issue can resolved by plotting out individual pixels as individual rectangles using matlab's patch command.

% take for example a non-uniform x-spacing, but uniform y-spacing – the
% pixel edges could be something like ...

X = [-140 -130 -105 -70 -10 0 10 25 110 130 140];
Y = X; 

% the midpoints of each ‘pixel’ 
    dX = diff(X);
    dY = diff(Y);
    x_mid = [X(1:end-1)+dX/2];
    y_mid = [Y(1:end-1)+dY/2];

% a simple, sample image
    IMG = zeros(length(x_mid),length(y_mid));
    IMG(3,8) = 1;

%Establish a template for vertices of the subsequent patch plots
%used o plot individual 'pixels'
    VerticesX = [-1 1 1 -1];
    VerticesY = [-1 -1 1 1];

%Loop over all the mid-points to plot a single pixel 
    figure
    for i = 1:10
        for j = 1:10
            Cx =  x_mid(i)+VerticesX*dX(i)/2;
            Cy =  y_mid(j)+VerticesY*dY(j)/2;
            patch(Cx, Cy,IMG(i,j)); hold on
            %plot3(x_mid(i),y_mid(j),IMG(i,j),'r.') % see commentary below
        end
    end
    colorbar    

%Extraneous adjustments to plot the 'edges' of the pixels for reference
    hold on
    % vertical or up-down lines, X-constant lines
    for i = 1:10
        line(X(i) * [ 1 1],[-140 140],'Color','red','LineStyle','--')
    end
    
    % horizontal or left-right, Y-constant lines
    for j = 1:10
        line([-140 140],Y(j) * [ 1 1],'Color','red','LineStyle','--')
    end

%Extraneous adjustments just to make tick-marks easier to read
    subaxis = gca;
    subaxis.XTick = X; 
    subaxis.XTickLabelRotation = 30; 
    subaxis.YTick = Y; 
    subaxis.YTickLabelRotation = 30; 

This will yield ...

This still only identifies a pixel by its boundaries, so that some the 'Data-tip' tool still doesn't give me the value of the pixel at its center, but rather only the coordinates of the vertices of the rectangle encompassing the 'pixel'

This might by somewhat addressed by adding something akin to plot3(x_mid(i),y_mid(j),IMG(i,j),'r.') right after the call to the patch() commmand. It is a little kludgy though IMO.