Generate 3D model from a 2D image

Hi friends,

I would like to generate a 3D model from a 2D image but I don't have any clue.

From some good instruction, I have successfully generated a binary image with my defined masks. Here the reason why I want a binary image is that the 3D printer only accepts binary slices. I would like to just extrude my pixels into a 3D model (all numbers 1 need to be given the same height, but 0 don't need a height or a neglectable height), and slice them with my printer software. Here is the first section to generate a good binary image. I know there is some other ways we can use to reconstruct a 3D model from a 2D image, but I just want do it in matlab.

Thank you for anyone who can give some idea.

clear
Pic=imread('MonaLisaSquare.jpeg'); % Read any image
%imshow (Pic); % Display the image

PicGray = im2gray(Pic); % convert RGB images to grayscale image
%imshow (PicGray); % Display the grayscale image

% Resize images to printable size (resolution 0.102mm, 3x3 array for halftone mask) 
% (0.102*3)*100=30.6mm is acceptable
L=length(PicGray); % Get the length of image
R=100./L; % Set resize retio

I=imresize(PicGray,R); % Resize image to the printed size
%imshow(I); % Display the resized image
[H,W]=size(I); % Get the new size of the resized image
%histogram(I);

% Set grayscale values
%G0=0-25, G1=26-51, G2=52-77, G3=77-102, G4=102-127, G5=127-152,
%G6=152-177, G7=177-202, G8=202-227, G9=227-255

G0 = [0 0 0; 0 0 0; 0 0 0];
G1 = [0 0 0; 0 0 0; 0 0 1];
G2 = [0 0 0; 0 1 0; 0 0 1];
G3 = [0 1 0; 0 1 0; 0 0 1];
G4 = [0 1 0; 1 1 0; 0 0 1];
G5 = [0 1 0; 1 1 0; 1 0 1];
G6 = [0 1 1; 1 1 0; 1 0 1];
G7 = [0 1 1; 1 1 1; 1 0 1];
G8 = [0 1 1; 1 1 1; 1 1 1];
G9 = [1 1 1; 1 1 1; 1 1 1];

% Set new image zero array with 3x3 arrays
H3=H.*3;
W3=W.*3;
NewPic=zeros(H3,W3);

% Give 3x3 grayscale values to the resized image based on 256 data

for j=1:H
    for i=1:W
        if (0<=I(i,j)&&I(i,j)<=25)
            NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G0;
        else
            if (25<I(i,j)&&I(i,j)<=51)
                NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G1;
            else
                if (51<I(i,j)&&I(i,j)<=77)
                    NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G2;
                else
                    if (77<I(i,j)&&I(i,j)<=102)
                        NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G3;
                    else
                        if (102<I(i,j)&&I(i,j)<=127)
                            NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G4;
                        else
                            if (127<I(i,j)&&I(i,j)<=152)
                                NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G5;
                            else
                                if (152<I(i,j)&&I(i,j)<=177)
                                    NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G6;
                                else
                                    if (177<I(i,j)&&I(i,j)<=202)
                                        NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G7;
                                    else
                                        if (202<I(i,j)&&I(i,j)<=227)
                                            NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G8;
                                        else
                                            NewPic((i*3-2):(i*3),(j*3-2):(j*3))=G9;
                                            
                                        end
                                    end
                                end
                            end
                        end
                    end
                end
            end
        end
    end
end

      
imshow(NewPic)

imwrite (NewPic,"MonaLisa_d0.102mm.tif")

[DGM]

Is there a particular reason you're using this dither pattern? There are existing tools for ordered and error-diffusion dithering.

Is there a reason that you're doing dithering at all? At least for a FDM process, this is going to create something that may be a challenge for many printers due to the absurd number of retractions (~30k retractions per layer).

You'd also need to be combining the extruded image with some other solid geometry, otherwise you'd just end up with a bunch of disconnected plastic bits on the build plate. So obviously there's more to this than just extruding the binarized image.

If you're going to slice the generated model anyway, I don't see why it needs to be binarized or dithered. What is the actual goal? Lithophanes?

[Haoqing Yang]

Hi DGM, I am using a DLP 3D printer instead of FDM printer, I know this seems weird, but that printer doesn't accept grayscale images, I just want to have a halftoned sample which may look like there is some grayscale. I created these masks to let the image smooth grayscale surface. I have manually made some simple models in Solidworks by this way, they all work well, but I just wanna see if there is a more general way for complex images.

Answer 1

[Haoqing Yang]

I found a way to generate the stl, but it seems that the isosurface fuction mess up my pixels. My thought is to generate a tiff stack with >1 image to genrate a 3D array and export faces and vertices with isosurface.

% Create stl file from the binary images

% First write a tiff stack with only 2 identical tif
imwrite(NewPic,"MonaLisa-1.tiff")
imwrite(NewPic,"MonaLisa-1.tiff","WriteMode","append")

% Import tiff stack and generate 3D  
V1=double(tiffreadVolume("MonaLisa-1.tiff"));

% Img information
whos V1
isosurface(V1)

[faces,verts]=isosurface(V1);

% Create triangulation
TR=triangulation(faces,verts);

% Save STL
stlwrite(TR,'MonaLisaSTL.stl');

% Check STL in matlab
TR2=stlread('MonaLisaSTL.stl');

figure
triplot(TR2);

Answer 2

[DGM]

I've always found that one of the biggest troubles with resin processes is getting decent reproduction of small islands/holes. Small islands tend to break off and end up welding to the print elsewhere, and small holes are always undersized. If the goal were to just produce a lithophane, I'd convert the image into a 3D surface (a height map), and then let the slicer do its job instead of trying to do the slicer's job for it.

If you want to do it with dithering, well here's one attempt, but I'll warn you that it has problems.

% read and strip color
inpict = imread('MonaLisaSquare.jpeg');
inpict = im2gray(inpict);

xyscale = 0.102; % mm/px
zscale = 0.2; % mm
maxwidth = 300; % pixels

% resize
inpict = imresize(inpict,min(maxwidth./size(inpict,1:2)));

% binarize
%binarized = imbinarize(medfilt2(inpict,[1 1]*5)); % something simple
binarized = dither(inpict); % dithered

imshow(binarized,'border','tight')

% generate position data
szo = size(binarized,1:2);
x = xyscale/2:xyscale:xyscale*(szo(2)-0.5);
y = xyscale/2:xyscale:xyscale*(szo(1)-0.5);
z = zscale/4:zscale/2:zscale*0.75; % model is 1 voxel thick

% expand and reorient
out = repmat(flipud(binarized).',[1 1 2]);

% write as voxelized STL
outputname = 'mona.stl';
CONVERT_voxels_to_stl(outputname,out,x,y,z,'binary')

This uses this tool from the FEX

https://www.mathworks.com/matlabcentral/fileexchange/27733-converting-a-3d-logical-array-into-an-stl-surface

The problem here is that two pixels which are only diagonally connected will form a non-manifold edge. You can expect the STL file to be relatively large (a few tens of MB), and you can expect it to have thousands of defects. Whether or not your slicer will put up with that, I don't know.

[Haoqing Yang]

This is so great! seems easier than my script. But I have some error when generating the stl. Let me find some way to fix it!

Error using CONVERT_voxels_to_stl (line 68)

The dimensions of gridDATA do not match the dimensions of gridX, gridY, gridZ.

Error in untitled4 (line 22)

CONVERT_voxels_to_stl(outputname,out,x,y,z,'binary')

[Haoqing Yang]

I still don't know why I always get this error, I just used your script and everything else was correct, just the error was there.

[DGM]

I'm not sure what changes you made. Note that the conversion tool expects the output to be a 3D image whose dimensions are ordered X,Y,Z, so for a standard 2D image, we need to flip dim1, transpose, and replicate so that it has at least two layers on dim3. If the binary image is 75x100x1, the variable called out is 100x75x2. Likewise, 100, 75, and 2 should be the lengths of the x,y,z vectors.

[Haoqing Yang]

I made no change on your script. I understand this, that's why you call repmat. And I read the instruction under CONVERT_voxels_to_stl to get some idea, so far x and y are 1x300 double, and z is a 1x2 array, nothing confilct with the tool. Just cannot generate a stl.

[Haoqing Yang]

CONVERT_voxels_to_stl says (line 67)

if ~isequal(size(gridDATA),[numel(gridX),numel(gridY),numel(gridZ)])
  error(' The dimensions of gridDATA do not match the dimensions of gridX, gridY, gridZ.')
end

size(gridDATA)=1, numel (gridx) or numel (gridx)=300 for sure, numel (gridx)=2 for sure, this if fail, but I still get this error.

[DGM]

Why is size(gridDATA) 1? I'm running R2019b, but I don't think that makes sense. Even a scalar input would return size = [1 1].

Mind you, the way this tool is performing this test is easy to break, but it shouldn't. Let me see if I can try it in the online version.

[Haoqing Yang]

Yes, I trid again, size(gridDATA)=[1,1], [numel(gridX),numel(gridY),numel(gridZ)]=[10 10 10], and ~isequal(size(gridDATA),[numel(gridX),numel(gridY),numel(gridZ)]) gives

ans =

logical

1

It means this error will always show up.

[Haoqing Yang]

Okay I fixed the problem, now it works well.

I appreciate so much for your help!

You made my day!!!!

[DGM]

Oh. What was the problem? Somehow the image got collapsed to 1px?

I'm still trying to get MATLAB Online to even work for me. It's pretty broken in my browser now.

[Haoqing Yang]

I think I called an old version function, I had the old version but never used it. I upgraded the new version function and restarted Matlab many times to get it to run. Matlab sometimes runs offline and we never recognize it.

Answer 3

[DGM]

While I'm waiting for MATLAB to load, I'm going to throw this out there.

https://www.mathworks.com/matlabcentral/answers/1926360-plotting-3d-surface-using-depth-map-and-grey-scale-image#comment_3031456

This is just a height map. For a lithophane, you'd just invert the image with imcomplement() so that the brightest areas are thinnest.

% read and strip color
inpict = imread('peppers.png');
inpict = im2gray(inpict);

% invert if making a lithophane
inpict = imcomplement(inpict);

% resize if you want to change the mesh resolution
inpict = imresize(inpict,0.5);

% make sure the image is full-scale and floating point
outpict = mat2gray(inpict);

% object scale parameters (mm)
% this does not do any resampling
reliefheight = 10; % the variation in thickness
basethickness = 2; % the minimum thickness
xwidth = 150; % y-width is calculated to preserve aspect ratio

% construct coordinate arrays
szi = size(outpict);
szo = xwidth*[szi(1)/szi(2) 1];
x = linspace(0,szo(2),szi(2));
y = linspace(szo(1),0,szi(1)); % shift origin to SW corner
[X Y] = meshgrid(x,y);
Z = im2double(outpict)*reliefheight + basethickness; % scale z

% display the surface
figure
hs = surf(X,Y,Z); hold on
hs.EdgeColor = 'none';
hs.FaceColor = [1 1 1]*0.8;
followerlight(hs); % attached

% construct a closed surface
% surf2solid() outputs face/vertex lists, but the native stlwrite()
% only accepts triangulation objects, so convert it
[F V] = surf2solid(X,Y,Z,'elevation',0); % SEE FEX #42876
TR = triangulation(F,V); % complains, but i'm going to ignore that

Warning: Some input points are not referenced by the triangulation.

% display it again
figure
hs = trisurf(TR);
hs.EdgeColor = 'none';
hs.FaceColor = [1 1 1]*0.8;
followerlight(hs);

% write it to a STL
stlwrite(TR,'monarelief.stl')

The slicer does all the work of binarizing the layers.

[Haoqing Yang]

Here I got a 3d slice by your method! It is the same as what I defined in matlab!!

[DGM]

Is that using this example or the other one? That looks like your original 3x3 custom dither pattern.

[Haoqing Yang]

I used both yours and my method, but I think both works well. I also got a good image from your method, and that works well for my printing as well.