Using TCPIP, snapshots of webcam are not being sent from server APP to client.

I want to send my webcam live streaming to client using tcpip, so far I have been able to send the webcam from the servers side using this code.

        function VIDSwitchValueChanged(app, event) %toggle switch
            value = app.VIDSwitch.Value;
            tcpipServer = tcpip('0.0.0.0', 30000,'NetworkRole','Server'); %tcpserver doesnt work for me
            if strcmp(value, 'On')
                
                app.cam = webcam;
                frame = snapshot(app.cam);  
                data = im2double(frame);
                s = whos('data');
                s.size;
                s.bytes;
                set(tcpipServer,'OutputBufferSize',s.bytes);
                fopen(tcpipServer);
                theSpeakerImage = image(app.UIAxes, zeros(size(frame), 'uint8'));
                axis(app.UIAxes, 'image');
                preview(app.cam, theSpeakerImage);
                fwrite(tcpipServer,data(:),'double');      

but I have not been able to read it from my client side while trying to unpack it. So far, this is the best I have reached without producing any error. However, no image or video is seen on client side and when I turn off the app, my server app gets stuck with ctrl+c not working

        function Switch_2ValueChanged(app, event)
            value = app.Switch_2.Value;
            tcpipClient = tcpip('0.0.0.0', 30000,'NetworkRole','Client');
            set(tcpipClient,'InputBufferSize',300000);
            %set(tcpipClient,'Timeout',30);
            
            if strcmp(value, 'On')
                fopen(tcpipClient);
                get(tcpipClient, 'BytesAvailable');
                tcpipClient.BytesAvailable; 
                DataReceived =[];
                while (get(tcpipClient, 'BytesAvailable') > 0) 
                    tcpipClient.BytesAvailable
                    rawData = fread(tcpipClient,30000/8,'double');
                    DataReceived = [DataReceived; rawData];
                    reshapedData = reshape(DataReceived,650,600,3);
                    imshow(reshapedData)
                    pause(0.1)

I have tried imshow since I tried to send the frame through tcpip and treated it like an image, still no luck. How do I go about sending the webcam feed from server side to client side? preferably using plot(app.UIAxes) to preview that image from the clients axis.

sorry for posting only a snippet of the code, trying to attach the mfiles fails every time for some reason.

I have also tried the follwoing for clients side and still no luck

                fprintf('0.0.0.0', '%d ', ndims(theSpeakerImage)); %no luck here
                
                DataReceived = [DataReceived; rawData];
                DataReceived = reshape(DataReceived,[1,1]); %throws error that the size of the data can not be changed by adding new elements
                
                preview(rawData, theSpeakerImage); %also nothing
                
                plot(app.UIAxes,rawData);% no results

[Stephen23]

https://web.archive.org/web/20211201130121/https://www.mathworks.com/matlabcentral/answers/1597229-using-tcpip-snapshots-of-webcam-are-not-being-sent-from-server-app-to-client

Answer 1

[yanqi liu]

yes，may be use http to build server and client process

[Garabani]

sorry if I seem low-level but I really am, I had no idea you could use http. is there any example of matlab code or could you share some websites with it please

[yanqi liu]

yes，sir，may be check：

https://ww2.mathworks.cn/matlabcentral/fileexchange/29027-web-server?s_tid=srchtitle

[Garabani]

Thank you so much sir

Answer 2

[Walter Roberson]

Each time through, you should construct a buffer that contains the image size (list of 3 doubles) followed by the image data. fwrite that to the socket.

on the receiving side, fread 3 double. That will be the image size. Now fread with that size and '*double' precision.

You might find that in practice you need to break the sending data into smaller chunks. If so then still have that buffer I mentioned, but fwrite smaller portions of it.

If you do that then on the receiving side have a small state machine. You start waiting for an image. loop fetching bytes available into a variable and fread that many bytes as *uint8, and put onto the end of your current input buffer.

If you are in the first state and there are less than 24 bytes in the buffer, loop back to waiting for bytes available.

If you are in the first state and your buffer has more than 24 bytes then typecast the first 24 as double to find the current image size, and remove those 24 from the buffer and calculate the product times 8... that's the number of bytes you need for the image. Enter the second state, where you are waiting for data to finish.

If you are in the second state and your buffer is less than the number of bytes you need then loop back to waiting for bytes available.

If you are in the second state and you have enough bytes, then pull the appropriate number of bytes from the front of the buffer and typecast to double and reshape to the size read. Then remove those bytes from the buffer. Display the image and change to the first state and loop back.

This needs a minor flow change such that if you are state 1 and there are at least 24 bytes in the buffer, you can skip waiting for bytes to be available and more on to extracting the size information and switching to state 2. Likewise at state 2 if you already have enough bytes you do not need to listen for more right now. This minor revision to what I described first makes it easier to deal with streams of bytes in the buffer, where you might already have received some of what you need for the next image.

You need to decide on a protocol to signal end of stream. That might be, for example, sending 0 as the image size.

Is there any particular reason why you want to send double precision images? Most webcam images are 8 bits per pixel, 16 at most, so double precision is usually a waste.

[Garabani]

Thank you, I will try this :D

[Garabani]

sorry, i thought the following was submitted, dont know why

I have tried to send it as unit8 and 16 but I think the server side reads it. The only problem is that the client doesnt post it in the UIAxes, I feel the video is passing through properly because the UIAxes on the client side changes from grid to empty screen. Do you know how to do it or any example code for it that I can look at?

[Walter Roberson]

Please post your current code

[Garabani]

Thank you for taking your time time, here is my code, Im running it in App Designer.

for server

classdef WEBCAM_FOR_DOCTOR < matlab.apps.AppBase

    % Properties that correspond to app components
    properties (Access = public)
        UIFigure                     matlab.ui.Figure
        UIAxes                       matlab.ui.control.UIAxes
        VIDSwitch                    matlab.ui.control.Switch
    end

im using a toggle switch to plot the webcam on UIAxes.

    % Value changed function: VIDSwitch
        function VIDSwitchValueChanged(app, event)
            value = app.VIDSwitch.Value;
            tcpipServer = tcpip('127.0.0.1', 20000,'NetworkRole','Server');
            if strcmp(value, 'On')
                
                app.cam = webcam;
                frame = snapshot(app.cam);  
                data=im2double(frame);
                s = whos('data');
                s.size;
                s.bytes;
                pic = buffer(5,5,2);
                set(tcpipServer,'OutputBufferSize',s.bytes);
                fopen(tcpipServer);
                
                theSpeakerImage = image(app.UIAxes, zeros(size(frame), 'uint16'));
                axis(app.UIAxes, 'image');
                preview(app.cam, theSpeakerImage);

%                 assignin('base','var',frame)
                fwrite(tcpipServer,pic,'double');
%                 fwrite(tcpipServer,data(:),'double');
 
            else
               fclose(tcpipServer);
                closePreview(app.cam);
                delete(app.cam);
                cla(app.UIAxes);

            end    

For client side.

classdef WEBCAM_FOR_Patient < matlab.apps.AppBase

    % Properties that correspond to app components
    properties (Access = public)
        UIFigure                      matlab.ui.Figure
        Switch_2                      matlab.ui.control.Switch
        UIAxes                        matlab.ui.control.UIAxes
    end

Using another swithc (Switch_2) to open tcpip for client side. trying to read the webcam from server to this sides UIAxes to act like a live stream

    % Value changed function: Switch_2
        function Switch_2ValueChanged(app, event)
            value = app.Switch_2.Value;
            tcpipClient = tcpip('127.0.0.1', 20000,'NetworkRole','Client');
            set(tcpipClient,'InputBufferSize',300000);
            %set(tcpipClient,'Timeout',30);
            
            if strcmp(value, 'On')
                
                fopen(tcpipClient);
                get(tcpipClient, 'BytesAvailable');
                tcpipClient.BytesAvailable; 
                
                %trying this code currently
                DataReceived =[];
                while (get(tcpipClient, 'BytesAvailable') > 0) 
                    tcpipClient.BytesAvailable
                    rawData = fread(tcpipClient,30000/8,'double');
                    DataReceived = [DataReceived; rawData];
                    reshapedData = reshape(DataReceived,650,600,3);
                    imshow(reshapedData)
                    pause(0.1)
                    
                    if strcmp(value, 'Off')
                        break;
                    end
                end
            end
            
            if strcmp(value, 'Off')
                fclose(tcpipClient);
                
            end
                %was trying these ones before
                
%                 theSpeakerImage = image(app.UIAxes, zeros(size(rawData), 'uint8'));
%                 fprintf('127.0.0.1', '%d ', ndims(theSpeakerImage));
                %DataReceived = [DataReceived; rawData];
                %DataReceived = reshape(DataReceived,[1,1]);
                %imshow(rawData, 'parent', app.UIAxes);
%                 preview(rawData, theSpeakerImage);

                %plot(app.UIAxes,rawData);


%                 imshow(reshapedData);
%                 pause(0.1)
             
           
                
%                 raw = fread(tcpipClient);
%                 
%                 rawData = uint8(raw);
%                 %rawData = reshape(raw,1);
% %                 theSpeakerImage = image(app.UIAxes, zeros(size(rawData), "int8"));
% %                 axis(app.UIAxes, 'image');
% %                 preview(rawData, theSpeakerImage);
% %                 obj=VideoReader(rawData); 
% %                 vid=read(obj); 
% %                 firstframe=read(obj,1); 
% %                 imshow(firstframe,'video',app.UIAxes);
%                 v = VideoWriter(rawData);
%                 video = read(v);
%                 imshow(video,'video',app.UIAxes);