Timer in matlab to repeat segments of code every few minutes?

Hi.

I have the following code which downloads a CSV file from a link and then saves it as a formatted file:

while(1) 
tic 

urlwrite('http://xweb.geos.ed.ac.uk/~weather/jcmb_ws/JCMB_2013_Sep.csv','Weather Data.csv');
data= readtext('Weather Data.csv');

f=2; 
disp('done') 

T=toc; 
pause(10-T) 
end

I've used 10-T as 10 seconds just to test the working program. This would actually be around 300-T for the correct value. I'm designing a model which requires me to download this file every 5 minutes.

How can I achieve this as some sort of loop or timer?

The updated data from the file is used for calculations and plotting graphs so I also want those calculations and other parts of my program to be automated every 3 minutes with the change in new file information.

Current code doesn't display any of my workspace variables and continues to loop with 'done' until i hit ctrl+c. Have used 'done' just to check if it was working correctly. This code should be almost like a background task so I still want to see my workspace variables.

Answer 1

[sixwwwwww]

Dear P, you can use pause command or cputime or etime according to your need:

See following links for more information:

http://www.mathworks.de/de/help/matlab/ref/pause.html

http://www.mathworks.de/de/help/matlab/ref/cputime.html

http://www.mathworks.de/de/help/matlab/ref/etime.html

I hope it helps. Good luck!

[P]

I was considering using a for loop such as:

for ...
    % code
    pause(300)
end

but I don't have any integer values to reference within the loop for the above code. Any suggestions? Thanks for your help

[sixwwwwww]

You can use while instead with a condition which never becomes false. Since you have to loop again and again. and in this while loop you can use pause command. I think it should work this way.

[Cedric]

Note that there is a timer class: http://www.mathworks.com/help/matlab/ref/timerclass.html

[P]

Okay so I tried:

 while(1)
      tic
     urlwrite('http://xweb.geos.ed.ac.uk/~weather/jcmb_ws/JCMB_2013_Sep.csv','Weather Data.csv');

      data= readtext('Weather Data.csv');

      T=toc;
      pause(10-T)
  end

For 10 seconds as a tester. But matlab remains 'busy' and none of my variables are appearing in the workspace. I also have a graph which is plotted well before this code and the graph doesn't appear. Is it stuck? How can I fix it. Thanks.

[sixwwwwww]

I tried the following code for testing and it is working smoothly for me:

x = -pi:0.01:pi;
y = sin(x);
figure, plot(x, y)
count = 1;
while 1
    tic
    count = count + 1
    x = -pi:0.01:pi;
    y = sin(x);
    figure, plot(x, y)
    t = toc
    pause(5 - t)
end

and also values are visible in the command window and variables are stored and updated in workspace. Are you getting some error or is it just stuck?

[Cedric]

Your loop isn't a background job and it's an infinite loop so you never get the command prompt back until you hit ctrl-c. Put a

 fprintf('.') ;

in the loop and you'll see points being displayed. Another thing is about your pause: what if T is greater than 10? So you should update the call for something like

 pause( max(10-T, 0) ) ;

The best way to create some sort of "background job", if you want to keep being able to use MATLAB a bit during the loop, is to use a timer object.

[P]

No. I have included an image using your test code to show you that I don't get any variables in the workspace window:

https://dl.dropboxusercontent.com/u/104069213/Test.png

Another colleague said that MATLAB won't show you the variables in the command window when you do this sort of loop. Seems very limiting/very frustrating to me as it's pretty much essential. Can you perhaps send a screenshot of yours working?

[sixwwwwww]

I was running this code in a script not on the command window. See attachment for screen shot. I assumed that you are running the code within script now command window. Try the code in a script then you can see all the information on command window and also workspace variables

[P]

@sixwwwwww

I am running my original code inside a script and still have that problem. I even tested your test code inside a script and not in the command window and it still doesn't show.

@cedric wannaz adding fprintf('.') eventually got part of my code to work as it displayed a graph which is coded previously in the script. I don't get my variables to display until ctrl+c is hit. I've tried using a timer but I think it calls a function. Might you suggest how to set this task up with a timer instead? Thank you for your time.

[sixwwwwww]

Can you show your code to find out the problem?

[P]

Have edited the main body of thread at the very top with my code. Thanks.

[Cedric]

The whole approach is disputable actually, because:

• You are always overwriting the same file, which makes it difficult to synchronize with whatever should read the data.
• The file that you download is 3.4MB, and it's not a good thing to request it every 10s.
• This is always the same file actually on the server; why do you want to request it over and over?
• Even if it were updated regularly on the server, requesting the whole data from start each time is probably not the way to go. In these situations, we often use APIs provided by the data provider, which allow us to request chunks of data, and not the whole stuff from the beginning each time.
• MATLAB is definitely not well suited for running a background job. Even with the timer, your MATLAB would be stuck each time the file is downloaded. If the download takes 20s and you request it every minute, you have 40s of command prompt use per minute, which makes MATLAB almost unusable.

I am attaching nonetheless two files which will illustrate my point about the timer. You'll see that you'll get the shell back only between downloads. I set a period of 60s, so you can better see it. I also generate file names with a time stamp so there is no problem with open files at this time. Just run P.m.

[P]

@Cedric

That file is actually an archived file, you're right. But I'm intending to use a file which is constantly updated on the server but it is being prepared for me by the dept at my university just now so I am using that archived file as a 'test' means until the actual file is ready.

I don't really have any other option as my goal is to extract info from this data file every 5 minutes (10 secs was just a tester to check code functionality) and is a requirement. The easiest way is to download it all. There is no user input to this program and I am not really concerned about speed of execution, more that it does the job consistently within a reasonable time period. There's no set time on repetition, it's just that 5 minutes would be an average. There will eventually be a G.U.I which shows information such as power/plots etc which will be calculated using this regularly updated data.

Just to clarify then, there's no way I can get my workspace variables while the script is running in the way I want?

[Cedric]

No way if you want to access them from the shell in parallel with your loop, but you can definitely implement whatever computation you need in the loop (moreover, you have time for that if there is a 5 minutes period). I'd simplify the whole process by getting rid of this local file though. You could URLREAD the CSV, and process the char array rather than URLWRITE it to file and then read the file.

If you have any control on the server side file generation (by talking to people who are generating these files for you), you could ask for file names to be redefined with a time stamp every 10 minutes or every hour, in order to minimize the transfer. On the server side, they could maybe even create a symbolic link to the last file each time they change, so they generate

 WeatherData_20131101_1530.csv
 WeatherData_20131101_1540.csv
 WeatherData_20131101_1550.csv

and always set

 WeatherData.csv

to point on the latest version .. this way you don't have to bother about time stamps if you just want the latest file.

[Image Analyst]

Cedric, you should start posting this stuff as your own answer so you get credit for it.

[P]

I only want to access those variables to check that they are producing the right values. The file they are creating for me will update with the last hour. That file is updated every few minutes. This will be considerably smaller and hence the time to execute should be much shorter. Apologies but some of the jargon you have used has gone over my head a bit since i'm new to MATLAB. Does it seem like my original code may work if this is the case with the new file they are building? I'm going to also want to extract the cell information downloaded from that csv and use it in my calculations on each 5 minute interval. My function readtext.m is formatting the csv file from URLWRITE so that it makes it easier for me to extract it once it's downloaded.

If you could sample or edit some of my code to show how I could get it to download and over-write itself every 5 minutes then that would be appreciated. Otherwise, I really appreciate your time, explanation and assistance!

[Cedric]

@ImageAnalyst: you are right, but I got lazy and went on with comments (!)

@P: you are already overwriting the same file. If it is every 5 minutes and only your loop in MATLAB is writing/reading it, there is no real problem. Problems could arise if you needed accesses at a higher frequency and/or if multiple "processes" (whatever they are) would have to access the file. If you are fine having your MATLAB dedicated to this loop, it is fine to implement it as you are doing. There would be an issue if you needed to be able to use MATLAB while it is downloading/processing.

Generally speaking, MATLAB is not the right tool for automatizing tasks like gathering data or serving data to other processes. Hence my comments. If you were designing an important tool which would require a sound design, flexibility, etc, I would advise you to perform the data download/extraction and early processing using e.g. Python to keep it simple enough. The Python "engine/process" could work as a server that could be queried with MATLAB, or, maybe simpler, it could just store data in a database that you would query with MATLAB.

At a MATLAB level, you would build a tool (e.g. with GUI) which doesn't run constantly in a loop, but that you could launch whenever you need it. This tool would allow you to interact with the database (e.g. ask for data availability, get relevant data), and display analysis results.

But all that would required a few weeks of full time work I guess, so it's probably not a solution that you'll wnt to implement.