big differences in saving variables

Hi,

i need to save some tables and variables as Mat and CSV files.

but I have faced a strange behavior on saving the files.

when I save my file and evaluate the time of each of them individually, the saving time much less than I evaluate total saving time.

I also should mention the true elapsed time is a bigger time.

here are my code and output,

folderAddress = 'D:\Nejatian\';
tic
delete([folderAddress 'signals.csv'])
delete([folderAddress 'LatestSignals.csv'])
t1 = toc;
tic
writetable(T,[folderAddress 'signals.csv']);
t2 = toc; 
tic
writetable(lT,[folderAddress 'LatestSignals.csv'])
t3 = toc;
tic
save([folderAddress 'visualData'],'visualData')
t4 = toc;
tic
delete('workspace.mat')
save('workspace.mat');
t5 = toc;

str1 = string(['Deleting CSV files    = ' num2str(t1) 'Sec']);
str2 = string(['Writing  CSV file one = ' num2str(t2) 'Sec']);
str3 = string(['Writing  CSV file one = ' num2str(t3) 'Sec']);
str4 = string(['Saving Mat file from a struct variable = ' num2str(t1) 'Sec']);
str5 = string(['Saving all workspace variables         = ' num2str(t1) 'Sec']);
disp([str1;str2;str3;str4;str5]);  

• Deleting CSV files = 0.0320444Sec
• Writing CSV file one = 1.99078Sec
• Writing CSV file one = 0.273289Sec
• Saving Mat file from a struct variable = 0.0120444Sec
• Saving all workspace variables = 0.00220444Sec

and the total time is

tic
folderAddress = 'D:\Nejatian\';
delete([folderAddress 'signals.csv'])
delete([folderAddress 'LatestSignals.csv'])
writetable(T,[folderAddress 'signals.csv']);
writetable(lT,[folderAddress 'LatestSignals.csv'])
save([folderAddress 'visualData'],'visualData')
delete('workspace.mat')
save('workspace.mat');
t=toc;
str = string(['Total time  = ' num2str(t) 'Sec']);
disp(str);

• Total time = 78.3331Sec

why this big difference happens and how i can speed up my File saving time?

i mean is there any way to parallelize or decompressed the output for reducing the saving file?

[Dana]

Hmm, this doesn't really make sense. The only differences in those two code snippets is the location of some tic/toc statements, which shouldn't meaningfully alter the run time. You're saying you've done this same exercise repeatedly and you keep getting the same result? Are you doing anything in between (e.g., manually altering any files)? Does it matter which order you run these in? Are you sure you've included the tic at the beginning when running the second snippet (if not, it'll compute the elapsed time using the previous call of tic, which could've been on a previous run).

What if you change your initial code to:

tstart = tic;
folderAddress = 'D:\Nejatian\';
tic
delete([folderAddress 'signals.csv'])
delete([folderAddress 'LatestSignals.csv'])
t1 = toc;
tic
writetable(T,[folderAddress 'signals.csv']);
t2 = toc; 
tic
writetable(lT,[folderAddress 'LatestSignals.csv'])
t3 = toc;
tic
save([folderAddress 'visualData'],'visualData')
t4 = toc;
tic
delete('workspace.mat')
save('workspace.mat');
t5 = toc;
t = toc(tstart)

str1 = string(['Deleting CSV files    = ' num2str(t1) 'Sec']);
str2 = string(['Writing  CSV file one = ' num2str(t2) 'Sec']);
str3 = string(['Writing  CSV file one = ' num2str(t3) 'Sec']);
str4 = string(['Saving Mat file from a struct variable = ' num2str(t1) 'Sec']);
str5 = string(['Saving all workspace variables         = ' num2str(t1) 'Sec']);
str = string(['Total time  = ' num2str(t) 'Sec']);
disp([str1;str2;str3;str4;str5;str]);

So we're doing a total time simultaneously with the individual component times. Do you still get the same problem?

[Mario Malic]

This is an interesting issue. What is size of each file you save?

You can remove delete('workspace.mat') line, as save will overwrite it, if the file exists.

[Abolfazl Nejatian]

Dear Dana

yep, I agree with you this doesn't make sense.

i think it does happen because tic toc function evaluates the processing time of CPU but first to end tic toc function evaluate Processing time plus writing file on my SSD drive.

and because they are almost big file (around 2 Gbyte each) this hypothesis may be true.

and this is out put of your results

• t =
• 82.5042
• "Deleting CSV files = 0.0119518Sec"
• "Writing CSV file one = 1.86071Sec"
• "Writing CSV file one = 0.160429Sec"
• "Saving Mat file from a struct variable = 0.019518Sec"
• "Saving all workspace variables = 0.019518Sec"
• "Total time = 82.5042Sec"

[Abolfazl Nejatian]

Dear Mario,

yep im agree with you.

Delete function should use for CSV or text files

this line of code can be omitted.

[Mario Malic]

There's an option -nocompression, but I don't know if it would make things different.

https://ch.mathworks.com/help/matlab/ref/save.html#bvmu1wo