Data Logging: Disk Vs Memory

I am having a hard time with different data logging modes during data acquisition. I am using an NI-DAQ PCI-6110E to acquire an analog input signal. I am triggering the acquisition repeatedly on an external digital TTL line connected to the PFI0 input of my DAQ card. I will post my code if there is interest.

The problem I am having is I want to acquire a large amount of data (1MS/sec for 4-6 sec) continuously. I am unable to acquire more than ~2 sec of data using the 'Memory' logging mode, I get a memory error. However, when I try to use the 'Disk' logging mode, my device misses almost 90% of the triggering events I ascribe to it (for example, if I ask the trigger to repeat 1000 times, the trigger will execute 9 or 10 times. If I repeat 2000 times, it executes about 17 or 18 times). The failure of the trigger to repeat occurs both when my TriggerType is Immediate and HWDigital signal. I do not get any run time errors in the Event Log. Any advice?

[Walter Roberson]

How large is a sample? Is it a single channel or multiple? Are you logging time stamps or are you letting the time be implicit?

[Ted]

a sample is 8 bytes, single channel (but the PCI-6110E allows for simultaneous acquisition on multiple channels), and I am not quite sure what it means to be logging time stamps, but I am collecting all the standard event information in the Event Log, which collects absolute time vectors for each trigger, among other things.

[Walter Roberson]

Could you calculate the disk space used per event logged? I am trying to get an estimate of how much memory you are using up per second. At the moment my estimate is that you would be using only on the order of 20 megabytes per second, which should be simple to sustain for the time you need.

[Ted]

Yea, I was getting a similar estimate. I am not super-familiar with data storage types in MATLAB, but the EventLog is a structure array, with 4 data points for each logged event. If each data point is stored as a 64-bit double (I am guessing the structure array requires more memory than just 4 doubles, but that's a start), then including the data I am acquiring with the EventLog data, there are 6 data points recorded for each sample (is that right? 4 EventLog data points, an analog data point, and the sample number associated with it). so that would be 8*6=48 bytes per sample, so for 1M samples per second that's 48Mbytes/second. So 3 seconds would have me at 144Mbytes. Is that something MATLAB can store in its memory? I don't know if that's a lot or a little. I also don't know how to check how much memory each sample point is taking up.

[Walter Roberson]

144 megabytes should not be a problem on any system with 3 Gb or more of RAM (as a rough estimate.) Though depending on exactly how it stores the data and exactly how it grows the buffer, it could temporarily need that much more. I definitely would not expect problems at ~100 megabytes unless your RAM is fairly limited and you do not have virtual swap space configured.