Main Content

Import Data into Diagnostic Feature Designer

Diagnostic Feature Designer is an interactive tool for processing ensemble measurement data and extracting features that indicate the condition, such as healthy or faulty, of the machines that produced the data. The data can come from measurements on systems using sensors such as accelerometers, pressure gauges, thermometers, altimeters, voltmeters, and tachometers. The main unit for organizing and managing multifaceted data sets in the app is the data ensemble. An ensemble is a collection of data sets, created by measuring or simulating a system under varying conditions. Each row within the ensemble is a member. Each member of the ensemble contains the same variables, such as Vibration or Tacho.

The first step in using Diagnostic Feature Designer is to import source data into the app from your MATLAB® workspace. You can import data from tables, timetables, cell arrays, or matrices. You can also import an ensemble datastore that contains information that allows the app to interact with external data files. Your files can contain actual or simulated time-domain measurement data, spectral models or tables, variable names, condition and operational variables, and features you generated previously. Diagnostic Feature Designer combines all your member data into a single ensemble data set. In this data set, each variable is a collective signal or model that contains all the individual member values.

Before importing your data, it must already be clean, with preprocessing such as outlier and missing-value removal. For more information, see Data Preprocessing for Condition Monitoring and Predictive Maintenance.

During the import process, you select which variables you want to import, specify the variable types, and perform other operations to create the ensemble that you want to work with. When you click import, the app applies your specifications and creates an ensemble that contains your selected data. The following figure illustrates the overall import flow.

Notional import workflow. Workspace data sources are on the left. Import dialog box is in the middle. Post-import app data browser is on the right.

After you import your data once, you do not need to import it again for subsequent sessions. Save your session to store both the initial data ensemble and any derived variables and features you compute prior to saving. You can also export your data set to the MATLAB workspace, and save your data as a file that you can import in a subsequent session.

If you have a large number of ensemble members, consider creating a representative subset of members when you first start exploring the data and potential features in the app. Because the app is interactive, importing a large number of members can result in slower performance. Instead, you can develop and rank features interactively with the smaller data set, and then generate code that repeats the computations on the original data set.

For more information on data sources, ensembles, and variable types in predictive maintenance, see Data Ensembles for Condition Monitoring and Predictive Maintenance.

Source Data Requirements

For signals, the app accepts individual member table arrays, timetable arrays, cell arrays, or numeric matrices, each member containing the same independent variables, data variables, and condition variables. For spectral data, the app accepts individual member table arrays or idfrd objects. The table describes the data requirements for variables within ensemble members.

Input ItemContentNotes
Signal dataTimetables, tables, cell arrays, or numeric arraysFor time-based data, timetables are recommended.
Signal independent variables (IVs)Double, duration, or datetime

For each signal variable, all member IVs must be of the same type, whether your signal is based on time or on another IV such as consumption or duty cycles.

If your member data is stored in a matrix, you must have only one IV that applies to the full set of signals that you are importing.

If your data was uniformly sampled in time and you do not have recorded timestamps, you can construct a uniform timeline during the import process.

Spectral dataNumeric (double) table or idfrd object

Each idfrd object must contain data for only one spectrum (the first dimension is 1).

You cannot import spectra from matrices.

Condition variables (CVs)Scalar — Numeric, string, cell, or categoricalYou can import condition variables along with your data in tables, timetables, or cell arrays, but not in matrices.
FeaturesScalar — Numeric, string, or cellYou can import features that you computed previously, either externally or within the app itself.
MatricesPurely numeric array that contains columns representing a single IV and any number of signals that share that IV. Cannot accommodate spectra. Cannot generally accommodate condition variables or features.

Matrices cannot accommodate variable names.

You can import condition variables and features from a matrix if your data set contains only scalars and does not contain signals.

You can import data members individually or as an ensemble that contains all your data members. This ensemble can be any of the following:

  • An ensemble table containing table arrays, cell arrays, or matrices. Table rows represent individual members.

  • An ensemble cell array containing tables, cell arrays, or matrices. Cell array rows represent individual members.

  • An ensemble datastore object such as a fileEnsembleDatastore or simulationEnsembleDatastore object that contains the information necessary to interact with files stored externally. Use an ensemble datastore object especially when you have too much data to fit into app memory. Reading and writing to external files during computations does impact performance, however. To create a representative subset of the ensemble datastore files to work with, see subset.

  • An ensemble matrix that contains only condition variables and features. Matrix rows represent individual members.

For more information about organizing your data for import, see Organize System Data for Diagnostic Feature Designer.

Import Ensemble Table

This workflow illustrates the steps for importing an ensemble table into the app.

Load Ensemble Table into MATLAB Workspace

Load your ensemble table into the MATLAB workspace. You can preview the data in the workspace variable browser, as this example shows. In this case, the data set contains two time-based signals and a scalar condition variable named faultCode.

Example ensemble table. Vibration signal is in the leftmost column. Tacho signal is in the center column. Fault code is in the rightmost column.

Open App and Start New Session

Open the app by entering diagnosticFeatureDesigner at the command line. Then, click New Session. This action opens the import dialog box.

New Session button

Select Ensemble Table from Source Variables

In the Select more variables section, select your ensemble table from the menu.

A menu displays the one available variable, which is dataTable.

View Source Variable Components and Change Variable Type and Units

The Source variables section displays the variables from your ensemble. In the following example figure, the app identifies the Vibration and Tacho variables as time-based signals that each contain Time and Data variables. The third variable in the variable set for each signal, Sample (Virtual), is not checked. Sample (Virtual) allows you to generate an IV, especially when your source data set does not contain an explicit IV. For more information on Sample (Virtual), see Import Signal with No Time Variable.

The app displays source variables Vibration, Tacho, and FaultCode in a column.

The icons identify the variable type that the app assumes. The icon next to faultCode, which illustrates a histogram, represents a feature. Both features and condition variables are scalars, and the app cannot distinguish between the two unless the condition variable is a categorical. To change a variable type, click the variable name to open the variable properties in the Source variable properties panel. Then, in Variable type, change Feature to Condition Variable.

In the source variables list on the left, the faultCode variable row is gray. In the source variable properties on the right, a menu displays Feature, Condition Variable, and Independent Variable.

The icon for faultCode now illustrates a label, which represents a condition variable.

The bottom of the variable list displays faultCode with the "label" icon.

For signal and spectrum variables, you can also change the units that the app uses for plotting and for other operations within the app. To do so, in the lower level variable list of the signal or spectrum variable, click the name of the IV or data variable. The Source variable properties panel provides a menu of options for each property that you can modify. In the following example figure, Source variable properties displays properties for the Time variable of the Vibration signal. The figure illustrates the selection of Minutes from the menu of Units options.

The Source variables panel is on the left. The Time variable in the Vibration signal is highlighted. The Source variable properties on the right displays options for variable name, variable type, IV type, and Unit. The unit menu ranges from seconds to years.

Preview Data Variables

In addition to providing options for variable properties such as Type, the Source variable properties displays a preview of your import data when you click the name of a signal or spectrum variable. The following example figure shows the preview of the Vibration data. The preview panel in the figure displays source data for the first ten Vibration samples of the first ensemble member, and includes values for IV, data, and the sample index.

The preview panel displays source properties only. The preview panel does not reflect any property modifications you make in the panel. For example, if you change the units of the Vibration signal from seconds to minutes, the preview panel still displays the source units of seconds. When you complete the import, the app converts the time data to minutes for use in the app.

The Source variables panel is on the left. The variable at the top of the panel, Vibration, is highlighted. The Source variable properties on the right displays the preview table.

Confirm Ensemble Specification and Execute Import

Confirm the ensemble specification in the Summary section at the bottom of the dialog box. Click Import to execute the data import.

Summary displaying the ensemble name near the top and three columns that contain, from left to right, Variable Name, Variable Type, and Independent Variable.

Confirm Successful Import into App

Confirm the import in the data browser. In this example, the two signals appear in the Signals & Spectra area. Click Datasets to view all the ensemble variables, including features and condition variables.

App Data Browser contains three sections in a column. These sections are, from top to bottom, Signals & Spectra, Feature Tables, and data sets.

Select Vibration/Data and click Signal Trace to plot the data and view the imported signals, as the following example figure shows. For more information on plotting data, see Import and Visualize Ensemble Data in Diagnostic Feature Designer.

Plot of the imported vibration signals.

Import Individual Members

This workflow describes the steps associated with importing ensemble members individually.

Initiate Import Process

Load the individual member variables into your MATLAB workspace, open the app, and click New Session. The Source menu displays a list of the files in your workspace. In the example figure, there are 10 member files and one additional file, sens2, that is not an ensemble member.

List of all workspace variables, arranged vertically in a menu. The list contains member files that start with the letter "d" and another file named "sens2".

Select Ensemble Members

Select one of the variables that represents an ensemble member. In this example, select d1. The app opens a list of all compatible workspace variables that contain the same internal variables. You can select any combination of these variables or click Select All to select all of them at once.

On the left is a list of variables that have similar format to selected variable "d1". On the right is the list of the variables—Vibration, Tacho, and faultCode—within d1.

Import Individual Timetables

If your individual members are packaged as timetables with a scalar value for each time point, specify Use as signal to have the app interpret the timetable variable as a signal.

Complete Import Process

Once you select your variables, the remaining import steps are the same as in Import Ensemble Table. The app combines the members you import into a single ensemble.

Import Matrix Data

This workflow describes the import of signals from individual member matrices. When you import data in matrices, you must share one independent variable, such as a time variable, with all signals in the ensemble. You cannot import condition variables, features, or spectral data.

Select and Preview Matrices to Import

Load your matrices into the MATLAB workspace, and then initiate the import process by clicking New Session. In the Select more variables section, select one of your matrices and click Select All. For matrices, this section also displays a Use as feature option. This option applies to the special case where a matrix contains only scalar condition variables and features, and no signal data.

Because matrices are numeric, the app identifies each variable column by its column index. In the example figure, there are four member matrices. The first column of each matrix contains the IV representing time, and the second and third columns contain the data values for vibration and tacho data. To preview the contents of the ensemble, in the Source variables section, select the Matrix row.

On the left is a list of the matrices to import. In the middle are the variables for each matrix, with the names "Col1", "Col2", and "Col3". On the right are the source variable properties, which include the variable type "Signal" and a preview of the matrix contents.

Confirm Variable Types

The app interprets Col1 as the IV because it is monotonic, and Col2 and Col3 as signal data variables.

Matrix variable assignments. Col1 is on the top. The properties display on the right contains "Independent Variable" for variable type. Col2 is on the bottom. The variable type on the right is "Data Variable".

If you cannot accurately represent your signals with a single time variable, convert your matrices into tables before import.

Complete and Confirm Import

Click Import to complete the import. Confirm that the data browser contains the signals you want, as shown in the following example figure.

Data browser. The top box contains Col2 and Col3. The middle box contains the message "No features available". The bottom box lists the data set and its signals. There are no features or condition variables.

The app merges the matrices into an ensemble data set that contains the four matrices.

This workflow demonstrates that you can import matrices, but only with limitations. If you want to identify your variables by name, import condition variables or features, or use independent timelines for independent signals, convert your matrices to tables or cell arrays prior to import. For an example of converting a set of matrices to an ensemble table, see Prepare Matrix Data for Diagnostic Feature Designer.

Import Spectral Data

This workflow illustrates how to import spectral data. You can import spectral data in two forms:

  1. An idfrd object that contains frequency and spectrum data for a single spectrum in the Frequency and SpectrumData properties, respectively

  2. A table that contains columns with the frequency and spectral data

When you import an idfrd object, the app recognizes that the data source is spectral, and defaults to the Spectrum variable type, as the following example figure shows.

When you import spectral data in a table, the app defaults to the Signal variable type, but provides additional options for the Spectrum and Order Spectrum variable types. The following example figure illustrates the import of spectral data in a table.

You can also import order spectra that contain order rather than frequency information. Order spectra are useful for analyzing rotating machinery. Each order is a multiple of a reference frequency, such as the primary-shaft rotational frequency. The following example figure illustrates the import of an order spectrum.

When you complete the import, the Data Browser displays the spectra, as the following example figure shows.

Import Signal with No Time Variable

This workflow illustrates how to generate a virtual IV if your data set includes signals that contain no IV. For instance, you might have a signal that was measured or generated in uniform time samples, but which does not include a vector of the actual time stamps. The app can generate a virtual timeline that contains the same sample rate.

Select Data Source and View Source Variables

Initiate the import process and select workspace variables to import. In the example figure, the data source is a table that contains Vibration and Tacho variables. However, these variables contain only measurement data and no time information. As always, the app provides a Sample (Virtual) option. In this case, since there is no IV in the data, the app automatically selects this variable.

View Virtual IV Properties

The default unit for a virtual IV is the sample index. You can modify this default setting by selecting the Sample (Virtual) name, which opens the source properties. The following example figure displays the default properties.

Reconstruct Signal Time Variable Using Sample Time

If you know the sample time of the signal, you can reconstruct the time variable. To do so, change Independent variable name to the name you want, IV type to Time, Unit to the time units you want, and Sampling interval to the sample time. For example, consider that you know the sample time for both Vibration and Tacho is 0.001 seconds. The following figure shows how to set this sample time for Vibration. Note that these settings do not affect Tacho.

Once you have reconstructed the IVs that you want, complete the import process. You can view your reconstructed timelines by plotting your imported signals in the app. The following figure shows plots for Vibration, which has the reconstructed timeline, and Tacho, which retains the default IV of Sample.

Specify Sample Index as Alternative IV

This workflow describes the steps for specifying the signal sample index as an alternative IV when you also import a time variable or some other signal IV.

Specify Sample Index as IV

Initiate the import and select the data to import. In the Source variables section, select Sample (Virtual) and view the properties. The following example figure illustrates this step. In this figure, Vibration now has all three lower level variables selected. The Source variable properties section displays the default IV type and unit for Sample (Virtual), which are Index and samples, respectively. Note that the figure also shows a checked selection box for Vibration but a filled selection box for Tacho. This difference is because Tacho contains a lower level variable that is not selected.

Perform the same operation on all variables for which you want to include the sample index. Complete the import process.

Switch to Sample Index the App

The app defaults to the IV type that you imported with the data. To switch to the sample index, in Computation Options, select Index, as the example figure illustrates.

All signals that include Index as an alternative IV type now use the sample indices rather than the time values. When you plot the data, the signal trace uses the sample data, as the following example figure shows.

Import Ensemble Datastore

This workflow describes the steps for importing a fileEnsembleDatastore object or a simulationEnsembleDatastore object. Ensemble datastore objects provide information that allow the app to interact with external files. They include specifications for the variables you want to read, the variable types, and the source file locations. When you import an ensemble datastore, you can choose whether to store results within the app memory or write the results back to the ensemble datastore. For more information about ensemble datastores, see Data Ensembles for Condition Monitoring and Predictive Maintenance.

Select Data Source and View Source Variables

Initiate the import process. In the Select more variables pane, select the ensemble datastore. The app uses the ensemble datastore properties for SelectedVariables to select the variables to display. The app also uses the DataVariables, IndependentVariables, and ConditionVariables properties to determine which variables belong to which of these variable types. The example figure illustrates the import of a simulationEnsembleDatastore object ens.

In the preceding example figure, the app interprets ens as follows:

  • The data variables Flow and Pressure appear identical in form to timetable-based variables extracted from tables.

  • ens includes the standard simulationEnsembleDatastore variable SimulationInput in the SelectedVariables property. However, the app does not support the SimulationInput data format and displays an orange warning icon. The app also automatically clears the selection and deletes SimulationInput from ens.SelectedVariables.

  • CombinedFlag appears as a condition variable in accordance with ens.ConditionVariables.

Choose How App Interacts with External Files

You can choose whether the app interacts with the external files referenced in your ensemble datastore. In the Select more variables pane, use Append data to file ensemble to specify your choice.

  • If you select this option, the app interacts directly with the external files and writes results, such as derived variables or features, to the same folder as the original data. If you are using a fileEnsembleDatastore object, the object must include a reference to a write function that is specific to your data structure. You do not need a write function if you are using a simulationEnsembleDatastore object.

  • If you clear this option, the app stores results in local app memory for the duration of the session. Select this option if, for example:

    • You want to keep your source files pristine at least until you have finalized your processing and feature generation.

    • You do not have write permission for your source files.

    • You do not have a write function, and you are using a fileEnsembleDatastore.

    • The process of writing the results back to the source files is slow.

    To retain your local results at the end of the session, use Save Session. You can also export your results as a table to the MATLAB workspace. From the workspace, you can store the results in the file, or integrate results selectively using ensemble datastore commands.

See Also

| | | | | |

Related Topics