Table of Contents


avg2ddt - convert an ERP average file to its derivative (d/dt)


avg2ddt inputfile outputfile [-d pp10units] [-v | -verbose]


avg2ddt calculates the derivative (d/dt) of the data in the input ERP average file inputfile and writes the result to outputfile. A derivative can be though of as how much a quantity is changing at any given point, and for the ERP data it is calculated point-by-point for each epoch by taking the slope from the previous data point to the current data point. outputfile has the same format as an ERP average file and can be used with other ERP programs as would an average file, but it must be kept in mind that the data contained in the file are no longer electric potential values in units of microvolts. avg2ddt may be run multiple times to generate first, second, third, etc. derivatives, but since the data increase in scale in each derivative only the first and second derivatives may be practical due to inherent limits in the ERP software.

avg2ddt sets the data type of outputfile, and conceptually changes the units of the data contained in the file, as follows:

if the input file is "average" (i.e., created with the avg(1) program), the output file is set to "ddt" and the output data values are first derivative of electric potential, in uV/s;

if the input file is "ddt" the output file is set to "ddt2" and the output data values are second derivative of electric potential, in uV/s/s;

if the input file is "csd" (i.e., created with the avg2csd(1) program), the output file is set to "csd-ddt" and the output data values are first derivative of current source density, in uA/m2/s (microamps per square meter per second);

if the input file is "csd-ddt", the output file is set to "csd-ddt2" and the output data values are second derivative of current source density, in uA/m2/s/s.

The data in outputfile can be plotted with ploterps(1) but almost certainly they will need smoothing or filtering to be viewable. Smoothing or filtering can be done as the data are being displayed using the -smooth or -filter options to ploterps(1) , or a separate file containing filtered data can be created from outputfile using the filter(1) program. See the EXAMPLES section for typical examples of filtering. Also, due to the increased scale of the output data, ploterps(1) will need the -cal and -scale options set appropriately when viewing the data. Typical values would be -cal 50 and -scale 100 for first derivative data and -cal 500 and -scale 1000 for second derivative data (note that ploterps(1) should set these parameters automatically when using the default configuration files, e.g. chan5, 26chan).

Data overflow must be considered when running avg2ddt. The derivative calculation increases the size of the data values, which can result in data overflow (i.e. values too large to fit their storage type). All overflow is detected and somewhat corrected by replacing the overflow value with the maximum valid value for the storage type (note this correction is merely the lesser of two evils - the maximum valid value is better than an invalid, potentially sign-changed, overflow value but it is still an incorrect value). Each overflow results in a warning message that indicates the bin number, channel number, and sample number associated with the overflow, as well as the magnitude of the overflow. A count of the number of overflows for each bin is also recorded in the bin headers of outputfile, so alternately the user can detect overflow by running headinfo(1) on outputfile. The user should inspect these messages (possibly plotting the associated data areas) to determine if the overflow is a problem. Some overflow may be acceptable, for example the overflow related to the steep edges of calibration pulses. Usually it is best to reduce or eliminate overflow errors by decreasing the resolution of the output data (see the -d option).


-d pp10units
This option specifies the resolution to be used for the data in outputfile, in points per 10 units (e.g. points per 10 uV/s if the file is of type "ddt"), and can be used to reduce overflow errors.

The data values are multiplied by the resolution (specifically, 1/10th of the pp10units value) to convert them to the values stored in outputfile, so a decrease in resolution results in a decrease in the size of the values being stored, which effectively reduces the chance of overflow.

If problematic overflow is occurring, check the resolution of the input file (using the headinfo(1) program for instance) and rerun avg2ddt with the -d option, specifying a resolution less than that of the input file. To avoid an unnecessary loss of information it is best to reduce the resolution only as much as needed; for example, if the input file has a resolution of 1000 points per 10 units, try specifying -d 900 and see how much the overflow is reduced; if it isn’t enough, try -d 800, etc.

The default is to set the output file to the same resolution as the input file.

-v | -verbose
This option specifies to echo the command line arguments and options and print the derivative calculations for each data point. It generates a LOT of output, so when using it standard output should be managed somehow: possibly piped to grep to select only the lines of interest; or redirected to a file. The file will probably be too big for a text editor to handle so it may be best to first more the file to get a sense of what it contains, then grep it for the bins/channels/samples of interest (and afterwards, remove the file when it is no longer needed to conserve disk space).


To help visualize how a waveform compares to its derivative, you can run avg2ddt on the file /usr/local/erp/lib/testwaves.avg, which is a mock average file comprised of bins containing simple test functions such as square waves, sine waves, etc. This example compares a sawtooth wave (bin 3) to its derivative (note the overflow errors reported by avg2ddt for the square waves in bin 0, which are due to the near-vertical rising and falling edges (i.e. very large slopes)):

avg2ddt /usr/local/erp/lib/testwaves.avg testwaves.ddt
ploterps xwin 26chan -files /usr/local/erp/lib/testwaves.avg -bins 3
ploterps xwin 26chan -files testwaves.ddt -bins 3

To create a derivative of a grand average, and compare bin 7 of the average to its derivative:

avg2ddt conaud.gnd conaud.ddt
ploterps xwin 26chan -files conaud.gnd -bins 7
ploterps xwin 26chan -files conaud.ddt -bins 7 -filter lp 5

To create a separate filtered file of the derivative, and view bin 7 of the derivative using the -nobaseline option to ploterps to maintain the intrinsic positivity and negativity:

filter lp conaud.ddt conaud.ddt.lp5 5
ploterps xwin 26chan -files conaud.ddt.lp5 -bins 7 -nobaseline

To create and view the first and second derivatives of bin 5 of a grand average, decreasing the resolution to reduce overflow errors:

avg2ddt clex.gnd clex.ddt -d 500
filter lp clex.ddt clex.ddt.lp5 5
avg2ddt clex.ddt.lp5 clex.ddt.lp5.ddt -d 100
ploterps xwin 26chan -files clex.gnd -bins 5
ploterps xwin 26chan -files clex.ddt.lp5 -bins 5
ploterps xwin 26chan -files clex.ddt.lp5.ddt -bins 5

To create and view the derivative of a current source density average:

avg2csd clex.gnd clex.csd 
filter lp clex.csd clex.csd.lp10 10
avg2ddt clex.csd.lp10 clex.csd.lp10.ddt -d 200
filter lp clex.csd.lp10.ddt clex.csd.lp10.ddt.lp5 5
ploterps xwin 26chan -files clex.gnd -bins 5
ploterps xwin 26chan -files clex.csd.lp10 -bins 5
ploterps xwin 26chan -files clex.csd.lp10.ddt.lp5 -bins 5



See Also

avg(1) , avg2csd(1) , filter(1) , headinfo(1) , normerp(1) , ploterps(1) for an interactive graph that helps in visualizing a waveform and its derivative.


Paul Krewski

Table of Contents