NT Data Logging

The information on this page refers to firmware v2.59e and higher.

A data logger, the NT Logger, can be connected to the NT bus, which allows us to record on a micro SD card a substantial amount of data. The data are recorded for each cycle of the controller loop, i.e., with the maximal possible time resolution. The NT Logger is quite small and can be easily mounted on any vehicle. This way one can get highly accurate and informative data taken during a real flight condition. The same data can also be recorded on the ground, without the need of a NT Logger, using the live-recording feature.

The data logging option allows us to diagnose any in-flight troubles, such as jello or micro vibrations in the video, and find solutions to resolve them. It also suggests many other applications. For instance, one may use it to dynamically balance the motors and propellers.

Comment: The data logger must be a NT Logger; any other logger, such as the widely available OpenLog and its derivatives, cannot be used.

NT Logger Modules

Micro NT Logger Module v1.32

The Micro NT Logger module has the RTC integrated and uses a replaceable battery cell.

NT Logger Module v1.3

This NT Logger module can be extended by connecting a RTC (real time clock) module to its I2C port. Any RTC module which is based on the DS3231 chip can be used. However, the "Raspberry Pi Mini RTC module" is most recommended, because of its low cost and tiny size. In fact, the NT Logger v1.3 PCB is designed specifically for that module.

Comment: The battery on these RTC modules lasts for about 1.5 years, but is not chargeable. That means that after that time the RTC will either not work anymore or needs to be replaced, which is quite a pain to do.

SD Card

The micro SD card must be formatted with FAT32. The firmware author uses the Windows tool. Using the SD Association formatting tool didn't yielded any noticeable advantage.

The micro SD card should be of very high speed. Unfortunately, not any card works equally well. This experience is available:

micro SD card	experience	reported by
SanDisk Extreme Micro SDHC 32GB Class 10 UHS-I UHS-Class 3 V30 A1-L (SDSQXAF-032G-GN6MA)	excellent, nearly no missing frames	OlliW
SanDisk Extreme Micro SDHC 16GB UHS-I Class 10 U3	very good, missing frames of ca. 0.1%	OlliW, GekoCH
SanDisk Ultra Micro SDHC 16GB UHS-I Class 10 HC1	very good, missing frames below 0.1%	OlliW
Kingston 8GB HC class 4	bad, missing frames of ca. 20%	OlliW
Kingston 32GB HC class 4	bad, missing frames of ca. 20%	OlliW

RTC

The NT Logger module can be equipped with a RTC (real time clock) module to its I2C port, and the Micro NT Logger has it already on board. This makes it possible to append the data file names with a precise date&time stamp, which makes identifying the data files on the SD card much simpler. This little feature is in fact so useful, that one wouldn't want to miss it once one has used it once :).

The RTC module must be configured before it can be properly used, i.e., the current date and time must be written to it at least once. This can be conveniently done via the GUI: Go to the [GUI:Tools] menu and call the [GUI:NTLogger RTC Tool].

NTLoggerTool

For analyzing the recorded data, the NTLoggerTool is available, providing tools such as FFT, and others. The NTLoggerTool also allows us to record directly the data on the NT bus using a USB-TTL adapter; a feature called live-recording. It enables many work-bench tests and procedures.

The NTLoggerTool is steadily advancing and any manual would be quickly outdated. For a discussion it is best to join the STorM32 NT thread at rcgroups.

The NTLoggerTool is open source (GPL3). Anyone is highly welcome to contribute to it.

Logged Data

The NT Logger "spies" the STorM32's NT Tx line and records all data transmitted from the STorM32 to the NT modules.

Controller

The points in the controller loop at which data are "sampled" are indicated in this sketch:

This is however not all data recorded by the NT Logger; it records many more.

Autopilot System Time

When BetaCopter/BetaPlane is used and a MAVLink connection is established, then the autopilot's system time is recorded in the NT logs (if the autopilot can provide this info, e.g., if a GPS is available and has a fix). This is highly useful for matching NT logs and autopilot logs.

GUI Settings

The actual data which are emitted by the STorM32 controller can be adjusted with the parameter NT Logging, found in the [GUI:Setup] window. It is a bit mask, and allows combining these options:

The individual options enable these data packets:

NT Logging	data
“Basic”	SetLogger, CmdAhrs, SetMotorAll, CmdEncoderData, CmdStorm32LinkData
“Acc + Gyro raw”	CmdAccGyroRaw for Imu1 & Imu2
“PID In”	CmdPidIn
“PID Out”	CmdPid
“Acc + Gyro”	CmdAccGyro for Imu1 & Imu2
“Inputs”	CmdFunctionInputValues
“PID I D”	CmdPidID
“STorM32 Link”	CmdStorm32Link

It can happen that not all data can be transmitted in one cycle, if the computational load in that cycle is too high to support the full data rate. This typically occurs with both 2nd IMU and full data logging enabled. In that case some data might be simply missing in the log file.

Also the data rate of the micro SD card represents an obvious limitation. If the SD card cannot keep up, frames (the data of one controller cycle) will be lost. Lowering the logging can alleviate that a lot. With modern state-of-the-art high-speed SD cards this is not much of an issue however.

For vibration analysis, the experience so far shows that the “acc + gyro raw” and “pid in” data are most useful. It is thus recommended to have these enabled. The “pid” and “acc + gyro” data are comparatively less useful for that purpose.

For analysing the stabilization performance of the gimbal, the “pid in” data are by far most useful, as it gives access to the PID error signals. It is thus recommended to have these always enabled.

Live Recording

It is also possible to "spy" on the STorM32's NT Tx line using a USB-TTL adapter, and record the data using the NTLoggerTool. The feature is somewhat analogous to the DataDisplay in the STorM32 GUI, except that it records much more data and moreover at full speed, with maximal time resolution.

The USB-TTL adapter must be capable of handling 2.000.000 bauds. Adapters with FT232RL (FTDI) work out of the box. Adapters with CP2102 may be configured to do so, see How to configure CP2102 USB adapters for high baud rates. The firmware author has much better results with a CP2102-based adapter. His FT232RL adapters appear to have problems with keeping up with the high data rate (it's about 1Mb/s), yielding many data gaps, while with a CP2102 adapter no data are lost.

The USB-TTL adapter is connected to the NT bus as follows:

USB-TTL adapter GND -> STorM32 NT GND
USB-TTL adapter Rx  -> STorM32 NT Tx
USB-TTL adapter Tx  -> do not connect

The USB-TTL adapter's Tx line must not be connected to the STorM32's NT Rx line, i.e., must be left floating.

Data Fields

Depending on the configuration, i.e., the NT Logging setting (see GUI Settings) and if it is a NT or T-STorM32 build, some data fields may not be available. In this case their values are set to zero.

Performance:
Various metrics measuring the execution time of some parts of the code.

Imu1 Pitch,Roll,Yaw:
Angles of IMU1, or the camera, respectively, as also visible in the GUI. These are the angles as they are coming out of the IMU1 AHRS and enter the PID and axis coupling calculations.

Imu2 Pitch,Roll,Yaw:
Angles of the 2nd IMU (IMU2), as also visible in the GUI. These are the angles as they are coming out of the IMU2 AHRS and enter the PID and axis coupling calculations.
(only for NT builds with 2nd IMU)

Encoder Pitch,Roll,Yaw:
Encoder angles, as also visible in the GUI.
(only for T-STorM32 builds)

PID Error Pitch,Roll,Yaw:
PID controller error input angles (= setpoint - feedback).

PID SetP Pitch,Roll,Yaw:
PID controller setpoint angles.

PID P I D Pitch,Roll,Yaw:
PID controller internal values of the P, I, and D term.

PID Pitch,Roll,Yaw:
PID controller output, probed directly after the PID controller.

Ahrs1:
Various metrics related to the IMU1 AHRS.

State:
State of the controller, as also visible in the GUI. 0 = StrtMotors, 1 = Settle, 2 = Calibrate, 3 = Level, 4 = MotorDir, 5 = Relevel, 6 = Normal, 7 = FastLevel

Error:
Error counts on the NT bus, as also visible in the GUI.

Voltage:
Battery voltage, as also visible in the GUI.

STorM32 Link:
Various data relevant to STorM32 Link:
- q0, q1, q3, q4: attitude quaternion received from flight controller
- vx, vy, vz: vehicle velocity received from flight controller
- SLStatus: status of the STorM32 Link processing

Acc1, Gyro1:
Accelerometer and gyroscope measurements of IMU1. These are the data as they are coming out after the calibration, bias, filter, and orientation corrections, and enter the IMU1 AHRS.

Acc2, Gyro2:
Accelerometer and gyroscope measurements of IMU2. These are the data as they are coming out after the calibration, bias, and orientation corrections, and enter the IMU2 AHRS.

Acc1 raw, Gyro1 raw:
These are the raw accelerometer and gyroscope data, as they are coming out of IMU1, before any other processing. Note that therefore the x,y,z axes refer to the orientation of the IMU1 chip, and not the STorM32's x,y,z axes from which the angles are derived, as the orientation correction has not been passed by these data. It is however possible to apply an orientation correction to compensate for this by using the [Imu Raw Data Rotation] options in the NTLoggerTool.

Acc2 raw, Gyro2 raw:
These are the raw accelerometer and gyroscope data, as they are coming out of IMU2, before any other processing. Note that therefore the x,y,z axes refer to the orientation of the IMU2 chip, and not the STorM32's x,y,z axes from which the angles are derived, as the orientation correction has not been passed by these data. It is however possible to apply an orientation correction to compensate for this by using the [Imu Raw Data Rotation] options in the NTLoggerTool.
(only for NT builds with 2nd IMU)

Temp 1+2:
Temperatures of IMU1, IMU2.

Mot Pitch,Roll,Yaw:
Motor angles as they are send to the motor drivers.

Vmax Pitch,Roll,Yaw:
Vmax for each motor axis (can differ from the Vmax values set in the GUI when voltage compensation is enabled).
(only for NT builds with 2nd IMU)

dU Pitch,Roll,Yaw:
dU for each motor axis.
(only for T-STorM32 builds)

Encoder raw:
Unitless, raw encoder values, their range is +-2^14.
(only for T-STorM32 builds)

dU raw:
Unitless, raw dU values, their range is +-2^15.
(only for T-STorM32 builds)

Inputs:
Inputs for all functions.

Camera:
Camera data flow.