Test Report Module

An in-depth description of the Test Report module for the Milli Manufacturing Tools Application

Introduction

When a Milli is integrated with a sensor, two operations are performed on the integrated device:

  • RF Test - This test validates the RF functionality of the Unit Under Test against a reference device called the Known Good Unit.
  • Personalization - This test configures the Milli to be enabled to communicate with the network. Configuration includes setting the Country Code, Network ID, CoAP URI, etc.

 

When these tests are run on many Milli devices in a manufacturing environment, it is important that the result data is organized and easily read. The reporting module of the Milli Manufacturing Tools application gives users the ability to execute these operations and view the details of these operations conducted on the Milli devices.

 

To Generate a Test Report 

  1. Launch the Milli Manufacturing Tools application.
  2. Click the Test Report tab.
     
  3. Click the dropdown under Start Date and select the date from which you want the test to include data.
     
  4. Click the dropdown under End Date and select the date at which you want the test to stop including data.
     
  5. Click the dropdown under Product and select the device you are running the test for.
     
  6. Click Generate Report. A CSV file called TestReport generates. You can save this file on your machine. ​

 

 

DB Schema

This section describes the DB schema defined for storing the test results. The reporting module generates the test report from database entries stored in this schema.

 

Product Table

The following values in report data are retrieved from Product table:

  • ProductID
  • ProductName
  • MachineNumber
  • SoftwareVersion

 

Primary Key: ProductID

 

StaticData Table

This table stores the standard values against which actual test data is compared.

The following values in report data are retrieved from StaticData table:

  • Standard Values
  • UpperLimit
  • LowerLimit

 

Primary Key: HeaderName, ProductID

Foreign Key: ProductID

 

TestResult Table

This table represents all the data retrieved from the tests performed during RF Test and Personalization. Users can analyze it further based on interpretation understanding (Refer to Report Data Interpretation section). When an RF test is performed on Milli units connected, the following columns are populated:

  • MacAddress
  • PacketSend,
  • PacketReceive,
  • RFTestStatus,
  • RSSI_Max,
  • RSSI_Type,
  • RSSI_Min,
  • RFTest_Failure_Reason

 

When Personalization is performed on the Milli units, the following columns are also populated:

  • MacAddress
  • PersonalizeStatus,
  • Personalize_Failure_Reason

 

Primary Key: TestID, MacAddress

Foreign Key: ProductID

 

ER Diagram

The diagram below depicts the entity- relationship diagram between the tables defined in the DB schema of the Reporting Module of the Milli Manufacturing Tools Application.

DB Utils APIs

The DBHelper class implements following APIs which are used to retrieve data fields for RFTest and Personalize operations.

 

AddTestResult(RMATestData objRMAData)

Signature: public static void AddTestResult(RMATestData objRMAData)

This API is used to insert RMA test result into database. It takes DBUtils.RMATestData object as a parameter.

 

AddTestResult(PersonalizeData objPerData)

Signature: public static void AddTestResult(PersonalizeData objPerData)

This API is used to insert personalize result into database. It takes DBUtils.PersonalizeData object as a parameter.

 

RMATestData GetTestDataByMac(string macAddress)

Signature: public static RMATestData GetTestDataByMac(string macAddress)

This API will return last test result of Milli having mac address passed as a parameter.

 

GetProductByProductID(int productID)

Signature: public static Product GetProductByProductID(int productID)

This API will return product details of product id passed in parameter.

 

GetProductList()

Signature: public static List<Product> GetProductList()

This API is used to get all the available products in database.

 

GetLimitsByProductID(int productID)

Signature: public static List<StaticData> GetLimitsByProductID(int productID)

This API returns the Limit details of product id. It takes product id as an argument and returns the standard value, standard upper limit and standard lower limit configured for the product.

 

GetTestResults(DateTime startDate, DateTime endDate, intproductID)

Signature: public static List<RMATestData> GetTestResults(DateTime startDate, DateTime endDate, int productID)

This API returns the test result of RF test and personalization within the specified range and product id.Report Data Interpretation.

In Milli Manufacturing Tools application, RF Test and Personalization is a single process which includes two sub steps:

  • STEP1: RF Test
  • STEP2: Personalization

 

Some guidelines on how these steps work and how data gets populated in the report:

  1. Each Milli unit will have a unique MAC address and multiple iterations for the same milli will be treated as unique entries in the report.
     
  2. If only Step1 is performed on milli, JUDGE column will display ‘STEP2: Personalization’ that means process failed at Step 2 as this step was not performed. Also, STEP2: Personalization column will store a null value. Here, NG count will be incremented by 1.
     
  3. If both Step1 and Step2 were performed and both went fine, JUDGE column will display ‘OK’, count in the OK column will be incremented by 1, and NG column count will remain same.
     
  4. If Step1 is interrupted in between and it failed, JUDGE column will display ‘Step1’ that means process failed at Step1 as this step was not completed. Also, STEP1: RF Test column will store a null value. Here, NG count will be incremented by 1.

 

Test Report

The final generated test report returns defintions for multiple values. The categories are described below. 

 

Report Format

Column Value
Significance
Time
Timestamp for each test step
TOTAL
Total iterations indicating the number of runs for STEP1 and STEP2
OK
Total count of successfull runs for STEP1 and STEP2
NG
The total number of failed steps
JUDGE
First failure is reported here. If both steps are successful, the value will be "OK"
MAC Address
Mac Address for the connected Milli unit. 
STEP1
RF Test.
OK = STEP1 completed successfully 
NG = STEP1 failed
NULL = STEP1 is not performed at all 
Packet Send
Number of packets sent to the Unit Under Test
Packed Received
Number of packets received from the Unit Under Test
RSSI Max[dBm]
Maximum value for reiceved signal strength indication (in dBm)
RSSmin[dBm]
Minimum value for received signal strength indication (in dBm)
STEP2
Personalization
OK = STEP2 completed successfully 
NG = STEP2 failed
Blank = STEP2 not performed at all

 

Sample Queries

The report fields described above represent one possible rendering of a report.  The data was compiled using the queries described below. The queries shown here are examples of how one can extract various results from the database for processing in other formats or for extraction and insertion into a different database, etc.

For the following samples, MAC Address = MAC Address of the Milli unit connected

 

Retrieve Test Results for Specific Mac Address 

select MacAddress, SerialNumber, pr.ProductName, pr.MachineNumber, pr.SoftwareVersion,

[DateTime], RFTestStatus, PacketSend, PacketReceive, RSSI_Max, RSSI_Min, RSSI_Type, RFTest_Faliure_Reason, PersonalizeStatus, Personalize_Faliure_Reason 

from TestResult TR 

inner JOIN Product PR on tr.ProductID = PR.ProductID

where MacAddress = '<MAC Address>';

 

Retreive Test Results for Specific Mac Address and Specific Date Range .

select MacAddress, SerialNumber, pr.ProductName, pr.MachineNumber, pr.SoftwareVersion

[DateTime], RFTestStatus, PacketSend, PacketReceive, RSSI_Max, RSSI_Min, RSSI_Type, RFTest_Faliure_Reason, PersonalizeStatus, Personalize_Faliure_Reason 

from TestResult TR  

inner JOIN Product PR on TR.ProductID = PR.ProductID

where MacAddress = '<MAC Address>' and [DateTime] between '2018-09-04' and '2018-09-15';
 

The End Date value should be one greater than the date value for which records need to be retrieved.
 

 

Retrieve Test Results for Specific Date range

select MacAddress, SerialNumber, pr.ProductName, pr.MachineNumber, pr.SoftwareVersion

[DateTime], RFTestStatus, PacketSend, PacketReceive, RSSI_Max, RSSI_Min, RSSI_Type, RFTest_Faliure_Reason, PersonalizeStatus, Personalize_Faliure_Reason 

from TestResult tr 

inner JOIN Product pr on tr.ProductID = pr.ProductID

where [DateTime] between '2018-09-04' and '2018-09-15';