HLA CONFORMANCE GUIDE

 

Version 1.3 (draft 1)

 

 

 

15 April 1998

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Prepared by:

 

Margaret L. Loper and Margaret M. Horst

Georgia Tech Research Institute

Georgia Institute of Technology

Atlanta, GA 30332-0832

margaret.loper@gtri.gatech.edu, margaret.horst@gtri.gatech.edu

Table of Contents

 

1. OVERVIEW *

1.1 Overview of the Conformance Process *

1.2 HLA Compliance Checklist *

2. Test Process *

2.1 SOM Conformance Testing *

2.1.1 Assumptions *

2.1.2 Parseability *

2.1.3 Completeness and Consistency *

2.2 Conformance Cross-Check Testing *

2.3 Interface Testing *

2.3.1 Assumptions *

2.3.2 Nominal Test Data *

2.3.3 Representative SOM Test Data *

2.3.4 IF Test Sequence *

3. Test Instructions *

3.1 Conformance Notebook *

3.2 Test Setup *

3.2.1 Object Model Test Set-Up *

3.2.2 Interface Test Set-Up *

4. Certification Summary Report *

5. References *

 

1. OVERVIEW

This guide contains the information needed by a federate to execute the HLA Conformance Tests.

1. Overview of the Conformance Process

Conformance is the process of verifying that an implementation performs in accordance with a particular standard [1]. HLA Conformance testing ensures that a federate performs in accordance with the Interface Specification (If Spec) [2] and the Object Model Template (OMT) [3] standards, per the HLA Compliance Checklist [4]. In order to determine if a federate conforms to the IF Spec and the OMT, a series of Conformance Tests are conducted. This process is shown in Figure 1.

 



 

Figure 1: HLA Compliance Test Process

 

 

The HLA Compliance Test process consists of four steps. In Step 1, the developers of a federate request information on the test process from the official Certification Agent (CA) by completing an HLA test application on the World Wide Web at <http://hlatest.msosa.dmso.mil>. The CA will check the official compliance database to determine the federate’s priority for Compliance Testing and, if approved, will respond with a user ID and password for conducting the test. It is important to note that the test process is initiated by the federate developer, not the CA, and it is the responsibility of the federate developer to ensure that the federate under test (FUT) represents a stable, mature release of code. Ideally, the test process should be initiated late in beta testing, so that the actual tests are performed on the release version of the code.

 

In Step 2 of the test process, the federate developer submits the Conformance Notebook, which includes the Simulation Object Model (SOM), the simulation Conformance Statement (CS), and optional Scenario Data, which is described further in Section 3.1 below. The CA checks the SOM for conformance to the OMT ("SOM Conformance Test") as described in Section 2.1 and, if successful, checks the SOM against the CS for consistency ("Conformance Cross-Check") as described in Section 2.2. Test results are then returned to the federate developer.

 

Assuming that the FUT successfully passes the SOM Conformance Test and Conformance Cross-Check, the CA also returns a Test Sequence to the federate developer for Interface (IF) Testing. The Test Sequence will be based on the Scenario Data submitted with the Conformance Notebook, if available. If the federate developer chooses not to submit Scenario Data with the Conformance Notebook, the CA will arbitrarily create a Test Sequence based on the SOM and CS. The Test Sequence is described in Section 2.3 below. The CA will propose a date and time for IF Testing based on other testing commitments in a schedule maintained by the CA.

 

In Step 3 of the test process, the federate developer will review the Test Sequence generated by the CA and submit Test Environment Data to the CA. The required Test Environment Data is described in Section 3.2. The federate developer and the CA confirm a test date and time. The FUT must use Runtime Infrastructure (RTI) version 1.0.2 or higher and must be prepared to conduct the Test Sequence multiple times.

 

In Step 4 of the test process, the IF Test (described in Section 2.3) is executed by the federate developer and the Certification Agent. The IF Test has two parts: the Nominal Test, which ensures that the FUT can invoke and respond to all services for which it is capable, per its CS; and the Representative SOM (RepSOM) Test, which ensures that the FUT is capable of invoking and responding to services using the range of data contained in its SOM. The CA will log service data from the test, analyze the data, generate results, and return a Certification Summary Report (CSR) to the federate developer. The CSR, described in Section 3.3, is the official record of HLA compliance for the specific version of the federate code tested.

2. HLA Compliance Checklist

The HLA Compliance Checklist [4] defines the requirements for a federate to be HLA compliant. Table 1 shows the mapping of the Compliance Checklist to the SOM Conformance, Conformance Cross-Check and IF Tests. The mapping demonstrates that the conformance process supports the checklist and that all Federate Compliance Checklist items are accounted for in the conformance process.

 

 

COMPLIANCE CHECKLIST ITEM	TEST
1. The federate shall have an HLA Simulation Object Model (SOM), documented in accordance with the HLA Object Model Template.	SOM Conformance Test
2. Federates shall be able to update and/or reflect any attributes of objects in their SOM and send and/or receive SOM object interactions externally, as specified in their SOM.	Conformance Cross-Check IF Test: Nominal IF Test: Representative SOM
3. Federates shall be able to transfer and/or accept ownership of attributes dynamically during a federation execution, as specified in their SOM.	Conformance Cross-Check IF Test: Nominal IF Test: Representative SOM
4. Federates shall be able to vary the conditions (e.g., threshold) under which they provide updates of attributes of objects, as specified in their SOM.	Conformance Cross-Check IF Test: Representative SOM
5. Federates shall be able to manage local time in a way that will allow them to coordinate data exchange with other members of a federation.	IF Test: Nominal
6. During a federation execution, federates shall interact with the Runtime Infrastructure (RTI) in accordance with the HLA Interface Specification.	IF Test: Nominal

 

Table 1: Compliance Checklist vs. Conformance Test Reference

 

2. Test Process

The conformance processes for the Object Model Template (OMT) and Interface Specification (IF Spec) are distinct but related. For each standard, a Federate Under Test (FUT) must complete several distinct tests. Then, to test the rules, a Consistency Test is performed between the OMT and IF. These tests are described below.

1. SOM Conformance Testing

In accordance with Federate Compliance Checklist Item 1, a federate must have a Simulation Object Model (SOM) in the OMT format. In order to facilitate automated testing, the SOM will be tested using the Data Interchange Format (DIF) [9]. The DIF is a standard file exchange format used to store and transfer object models between OMT development tools. The SOM Conformance Test process has three parts:

 

Parseability: to ensure that the SOM DIF file is readable

Completeness: to ensure that all required data in tables have been completed

Consistency: to ensure that data are consistent across tables

 

The parseability test is not strictly part of conformance, but is a requirement of the automated testing process to ensure that the DIF file is readable. The completeness and consistency checks will be conducted in accordance with the OMT Test Procedures [5].

 

The SOM Conformance Test process is shown in Figure 2.

 

 



 

Figure 2: SOM Conformance Test Process

 

1. Assumptions

This section states the assumptions made by the SOM Conformance Test process in order to conduct the test:

 

This test does not ensure that the data contained in the SOM is valid, only that it is complete and consistent.

2. Parseability

The parseability tests ensure that the Object Model conforms to the DIF standard [9]. Because of the specificity of the DIF, many potential problems in an object model can be checked simply by parsing the DIF file. The parseability test is completed simply through a parse tool that is programmed to accept only DIF format compliant data.

 

3. Completeness and Consistency

The completeness of an object model ensures that it is self-sufficient. That is, it goes beyond what is specifically required in the DIF format definition to ensure that each element of the object model is fully defined and all dependencies are resolvable. For example, if a class is defined as having a superclass, the completeness checks ensure that the definition of that superclass exists.

 

Consistency checks enforce context-dependent rules on the object model that are not defined in the DIF. The consistency checks are designed to catch logic errors in the definition of an object model. The following sections describe the specific checks for completeness and consistency in the Object Model Development Tool (OMDT) developed under DMSO sponsorship. [6]

1. Object Model

1. Check for the existence of at least one class [REF 3, section 4]

1. Metadata

1. Check for a valid object model name.
2. Check for a valid federation name, which is needed for writing a valid FED file. [REF 10, section A.2]
3. Verify that the required object model metadata has been specified. [REF 3, section 4.1.2]

1. Classes

1. The class name is checked to make sure it conforms to the OMT DIF Specification. [REF 9, section E.1.3]
3. The class name is checked to make sure it does not conflict with another class. [REF 3, section 4.2.2]
4. The number of superclasses is checked to ensure that the class does not inherit from more than one superclass. [REF 3, section 4.2.1]
5. The Publish and Subscribe options are checked depending on the model type of Federation Object Model (FOM) or SOM. [REF 3, section 4.2.2]
6. Class definitions are checked to ensure that each class has its own textual description. [REF 3, section 4]

1. Attributes

1. The attribute name is checked to make sure it conforms to the OMT DIF Specification. [REF 9, section E.1.3]
2. The attribute name is checked to make sure it does not conflict with another attribute in its owning class. [REF 3, section 4.4.2]
3. The attribute name also is checked to make sure it does not conflict with another attribute inherited by its owning class. Subclass attributes are not checked at this point because, as the classes are checked, all superclasses and subclasses will be checked, and any duplicate attribute will be detected in the hierarchy. [REF 3, section 4.4.2]
4. Transferable and Acceptable options are checked. For a SOM, the legitimate combinations are T-Transferable, A-Acceptable, TA, and N-Neither. For a FOM, the legitimate combinations are TA and N. [REF 3, section 4.4.2]
5. Updateable and Reflectable options are checked. For a SOM, the legitimate combinations are U-Updateable, R-Reflectable, and UR. For a FOM, the only legitimate combination is UR. [REF 3, section 4.4.2]
6. Attribute definitions are checked to ensure that each attribute has its own textual description. [REF 3, section 4]

1. Interactions

1. The interaction name is checked to make sure it conforms to the OMT DIF Specification. [REF 9, section E.1.3]
2. The interaction name is checked to make sure it does not conflict with another interaction. [REF 3, section 4.3.2]
3. Initiate, Sense and React options are checked. For a SOM, the legitimate combinations are I-Initiate, S-Sense, R-React, IS and IR. For a FOM, the legitimate combinations are IS and IR. [REF 3, section 4.3.2]
5. Interaction definitions are checked to ensure that each interaction has its own textual description. [REF 3, section 4]

1. Parameters

1. The parameter name is checked to make sure it conforms to the OMT DIF Specification. [REF 9, section E.1.3]
2. The parameter name is checked to make sure it does not conflict with another parameter in its owning class. [REF 3, section 4.5.2]
3. The parameter name also is checked to make sure it does not conflict with another parameter inherited by its owning class. Subclass parameters are not checked at this point because, as the interactions are checked, all superclasses and subclasses will be checked, and any duplicate parameter will be detected in the hierarchy. [REF 3, section 4.5.2]
5. Interaction parameter definitions are checked to ensure that each parameter has its own textual description. [REF 3, section 4]

1. Datatypes

1. The datatype name is checked to make sure it conforms to the OMT DIF Specification. [REF 9, section E.1.3]
2. The datatype name is checked to make sure it does not conflict with another datatype.
3. Enumerated datatypes are checked for the existence of at least on Enumerator.
4. Complex datatypes are checked for the existence of at least one field.

1. Conformance Cross-Check Testing

The Conformance Cross-Check process is illustrated in Figure 3 below. Table 2 contains the Conformance Cross-Check rules, showing each OMT table with its required Interface services, corresponding IF Spec [2] and OMT [3] reference. For this test, the services asserted in the FUT’s Conformance Statement (CS) are matched to services implied by the FUT’s SOM, to determine if the two specifications are consistent.

 



 

Figure 3: Conformance Cross-Check Test Process

 

 

 

OMT Table	IF Service Name	OMT Reference	IF Reference
Object Class
(P)	Publish Object Class	sec 4.2.2, page 8	5.2
	Register Object Instance	sec 4.2.2, page 8	6.2
(S)	Subscribe Object Class Attributes	sec 4.2.2, page 8	5.6
	Discover Object Instance	sec 4.2.2, page 8	6.3
Object Interaction
(I)	Publish Interaction Class	sec 4.3.2, page 15	5.4
	Send Interaction	sec 4.3.2, page 15	6.6
(S)	Subscribe Interaction Class	sec 4.3.2, page 15	5.8
	Receive Interaction	sec 4.3.2, page 15	6.7
(R)	Receive Interaction	sec 4.3.2, page 15	6.7
	Update Attribute Values	sec 4.3.2, page 15	6.4
	Publish Object Class	sec 4.3.2, page 15	5.2
Attribute/Parameter
(T) Active	Unconditional Attribute Ownership Divestiture	sec 4.4.2, page 21	7.2
	Negotiated Attribute Ownership Divestiture	Sec 4.4.2, page 21	7.3
	Attribute Ownership Divestiture Notification†	sec 4.4.2, page 21	7.5
	Update Attribute Values	sec 4.4.2, page 21	6.4
	Publish Object Class	sec 4.4.2, page 21	5.2
(T) Passive	Request Attribute Ownership Release†	sec 4.4.2, page 21	7.10
	Attribute Ownership Acquisition Notification†	sec 4.4.2, page 21	7.6
	Update Attribute Values	sec 4.4.2, page 21	6.4
	Publish Object Class	sec 4.4.2, page 21	5.2
(A) Active	Attribute Ownership Acquisition Notification†	sec 4.4.2, page 21	7.6
	Update Attribute Values	sec 4.4.2, page 21	6.4
(A) Passive	Request Attribute Ownership Assumption†	sec 4.4.2, page 21	7.4
	Attribute Ownership Acquisition Notification†	sec 4.4.2, page 21	7.6
	Update Attribute Values	sec 4.4.2, page 21	6.4
(U) DM services	Update Attribute Values	sec 4.4.2, page 21	6.4
	Publish Object Class	sec 4.4.2, page 21	5.2
(U) DDM services	Create Region	sec 4.4.2, page 21	9.2
	Associate Region for Updates	sec 4.4.2, page 21	9.6
	Request Attribute Value Update with Region		9.13
(R) DM services	Reflect Attribute Values	sec 4.4.2, page 21	6.5
	Subscribe Object Class Attributes	sec 4.4.2, page 21	5.6
(R) DDM services	Subscribe Object Class Attributes with Region	sec 4.4.2, page 21	9.8

 

Table 2: Conformance Cross-Check (v1.3, 12 April 1998)

2. Interface Testing

In accordance with Federate Compliance Checklist Items 2-6, a federate must be capable of supporting the services in the IF Spec and the capabilities specified in its SOM. In this context, capability refers to those services a federate can invoke and/or respond to during a federation execution. The Conformance Statement (CS), shown in Table 3, is a list of all Interface services with their corresponding IF Spec [2] reference, OMT [3] reference, HLA Compliance Checklist [4] reference, and whether the service is optional or mandatory, per the Compliance Checklist.

 

Before beginning Interface Testing, a FUT is required to complete the CS, indicating which services the federate supports and therefore will require testing. The CS will be cross-checked against the SOM DIF to ensure consistency (cf. Conformance Cross-Check Test, Section 2.2 above). Only services specifically asserted in the CS will be tested using the procedures outlined in the IF Spec Test Procedures [7].

 

Two types of input data will be used to verify the correct implementation of the services asserted in the CS. The Nominal Test data ensures that the FUT can invoke and respond to all services for which it is capable, per its CS. The Representative SOM (RepSOM) Test data ensures that the FUT is capable of invoking and responding to services using the range of data contained in its SOM, as suggested by Scenario Data (if supplied by the FUT in its Conformance Notebook).

 

In order to pass the Interface Tests, the federate must execute a Test Sequence (scenario) that proves it can invoke and respond to the services as specified in the Nominal and RepSOM Test data. The Test Sequence can be based on a federate’s existing Scenario Data (if submitted), or it can be generated arbitrarily by the Certification Agent. These tests are described more fully below.

1. Assumptions

This section states the assumptions made by the Interface Test process in order to conduct the test:

 

Federate Interface Testing is not dependent on a federation. However, because Create Federation and Join Federation are mandatory services, the FUT must create and join a federation in order to conduct Federate Testing.

 

In addition to the services a federate supports, it also must accept all RTI-invoked services without adversely affecting its operation.

 

 

 

SERVICE GROUP	SERVICE	IF Ref	OMT Ref	Check List	M/O
Create/Destroy	Create Federation Execution	4.2	None	Item 6	M
	Destroy Federation Execution	4.3	None	Item 6	M
Join/Resign	Join Federation Execution	4.4	None	Item 6	M
	Resign Federation Execution	4.5	None	Item 6	M
Synchronize	Register Federation Synchronization Point	4.6	None	Item 6	O
	Confirm Synchronization Point Registration †	4.7	None	Item 6	O
	Announce Synchronization Point †	4.8	None	Item 6	O
	Synchronization Point Achieved	4.9	None	Item 6	O
	Federation Synchronized †	4.10	None	Item 6	O
Save/Restore	Request Federation Save	4.11	None	Item 6	O
	Initiate Federate Save †	4.12	None	Item 6	O
	Federate Save Begun	4.13	None	Item 6	O
	Federate Save Complete	4.14	None	Item 6	O
	Federation Saved †	4.15	None	Item 6	O
	Request Federation Restore	4.16	None	Item 6	O
	Confirm Federation Restoration Request †	4.17	None	Item 6	O
	Federation Restore Begun †	4.18	None	Item 6	O
	Initiate Federate Restore †	4.19	None	Item 6	O
	Federate Restore Complete	4.20	None	Item 6	O
	Federation Restored †	4.21	None	Item 6	O
Publication	Publish Object Class	5.2	4.2	Item 2	O
and	Unpublish Object Class	5.3	4.2.2	Item 2	O
Subscription	Subscribe Object Class Attributes	5.6	4.2.2	Item 2	O
	Unsubscribe Object Class	5.7	4.2.2	Item 2	O
	Publish Interaction	5.4	4.3.2	Item 2	O
	Unpublish Interaction Class	5.5	4.3.2	Item 2	O
	Subscribe Interaction	5.8	4.3.2	Item 2	O
	Unsubscribe Interaction Class	5.9	4.3.2	Item 2	O
Flow Control	Start Registration For Object Class †	5.10	None	Item 6	O
	Stop Registration For Object Class †	5.11	None	Item 6	O
	Turn Interactions On †	5.12	None	Item 6	O
	Turn Interactions Off †	5.13	None	Item 6	O
Object	Register Object Instance	6.2	4.2.2	Item 2	O
Representation	Discover Object Instance †	6.3	4.2.2	Item 2	O
	Update Attribute Values	6.4	4.4.2	Item 2	O
	Reflect Attribute Values †	6.5	4.4.2	Item 2	O
Object	Send Interaction	6.6	4.3.2	Item 2	O
Interactions	Receive Interaction †	6.7	4.3.2	Item 2	O
Delete/Remove	Delete Object Instance	6.8	None	Item 6	O
Objects	Remove Object Instance †	6.9	None	Item 6	O
	Local Delete Object Instance	6.10	None	Item 6	O
Attribute Transport	Change Attribute Transportation Type	6.11	None	Item 6	O
Interaction Transport	Change Interaction Transportation Type	6.12	None	Item 6	O
Attributes	Attributes In Scope †	6.13	None	Item 6	O
In/out Scope	Attributes Out Of Scope †	6.14	None	Item 6	O
Request/Provide Attribute	Request Attribute Value Update	6.15	None	Item 6	O
Value Update	Provide Attribute Value Update †	6.16	None	Item 6	O
	Turn Updates On For Object Instance †	6.17	None	Item 6	O
	Turn Updates Off For Object Instance †	6.18	None	Item 6	O
Attribute Ownership	Unconditional Attribute Ownership Divestiture	7.2	4.4.2	Item 3	O
	Negotiated Attribute Ownership Divestiture	7.3	4.4.2	Item 3	O
	Request Attribute Ownership Assumption †	7.4	4.4.2	Item 3	O
	Attribute Ownership Divestiture Notification †	7.5	4.4.2	Item 3	O
	Attribute Ownership Acquisition Notification †	7.6	4.4.2	Item 3	O
	Attribute Ownership Acquisition	7.7	4.4.2	Item 3	O
	Attribute Ownership Acquisition If Available	7.8	4.4.2	Item 3	O
	Attribute Ownership Unavailable †	7.9	4.4.2	Item 3	O
	Request Attribute Ownership Release †	7.10	4.4.2	Item 3	O
	Attribute Ownership Release Response	7.11	4.4.2	Item 3	O
	Cancel Negotiated Attribute Ownership Divestiture	7.12	4.4.2	Item 3	O
	Cancel Attribute Ownership Acquisition	7.13	4.4.2	Item 3	O
	Confirm Attribute Ownership Acquisition Cancellation †	7.14	4.4.2	Item 3	O
Ownership Query	Query Attribute Ownership	7.15	None	Item 6	O
	Inform Attribute Ownership †	7.16	None	Item 6	O
	Is Attribute Owned by Federate	7.17	None	Item 6	O
Enable/Disable	Enable Time Regulation	8.2	None	Item 5	O
Time	Time Regulation Enabled †	8.3	None	Item 5	O
	Disable Time Regulation	8.4	None	Item 5	O
	Enable Time Constrained	8.5	None	Item 5	O
	Time Constrained Enabled †	8.6	None	Item 5	O
	Disable Time Constrained	8.7	None	Item 5	O
Advance	Time Advance Request	8.8	None	Item 5	O
Request	Time Advance Request Available	8.9	None	Item 5	O
Next Event	Next Event Request	8.10	None	Item 5	O
	Next Event Request Available	8.11	None	Item 5	O
Flush Queue	Flush Queue Request	8.12	None	Item 5	O
Request	Time Advance Grant †	8.13	None	Item 5	O
Asynchronous Delivery	Enable Asynchronous Delivery	8.14	None	Item 5	O
	Disable Asynchronous Delivery	8.15	None	Item 5	O
Query	Query LBTS	8.16	None	Item 5	O
	Query Federate Time	8.17	None	Item 5	O
	Query Minimum Next Event Time	8.18	None	Item 5	O
Lookahead	Modify Lookahead	8.19	None	Item 5	O
	Query Lookahead	8.20	None	Item 5	O
Retract	Retract	8.21	None	Item 5	O
	Request Retraction †	8.22	None	Item 5	O
Change	Change Attribute Order Type	8.23	None	Item 6	O
Order Type	Change Interaction Order Type	8.24	None	Item 6	O
Region	Create Region	9.2	None	Item 6	O
	Modify Region	9.3	None	Item 6	O
	Delete Region	9.4	None	Item 6	O
Register Object With Region	Register Object Instance With Region	9.5	4.4.2	Item 2	O
Associate/ Unassociate	Associate Region For Updates	9.6	4.4.2	Item 2	O
Region	Unassociate Region For Updates	9.7	4.4.2	Item 2	O
Subscribe Object with	Subscribe Object Class Attributes With Region	9.8	4.4.2	Item 2	O
Region	Unsubscribe Object Class With Region	9.9	4.4.2	Item 2	O
Subscribe Interaction with	Subscribe Interaction Class With Region	9.10	4.4.2	Item 2	O
Region	Unsubscribe Interaction Class With Region	9.11	4.4.2	Item 2	O
Send Interaction With Region	Send Interaction with Region	9.12	4.4.2	Item 2	O
Attribute Updates With Region	Request Attribute Value Update With Region	9.13	None	Item 6	O
Advisory Switches	Enable Class Relevance Advisory Switch	10.23	None	Item 6	O
	Disable Class Relevance Advisory Switch	10.24	None	Item 6	O
	Enable Attribute Relevance Advisory Switch	10.25	None	Item 6	O
	Disable Attribute Relevance Advisory Switch	10.26	None	Item 6	O
	Enable Attribute Scope Advisory Switch	10.27	None	Item 6	O
	Disable Attribute Scope Advisory Switch	10.28	None	Item 6	O
	Enable Interaction Relevance Advisory Switch	10.29	None	Item 6	O
	Disable Interaction Relevance Advisory Switch	10.30	None	Item 6	O

 

Table 3: Federate Conformance Statement
(Interface Specification v1.3, 5 February 1998)


2. Nominal Test Data

The Nominal Test data ensures that the FUT can invoke and respond to all services for which it is capable, per its Conformance Statement (CS). Figure 4 below shows that the CS is compared to the Master Sequence to create the Nominal Sequence of services that the FUT can support. The Master Sequence is a dependency tree of all services defined in the IF Spec [2]. Where true dependencies exist (e.g., Publish before Update), the Master Sequence shows a mandatory ordering. If no dependency exists (e.g., Update and Pause), the Master Sequence ordering is arbitrary.

3. Representative SOM Test Data

The RepSOM Test data ensures that the FUT is capable of invoking and responding to services using the range of data contained in its SOM. For example, a FUT may be capable of representing multiple objects, attributes, and interactions. Rather than attempting an exhaustive test of all combinations of the objects, attributes, and interactions, a subset of the SOM is chosen by the Certification Agent (CA) to represent the range of SOM data. This "logical subset" forms the FUT’s basis for RepSOM Testing.

 

The RepSOM Test data is generated from the SOM and available Scenario Data submitted with the Conformance Notebook, as shown in Figure 4. A logical subset is derived from the SOM by a set of rules. These rules select one to three instances from each object, attribute, and interaction table. The RepSOM Test data requires that the federate invoke the IF services (specified in the Conformance Cross-Check Test, Section 2.2 above) associated with each OMT table for each instance requested.

 


4. IF Test Sequence

After the FUT has successfully passed both the SOM Conformance and Conformance Cross-Check Tests, the IF Test Sequence is generated by the CA and provided to the federate developer, along with the RepSOM Test data. The final Test Sequence is generated by taking the Nominal Sequence and expanding it by the RepSOM, as shown in Figure 4. The FUT should be prepared to execute the Test Sequence multiple times within the Test Federation, as specified by the CA. The CA will log service interactions via Management Object Model (MOM) interaction reports for later analysis and report generation. [2]

 

1. Test Instructions

This section describes how to prepare for the Conformance tests.

1. Conformance Notebook

The Conformance Notebook is the information a FUT brings to the test. It is comprised of three items: the Simulation Object Model (SOM), Conformance Statement (CS), and (optional) Scenario Data, as shown in Figure 5.

 



 

Figure 5: Conformance Notebook

 

The easiest way to develop a SOM is through the DMSO-supplied Object Model Development Tools (OMDT), which include a Consistency Checker for verifying completeness and consistency. The OMDT can automatically output the appropriate .omt Data Interchange Format (DIF) file. However, if the SOM tables were developed manually, they must be converted to an ASCII text DIF file as described in Ref. [2].

 

The CS (shown in Table 3) can be submitted electronically or online. If submitted electronically, the CS should be an ASCII text file that contains the name of each IF service, along with "Yes" or "No," to indicate whether the FUT supports that particular service.

 

CreateFederationExecution Yes

DestroyFederationExecution Yes

JoinFederationExecution Yes

ResignFederationExecution Yes

RegisterFederationSynchronizationPoint No

ConfirmSynchronizationPointRegistration No

etc . . .

 

 

To submit the CS online, go to the main page of the DMSO HLA Federal Conformance Testing Web site, located at <http://hlatest.msosa.dmso.mil/>. Choose Step 2: Provide a Conformance Notebook. Enter your user ID and password, then choose the "Complete CS Online" option. Follow the instructions to fill out the application provided, as shown in Figure 6, then choose the "Submit" option at the bottom of the page.

 



Figure 6: Conformance Statement Application

 

 

The last part of the Conformance Notebook is the Scenario Data, which is optional and not required for Conformance Testing. The purpose of Scenario Data is to help the Certification Agent (CA) develop the RepSOM for the FUT. It is recognized that developing a scenario that satisfies the requirements of the Test Sequence can be both labor-intensive and costly for some federates. It is not the intent of the RepSOM Test to add cost and complexity to Conformance Testing. Therefore, a federate can submit existing Scenario Data in its Conformance Notebook to help the CA develop a RepSOM Test that meets the objectives of the Conformance Tests.

 

Scenario Data is submitted as an Object Model Template (OMT) DIF file containing only those objects, interactions, attributes, and parameters included in the scenario. The CA will evaluate the Scenario DIF file to confirm that the proposed scenario meets the criteria for the RepSOM.

2. Test Setup

The following sections describe the setup procedures for the OMT and IF Tests.

1. Object Model Test Set-Up

Object model testing on the OMT DIF files is conducted using the Aegis OMDT tool [6] as follows:

 

1. Load the DIF file. (File must load without errors.)
2. Open the Consistency Checker (under "Tools") as shown in Figure 7.
3. Set the following options in the Consistency Checker:

General: Enforce Definitions

Validate Metadata

Class:^* Validate PS Settings

Interactions: Verify ISR Settings

Attributes: Verify Unique in Class Tree

Validate TA Settings

Validate UR Settings

Parameters: Verify Unique in Class Tree

4. Execute Consistency Checker.
5. Verify that Consistency Checker Status output is "Model is Valid."

 

Figure 7: OMDT Consistency Checker Default Settings

 

1. Interface Test Set-Up

Prior to the IF Test period, the applicant must have submitted a CS and a SOM for evaluation, which will be used by the testing agency to prepare the Test Sequence and the test log post-processing. On test day, the FUT should be prepared to execute the Test Sequence using version 1.0.2 of the RTI (or higher). The applicant should upload all configuration files relevant to the test.

 

To conduct a test for the Interface Specification (IF Spec), at least two federates are required. The first federate is the FUT, which contains the implementation being tested. The second and subsequent federates are present as required to demonstrate services asserted and to perform logging of the test period. For the purpose of Conformance Testing, a specially configured federate, referred to as the Test Utility, will be present and will perform federation management activities during the test period. The Test Utility will subscribe to Management Object Model (MOM)-level attributes to record service interactions between the FUT and the RTI. The FUT and Test Utility are connected via the RTI, as shown in Figure 8.

 

The test applicant is responsible for execution of all federates required to complete a test period. The applicant also may be required to provide a platform upon which to run the Test Utility.

 

 

 

Figure 8: Interface Test Configuration

 

 

A FUT is expected to perform all its testable activities while joined to an execution with federates for which it was designed. For example, if the FUT is designed to function in a simulation system with others like itself, these are suitable auxiliary federates. If, on the other hand, a FUT is designed to work as a specialized subsystem within a broader framework, the FUT would require a sufficiently sophisticated set of auxiliary federates to provide all necessary input and output. In either case, it is the responsibility of the applicant to ensure that all necessary federates are present and operating during each test period.

 

In Step 3, the FUT submits the Test Environment Data, which lists the RTI Federation Execution (fedex) Host, Application Programmer’s Interface (API), federate hardware, and federate software used. The FUT also submits the .rid and .fed (or equivalent) file used to initialize the RTI. The Test Utility will be configured to operate in the execution with the FUT.

 

During the start of the test period, the Test Utility will be initialized and will join the execution after the FUT has created and joined the federation execution. The Test Utility will initialize and begin logging service invocations via MOM [2] service interaction reports. Upon completion of a scenario, which is used to meet the objectives of the Test Sequence, the logging may be stopped at any time. The CA will immediately label and store the test log for post-processing.

 

The final step of Interface Testing involves the post-processing of the test logs. The Interaction Report Post-Processor (IRPP) tool is designed to reduce and analyze Service Interaction Report Logs (SIRLs) to determine whether each asserted service was demonstrated and whether the classes, interactions, and attributes specified in the RepSOM were demonstrated. Operation of this tool will be described in a separate document. [10]

 

1. Certification Summary Report

Once the Conformance Tests are completed, the Certification Agent will generate a Certification Summary Report, which contains four types of information: federate under test (FUT) Information, Test Configuration, Test Results, and Supporting Information. Each is described below.

 

FUT Information

Name

Version Number

Developer’s Point of Contact for Testing

 

Test Configuration

RTI Federation Executive (fedex) Host

Application Programmer’s Interface (API) Used

Federate Hardware

Federate Software

.fed and .rid Files

 

Test Results

Object Model Tests

Pass/Deficiencies with comments

Conformance Cross-Check Tests

Pass/Deficiencies with comments

Interface Tests

Pass/Deficiencies with comments

 

Supporting Information

SOM (in DIF)

Conformance Statement

Test Sequence

 

2. References

[1] Knightson, K.: OSI Protocol Conformance Testing: IS 9646 Explained, McGraw-Hill, New York, 1993.

[2] Defense Modeling and Simulation Office, High Level Architecture Interface Specification, Version 1.3, 5 February 1998.

[3] Defense Modeling and Simulation Office, High Level Architecture Object Model Template, Version 1.3, 5 February 1998.

[4] Defense Modeling and Simulation Office, High Level Architecture Compliance Checklist, Version 1.3, 10 April 1998.

[5] Defense Modeling and Simulation Office, High Level Architecture Object Model Template Test Procedures, Version 1.3, 6 April 1998.

[6] Aegis Research Corporation, Software Design Specification for OMDT Version 1.3.9, Module Consistency Checker, 18 March 1998.

[7] Defense Modeling and Simulation Office, High Level Architecture Interface Specification Test Procedures, Version 1.3, April 1998.

[8] Turnkey User Guide, Version 1.1, 31 October, 1997.

[9] Annex E OMT Data Interchange Format (DIF), v1.3 Draft 3, 2 February 1998.

[10] Annex A FED Data Interchange Format, V1.3 Draft 8, 30 January 1998.