Reverse engineering the innards of System Platform #1 - Packages, Templates, Objects, Primitives and Attributes

In order to try and understand various aspects of System Platform a little better, I have been poking around the inner workings of the software and discovering many interesting things, which I intend to share in a series of posts on the subject.  So here goes with the first one about Packages, Templates, Primitives and Attributes - settle in because it's a long one.

Firstly, we must must realise that the templates, primitives and attributes that we are going to talk about here are not the same as the templates and attributes that we are used to dealing with in the IDE which I will call IDE templates and IDE attributes from now on.

Templates are a base object type, such as $UserDefined, $Symbol, $DiscreteDevice and so on.  This list of templates is in the database in table template_definition, and for example mine looks like this:

template_definition_id base_gobject_id original_template_tagname
1 1 $Galaxy
2 3 $_AutoImport
3 4 $_DiCommon
4 5 $WinPlatform
5 6 $AppEngine
6 7 $Area
7 8 $AnalogDevice
8 9 $DDESuiteLinkClient
9 10 $DiscreteDevice
10 11 $InTouchProxy
11 12 $OPCClient
12 13 $RedundantDIObject
13 14 $UserDefined
14 15 $Logic
15 16 $Symbol
16 17 $Layout
17 18 $ScreenProfile
18 19 $ViewEngine
19 20 $InTouchViewApp
20 21 $ViewApp
21 22 $Sequencer
22 23 $ClientControl
23 687 $SQLData
24 691 $DisplayModule
25 1494 $Window
26 1495 $Screen
27 1496 $NavigationModel
28 1497 $ViewApplication
29 1498 $DisplayInstance
30 1499 $IOManager

An object is a template, and anything derived from a template.  Which is pretty much everything.  Objects are stored in the gobject table.  A couple of key columns on this table are:

  • template_definition - the template from the template_definition table that this object is derived from
  • derived_from_gobject_id - this is the user template (IDE template if you like) that the object is derived from
  • tag_name - the object name as seen in the IDE
  • is_template - true is the object is a user template

A package is one particular version of an object.  When you deploy an object you are deploying a particular version, and when you create a new version in the IDE, a new package is created.  You can sometimes have several packages for the same object id in the database, but eventually the old ones get cleaned up when they are no longer needed.  Packages are stored in the package table.

Make sense so far?  Good, because that was the easy stuff.  Now things get interesting.

A primitive is a grouping of functionality, and is specific to a template.  For example, there is a primitive for scaling (an analogue value), a primitive for inputs, a primitive for alarms and so on.  Primitives can have child primitives too, and each type of primitive has an entry in the primitive_definition table which describes which template it belongs to, and what its parent primitive id is.  The first few entries in my database look like this:

primitive_definition_id template_definition_id mx_primitive_id parent_mx_primitive_id primitive_name
1 1 100 2
2 1 2 0
3 1 3 2 ScriptExtension
4 1 4 3 ScriptExtension.ExecutionError
5 1 5 3 ScriptExtension.State
6 1 6 2 InputExtension
7 1 7 2 OutputExtension
8 1 8 2 AlarmExtension
9 1 9 2 HistoryExtension
10 1 10 2 InputOutputExtension
11 1 11 10 InputOutputExtension
12 1 12 2 SymbolExtension
13 1 13 12 SymbolExtension
14 1 14 2 DisplayExtension
15 1 15 14 DisplayExtension
16 1 16 2 AnalogExtension
17 1 17 16 AnalogExtension.AnalogStatistics
18 1 18 16 AnalogExtension.DeviationAlarms
19 1 19 18 AnalogExtension.DeviationAlarms.Minor

A point of note here is that while the primitive_definition_id is unique, the mx_primitive_id is not, and primitives on different templates may have the same id.  Every instance of a primitive is stored in the primitive_instance table.  Some of the key columns of the table are:

  • gobject_id - the object that the primitive belongs to
  • package_id - the package which the primitive belongs to
  • mx_primitve_id
  • primitive_definition_id - the type of primitive
  • primitive_name - the name of the primitive.  This is usually blank (eg for the root primitive for an object), or the name of the IDE attribute.
  • parent_mx_primitive_id - the parent primitive id
  • extension_type - eg script extension, scaling extension, input extension
  • primitive_attributes- the value of the primitives non dynamic attributes (more on that below)

Each primitive has a set of attributes, of which all or some or none may exist on the object at runtime.  Attributes can be either of the dynamic type, or non dynamic type.  Non dynamic attributes have definitions stored in the database in a similar fashion to primitives, and they are stored in the attribute_definition table.  For example, if we lookup the attributes for the InputExtension in the table above (primitive defition id = 6) then we get the following:

attribute_definition_id primitive_definition_id attribute_name mx_attribute_id mx_data_type
205 6 ReadStatu 100 9
206 6 InputSource 101 8
207 6 _refAttrKey 102 14
208 6 _ExtensionAttributeDatatypes 103 10
209 6 _ExtensionAttributeCategories 104 5
210 6 _InternalName 2 5
211 6 _ExternalName 1 5

Note that attributes also have unique definition ids, as well as non unique mx_attribute_ids, just like primitives.  They also have a particular data type.  The data type is the integer value of the Mx_DataType enum used in the GRAccess toolkit.  The full list of datatypes is as follows:

MxDataTypeUnknown -1
MxNoData 0
MxBoolean 1
MxInteger 2
MxFloat 3
MxDouble 4
MxString 5
MxTime 6
MxElapsedTime 7
MxReferenceType 8
MxStatusType 9
MxDataTypeEnum 10
MxSecurityClassificationEnum 11
MxDataQualityType 12
MxQualifiedEnum 13
MxQualifiedStruct 14
MxInternationalizedString 15
MxBigString 16
MxDataTypeEND 17

The value of static attributes is stored in the primitive_instance table, as a binary value.  This binary value is just a list of concatenated tuples of <attribute_id,mx_datatype,value>.  For example, the binary blob for this entry

gobject_id package_id mx_primitive_id primitive_definition_id primitive_name extension_type primitive_attributes
939 952 250 643 Monoblock1.Voltage.PV scalingextension 0xFA000F000000010005300000002C000000...

can be broken down as per the following image:

The hex view of the data is on the left, and ascii view is on the right. The data is highlighted as follows:

  • Yellow: Attribute id
  • Green: Data type, note that 0x4x indicates that the variable is an array
  • Blue: Metadata, such as string lengths, index (e.g. for arrays and enums)
  • Purple: Value

Note that the first value is actually the primitive id rather than an attribute id.

Dynamic attributes are stored individually in the dynamic_attribute table.  The attribute_name value is appended to its parent primitive name to get the full name.  The mx_value field stores the value of the attribute with the data type in the first byte.

So now we know how IDE objects and their attributes are stored in the database, we can probably appreciate why the IDE is so slow, and why galaxy operation in general seem to take such a long time.  Just loading an object from the database will involve a very large number of queries to the database.

| June 16th, 2018 | Posted in Reversing, SCADA |

Leave a Reply