raggles.kiwi

Reverse engineering the innards of System Platform #1 - Packages, Templates, Objects, Primitives and Attributes

In order to try and understand various aspects of System Platform a little better, I have been poking around the inner workings of the software and discovering many interesting things, which I intend to share in a series of posts on the subject.  So here goes with the first one about Packages, Templates, Primitives and Attributes - settle in because it's a long one.

Firstly, we must must realise that the templates, primitives and attributes that we are going to talk about here are not the same as the templates and attributes that we are used to dealing with in the IDE which I will call IDE templates and IDE attributes from now on.

Templates are a base object type, such as $UserDefined, $Symbol, $DiscreteDevice and so on.  This list of templates is in the database in table template_definition, and for example mine looks like this:

template_definition_id	base_gobject_id	original_template_tagname
1	1	$Galaxy
2	3	$_AutoImport
3	4	$_DiCommon
4	5	$WinPlatform
5	6	$AppEngine
6	7	$Area
7	8	$AnalogDevice
8	9	$DDESuiteLinkClient
9	10	$DiscreteDevice
10	11	$InTouchProxy
11	12	$OPCClient
12	13	$RedundantDIObject
13	14	$UserDefined
14	15	$Logic
15	16	$Symbol
16	17	$Layout
17	18	$ScreenProfile
18	19	$ViewEngine
19	20	$InTouchViewApp
20	21	$ViewApp
21	22	$Sequencer
22	23	$ClientControl
23	687	$SQLData
24	691	$DisplayModule
25	1494	$Window
26	1495	$Screen
27	1496	$NavigationModel
28	1497	$ViewApplication
29	1498	$DisplayInstance
30	1499	$IOManager

An object is a template, and anything derived from a template.  Which is pretty much everything.  Objects are stored in the gobject table.  A couple of key columns on this table are:

• template_definition - the template from the template_definition table that this object is derived from
• derived_from_gobject_id - this is the user template (IDE template if you like) that the object is derived from
• tag_name - the object name as seen in the IDE
• is_template - true is the object is a user template

A package is one particular version of an object.  When you deploy an object you are deploying a particular version, and when you create a new version in the IDE, a new package is created.  You can sometimes have several packages for the same object id in the database, but eventually the old ones get cleaned up when they are no longer needed.  Packages are stored in the package table.

Make sense so far?  Good, because that was the easy stuff.  Now things get interesting.

A primitive is a grouping of functionality, and is specific to a template.  For example, there is a primitive for scaling (an analogue value), a primitive for inputs, a primitive for alarms and so on.  Primitives can have child primitives too, and each type of primitive has an entry in the primitive_definition table which describes which template it belongs to, and what its parent primitive id is.  The first few entries in my database look like this:

primitive_definition_id	template_definition_id	mx_primitive_id	parent_mx_primitive_id	primitive_name
1	1	100	2
2	1	2	0
3	1	3	2	ScriptExtension
4	1	4	3	ScriptExtension.ExecutionError
5	1	5	3	ScriptExtension.State
6	1	6	2	InputExtension
7	1	7	2	OutputExtension
8	1	8	2	AlarmExtension
9	1	9	2	HistoryExtension
10	1	10	2	InputOutputExtension
11	1	11	10	InputOutputExtension
12	1	12	2	SymbolExtension
13	1	13	12	SymbolExtension
14	1	14	2	DisplayExtension
15	1	15	14	DisplayExtension
16	1	16	2	AnalogExtension
17	1	17	16	AnalogExtension.AnalogStatistics
18	1	18	16	AnalogExtension.DeviationAlarms
19	1	19	18	AnalogExtension.DeviationAlarms.Minor

A point of note here is that while the primitive_definition_id is unique, the mx_primitive_id is not, and primitives on different templates may have the same id.  Every instance of a primitive is stored in the primitive_instance table.  Some of the key columns of the table are:

• gobject_id - the object that the primitive belongs to
• package_id - the package which the primitive belongs to
• mx_primitve_id
• primitive_definition_id - the type of primitive
• primitive_name - the name of the primitive.  This is usually blank (eg for the root primitive for an object), or the name of the IDE attribute.
• parent_mx_primitive_id - the parent primitive id
• extension_type - eg script extension, scaling extension, input extension
• primitive_attributes- the value of the primitives non dynamic attributes (more on that below)

Each primitive has a set of attributes, of which all or some or none may exist on the object at runtime.  Attributes can be either of the dynamic type, or non dynamic type.  Non dynamic attributes have definitions stored in the database in a similar fashion to primitives, and they are stored in the attribute_definition table.  For example, if we lookup the attributes for the InputExtension in the table above (primitive defition id = 6) then we get the following:

attribute_definition_id	primitive_definition_id	attribute_name	mx_attribute_id	mx_data_type
205	6	ReadStatu	100	9
206	6	InputSource	101	8
207	6	_refAttrKey	102	14
208	6	_ExtensionAttributeDatatypes	103	10
209	6	_ExtensionAttributeCategories	104	5
210	6	_InternalName	2	5
211	6	_ExternalName	1	5

Note that attributes also have unique definition ids, as well as non unique mx_attribute_ids, just like primitives.  They also have a particular data type.  The data type is the integer value of the Mx_DataType enum used in the GRAccess toolkit.  The full list of datatypes is as follows:

MxDataTypeUnknown	-1
MxNoData	0
MxBoolean	1
MxInteger	2
MxFloat	3
MxDouble	4
MxString	5
MxTime	6
MxElapsedTime	7
MxReferenceType	8
MxStatusType	9
MxDataTypeEnum	10
MxSecurityClassificationEnum	11
MxDataQualityType	12
MxQualifiedEnum	13
MxQualifiedStruct	14
MxInternationalizedString	15
MxBigString	16
MxDataTypeEND	17

The value of static attributes is stored in the primitive_instance table, as a binary value.  This binary value is just a list of concatenated tuples of <attribute_id,mx_datatype,value>.  For example, the binary blob for this entry

gobject_id	package_id	mx_primitive_id	primitive_definition_id	primitive_name	extension_type	primitive_attributes
939	952	250	643	Monoblock1.Voltage.PV	scalingextension	0xFA000F000000010005300000002C000000...

can be broken down as per the following image:

The hex view of the data is on the left, and ascii view is on the right. The data is highlighted as follows:

• Yellow: Attribute id
• Green: Data type, note that 0x4x indicates that the variable is an array
• Blue: Metadata, such as string lengths, index (e.g. for arrays and enums)
• Purple: Value

Note that the first value is actually the primitive id rather than an attribute id.

Dynamic attributes are stored individually in the dynamic_attribute table.  The attribute_name value is appended to its parent primitive name to get the full name.  The mx_value field stores the value of the attribute with the data type in the first byte.

So now we know how IDE objects and their attributes are stored in the database, we can probably appreciate why the IDE is so slow, and why galaxy operation in general seem to take such a long time.  Just loading an object from the database will involve a very large number of queries to the database.