04.6.17

Intouch Multimonitor - version two

Following on from my last post on the subject, I have now made my multimonitor application work for arbitrary screen layouts (but still limited to only two monitors).  Additionally, a number of subtle bug fixes have been implemented, such as the windows correctly showing following a NAD update.  Grab the updated package (for InTouch 2017) here.  To set the screen layout, simply select the picture that fits your situation on the settings page:

Multimonitor selection

Multimonitor selection

| Posted in SCADA | 6 Comments »
02.21.17

System Platform 2017 (Omni) Thoughts

This has to be the biggest overhaul System Platform has seen in some time, and there is some awesome new stuff there.  Lets take a look at the major new features.

Wizard based object templates

This is the thing I am most excited about.  I have long wanted an equivalent of 'Interfaces' in a programming language in ArchestrA, and with the addition of 'choices' to the templating system in Omni this allows us to do basically the same thing.  I have been able to reduce the large amount of template sprawl that was occurring in 2014R2SP1 and previous and also reduce the memory footprint of objects by not having unused attributes present.  Of course there is some learning to do with this, but I found it all very natural and didn't need to consult the manual at all.

New ViewApp container

This looks like it is going in the right direction.  Flexible layouts, pan and zoom, resolution independence and lots of other good stuff.  I have only had a brief play with it, but it certainly looks like a good start although it has a long way to go before it will be ready for use in a complex production environment.

Historian security improvements

Lots of work done here to make sure that the sql accounts created for historian are no longer using default passwords, and the unnecessary creation of accounts is no longer.  Might seem like an unimportant detail, but I think it's great as it saves a lot of hassle afterwards if you are security conscious.

Preserve runtime changes

This is a super handy feature that allows the runtime changes to be preserved when deploying changes.  It means that you don't have to go through the hassle of writing values out to disk, or remembering to do an 'upload runtime changes' before doing a redeploy.

The Bad

Unfortunately there is some bad news as well.
The new graphics container does not look to me like it will be ready for production, except for the simplest of installations.

  • .NET controls do not work in the current build, and from what I've seen there isn't much priority going into fixing this.  This alone makes the new ViewApp container useless to many.
  • The new Apps (basically a turbo charged .Net control) only work in isolation.  You can't script them, and can't make them interact with anything unless it was part of the controls design.  This too is a deal breaker for many.
  • You can no longer have a modeless popup.  They have crudely sacrificed this function to show graphics in the new ViewApp container instead of creating a new animation for this purpose.
  • UPDATE: It turns out modal popup graphics don't link the symbol wizard options properly either just as they don't work in InTouch!
  • Changes made to linked graphics in templates do not propagate down to existing graphics.  You must reinsert these graphics for them to display correctly.
  • The map app does not support custom tile layers.  Allegedly base layers are supported but I couldn't get it to work.  Custom tile overlay layers are not supported, which is a major oversight in my opinion.

More bad news, is that you can't practically use the new object based templates with InTouch either, because any graphic associated with one of these object will not display correctly if it is directly embedded in InTouch, or shown as a popup.  They only show correctly when shown as a non popup graphic embedded in a parent symbol.  Wonderware have confirmed that this behaviour will not be fixed until a later release of System Platform.

Conclusion

I am sadly forced to conclude, that I will be unable to take advantage of any of the major new features in this release - the view app container isn't up to scratch yet, and the new wizard based objects do not show correctly in InTouch.  At least I can design all my new templates so that they will be ready to go when the time comes that I will be able to use them.

| Posted in SCADA | No Comments »
01.25.17

InTouch/Archestra/SCD5200 Tag Checker

I have recently been working on a project to upgrade our InTouch SCADA to ArchestrA.  This has a number of challenges, but a big one is how to be sure that you got all the right tag names going to the correct IO.  We use DNP and SCD5200 RTUs almost exclusively for large sites, so I have started writing a tool that parses all of our SCD5200 RTU configurations, our InTouch tag database, and the ArchestrA database and produces a spreadsheet with a comparison of all tags and addresses.  It can take also take a map file that maps RTU names to ArchestrA areas to InTouch topic names.  You can then review this report to compare any glaring differences - still a time consuming process but damn site quicker than doing it manually.  The tool is fairly functional at present and parses all my data correctly, but I've no doubt that there are a lot of edge cases not taken care of yet.

Source is on Github.

| Posted in Foxboro, SCADA | 4 Comments »
11.30.16

I like ArchestrA, but...

There are just so many bugs, and annoyances that it makes me want to tear my hair out on a continual basis.  I'm not a full time software engineer, but I code enough to know that so many of these things could be fixed with relatively little effort.  Here is a list of things that really get my goat, and if some Schneider code monkey ever sees this then please feel free to sort them out.  Or, just send me a link to the repo and I'll send you back some pull requests (as if that would ever happen!).

  1.  Opening objects in the IDE is painfully slow, and can only be done one at a time.  If you open objects too quickly one after another then the frame sizes (attribute list and attribute settings) get all messed up.
  2. Buffered attributes are a great concept but they have so many subtle bugs associated with them that they are unusable.  They cause alarms that can't be acknowledged following subsecond value changes, the times in the alarm database for these attributes are all in UTC and not local time like the rest, and they don't always initialize correctly when they also have the inverted flag set.
  3. Plain weird shit happening/object corruption.  Like this:aabugs1The attributes in the above image are all boolean tags, have nothing to do with voltage, but somehow I am getting random events from other attributes mixed in with these ones.  In this case the problem was solved by un-logging the attribute in the template, redeploying, and enabling the logging again and redeploying but there is no way that this sort of stuff should ever happen, and I could see it leading to some very expensive mistakes by operators.
  4. Failovers barely ever work well.  The only times I have ever seen a AppEngine failover work properly is when it was graceful, ie I asked the engine to failover via the SMC.  If I simply restart the machine, all hell breaks loose.  Objects only half work, alarms going off that were never configured, some objects get corrupted and need to be recreated from a galaxy dump.
  5. There is no where to report bugs.  There is nowhere to make suggestions.  The forum is great, but I sometimes my question can only be answered by someone that writes the code or at least has access to it.
  6. Deploying objects takes over the IDE.  In fact lots of basic operations (think import/export objects) do this.It would be nice to be able to deploy objects in the background and continue working on other objects, then show the outcome in the status bar or something so I can get on with other things in the mean time.
  7. When deploying half the time it appears the deployment has failed, when it has actually completed successfully.
    aabugs2
  8. Multiple monitor support in InTouch is about as useful as the tits on a bull.  I mean I got it to work quite well, but holy balls it took a quite a bit of custom code, testing, and more testing.  I am very much looking forward to the next release which I understand has proper multimonitor support. (UPDATE: have beta build, and it has mult-monitor support by way of a new view container, intouch remains the same.)
  9. Renaming attributes in both graphics and in objects will break in either animations (for graphics), or will reset your IO references (in objects).  This creates an aversion to creating good naming conventions, because if you try to correct an error after you have created many instances you are in for a crap time.

Lets hope the System Platform 2017/Omni addresses some of these.

| Posted in SCADA | No Comments »
10.30.16

ArchestrA - Disabled and Silenced Alarms

It can be very useful to disable or silence alarms in ArchestrA, for example during commissioning, testing and maintenance.  However when you inhibit or silence alarms on an object, it is handy to keep track of these as they won't come up in the alarm list.  The last thing that you want is an operator leaving the alarms disabled after testing and subsequently missing an important event.

Here is one method of keeping track of which objects have their alarms silenced/disabled/inhibited.

I got the idea from here, which performs a query on the alarms database on the historian.  The alarms database (A2ALMDB) records every write to a tag performed by a user, so in theory you can trawl the database for all writes done to tags involved in the alarm disabling process.  Note that by default this does not include tags that are written to in object scripts, so if you disable tags this way (which I used to) then this won't work out the the box.  I moved the logic from the object to the graphics for this reason, as graphics scripts are treated as user writes, not object writes.

The query from the post above was as follows

WITH dateorderedcommandevents AS 
(
    SELECT *, RNum=ROW_NUMBER() OVER (PARTITION BY [TagName] ORDER BY EventStamp DESC)
    FROM dbo.v_EventHistory
    WHERE [TagName] LIKE '%AlarmModeCmd'
    AND [Description] LIKE '%Write success%'
)
SELECT EventStamp, LEFT(TagName, LEN(TagName) - 13) , Area, Value, Operator, OperatorNode
FROM dateorderedcommandevents
WHERE RNum=1 AND NOT Value='Enable'

This doesn't cover all the use cases that I wanted, so I modified it (heavily) to the following:

Select EventStamp, LEFT(TagName, LEN(TagName) - CHARINDEX('.', REVERSE(TagName), 0)) as Item, 
RIGHT(TagName, CHARINDEX('.', REVERSE(TagName), 0) - 1) as Method, Value, UserFullName from
(
    SELECT Max(EventStamp) as ev, TagName as tn
    FROM dbo.v_EventHistory
    WHERE TagName LIKE '%AlarmInhibit' or TagName like '%AlarmModeCmd' or TagName like '%PlantState'
    AND [Description] LIKE '%Write success%'
    Group by TagName
) as tb1
left join dbo.v_EventHistory on EventStamp = tb1.ev and TagName = tb1.tn
Where not value = 'Running' and not value = 'Enable' and not value = 'false'

It searches for all tags that end in AlarmInhibit, PlantState, or AlarmModeCmd.  These are some of the ways that one can control alarms on objects and areas.  It filters out the list to only include the most recent write for each tag, and then checks the state of this tag.  Note that I have an additional plant state called "Testing" that I use to silence an entire area.  We can wrap this up in a SqlDataGrid object and present it as a list, or use it to set an alarm to remind the operator that there are inhibited alarms in the system.

UPDATE 24/1/17:  When using the Historian as the backing store for the alarms, the above query won't work as the Historian has several limitations when it comes to SQL query syntax.   The following, much longer and more cumbersome query will work.  If anyone comes up with a shorter version that works I'd be interested to see it.

select EventStamp, LEFT(TagName, LEN(TagName) - CHARINDEX('.', REVERSE(TagName), 0)) as Item, 
RIGHT(TagName, CHARINDEX('.', REVERSE(TagName), 0) - 1) as Method, Value, UserFullName as Operator from
(
    SELECT Max(EventStamp) as ev, TagName as tn from
    (
        SELECT  * FROM dbo.v_EventHistory WHERE TagName LIKE '%PlantState' 
        AND [Description] LIKE '%Write success%' and  EventStamp > '20160101'
        union 
        SELECT  * FROM dbo.v_EventHistory WHERE TagName LIKE '%AlarmModeCmd' 
        AND [Description] LIKE '%Write success%' and  EventStamp > '20160101' 
        union 
        SELECT  * FROM dbo.v_EventHistory WHERE TagName LIKE '%AlarmInhibit' 
        AND [Description] LIKE '%Write success%' and  EventStamp > '20160101'
    )
    tb3 group by TagName
)
as tb1 left join 
(
    select * from  dbo.v_EventHistory where EventStamp > '20160101'
)
tb2 on EventStamp = tb1.ev and TagName = tb1.tn Where not value = 'Running' and not value = 'Enable' and not value = 'false'";

| Posted in SCADA | No Comments »
03.11.16

Using the InTouch Script Toolkit to get design time window offsets

Well that title is quite a mouth full, so perhaps I should explain what is going on here.  During the course of a previous post I created an InTouch application that worked on multiple monitors.  An improvement on that application was to create a Quick Function MMPopup(WindowName as Message) such that it shows a popup window using the offsets specified at design time, but on the correct monitor. itstk6This seems like a trivial problem at first glance, but on closer inspection it is not so straightforward.  There is no way to get the design time window offset, without showing the window and getting it from GetWindowPosition().  This is a truly abominable approach, and I found it worked poorly at best.  I decided that this information could surely be obtained from the *.win file, and so set about reverse engineering the file format to a point where I could get this data.  It proved to be a fairly straightforward task, but as it involves reading and converting binary data it can't be done inside a simple QF.  Hence I set about obtaining the script toolkit (part of the FSK2000 toolkit) which my local Wonderware distributor kindly supplied.
Read the rest of this entry »

| Posted in SCADA | 3 Comments »
03.7.16

Creating a multimonitor application in InTouch

For a long time our operators have been asking for multi-monitor support in our InTouch application, they want to be able to have alarms showing on one monitor and a substation on another for example.  InTouch does support multiple monitors, but it makes you do a lot of work to get things the way you want.  In our case we wanted to have two independent monitors, each showing a different window in the application.  Additionally we wanted to have a history of windows for each screen that you could scroll back through like a web browser, and for the application to work when scaled for monitors on different resolutions.

These are some of the things that you need to be aware of when developing a multi monitor application:

  • Each window can only be shown on one monitor at a time (InTouch limitation)
  • If you try to show a window that is already open on a different monitor, it will close the window on the monitor it was originally on, leaving you with a blank screen.
  • $ObjHor and $ObjVer are given in pixels, offset from the top left corner of the view window client area.  This is important as when you are viewing the application on a monitor with a different resolution to the design resolution this must be taken into account.
  • When using the Show* functions on a monitor of different resolution to the design resolution, same applies as above.
  • Calling windowstate() on a window right after (ie in the same script) showing or closing the same window will not return the result you are expecting!

This is a rough guide as to how I went about achieving these goals.

 Modify win.ini (twice!)

You need to configure InTouch to use multiple monitors, you do this in the win.ini file, but here's the trick - there are two win.ini files!  The one in C:\Users\<user>\AppData\Wonderware\win.ini  is used by view and window maker, but the wwtechsp.dll library uses the system win.ini file, the location of which varies by operating system and other settings.  The correct file to use is which ever one has been modified by the InTouch installer.

  • C:\Users\<user>\WINDOWS\win.ini
  • C:\Windows\win.ini

Append/add a configuration section such as the below to these files, using the appropriate figures for your monitors, NOT your application design resolution (although on the machine you develop the application, these will be the same).

[InTouch]
MultiScreen=1
MultiScreenWidth=1920
MultiScreenHeight=1080

Read the rest of this entry »

| Posted in SCADA | 31 Comments »
11.27.15

aaAttributeBrowser learnings

In my undertaking to develop aaAttributeBrowser I have learned a few things, and as a result I significantly changed the design from the original plan.  Here are some things that were not immediately obvious to me as a novice AOT developer:

  • GRAccess requires an IDE License.  IDE licences are issued out per session.  ArchestrA objects run in session 0 (the session used by services and other system bits), and the IDE runs in session 1 (or higher), which is your UI session.  Hence if you run the app from your desktop it will not gobble an additional IDE license to the one you are using for the IDE.  However as soon as you put GRAccess into an ArchestrA object you will require an IDE license for session 0 as well as your user session.
  • When accessing Arrays with the MXAccess toolkit, you need to add [] to the attribute name to get the entire array!  You can also get individual items using the [1] notation, indexing starts at 1!
  • Using the MXAccess toolkit inside an ArchestrA object seemingly does not go well, and typically results in errors when receiving datachange events.  I am yet to work out the cause of this.
  • For some reason, you can only build ArchestrA objects in visual studio when the objects have not been deployed to the GAC.  This means, that to package a new version, you must undeploy and delete the object from the Galaxy, delete them from the GAC (the .NET 4 GAC is at C:\Windows\Microsoft.Net\assembly$$.  You may have to shutdown the engine that the objects were deployed to, and restart the aaGR service before you can delete the files from the GAC.
  • Any additional .NET assemblies that are packaged with the object, seemingly must be strong named.
  • AOT objects can be debugged!  Just attach visual studio to the correct aaEngine instance for your object and set your breakpoints!
  • The vendor name in the object designer is tied to the company field in the assembly info.  If you change the vendor/company in one place, make sure it is changed everywhere, as mismatches can result in a big mess that ends up resulting in a Galaxy rebuild!  Also, best not to change the vendor once it has been deployed to a galaxy, it really doesn't like it!

With the above learnings, I made the following executive decisions:

  • Ditch GRAccess, as it is not acceptable to expect people to pay upwards of $10k for this simple bit of software.  For enumerating attributes, use MXAccess to read the _Attribtes[] attribute.
  • Because MxAccess doesn't work inside the AOT, don't use it.  Scripting in a regular object is easy enough to start and stop an external process.

Note that there is no technical reason for having a client/server architecture any more.  However, to keep the client control as light as possible I decided to keep the client/server model, as it does provide the handy function of making any object attributes available in a web browser.

| Posted in SCADA | No Comments »
11.22.15

aaAttributeBrowser

Recently I have been working on an ArchestrA system where unfortunately many of the objects do not conform to any sort of standard.  This makes designing graphic templates difficult.  Often it is desired to only show a small amount of information via a graphic, and then have a popup window with a complete list of IO attributes for an object.  In InTouch we did all of this manually, but with ArchestrA we wanted to automate this tedious process.

The problem boils down to three main steps:

  1. Enumerate the IO attributes (by filtering for those with "InputSource" at the end.
  2. Retrieve the runtime values
  3. Display the values.

Enumerating the attributes of an object can be achieved with the GRAccess toolkit.  Runtime values are fetched using the MXAccess toolkit and displaying the values can be done by writing a windows forms control.

To prove the concept, I wrote a user control that performed all three steps above, but it proved to be very slow to create a connection to the galaxy, retrieve the attributes and query the values.  This created an unacceptable delay when showing a popup before the data showed up in the list, nevertheless it did work rather well.  Furthermore, the GRAccess toolkit only works on the Galaxy node, so some sort of server/client architecture is required for a production implementation.

Version 2:

Version 2 consists of a server component that resides on the galaxy, and a client control that communicates with it.  The server component maintains one connection to the galaxy for all objects and attributes, and serves data over http in the JSON format.  This has the added benefit that one can simply browse to to http://servername:port/myobject and recieve a list of the objects IO attributes.  The end result of this is very snappy opening and closing times for the popup, with only a slight delay when first requesting data from an object.  However, there is still room for improvement...

Left - using a web browser to access attributes, and an ArchestrA/InTouch popup on the right.

Left - using a web browser to access attributes, and an ArchestrA/InTouch popup on the right.

Version 3:

So Version 2 works well, but there are some improvements that could be made.  These are the things that I have in mind for future development:

  • Turn the server side component into an ArchestrA object (written using the ArchestrA Object Toolkit), this would make for much simpler deployment.
  • Add more options to the query string, for example myobject?attributeType=output to get output attibutes only.
  • Fetch custom labels from galaxy and use them in place of True/False for boolean types.
  • Fetch units for values
  • Fetch descriptions for attributes

Source code for this project is available at https://github.com/Raggles/aaAttributeBrowser, read the readme for usage instructions.

| Posted in SCADA | No Comments »
08.16.15

Changing a DNP Master from UDP to TCP in System Configurator

A frustrating situation that may occur from time to time when programming SCD5200's is creating a DNP master and adding it to a UDP Master Group, only to discover that the device doesn't support UDP, only TCP. Normally one would have to detach the device, reattach it to a TCP Master Group, and remap any points to slaves. This can be a monumental time waster, so I will share how we can cut some corners and move it with a bit of database back door banter.

  1. Navigate to your cygwin directory and start bash. (set your window width buffer to 2000 to make things look nice)syscfgtcpudp1
  2. Login to the SYSFG database
    psql -d SYSCFG
  3. Enumerate your dnp protocol masters
    select * from protocol_master_dnp;

    Note the protocol_type_oid column. For me, tcp devices show up as 115, and udp masters are 552 (for dnp slaves udp is 553 and tcp 99). Your version of System Configurator may have different numbers here.

  4. Enumerate your dnp protocol master groups
    select * from protocol_master_grp_dnp;

    Note the prot_grp_type_oid column. Mine shows 551 for UDP, and 64 for TCP.

  5. Change the DNP Master type, and change its protocol group. Write down the oid for the UDP Master Group and the oid for the UDP Master that you want to change to TCP. For me those numbers are 201506300351335920 and 201505120209572450. Now:
     update protocol_master_dnp set protocol_type_oid = 115, protocol_grp_oid = 201506300351335920 where oid = 201505120209572450;

And that's it!

| Posted in Foxboro, SCADA | No Comments »