hansgschossmann/iot-edge-opc-publisher

By hansgschossmann

Updated about 7 years ago

Repo for the Azure IoT Edge OPC Publisher (dev version of hansgschossmann)

Image
2

4.3K

hansgschossmann/iot-edge-opc-publisher repository overview

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments

OPC Publisher

This reference implementation demonstrates how to connect to existing OPC UA servers and publishes JSON encoded telemetry data from these servers in OPC UA "Pub/Sub" format (using a JSON payload) to Azure IoT Hub. All transport protocols supported by the Azure IoTHub client SDK can be used, i.e. HTTPS, AMQP and MQTT.

This application, apart from including an OPC UA client for connecting to existing OPC UA servers you have on your network, also includes an OPC UA server on port 62222 that can be used to manage what gets published and offers IoTHub direct methods to do the same.

The application is implemented using .NET Core technology and is able to run on the platforms supported by .NET Core.

OPC Publisher implements a retry logic to establish connections to endpoints which have not responded to a certain number of keep alive requests, for example if the OPC UA server on this endpoint had a power outage.

For each distinct publishing interval to an OPC UA server it creates a separate subscription over which all nodes with this publishing interval are updated.

OPC Publisher supports batching of the data sent to IoTHub, to reduce network load. This batching is sending a packet to IoTHub only if the configured package size is reached.

This application uses the OPC Foundations's OPC UA reference stack as nuget packages and therefore licensing of their nuget packages apply. Visit https://opcfoundation.org/license/redistributables/1.3/ for the licensing terms.

BranchStatus
masterBuild status Build Status

Building the application

The application requires the .NET Core SDK 2.1.

As native Windows application

Open the opcpublisher.sln project with Visual Studio 2017 and build the solution by hitting F7.

As Docker container

Depending if you use Docker Linux or Docker Windows containers, there are different configuration files (Dockerfile or Dockerfile.Windows) to use for building the container. From the root of the repository, in a console, type:

docker build -f <docker-configfile-to-use> -t <your-container-name> .

The -f option for docker build is optional and the default is to use Dockerfile. Docker also support building directly from a git repository, which means you also can build a Linux container by:

docker build -t <your-container-name> https://github.com/Azure/iot-edge-opc-publisher

Note: if you want to have correct version information, please install gitversion and run it with the following command line in the root of the repository: gitversion . /updateassemblyinfo /ensureassemblyinfo updateassemblyinfofilename opcpublisher/AssemblyInfo.cs

Configuration of the OPC UA nodes to publish

Configuration via configuration file

The easiest way to configure the OPC UA nodes to publish is via configuration file. The configuration file format is documented in publishednodes.json in this repository. Configuration file syntax has changed over time and OPC Publisher still can read old formats, but converts them into the latest format when persisting the configuration. An example for the format of the configuration file is:

    [
      {
        "EndpointUrl": "opc.tcp://testserver:62541/Quickstarts/ReferenceServer",
        "UseSecurity": false,
        "OpcNodes": [
          {
            "Id": "i=2258",
            "OpcSamplingInterval": 2000,
            "OpcPublishingInterval": 5000,
            "DisplayName": "Current time"
          }
        ]
      }
    ]

If you want to publish node values even they have not changed after a certain amount of time (with this you can also publish static node values regularily), then you can add the HeartbeatInterval key to a node configuration. This specifies a value in seconds where the last value should be resent, even it has not changed:

            "HeartbeatInterval": 3600,

To prevent sending the value of a node at startup, you can add the SkipFirst key to a node configuration:

            "SkipFirst": true,

Both configurations settings can be enabled for all nodes in your configuration file via command line options as well.

Configuration via OPC UA method calls

OPC Publisher has an OPC UA Server integrated, which can be accessed on port 62222. If the hostname is publisher, then the URI of the endpoint is: opc.tcp://publisher:62222/UA/Publisher This endpoint exposes five methods:

  • PublishNode
  • UnpublishNode
  • GetPublishedNodes
  • IoTHubDirectMethod (see below)
Configuration via IoTHub direct function calls

OPC Publisher implements the following IoTHub direct method calls, which can be called when OPC Publisher runs standalone or in IoT Edge:

  • PublishNodes
  • UnpublishNodes
  • UnpublishAllNodes
  • GetConfiguredEndpoints
  • GetConfiguredNodesOnEndpoint
  • GetDiagnosticInfo
  • GetDiagnosticLog
  • GetDiagnosticStartupLog
  • ExitApplication
  • GetInfo

The format of the JSON payload of the method request and responses are defined in the file opcpublisher/HubMethodModels.cs.

If you call a unknown method on the module, it responds with a string, saying the method is not implemented. This can be used to ping the module.

Configuring the telemetry published to IoTHub

When OPC Publisher gets notified about a value change in one of the configured published nodes, it generates a JSON formatted message, which is sent to IoTHub. The content of this JSON formatted message can be configured via a configuration file. If no configuration file is specified via the --tc option a default configuration is used, which is compatible with the Connected factory Preconfigured Solution.

If OPC Publisher is configured to batch messages, then they are sent as a valid JSON array.

The data which is ingested is taken from three sources:

  • the OPC Publisher node configuration for the node
  • the MonitoredItem object of the OPC UA stack for which OPC Publisher got a notification
  • the argument passed to this notification, which provides details on the data value change

The telemetry which is put into the JSON formatted message is a selection of important properties of these objects. If you need more properties, you need to change the OPC Publisher code base.

The syntax of the configuration file is as follows:

    // The configuration settings file consists of two objects:
    // 1) The 'Defaults' object, which defines defaults for the telemetry configuration
    // 2) An array 'EndpointSpecific' of endpoint specific configuration
    // Both objects are optional and if they are not specified, then publisher uses
    // its internal default configuration, which generates telemetry messages compatible
    // with the Microsoft Connected factory Preconfigured Solution (https://github.com/Azure/azure-iot-connected-factory).

    // A JSON telemetry message for Connected factory looks like:
    //  {
    //      "NodeId": "i=2058",
    //      "ApplicationUri": "urn:myopcserver",
    //      "DisplayName": "CurrentTime",
    //      "Value": {
    //          "Value": "10.11.2017 14:03:17",
    //          "SourceTimestamp": "2017-11-10T14:03:17Z"
    //      }
    //  }

    // The 'Defaults' object in the sample below, are similar to what publisher is
    // using as its internal default telemetry configuration.
    {
        "Defaults": {
            // The first two properties ('EndpointUrl' and 'NodeId' are configuring data
            // taken from the OpcPublisher node configuration.
            "EndpointUrl": {

                // The following three properties can be used to configure the 'EndpointUrl'
                // property in the JSON message send by publisher to IoTHub.

                // Publish controls if the property should be part of the JSON message at all.
                "Publish": false,

                // Pattern is a regular expression, which is applied to the actual value of the
                // property (here 'EndpointUrl').
                // If this key is ommited (which is the default), then no regex matching is done
                // at all, which improves performance.
                // If the key is used you need to define groups in the regular expression.
                // Publisher applies the regular expression and then concatenates all groups
                // found and use the resulting string as the value in the JSON message to
                //sent to IoTHub.
                // This example mimics the default behaviour and defines a group,
                // which matches the conplete value:
                "Pattern": "(.*)",
                // Here some more exaples for 'Pattern' values and the generated result:
                // "Pattern": "i=(.*)"
                // defined for Defaults.NodeId.Pattern, will generate for the above sample
                // a 'NodeId' value of '2058'to be sent by publisher
                // "Pattern": "(i)=(.*)"
                // defined for Defaults.NodeId.Pattern, will generate for the above sample
                // a 'NodeId' value of 'i2058' to be sent by publisher

                // Name allows you to use a shorter string as property name in the JSON message
                // sent by publisher. By default the property name is unchanged and will be
                // here 'EndpointUrl'.
                // The 'Name' property can only be set in the 'Defaults' object to ensure
                // all messages from publisher sent to IoTHub have a similar layout.
                "Name": "EndpointUrl"

            },
            "NodeId": {
                "Publish": true,

                // If you set Defaults.NodeId.Name to "ni", then the "NodeId" key/value pair
                // (from the above example) will change to:
                //      "ni": "i=2058",
                "Name": "NodeId"
            },

            // The MonitoredItem object is configuring the data taken from the MonitoredItem
            // OPC UA object for published nodes.
            "MonitoredItem": {

                // If you set the Defaults.MonitoredItem.Flat to 'false', then a
                // 'MonitoredItem' object will appear, which contains 'ApplicationUri'
                // and 'DisplayNode' proerties:
                //      "NodeId": "i=2058",
                //      "MonitoredItem": {
                //          "ApplicationUri": "urn:myopcserver",
                //          "DisplayName": "CurrentTime",
                //      }
                // The 'Flat' property can only be used in the 'MonitoredItem' and
                // 'Value' objects of the 'Defaults' object and will be used
                // for all JSON messages sent by publisher.
                "Flat": true,

                "ApplicationUri": {
                    "Publish": true,
                    "Name": "ApplicationUri"
                },
                "DisplayName": {
                    "Publish": true,
                    "Name": "DisplayName"
                }
            },
            // The Value object is configuring the properties taken from the event object
            // the OPC UA stack provided in the value change notification event.
            "Value": {
                // If you set the Defaults.Value.Flat to 'true', then the 'Value'
                // object will disappear completely and the 'Value' and 'SourceTimestamp'
                // members won't be nested:
                //      "DisplayName": "CurrentTime",
                //      "Value": "10.11.2017 14:03:17",
                //      "SourceTimestamp": "2017-11-10T14:03:17Z"
                // The 'Flat' property can only be used for the 'MonitoredItem' and 'Value'
                // objects of the 'Defaults' object and will be used for all
                // messages sent by publisher.
                "Flat": false,

                "Value": {
                    "Publish": true,
                    "Name": "Value"
                },
                "SourceTimestamp": {
                    "Publish": true,
                    "Name": "SourceTimestamp"
                },
                // 'StatusCode' is the 32 bit OPC UA status code
                "StatusCode": {
                    "Publish": false,
                    "Name": "StatusCode"
                    // 'Pattern' is ignored for the 'StatusCode' value
                },
                // 'Status' is the symbolic name of 'StatusCode'
                "Status": {
                    "Publish": false,
                    "Name": "Status"
                }
            }
        },

        // The next object allows to configure 'Publish' and 'Pattern' for specific
        // endpoint URLs. Those will overwrite the ones specified in the 'Defaults' object
        // or the defaults used by publisher.
        // It is not allowed to specify 'Name' and 'Flat' properties in this object.
        "EndpointSpecific": [
            // The following shows how a endpoint specific configuration can look like:
            {
                // 'ForEndpointUrl' allows to configure for which OPC UA server this
                // object applies and is a required property for all objects in the
                // 'EndpointSpecific' array.
                // The value of 'ForEndpointUrl' must be an 'EndpointUrl' configured in
                // the publishednodes.json confguration file.
                "ForEndpointUrl": "opc.tcp://<your_opcua_server>:<your_opcua_server_port>/<your_opcua_server_path>",
                "EndpointUrl": {
                    // We overwrite the default behaviour and publish the
                    // endpoint URL in this case.
                    "Publish": true,
                    // We are only interested in the URL part following the 'opc.tcp://' prefix
                    // and define a group matching this.
                    "Pattern": "opc.tcp://(.*)"
                },
                "NodeId": {
                    // We are not interested in the configured 'NodeId' value, 
                    // so we do not publish it.
                    "Publish": false
                    // No 'Pattern' key is specified here, so the 'NodeId' value will be
                    // taken as specified in the publishednodes configuration file.
                },
                "MonitoredItem": {
                    "ApplicationUri": {
                        // We already publish the endpoint URL, so we do not want
                        // the ApplicationUri of the MonitoredItem to be published.
                        "Publish": false
                    },
                    "DisplayName": {
                        "Publish": true
                    }
                },
                "Value": {
                    "Value": {
                        // The value of the node is important for us, everything else we
                        // are not interested in to keep the data ingest as small as possible.
                        "Publish": true
                    },
                    "SourceTimestamp": {
                        "Publish": false
                    },
                    "StatusCode": {
                        "Publish": false
                    },
                    "Status": {
                        "Publish": false
                    }
                }
            }
        ]
    }

Running the application

Command line options

The complete usage of the application can be shown using the --help command line option and is as follows:

    Current directory is: /appdata
    Log file is: <hostname>-publisher.log
    Log level is: info

    OPC Publisher V2.3.0
    Informational version: V2.3.0+Branch.develop_hans_methodlog.Sha.0985e54f01a0b0d7f143b1248936022ea5d749f9

    Usage: opcpublisher.exe <applicationname> [<iothubconnectionstring>] [<options>]

    OPC Edge Publisher to subscribe to configured OPC UA servers and send telemetry to Azure IoTHub.
    To exit the application, just press CTRL-C while it is running.

    applicationname: the OPC UA application name to use, required
                     The application name is also used to register the publisher under this name in the
                     IoTHub device registry.

    iothubconnectionstring: the IoTHub owner connectionstring, optional

    There are a couple of environment variables which can be used to control the application:
    _HUB_CS: sets the IoTHub owner connectionstring
    _GW_LOGP: sets the filename of the log file to use
    _TPC_SP: sets the path to store certificates of trusted stations
    _GW_PNFP: sets the filename of the publishing configuration file

    Command line arguments overrule environment variable settings.

    Options:
          --pf, --publishfile=VALUE
                                 the filename to configure the nodes to publish.
                                   Default: '/appdata/publishednodes.json'
          --tc, --telemetryconfigfile=VALUE
                                 the filename to configure the ingested telemetry
                                   Default: ''
      -s, --site=VALUE           the site OPC Publisher is working in. if specified
                                   this domain is appended (delimited by a ':' to
                                   the 'ApplicationURI' property when telemetry is
                                   sent to IoTHub.
                                   The value must follow the syntactical rules of a
                                   DNS hostname.
                                   Default: not set
          --ic, --iotcentral     publisher will send OPC UA data in IoTCentral
                                   compatible format (DisplayName of a node is used
                                   as key, this key is the Field name in IoTCentral)
                                   . you need to ensure that all DisplayName's are
                                   unique. (Auto enables fetch display name)
                                   Default: False
          --sw, --sessionconnectwait=VALUE
                                 specify the wait time in seconds publisher is
                                   trying to connect to disconnected endpoints and
                                   starts monitoring unmonitored items
                                   Min: 10
                                   Default: 10
          --mq, --monitoreditemqueuecapacity=VALUE
                                 specify how many notifications of monitored items
                                   can be stored in the internal queue, if the data
                                   can not be sent quick enough to IoTHub
                                   Min: 1024
                                   Default: 8192
          --di, --diagnosticsinterval=VALUE
                                 shows publisher diagnostic info at the specified
                                   interval in seconds (need log level info).
                                   -1 disables remote diagnostic log and diagnostic
                                   output
                                   0 disables diagnostic output
                                   Default: 0
          --ns, --noshutdown=VALUE
                                 same as runforever.
                                   Default: False
          --rf, --runforever     publisher can not be stopped by pressing a key on
                                   the console, but will run forever.
                                   Default: False
          --lf, --logfile=VALUE  the filename of the logfile to use.
                                   Default: './<hostname>-publisher.log'
          --lt, --logflushtimespan=VALUE
                                 the timespan in seconds when the logfile should be
                                   flushed.
                                   Default: 00:00:30 sec
          --ll, --loglevel=VALUE the loglevel to use (allowed: fatal, error, warn,
                                   info, debug, verbose).
                                   Default: info
           --ih, --iothubprotocol=VALUE
                                  the protocol to use for communication with IoTHub (
                                    allowed values: Amqp, Http1, Amqp_WebSocket_Only,
                                     Amqp_Tcp_Only, Mqtt, Mqtt_WebSocket_Only, Mqtt_
                                    Tcp_Only) or IoT EdgeHub (allowed values: Mqtt_
                                    Tcp_Only, Amqp_Tcp_Only).
                                    Default for IoTHub: Mqtt_WebSocket_Only
                                    Default for IoT EdgeHub: Amqp_Tcp_Only
          --ms, --iothubmessagesize=VALUE
                                 the max size of a message which can be send to
                                   IoTHub. when telemetry of this size is available
                                   it will be sent.
                                   0 will enforce immediate send when telemetry is
                                   available
                                   Min: 0
                                   Max: 262144
                                   Default: 262144
          --si, --iothubsendinterval=VALUE
                                 the interval in seconds when telemetry should be
                                   send to IoTHub. If 0, then only the
                                   iothubmessagesize parameter controls when
                                   telemetry is sent.
                                   Default: '10'
          --dc, --deviceconnectionstring=VALUE
                                 if publisher is not able to register itself with
                                   IoTHub, you can create a device with name <
                                   applicationname> manually and pass in the
                                   connectionstring of this device.
                                   Default: none
      -c, --connectionstring=VALUE
                                 the IoTHub owner connectionstring.
                                   Default: none
          --hb, --heartbeatinterval=VALUE
                                 the publisher is using this as default value in
                                   seconds for the heartbeat interval setting of
                                   nodes without
                                   a heartbeat interval setting.
                                   Default: 0
          --sf, --skipfirstevent=VALUE
                                 the publisher is using this as default value for
                                   the skip first event setting of nodes without
                                   a skip first event setting.
                                   Default: False
          --pn, --portnum=VALUE  the server port of the publisher OPC server
                                   endpoint.
                                   Default: 62222
          --pa,

Tag summary

Content type

Image

Digest

Size

41.5 MB

Last updated

about 7 years ago

docker pull hansgschossmann/iot-edge-opc-publisher:hb