Wireshark Now Includes an MS/TP Capture Tool!

Wireshark is a fantastic tool for network diagnostics. It’s been the best tool for BACnet/IP network troubleshooting for years. It’s even been able to dissect MS/TP traffic dumps if you knew how and had the right tools. Well, those tools come standard now! Check out the new Wireshark BACnet MS/TP capture tool, included in 2.4.4 (and possibly earlier):

A screengrab of Wireshark's MS/TP capture tool

You’ll have to be connected to the BACnet MS/TP network directly. BACnet/IP routers will not work. I’ve used StarTech‘s successfully and was very happy with it. An MS/TP to IP gateway won’t do the job, since it hides the MS/TP side of things from you. You can use MS/TP capture with Wireshark to diagnose communication problems. It will also reveal if your token passing is being handled efficiently. Wireshark won’t do any automatic diagnostics for you unfortunately, you’ll have to use your understanding of the protocol to troubleshoot or identify problems. Wireshark does have some handy statistics generating features that can be helpful at a glance, though. It’ll break down the number of packets by the type of message they are sending: Who-Is, I-Am, etc., and what devices are communicating the most.

What Is BACnet?

The Basics – What is BACnet?

BACnet is a popular building automation protocol used worldwide to enable communication between devices in commercial automation systems. Industrial automation has historically used other protocols such as OPC and Modbus. BACnet standardizes communication across vendors and simplifies the integration process.

The name BACnet stands for building automation and control network. It was created by the American Society of Heating, Refrigerating and Air-Conditioning Engineers by a committee of industry professionals to modernize communication for automation systems using an open protocol.

Open – To Intepretation

It’s a open standard, anyone can get a full copy of it if they are willing to pay for it. Closed standards are proprietary communication systems created by vendors for their own devices. Open systems allow multiple vendors to utilize a common communication backbone and automation front end. It’s also a very easy standard to comply with, as it requires very little of you. This can be a challenge for system integrators.

A device does not need to support much of the BACnet standard to be a compliant BACnet device. So long as it knows how to say hello to other devices, it’s a BACnet device. This flexibility presents a challenge for integrators, as vendors can implement as much or at little of the standard as they like. Sometimes a vendor won’t bother to put any useful metadata in the standard fields for points. You’ll see analogObject_1 through 500 with no way of determining what anything represents. Lazy BACnet might as well be Modbus.

When implemented well by the vendor, BACnet is damn near pleasurable to work with. It makes some hard things easy since it is designed for our industry. The old school controls guys are not fans of BACnet because larger systems get into the realm of computer networking.

 

Overview of Commercial Automation Protocols

In automation there are several communication protocols. Some are entirely proprietary and others are open. Despite the friendliness of the word, “open” isn’t always good news. Some of the older proprietary protocols are easy to integrate with off the shelf integration products. Two versions of the same protocol often look different. It’s a zoo out there, so here’s an overview of commercial automation protocols to help you sort it out.

Modbus

Modbus is about as bare bones as a protocol can be. It’s simplicity is a strength and a weakness. Since it requires so little it’s easy for vendors to support Modbus, and as a result it’s quite popular. Modbus does little work for you, which makes it difficult to integrate. To integrate a Modbus device you’ll need good documentation and a good understanding of how computers store numbers. If your tools allow you to copy integrations you’ll only have to do it once, as the addresses don’t vary between identical devices. Modbus is more of a simple data transfer protocol without much in common with other automation protocols.

We wrote an article that goes into more depth on Modbus.

OPC

Like Modbus, OPC is common in industrial environments. It can be straightforward to integrate as it doesn’t require you to know low-level details. Discovery is possible, and advanced features such as alarms and historical data access are possible for devices that support them. The tricky part with OPC is connecting devices. The usual form uses a proprietary Microsoft technology that relies on DCOM. DCOM settings can be difficult enough that Matrikon developed an OPC Tunneler that allows you to skip all of that headache for any OPC client-server relationship.

LONWorks

Lonworks is a protocol developed by Echelon. The protocol understands an air handler and what kind of points they should have. Many vendors implement their Lonworks interfaces in a generic manner though and don’t take advantage of this standardization. Other times they will use the standardization, but some of the points will be duds or their implementation will be more confusing. A Lonworks installer needs to understand the protocol deeply since it’s an advanced protocol. If you work in a company with Lon products, you’ve probably had someone bring an entire building down because they synchronized the database the wrong way. It can be a very labour intensive protocol.

BACnet

BACnet is a popular protocol created by ASHRAE that is “open” but not very standard in practice. It’s replacing other commercial automation protocols. There’s a grab-bag of object types and parameters in the protocol for fire monitoring, lighting, scheduling, and so on. BACnet only requires that vendors are implement a few object types. Some will implement the bare minimum in strange ways. When done right, BACnet is a pleasure to work with. Objects are discoverable and contain lots of data that help the integrator understand what they are looking at. Points can be overridden on different priority levels, allowing for schedules to command a fan directly and an operator to override that without any extra programming logic.

Add a Remote Device Without a BBMD

You may encounter a situation where you must add a device on a different network. You may have no control of the BBMD configuration or you may have no BBMDs available due to poor device support. In these situations, you may have to add a remote device without a BBMD. It’s a handy trick, but not an ideal long-term situation.

How is that possible?

The BACnet Broadcast Management Device is required to route broadcasts between BACnet networks separated by routers. BACnet uses broadcast messages for device and point discovery, which are public requests by nature. However, if you know what devices are out there you don’t need discovery! You can add a remote device without a BBMD so long as you know the MAC addresses of the device in question.

What do we need?

You’ll need the device’s BACnet MAC address and BACnet instance number. The BACnet MAC address isn’t the same as the usual MAC address we’re familiar with in IT. A quick way to find it is to discover the device while you’re connected to its BACnet network directly and your BACnet client should tell you directly. Tridium’s N4 platform has a column dedicated to this field. For BACnet/IP devices, the MAC address is a combination of the IP address in hexadecimal and the BACnet/IP port in use.

For example, for an IP address of 192.168.1.6 and a default BACnet/IP port of 47808 (“BAC0”), the MAC address is C0A816BAC0 (192 -> 0xC0, 168 -> 0xA8, 1 -> 0x1, 6 -> 0x6).

Just add the device with the appropriate instance number and MAC address. With any luck, you should be fully communicating. However, the implementation specifics will vary wildly depending on what software you’re using.

Potential Problems

Though this technique can work well for diagnostic purposes, some devices will continue to assume good broadcast communication. If that communication isn’t possible, they’ll assume something is wrong. It’s common practice to use a Who-Is broadcast to check the network status of all devices on the network. If you manually add a remote device without a BBMD handling broadcasts, it will not see or respond to these requests and appear to be offline. You’ll need to get a BBMD in place to ensure your network runs smoothly in that scenario.

 

BACnet Broadcast Distribution Table Example

Setting up a BBMD-connected network is a manual and somewhat arcane process. The broadcast distribution table is a list of the BBMDs on the network. Each BBMD on each network should have the same broadcast distribution table. Here’s a quick example of a BACnet broadcast distribution table (BDT) that might help clear things up.

The Setup

With the above illustration we have three IP networks separated by routers on a backbone. Let’s say that we have a BBMD on each network, and here are their IP addresses: 192.168.1.50, 192.168.2.50, 192.168.3.50. Let’s also grant that we are using the standard BACnet port, 0xBAC0 (i.e.: 47808).

Example BACnet Broadcast Distribution Table

BBMD Address Subnet Mask
192.168.1.50 255.255.255.255
192.168.2.50 255.255.255.255
192.168.3.50 255.255.255.255

The subnet mask is a bit mask that is applied against the BBMD address. The combination of the two allows the device to find the address to send broadcast packages to. Unless you have routers that permit remote broadcasts to their local networks (rare), you’ll need to broadcast directly to your BBMD. The subnet mask value 255 is the same as saying “I’m a bit mask that does nothing”, so that applied against the BBMD address and a bunch of ones is the same as the BBMD address. If you use the network mask the BBMD will send the messages to the router, which will block them.

Example BACnet Network Number & Instance Number Convention

When I assign a BACnet instance number or BACnet network number, I never use the defaults provided by whatever product I’m using.

A few moments of thought about some kind of standard for your site can save you a lot of time down the road when you want to expand your system to include more devices. You don’t know quite how your networks will expand; I’ve had to merge two separate hospital BACnet networks that were addressed without any particular guideline and it was a mess. Nobody thought that the hospitals would merge when the networks were created, so people felt free to use random numbers for every new device.

Here’s a sort of guideline you can use to decide how to plan your own network. It’s certainly not gospel, and if you think of a better way of going about it please let me know.

BACnet Network Number Convention

Keep in mind that the maximum valid network number is 65535.

Building # (2 digit) – Floor # (2 digits) – Trunk # (1 digits)

Example: 01253

This would be a BACnet network in building 1, on floor 25, and it’s trunk number 3 on that floor.

BACnet Instance Number Convention

Keep in mind that the maximum instance number is 4,194,302.

Building # (2 digits) – Floor # (2 digits) – Device Marker (1 digit) – Device Index (2 digits)

For device marker, I use 0 as some kind of supervisory device.

Supervisory devices are primarily used as a graphics interface or historian. They aren’t directly responsible for control.

I use 1 as some kind of field controller directly responsible for control or monitoring.

I use 2 as some kind of gateway that converts to kinds of media, like MS/TP to BACnet/IP.

You can be creative here and think of ways to describe the particular kind of hardware you’re using. I think it’s useful to have that described in the instance number, but that’s arguable.

The device index is something like a controller number or some kind of arbitrary device index number that makes it unique.

Summary

If you spend a few minutes working on a device and network number convention, you’ll have a lot less trouble in the future. Only doing one controller in a building? Still follow a convention like the above. When it comes time to integrate that building with a wider network you’ll surprise the people that come after you with your uncanny abilities of foresight.

It costs you little, and saves so much time later.

 

When and Why Do We Need an End of Line Termination?

End of Line Termination Concept

Most of the buses we use are specified with end of line termination in mind. In the field these terminations are not always present, especially in short runs. Do we need them?

Here’s an analogy I like that illustrates why we need end of line terminations: a bus behaves like a rope held between two people. When one person wants to talk they whip pulses down the rope towards their partner. The other person feels these pulses and figures out what the message means from them.

Those pulses don’t go away, though – they’ll reflect back at the person who whipped it down the line. It’ll keep reflecting back and forth, a little weaker each time as losses eat it up. These reflections are confusing to both people, who waste time trying to decode this false message over and over again.

A bus terminated on both ends is like a rope fixed on either side with springs. When the pulses reach the end, the person there can feel them and then the pulses are absorbed by the springs. If the springs are appropriately sized, the pulses will be “fully dampened” and there will be no reflection back.

RS-485/RS-422 End of Line Termination

Since we’re typically using RS-485, the end of line termination is selected based on something called the characteristic impedance of the cable you’re using. This is something the manufacturer stamps on the box the cable comes in, so it’s no secret. Typically we use something around 108-120 ohms. You don’t need to use exactly that value, but try to be close and don’t use resistors that are much less than it! Some other bus electrical standards will use more complicated devices as their end of line terminations.

Do you always need an end of line termination? Not really, no. It depends on two things: baud rate and bus length. The lower your baud rate, the less reflection will matter. The greater the distance, the more reflection will matter.

High speed buses much more sensitive because everything happens at a much lower time scale – a single bit transferred at 38400 baud lasts only 26 microseconds. For a 9600 baud bit, it’s 104 microseconds. A voltage spike in the wrong place that lasts the length of one bit length can cause communication errors.

Longer buses are more sensitive to reflection because any reflection will tend to dissipate after several trips back and forth down the bus; for a longer bus, that dissipation takes longer. If it’s long enough, controllers could send new messages on the bus while the older reflections are still bouncing around.

Getting Away With Lazy

You can reach a ballpark estimate of when end of line termination is needed can by calculating two things: the propagation delay and the bit pulse width.

Propagation delay is how long a bit takes to go down the cable. Since a voltage pulse travels at a fraction the speed of light (let’s say it’s 66% the speed of light) the propagation delay is 0.66 * c / (length of cable).

The bit pulse width is how long a single bit pulse lasts in time, which we addressed earlier. It’s as simple as 1 / (baud rate). Since baud rate is in Hertz, which is cycles per second, 1/(baud rate) gives us the seconds per cycles (bits).

We now have two time values we can compare. A general guideline is that you can avoid using end of line termination when your propagation delay is much less than the bit pulse time. I.E.: pulses travel so fast across the bus that a reflection will dissipate before it interferes with the interpretation of future bits.

It’s not pure science and could still depend on other factors. Generally speaking, there’s not much reason to cheap out on a few 108 ohm resistors. Bus problems are very labour intensive to troubleshoot.

If you want to be cheap, here’s a table of rough estimates I’ve made of maximum lengths for different baudrates before termination is required:

 

Baudrate Max Length for PD << BP (m) Max Length for PD << PD (ft)
9600 1440 3740
38400 400 1300
76800 200 650
115200 120 400

 

What is a BACnet Instance Number?

A BACnet Instance Number Is a Globally-unique Address

An oft misunderstood and neglected concept is the BACnet instance number of a BACnet device. Neglecting this number at installation might not bite you today, but it remains as technical debt that complicates future work. It’s simply an address number that identifies that device uniquely on the entire interconnected BACnet network. No other device anywhere on the network may have the same BACnet instance number, even if it’s on a different bus in a different building. If they can see each other, they can’t have the same address. This is in contrast to the MS/TP address, which doesn’t have to be globally unique.

More Technical

The instance number is an unsigned decimal number that can range from 0 to 4,194,302. Every device on a BACnet network gets an instance number, and two devices must not have the same number. Devices on different physical network segments that can talk to each other are on the same BACnet network – they can’t have the same instance numbers!

Don’t neglect the instance number uniqueness just because you’re installing MS/TP devices. MS/TP devices also have MAC addresses and need to communicate to BACnet/IP devices through a gateway, but they still have unique BACnet instance numbers that cannot be duplicated anywhere else on the network.

If you have duplicate instance numbers on your network you’ll have a similar problem that you’d see if you have duplicate IP addresses. Some devices on the network will talk to one and not the other, and the devices themselves will have unreliable communication with the overall network.

To avoid problems, start your installation with a clear numbering convention. Here’s an example one I like to use:

1 – 23 – 03 – 23 = 1230323

SITE/Building – Floor – Bus/Network Number – MAC Address

I keep to this even if it’s only for one building as it makes future expansion easy. The numbers are a bit limited, you can only expand to 4 sites/buildings this way. If you have a broader campus/etc. you’d probably want to arrange a different numbering convention. Just spend the time to think carefully about it and you’ll save yourself in the long run!

RS-422 – The Other Bus-meat

I encountered an RS-422 bus for the first time recently and it threw me through a bit of a loop! The device was a Modbus/RTU slave device and so I anticipated a RS-485 connection as per usual, but that was not the case. It’s the same cable, and the same essential signalling standard, but if you try to interface with RS-422 as if it’s RS-485 you’re going to have a bad time.

What’s RS-422?

It’s similar to RS-485, but it has some key differences:

  • Only one device gets to be the big talker, and everyone else has to listen. If you have multiple receivers, they can’t talk back. If you only have two devices, no problem. They can actually talk at the same time, which is faster than your usual half-duplex RS-485.
  • It can have one driver (talker) and 10 receivers, RS-485 can have 32 “unit loads” which can mean many more than 32 controllers (not many manufacturers would recommend it, though)
  • It’s a four-conductor bus. Just like RS-485 has a single balanced pair, RS-422 has two balanced pairs: one for transmission, and the other for reception.
  • You wire the driver (talker)’s transmit to the receive terminals of all of the slave devices

Some RS-485 adapters support RS-422, but don’t assume it.

Based on the standard, RS-485 can be “full-duplex” too. An extra pair of wires allows a bus to send and receive at the same time by using four wires, but it’s unnecessary and rare for RS-485.

RS-485 Bus Details

Low-voltage communication buses are critical to networked automation systems and when they fail they are a pain to troubleshoot. We use them because the cables are cheap and you don’t need expensive switches, but what you save in parts you may pay in labor later. Ethernet/IP based communication is on the rise, but is still expensive as an up-front cost. RS-485 is the typical low-voltage communication electrical standard used for the bus we see in commercial and industrial automation. You’ll see it used for BACnet MS/TP, Modbus, and various proprietary protocols like Honeywell’s C-Bus and JCI’s N2 bus.

What is RS-485?

RS-485 isn’t a protocol, it’s an electrical standard that describes how to transmit bits with volts. The protocol allows multiple devices to co-exist on a long, fast bus. It’s similar to the RS-422 protocol, except with RS-485 any device can talk, and any device can listen.

Is RS-485 a Two-wire Bus?

When you want to measure a voltage with your multimeter you need both probes because voltages only exist between things. The parts that “do” RS-485 aren’t looking between your A and B ports, they are looking at each wire relative to a common reference point. The voltage reference is often the power source’s ground, earth ground, a third independent reference wire. Some choices are much better than others.

RS-485 is polarity sensitive, having an “inverting” (negative, “A”) and “non-inverting” (positive, “B”) wire. Depending on the application the conductors will be shielded or twisted-pair.  Some installers will use the shield as a signal ground to save on cable costs. This can work well, but there are caveats. Not all grounds are equal, so be careful how you terminate your shield; one lightning storm and your bus could explode. If you forget to terminate your shield in one place and you’ll offset all your signals by the noise the shield picks up, as we found in this case study.

The two-wire applications that have no ground may actually be using the power source for the controllers as the voltage reference, but that can be problematic if you mix up the transformer polarity or if you are dealing with multiple power sources. It also looks freaky on an oscilloscope – you’ll see the bus riding on a 60 Hz carrier! It’s best to avoid this kind of setup if you can, but it’s cheaper.

It takes a lot more sustained thought and consistent effort to make two wires work. When you add a different controller or use a different source of power you may find yourself with an expensive problem. If you want a solid bus, it’s best to use a twisted pair of conductors for A+B, a free conductor for the reference, and a shield to improve noise immunity.

Interference – Why We Twist Our Pairs and Use Shielding

As you can see in the above case study twisted pair works extremely well; despite a noisy environment the bus signal can still punch through pretty clearly.

Slower buses (~9,600 baud) tend to be less sensitive to interference. Twisted pair alone works fine for slower buses especially in low-noise environments. BACnet MS/TP which typically runs at 38,400 baud is much more sensitive to interference and must be properly shielded, especially in a high-noise environment.

Interference can come from most electronic equipment but big offenders are VFDs and any kind of switches (MCC cutoffs, relays, etc.). Devices that are improperly shielded or have failing components emit interference. Shielding your cable correctly protects your bus from these difficult to locate and potentially impossible to fix issues.

If you look at your shield by itself on an oscilloscope you’ll see the stray voltage picked up by your bus, and it’s largely just sharp spikes that usually do not last very long. Higher frequency buses are more sensitive to noise. Take 38,400 baud for example. At 38,400 bits per second one bit is sent down the bus in a flash of 26 microseconds. Any interference that lasts that long will screw up the data stream; for 9,600 baud one bit lasts for 3 times as long as that. Many applications can sacrifice some bus speed without a noticeable change in performance, but wiring your bus right is a lot more effective.

Hidden Depths

You know how we talk about how RS-485 is a two-wire bus? It’s not true. Not even 2.5, or 3, or 3.5! RS-485 can communicate in full-duplex mode using four signalling conductors in several typologies. I’ve yet to see this or a controller in the field that supports it, but the transceivers exist.