Establishing Principles for the Internet of Things (IoT) Security

Hello, and welcome. My name is Donald Schleede. I'm the Information Security Officer at Digi International. Today, we're gonna talk about establishing principles for the Internet of Things security. The intent of this talk is to kind of describe with what Digi does within security on our products. First, we're gonna kind of talk about the industry as a standard at this point, what is out there, what is being to expected, what we're seeing from the manufacturers' standpoint. And that's driven from compliance needs of our customers and those type of aspects and then how we're going to kind of respond for that.

The intent of this is to kind of eventually boil up to something we call Digi TrustFence which is kind of our promise on security that kind of sums up the tenets and the issues that we're trying to drive within our security to be able to deliver this as a value add to our customers. So as you know, IOT security is highly visible specifically into the public. There has been many hack attempts that have gotten front page newspapers, FBI has issued this at Federal Trade Commission as establishing reports. There's many just drivers of this as you're probably familiar with HVAC control, there's Jeeps, cars. The whole entire industry and verticals are being attacked.

So the challenge becomes is what does that mean from security aspects as Digi's responsibility is the customers' responsibility and anywhere in between. So the first thing we wanna kind of talk about is what I've actually personally seen with this is how do we deliver this? What is actually being driven? And there's two folds. And security that we'd like to talk about. There's compliance and then there's the effectively the technical parts of acting of security. The technical parts being the confidentiality, integrity, availability. But then there's the compliance parts which people may have heard like PCI DSS for the payment card industry, there's NERC CIP for the power industries and HIPAA. I'm sure people are familiar with the industry for medical.

So there's a lot of pieces on there and a lot of it does come through and again, driven by some of the U.S. which happens to be kind of a good framework is some of the NIST frameworks. So NIST has been working on some IOT frameworks which we'll kind of really just quickly kind of show what they're working on. But again, there are some pieces out there that don't really address IOT specifically but that's where the direction is going. So when this started… And we realized that IOT does have a challenge, you know, this is not traditionally the secure the server in a back room challenge. Now we have devices that are located all over. 

There was a…the Federal Trade Commission held a workshop a few years back and I believe it was 2015, 2016 to kind of talk about these things and they kind of hit some different parts here. What is it that we need to solve? These are some standardized interfaces, configuration, privacy and safety, instrumentation and feedback as well as dealing with software errors. I think that's a big one. I want to pull that out. This generally means updates in a field, vulnerabilities, and how do you get those things out there in case there is a problem. Not just to say, "Oh, well there's a problem. No company is perfect. No company makes a device that is perfectly secure through the entire lifetime."

So you have to address and have a good response to that and I would always challenge any company whether it's Digi or not is that's actually a very good metric and to tell how well a company responds to their security. Is it true they understand yes, there will be issues and how do they respond to those. If they acknowledge it, have a methodology and a place to fix those securities and move forward, that's a great response. So it's really how companies respond. And I think that's really what that was trying to call. So feel free to look at it at the link below. You can read a little bit more about that workshop.

This is just another example of kind of showing some of the other compliance pieces. And the reason why I'm going through these by the way is I'm trying to show these are the drivers of the compliance but at this point, I'm not very familiar with a really good IOT framework that exists that covers all different industries. The NIST report is one again from the...specifically this one is one that is from the Federal Trade Commission which then contacted the Federal Government in the United States to be able to look into the energy sector. And that's what this was covering, to try to figure out how secure is energy.

So they came up with, again, a few tenets that they see. If you notice, these are all kind of falling into the same type of methodology that we're finding. This is the first initial attempts, I'll just show an example, of the NIST IOT cybersecurity framework which is being updated but they kind of have a lot of these pieces in play. Keep in mind they're using...they're borrowing a lot of these elements from...many of these things from a traditional server farm which is good but IOT is fundamentally different so we have to understand how that's going to work within respect. 

The other aspect, what I'd like to also adjust and describe that is a really key point to make is I've had many people within IOT and actually just security in general talk about where does security exist? Whose responsibility is security? If you do ever get that answer, the answer is everyone. And that's kind of what this is talking about. For our products and the way we view the development of our products is we're going to take certain steps to be able to build a secure product or it may even be a service but the intent is is that we can't do it all. 

We can build, for example, we can we can build an LDAP authentication module within a device but it is not our responsibility to run the LDAP server that it goes against. LDAP, for example, is a really good example as a protocol that helps identify people logging into a device. But since there's many different compliance standards and how those people are supposed to do, generally it's supposed to be centrally authenticated. That's where the LDAP server comes in. And many people use something like maybe Microsoft Windows to be able to do some of these pieces. 

But the intent of us as a manufacturer is we need to be able to build and make sure that our device can support LDAP so that people can do that centralized management. And that's kind of what our commitment to security is, I'm just giving you an example. But ultimately, it is the manufacturer to build, for example, in this case an LDAP authentication agent within a device. The corporate user or I should say the corporate…maybe your corporate IT department, it's their responsibility to run and manage the Active Directory and the LDAP server for example. 

And then the end user of course, they do have a responsibility as well. They have a responsibility of protecting their identity, making sure their password meets compliance standards, and not just giving that out. So it really is a whole complete subject and its different facets of that in security and being able to completely have a secure solution in the field. Case in point, the one great example I've seen of this is in the payment card industry, there's something called PCI DSS. With their new, I believe it is 3.1 and above control frameworks, they actually have where certain, as you can see here, certain controls like the 1.1.1 control is that if…and particularly if you're in a cloud environment, this is…would be big one is, well, is it your data center's responsibility? Is it your responsibility as running the OS? Or is it your application responsibility or is it the end user customer? So they actually, and it is a required mandate that you actually have a matrix. This is an example of actually one of our control matrix to show as a document who's responsible for different pieces and aspects of that. This is something that I would always, if you're doing an evaluation of an IOT device out in the field, think about this. So who's really responsible for certain control points because it does make a big difference?

You may not be able to be able to do them but these are things, for example, if you are being hosted in an environment at some point, some of those controls go away because you've signed a contract and that's a way of getting the control in places. You've got a contractual contract that they're going to provide that control for you. So these are kind of examples where we see that this is important and this is kind of where we're coming from when we try to build secure devices, that we're thinking about the whole entire chain. 

And again, you can see the Federal Trade Commission, other things that they're trying to go through, understand these things. Security has to be built in and part of the process. This is getting very true for medical devices for example to be able to make sure that those pieces are actually tested prior to being shipped. And understand security as things like you'll hear defense in depth which means don't just rely on one control so the whole thing opens up and now, all these issues are a huge issue. If you have a few different levels of control, that's great because assume they are going to be broken. 

That assumption is very key and it helps you design and understand your risk profile. And really, security is not necessarily about doing really cool things. It really is about reducing risk or understanding your risk and sometimes, there are secure things that you may not care about. Is it really a thing that you need to have a password changed every 30, 60, 90 days if it's we're not really protecting an asset? Then you've understood the risk profile, that may be one choice that you've made. Because there is a trade-off of security and ease of use.

So these are, again, I just wanted to throw up some things. These are our other ideas that are kind of rapidly coming together in the organization, you know, things that you need to do. Understand encryption in flight, encryption in a rest, logging, configuration management, identification of the device and of the people, doing that risk analysis. And again, over the air upgrades which is that firmware upgrade field. So rapidly, we're kind of coming to a solution of these are the security features and functions that need to be in devices at least need to be [inaudible 00:11:20] in the field, hopefully device supports these, and then can be implemented by the end user and customers.

So one of the things that I kind of wanted to start before we go into what Digi is really doing, again, this is another aspect that we've been trying to push out is from a manufacturer responsibility, we've kind of taken the stance of secure by default. What that means is that when you get a device, that it is secure out of the box. Now that is a very difficult thing to do. I will put that out because the issue as I just said is security can be diametrically opposite, if you will, of usability. We also try to make our devices usable out of the box. 

So if you had to open a box and two days later, you finally got it configured because of all the security parameters to use it, we would understand that you might not wanna use that device. So we try to make it very usable but yet secure. So there's many different methods that we try to do that. So what I will tell you, it is not an easy thing but that is kind of our intent, is that we understand customers may not go through those process. They just wanna get quick and dirty logins and start using the device. But if we can make it even if it's just a small, minor annoyance occasionally, that would be great but that's kind of our intent and hopefully nothing.

So if it comes out and it's secure and you've got it and use it and you know and you can have trust, that's our ultimate intent. One thing I will also say is that there is some legislation being driven at this point, at the time of this recording. I know some in Massachusetts and potentially California. I believe in England, there has been some cases as well that they're looking for kind of securing some of these. And traditionally, these laws have been around two different things. No default user IDs or passwords. We found that's horrendously horrible because nobody changes those. Again, secure by default. 

But the other aspect is firmware updates, ability to update within a field if there is a security issue. And that's kind of where the legislation...mostly that's in consumer but that's where a lot of legislation is trying to hit at this point. It may become a legislation, I don't know but it is definitely being talked about to be able to push... And most notably, that is in the consumer space but it also will be hitting industrial spaces as well. So first we wanna talk about this concept as I said is, this is our promise. What is Digi's promise to our customers?

We have something, a trademark called Digi TrustFence. And really, it is our attempt to be able to say, "This is what our promise is to our customers that we do for security." Again, it's not to make a device completely secure so you can just take it out of the box and stick it in. There's responsibility. We're trying to reduce as much responsibility on the customer as possible but they all still have a little bit of that. But what we've kind of done is that we've taken this framework that it applies to IOT devices and effectively, we understand the risks of what's being attacked out in the field. If you will, the Pareto principle of IOT security, the 80/20 rule as they call it.

We wanna know what… Some people call that low-hanging fruit. What are they gonna be, the attacks? Case in point, default passwords was a very huge attack. If you're not familiar with the Mirai botnet. That is actually one. I think there was 31 or 32 passwords, maybe more but not a lot that were tested. These are default passwords on a bunch of different devices and it turns out that almost nobody out there changes the passwords. They're able to get in and they know, of course now know hacks and they can install malware on your device and now you have what they call a bot on a botnet that you can now do bad traffic and bad things and they effectively control you.

If you were to not have a default username and password on the device, you would have basically stopped the Mirai botnet at least in many different forms than I'm aware of. So that's kind of the intent is that we do with certain things. So the intent is and I'll talk about this second part because this is really critical too, the intent is is that from a security perspective, there's something called, and I think I have a link on the bottom here from Lockheed Martin, it's a great white paper talking about the cyber kill chain. The intent of the cyber kill chain is it's not just one issue that breaks your entire system, one security issue that breaks your entire system, it's about a number of different issues.

You make one mistake, you make one other mistake and they may be small mistakes, and eventually, the piece falls together and you have a complete attack. So the intent is not necessarily to make the perfect device out there. The intent is to stop the cyber kill chain. And we kind of picked these pieces, if you, these tenants which we'll cover in a second but these tenants that will say, "How do we stop the cyber kill chain so that those attacks don't happen?" I'm not saying the device is, you know, I'm not saying that persistent. They call it advanced persistent threat, if you've heard of that, APT. You know, as an APT attack with someone who has lots of money and years of time, they will get in and that that is just impossible to be able to address that issue.

But that's not what we're talking about. We're talking about that one simple thing that's out there that has a misconfiguration that basically leaves it open. We're trying to redo those and do a good-faith best effort security and you'll find that it will stop 99.99% of these attacks that you see out there. So we're gonna cover those pieces as you see here. So those pieces are for specifically, there's six actual tenets that we have. And the first one I'll talk about is kind of very quickly, secure boot. The intent is...I honestly don't like that term so I'll put that out. Everybody is very familiar with secure boot when we talk about it but the intent is that any code that runs on that IOT device, even if it's code that you put on for example in a Python interpreter, you know that it's valid, it came from you. 

That may mean like code-signing things or that aspect or at least a traceability to understand what's running on it is actually your code. If it is a box product as we call these in Digi which is a product that we do where we're building the firmware, we wanna make sure that nobody can install their own firmware to do bad things or alter the firmware and deploy it out there in a remotely possible way to get many devices out there. So that's kind of our intent is we have signing, we have things that are in many different ways that we do it for different products. But the intent is you can't just go and install edited or modified firmware to be able to do what you wanna do.

The second part is authentication and that really kind of hits two things. It's the device authentication as well as the user's authentication. Case in point, so we talked about how do you authenticate that user. Is it a radius user doing authentication, things to that effect, so that we know we have appropriate use of users and traceability. The next two parts I'm gonna cover I think kind of go in together, the protected hardware ports. Protected hardware ports and I would actually say secure connections as well kind of go hand in hand. The intent is that there are no hardware ports you can just plug in and now get full access to the device. You still have to use the password. You still have to do certain things even if that means you open up the device.

Some of the engineers may know about JTAG. JTAG attack is an easy way to get the configuration out that may include passwords so we do protections on that. Serial ports as well. But the secure connection is the same thing. It's just a logical connection. We're not using telnet for example. We're using SSH and a good version of SSH to make sure that data is encrypted basically in motion if you will out of the box. So all the protocols are encrypted in some form or fashion, that we're using the encrypted protocols, not the unencrypted versions of that. There are many cases I will tell you though. There are boxes that do support the unencrypted protocols and that is for legacy issues. 

Where we're standing on that is things that we may still support that but they're not turned on automatically out of the box, aka, secure by default. So it will be a secure service on it. If you need that service, then that's something you can go in as a customer. You can go change that configuration to be able to reduce the security. And again, we understand it's not...is although we help you in establishing that risk, it ultimately the risk association, the risk profiling and the risk work is something that needs to be done by our end customers whether they choose the features or not. 

That's something we just can't make that choice for our customer. That's something that they have to do. The other two aspects we'll talk about, the secure storage one is a unique one. And what we're saying really with that is it's not just necessarily storage per se but is we have secure hardware on board and that secure hardware is going to do processing, it may help for authentication, it may help with securing, securing your keys in your device, that it's not just trivial to plug in and read an NVRAM configuration out of a device and now you've got some password for everything. We're trying to make it more secure. 

And so we have actual hardware elements based on the board to be able to help us answer all of these problems. And the last one which is really important, I'm going to hit on quickly, is really a life cycle. Some people call it SDLC, system design life cycle, that's kind of the ongoing monitoring support, is we're gonna be looking at the device. And when we build the device for security, we're not thinking a device that we ship now and that's it. We're thinking about security if there's a vulnerability of it eight months down the road, we're gonna patch that vulnerability, we're gonna put new firmware, and we're gonna support that.

So we're doing security audits and a lot of other things in that aspect. So let's talk a little bit more and kind of go through these really quickly. Secure boot. Exactly. What we're trying to do is ensure the authenticity and the integrity of the code running on the device at boot time. This is in updates and at boot time. So we validate that, we have add some…a lot of digital signature. There's some what they call ECDSA, elliptic curve digital signature algorithms, that kind of help with those pieces. But what it effectively is is that if someone takes our firmware and tries to change it, we are not going to allow it to be installed, at least not on a remote aspect where you can do it on several hundreds of thousands of devices.

This is an example here which we talk about is how we do this process. I just wanted to go in a little bit but effectively what it is is that we take our images in binary. We actually have a secure signing server that we go through that has been secured. When we build code in an environment, we actually do an encrypted signing of that so that we know that we have traceability from the entire process of the firmware, of literally what some people go...in some of our routers for example, we do use an embedded Linux but those objects that are running inside that embedded Linux, each one of those is validated through as a whole binary object so we know that none of them have been altered.

And in many cases, this is actually forced down from the CPU. The CPU doesn't even boot unless it's properly signed. And a lot of those signatures are actually programmed at the time in the factory and when we make this device to be able to make sure that these things and those signatures are there so it's effectively unable to be able to run anything other than software that we have signed. Again, getting into authentication, one thing I will say is a lot of these aspects, by definition, the Digi TrustFence is...some people like to say nebulous. 

We don't necessarily try to drive into very specific examples, a technical example. It's a framework. It's a framework to be able to say these are the things that we're thinking about because these are the attacks. So you can see, case in point, the authentication, what we're looking for is the ability to support local users, things that might mean is for example the password that's stored locally is not actually just stored. We do hashing methods where we can and appropriate hashing methods. And if you're not familiar with that term, feel free how to store passwords. We make sure we salt and hash passwords in an appropriate algorithm if we have to store them so that they really can't be attacked from many different ways. And again, we're always trying to figure out what those algorithms are and update them to make sure that they're secure.

Again, we talked about no default password. We've also talked about radius/TACACS/LDAP which we're looking at. And that's kind of that support, the centralized authentication. We understand that that's a very common thing to support with many of the compliance standards out there, case in point, you know, many organizations have many different methodologies on how the complexity of the password, the length of the password, how long does it expire, where it is. Well, we just can't implement all those controls on a device. It would be very difficult to do because devices are limited in space and CPU power. 

So that's why we support the central authentication and that's actually really where you should be doing that because you also should be logging those actions, you know, failed and as well as authenticated. True authenticated sessions should be logged and used for other methods. If you have a SIM or something like that, that may be an example where you would put that into your Active Directory, you look at that, you make reports to be able to see if there's any attacks. We're also looking at device level authentication. There's some pieces that we do have in play now but we're still working on this one honestly but the intent is that we wanna make sure that the device has been authenticated and nobody has changed the device or is trying to spoof the device and that you know that a device in a field, if that value that say if it's a some type of monitor of a tank, for example, that the value that it's gathering really came from that device and we can prove that authenticity.

That's not encryption. That's authenticity. We wanna make sure that yes, that value was read by that device in a field and I feel confident about that. Encryption is we're gonna protect who can see that value. But if it's a tank reading, maybe you don't care who reads that. The other aspects we also look at is two-factor authentication. We wanna make sure we support that. That generally does fall under radius. And then, of course, we wanna make sure that we do have a set of logging success and deny events on our devices so we can actually report that and have that roll up into a security office or something like that that may be looking for attack vectors.

Secure connections. Again, we talked a little about this JTAG, SSH, HTTPS. This means support of certificates on a web server. Many devices have web servers that are front-end that you need to work on and that might be a case that you have to be able to deal with things like that. We also make sure that we also have other encrypted types of protocols that we support, some NTP, Syslog as well, as well as the ability to reduce, disable services that are not used. That's a pretty common thing. As well as some of our pieces that we're looking at, support of something called Digi Remote Manager. Digi Remote Manager is a Digi product. It's a service. And the intent is you can literally manage your device through a remote portal and actually turn off authentic...turn off things like HTTPS and SSH on a device.

Devices will only do a very secure reach out to Digi Remote Manager but the cool part about that is that you can actually manage all the aspects of the device from that portal on a remote site. It gives you a centralized authentication, a centralized authenticated model that you can go in and control the configuration. The intent is Digi Remote Manager will help you with a new security to provide these other aspects of security. So let's talk about secure hardware. Secure hardware is not something necessarily that we're trying to meet a specific tenet of Digi TrustFence but it's really there to support the internal features of Digi TrustFence as implemented that we do.

Case in point, and this is one that I'll talk about as we look at things like entropy and random number generation. Effectively to boil this down, a hardware random number generation and this is not a software function. It actually has to be a hardware, what they call true hardware random number generator is required for any cryptographic operation. That's just one of those, if you will, tenets of security. You need to have that. There's been attacks on those things, and you can read the papers as well, that identified. But these are things that we think about that truly, we wanna make sure that this is something that's out there. So that we know that the encryption or all the authentication or any of the algorithms that we use for security actually have a good background and can be truly secure.

The hardware that we use, there's a bunch of different chips that we're using at this point but that's kind of the intent of them. And we kind of use those to force that through and to kind of provide this as a feature on the device. That also, a chip also does secure storage, case in point, that we talk about key storage. You may have a password that needs to be stored. Now normally, a password, a user password, you could say, well, that's hashed, right? That there's no way of getting back the original password. Unfortunately in IOT devices that have to talk out to legacy services, they may need to know and store a password to authenticate the device back to that service and that's where our password may need to be stored. 

And this is kind of what that provides us is an ability that it's very difficult to get that password out of the device. The hardened pieces we talked about, the actual hardware have protection against attacks on the hardware level so it's not easy to get out those easy…instead of just connecting up like say, for example, a JTAG and reading it out of memory. There are things that we're trying to mitigate and try to provide that process and those hardened pieces. This also helps for some cases within our secure boot as well. So this is the security life cycle that I talked about as well which is, again, our guarantee to our customers about security on the device and it's really through the life cycle of the device.

And really what that means is a few different things I'm gonna go through here. Continuous vulnerability testing. If there's a new vulnerability out there that impacts source code and things that we've used and say from the open source community, we're gonna validate, we're gonna look at that, we're gonna get that patched if it's pretty critical, and we're gonna make those available to our customers. Case in point, we do have in our www.digi.com/security, the security webpage, it talks about common things that are out there as well as an RSS feed that kind of directs you to things to get updates on your devices from the support portal so you know that when these issues occur, you will get an alert that new firmware has been upgraded and it may or may not have a security issue that you need to review.

We also do pen testing of our devices. A great thing about vulnerability testing is they test the known things. If it's an unknown that's very unique to us, that's gonna be done by our pen testing in internal as well as external teams. We do have trained people on staff that are certified pen testers that go through and identify devices. So we continuously do that. That is a very hard...by the way, I will say that's a very hard thing to do because pen testing is effectively testing for all possible inputs which could be infinite but we do have some skilled people on that and we try different things in many different ways. Likely as well is that we have external people and teams that we work with. 

In fact, I would say at this point of the year, that is that we're very much willing if you have, as a customer, you find a security vulnerability, we're very happy. There's ways to report that from our website. You can report that to us and then we will definitely take a look and work with you to kind of mitigate that control and kind of see if we can close that security down. One thing I will ask though is if you're running a vulnerability test and just a scan sending it to us as a vulnerability scanner, they are notorious for lots of false positives. We'd love to see if you truly do have a true vulnerability and you've been able to demonstrate that. That's definitely something we'll address.

Unfortunately with lots of vulnerabilities and false positives, it's really hard to work with tools. Some tools will literally produce a thousand pages of output that is false. They're difficult tools to use but we definitely...we run them internally. We actually run multiple tools internally to try to get a continuous look and feel across the industry of those tools. We also do static analysis which is actually looking at our code base, looking for security issues within the code base and this is looking for programmer mistakes or things to that effect. In some cases, we do a lot of fuzzing as well. 

These are kind of just throwing input, bad input specifically, in a very known way into a device to be able to see if it will tip over or attack or if it exposes a security vulnerability. Again, we're also looking at is a certain algorithm X or Y, now not secure, considered secure. For example, about a year and a half ago, the SHA-1 hash methodology was broken although in a limited form. So we're very familiar with those things and we kind of look at and evaluate our products if they rely too much, that that could be considered an attack. We'll have to figure out ways to work on that. So those are the things that we look at.

We also kind of look at what are those other security features that we need to get within our devices to make sure that they're on our roadmaps. And again, secure configurations falls into that. Although we want to do secure by default, we have recommended security settings that we have for devices to make sure that they're fully hardened and secured out there and what those recommendations are as well as we are also a member of the Center for Internet Security and we kind of help...we're trying to drive templates out there so people can use some more automated tools. Our Digi Remote Manager as well will do that for you too as well as it will look for configurations and what you call a gold standard to be able to validate against things like that.

So other elements I wanted to cover, again, it's one of those things we do, again over the air upgrades. We wanna make sure that firmware can be done out there. You can deploy in a mass. Digi Remote Manager is awesome for that. It allows you to kind of say, "Put this firmware out there. It tracks it. It's great." Again, we talked about the configuration management. Network time protocol, we do support...we actually have a direct authenticated access to NIST time servers. We provide that as time.devicecloud.com. We have other things that we look into the OS and this is not true for all devices but certain things. 

For example, Linux, if you're familiar with this, there's things called AppArmor. We have profiles that are loaded just to try to...it's not necessarily if you're gonna have an issue with a vulnerability in a firm but when if there's a known exposure, the great part about AppArmor is that it may take, if you will, the teeth out of the attack and that's kind of the intent. Things like ASLR and data execution protection which is protecting the stack against execution. So these are the other things that we build into the firmware and try to be able to make a hardened device.