Securing operational technology: ICS, IoT, AI and more

[INTRODUCTION]

[00:00:00] CS: Every week on Cyber Work, listeners ask us the same question, "What cyber security skills should I learn?" Well, try this. Go to infosecinstitute.com/free to get your free cybersecurity talent development ebook. It's got in-depth training plans for the 12 most common roles, including SOC analyst, penetration tester, cloud security engineer, information risk analyst, privacy manager, secure coder and more.

We took notes from employees and a team of subject matter experts to build training plans that align with the most in-demand skills. You can use the plans as is or customize them to create a unique training plan that aligns with your own unique career goals. One more time, just go to infosecinstitute.com/free. Or click the link in the description to get your free training plans, plus many more free resources for Cyber Work listeners. Do it. Infosecinstitute.com/free.

Now, on with the show.

Today on Cyber Work, I’m joined by Francis Cianfrocca, CEO of Insight Cyber to talk about security problems around OT and IoT systems. You know this topic is one of my pet concerns, and Francis treats us to some very surprising stories of intruders in the electrical grid. Why it's so hard to secure a set of machines that often predate computer technology? And the small changes in your community that can make huge differences in the entire security industry. It's a bit of bad news and a bit more good news today on Cyber Work.

[INTERVIEW]

[00:01:34] CS: Welcome to this week's episode of the Cyber Work with InfoSec podcast. Each week, we talk with a different industry thought leader about cyber security trends the way those trends affect the work of infosec professionals while offering tips for breaking in or moving up the ladder in the cyber security industry.

Francis Cianfrocca the founder and CEO of Insight Cyber, a cyber security startup developing a new AI-powered security service to provide insights and protection against a wide range of threats in cyber physical environments.

An inventor of key technologies at Insight Cyber, as well as a previous company he founded, Cianfrocca is a noted expert in the fields of data security, computer language design, compiler implementation, network communication and large-scale distributed application architectures. Having a background in music, Cianfrocca attended the Eastman School of Music at the University of Michigan. Yeah, today our theme is AI solutions in both the IoT and the OT sphere.

Francis, thank you for joining me today. Welcome to Cyber Work.

[00:02:36] FC: Thank you so much. Great to be with you.

[00:02:38] CS: To help our listeners get a sense of your personal history, let's start out by asking how you got interested in computers and tech in the first place? And how that interest uh eventually expanded to include things like cyber security and AI?

[00:02:52] FC: Yeah. Thank you. Well, I’m one of those kids, and you've met many of them, that started off with interest in computers just kind of born in. And I was always been interested in them and always also been interested in electronics. And so, that kind of led me to what we call the cyber physical world. In other words, connected machines. Not just computers. But non-traditional devices and machines that are connected to networks. And I was in that real early. And of course, artificial intelligence.

And cyber security came along when we were building some really large-scale systems for enterprise data handling. And security just got to be a major thing when the Internet got big.

[00:03:40] CS: Mm-hmm. Oh, yeah.

[00:03:42] FC: And artificial intelligence, there's just so much to say about that. It's so transformative in many, many ways. I do believe that it changes the way every human activity proceeds. And so, that's just a real important part of the background, too.

[00:04:01] CS: Yeah. Okay. We're going to talk way more about that, too. Because, yeah, we've had a couple different people talk about AI sort of tangentially to other things. But I’ll be excited to kind of go right into the heart of the beast here. Whenever I do my pre-prep for these, I like to peek on my guests LinkedIn profiles, as it gives me kind of a quick shorthand to understand your career journey.

Yours is quite interesting, as it seems that you went right from your schooling into founding the IT and OT solutions company, Bayshore Networks. Were there any other formative work or learning experiences before that?

[00:04:34] FC: Yeah. I mean, I left out a bunch of stuff. I worked on Wall Street for a few years. I’m a New Yorker. And I also did a lot of work on what industrial controls engineers would recognize as the very early PLCs and what we call distributed control systems back then. That's why I said, connecting computers to machines has always been big in my career.

And I also founded a company, prior to Bayshore, called Tempest Software. And we built a very large-scale messaging bus for enterprises. And this is right around the time that people were becoming very, very interested in the cyber security risks. And that just came to the fore. And just because we had to solve those problems, that ended up becoming my career.

[00:05:22] CS: Now, can you talk about like the early days of sort of the ICS systems that you mentioned there? Has it changed a lot? I mean, I’m always looking for people to talk about ICS, and infrastructure security, and all that, because to me it's just like one of the hugest imaginable threats just to think of all these different municipal facilities and stuff that are like way wide open because there's no one there to – But what was it like back in the day? Because now it seems like – With more tech comes more possibilities to hack it, and more worries, and so forth. But what was it like back then?

[00:05:59] FC: Yes. Well, way back in the early days, it was all about measuring industrial processes, temperatures, pressures, voltages, and feeding those data points, that telemetry, into computerized computer programs that could do more advanced analytics. And so, this would give you a view into what processes we're doing and also an ability to control them.

And I remember some of the earliest things I worked on, the oil business. Controlling new oil refineries. You realize, we haven't built a brand-new oil refinery to any great extent in the United States for over 30 years. That's a different story. But 30 and 40 years ago, we were building new refineries that had these computerized control systems. All right?

And a lot of them – Well, way, way, way back then, the kind of sensors, the way that you connected the computer to the physical world was through – Well, we call them field bus. That's what engineers call it. It was serial, RS-485 protocol serial bus. Differential voltages. The kind of thing you'd find in a telephone modem from way, way back then.

And about 30 years so – 30, 25, 20 years ago, there was a big motion to replace all that stuff with ethernets, with ethernet computer networking. Okay? And the motivation for that at the time was cost, because it's a lot cheaper, a lot easier to manage. And what happened at that time at the same time, people said, "Wait a second. All of a sudden all of our industrial control protocols are now – We can connect into our computer applications and we can feed them outside of the shop floor, or the substation, or the water processing plant. Feed them into our other computer systems." And that just opens up many, many, many wonderful opportunities to manage your processes better, save money, control them. And this is all really – You won't put that genie back in the bottle, because there's way too much business value that is created by it, right? And that's another long, long story that many people have told. But, of course, the security aspect of it comes in in a big way, right? Because, now, all of a sudden, all the bad stuff that is out there has access, if you will. Sort of jump the gap, right?

And one of the things – I don't want to go too deep, because you got a lot of questions, Chris. But I think it's important to recognize, the vulnerabilities – You talk a lot about cyber security, and everybody's had the cyber security training for computers. And we know what it all is; firewalls, change your passwords, don't click on emails – On links in emails, right? All that's good advice, we all have to follow it.

Think a little bit about, if you compromise a computer, okay? You can steal information. Really, really bad. You can damage your reputation. You can damage your customer service. All kinds of bad stuff um. Or you can take down your email. That's really bad for business. You can mess up your databases. Take them offline if you're really good. That's bad for business.

But think about if I can destabilize or make a robot or a water filtration system do things they're not allowed to do or they shouldn't do, okay? The impact of that is at a whole different level. Now, damaged information. Bad stuff. You recover from it. But now we're talking about maybe hurting people, killing people.

[00:09:52] CS: Poisoning people. Yeah, absolutely. Shutting down power grids. And creating – Well, whatever. We can avoid the sort of Doctor Strange loaf comparisons. But, yeah, everything up to that point. For sure, we can also still talk about very cogently as always, an option to be watched out for.

[00:10:13] FC: Oh, yeah.

[00:10:14] CS: Yeah. Before we get into some of the OT and IoT stuff we're going to discuss, I like to give people a sense of what a person's day-to-day job is for our guests, because I think a lot of our listeners like to imagine themselves in these different roles. Now that you're the CEO of Insight Cyber, can you tell me about the day-to-day work that you do in that role? Do you have certain common tasks that you work on every day? And how regimented is your everyday? Do you have anything that gives you the Sunday night blues that you worry about? Lose sleep over? Things like that?

[00:10:49] FC: Well, we're a startup. We're doing very, very well. We're getting a lot of new customers in. But I spent a lot of my time building my team, all right? The most important thing for a CEO of any companies, but especially a startup, is to build a great team. And everything that that involves. That's a lot of my time. But most of what we're doing on a daily basis is we're just working with new customers. Bringing them into our system and showing them – What we focus on is helping people to see what they're not seeing, all right? See what they're missing. Because that's really important in cyber, all right?

And a lot of the problems that people have with managing their cyber defenses is because they don't have good enough visibility or deep enough. And that's a very, very big job. If there's anything that keeps me up at night, it's just how bad it is out there, right?

As we are looking in our customers environments and we're helping them to see to gain more visibility, and more detail, and more insight as to what's going on. All kinds of stuff is just coming out of the woodwork. And you can use – Your listeners use their imagination about how bad it is. But there's a lot more going on. The attacks are determined, intelligent, stealthy. And that's where people kind of miss the problem. That's what we find ourselves working on a lot.

[00:12:19] CS: Yeah. Whenever I talk to ICS people and, really, the people on here, it just feels like I’m talking to like a locksmith that's like walking into a town where no one has any locks on their doors. And you're like, "What are you all doing?" I mean, I got to ask about that. Like, how did we let this get so bad in the first place?

[00:12:38] FC: Well, I think the one word, if there is a one-word answer to that, it's convergence, right? I told you the story about how people saw great value initially for financial and management reasons and then later, just because so much good business insight and value comes from connecting systems together. And the industrial control systems – And I love talking about industrial control. So, don't let me bore you with this.

[00:13:07] CS: No, please. I love hearing about it.

[00:13:09] FC: Industrial controls, as we understand them now, the mathematics underneath them, which is quite sophisticated. It's been around for a little more than 100 years.

Well, computers have only been around for 60, 70, like that, and in computerized control systems for much less than that. And control systems technology is exceptionally mature from a mathematical perspective. It's fascinating stuff.

The ways that control systems fail are very well-defined mathematically. And engineers work with these well-defined situations. As soon as you bring bad guys, as soon as you bring hackers and malicious actors into the mix, all that lovely math flies out the window. And you're talking about stealthy people. We're trying to hide their tracks. And the control systems were designed for robustness. They were designed for safety? Okay. They weren't designed for security. And it's an old story, right? But it is really true. Convergence of these systems, when it happened, there were no additional controls built in to apply security, right?

Really, it's hard – Perhaps people who aren't so well versed in ICS, it's hard for them to understand that adding computer style security controls to industrial environments, exceptionally difficult to do, because the processes like to be naked on the metal. They don't like to have extra packets added in. They don't like things like firewalls, because it changes the timing of the processes. And so, there's very little you can do.

And even patching. One of the big, big things, everyone, you always patch your systems to the latest security fixes. That's almost impossible to do with hardcore industrial controls for a really important reason. They're designed for safety. They're tested for safety, okay?

And so, as soon as – Even if you change an operating system version just by a little bit, just to patch it, you've invalidated all your safety testing. People resist doing that. So, you need another approach to add the security.

[00:15:32] CS: So, some of the instability – Or not the instability. The insecurity of it is almost kind of baked into the process in a way that it's kind of hard to undo. You would need to kind of rethink the –

[00:15:42] FC: You're absolutely right. You're 100% right. Well, a more positive way to say that is we just need a different approach. Okay? The approaches and techniques that we use to add security to IT t systems are not as applicable on the ICS or industrial, or the IoT side. And so, we just need better methodology or different methodology.

[00:16:05] CS: All right. Well, let's get into that is my next question anyway. What would it take to make a major improvement in the state of our infrastructure security legacy systems and our insecure or un-patchable IoT devices? I mean, if this is a way to go – If there's a way to go forward with the massive initiative of some sort, what would that look like? Or is that doomed to just be a forever a piecemeal repair here and there?

[00:16:29] FC: Yeah, Chris, I think that is just a wonderfully framed question. And as I think about it, it's like we all would love to have a – Forgive me for oversimplifying. But a silver bullet. A major thing, like a moon shot. Something that we could spend –

[00:16:49] CS: Yeah. I use the phrase a magic gavel. You know? You sign something in the law, and everything gets changed, and it's all fine.

[00:16:58] FC: No matter how much it costs. Let's recognize, it’s a problem worth solving. It would us billions of dollars. But let's do that. Well, I’m not quite so sure there is one single approach. And we can talk about this, because people have proposed things like massive isolation of industrial systems.

[00:17:20] CS: In what sense?

[00:17:22] FC: In the sense of literally closing off the computer networks. This has been discussed a fair amount in the power grid, in the specific context of electric power, where you take all of those assets. And they're distributed geographically. Electric power is different from most other ones, where like a factory floor, it's all within four walls, right? You've got one chokepoint ideally where you could close it off. Power grid, it's over thousands of miles of geography, okay? Much harder to do. And there has been a lot of thought given by smart people to standing up a completely separate closed computer network, right? Or systems of computer –

[00:18:05] CS: Like an electrical intranet or something?

[00:18:08] FC: An electrical intranet. Yeah, pretty much. If you think about it, it doesn't convince me. Because, again, all those assets are out there. And you can physically access them.

[00:18:25] CS: You've closed one door and opened many others. Yeah. Yeah.

[00:18:31] FC: I think we need to get better at monitoring and understanding what the threats are. And one of the things I’d like to say very much by way of – To answer your question, right? What is a major improvement we can make? What is a methodology improvement we can make? We talk a lot about looking for vulnerabilities, right? A standard process with cyber defense is to look at all the machines and all the assets you have, map them against known vulnerabilities. Okay, that particular operating system, or that application, or that particular industrial machine is vulnerable to this kind of an attack.

And there are databases, big ones and really well-done ones. Here's the problem with that. Scale, right? I can look across an infrastructure with millions of connected devices in it and I’ll find tens of millions of vulnerabilities. It doesn't give me a pathway to become more secure, all right? It gives me tens of millions of things to look at. And I’m not going to patch them all. Okay? Yeah, let's get on that. Three weeks from now, we'll patch everything. Okay. Forget about how much that's going to cost us and how many additional problems it creates. Oh, and then three weeks from now, we'll have a new list. Okay, we'll start up. No. It's not plausible.

In my company, among my friends, we like to talk about don't look for vulnerabilities. Look for attacks. Look for what is really going on that is really problematic. Catch it early so you can do something about it. And there's some interesting things to that. Some aspects of that matter with industrial. Because with industrial, you have the ability to catch things early because here's why. And this is subtle. And this is interesting um.

To attack a power station, or an auto assembly plant, or a telephone central office, the bad guy needs to know what's in there. Okay? If you are just attacking windows computers and you find a new vulnerability in windows, that's all you need. Just find somebody to click on a link in an email and you're in, and you can attack them. Okay? With industrial controls, you kind of need to know what's in there so that you can – You find a vulnerable Windows computer. That doesn't change. That's still part of the attack methodology. But once you're in there, you want to do a lot of damage.

People do recon, reconnaissance, and they snoop around those networks looking for stuff. And when they find something, they have to apply some pretty special knowledge to know what they can do to make some damage. That takes a little bit of time. And that opens a vulnerability for the bad guy. Okay? That's a way to attack him. Instead of looking for vulnerabilities, because those are everywhere, we look for traces where people are already trying to recon you, and we close those off, right?

Now, what does that mean? All right. That's another huge scale problem. And that requires very specialized knowledge. You got to know what you're looking for. Otherwise, you're boiling the ocean. Okay? So that's why we said that's a job for artificial intelligence. That's the kind of thing that if you had an AI that was well enough training, or a suite of AIs, each with slightly different training looking at different things, you could approach that. And that turned out to be promising and fruitful for us.

[00:22:16] CS: Yeah. Ai is almost like – I keep thinking of this in terms of like breaking into like Fort Knox or something. Like, if they find an open window, but the room that they get into has a locked door, or no door, then it's no use. We've had another guest that said vulnerable doesn't necessarily mean exploitable. And then there's other people who have talked about selling vulnerabilities that you found to hackers. Like, they just find vulnerabilities to find vulnerabilities and leave it to someone else to try and get in there and so forth.

I mean, AI solution almost sounds like it's not quite like putting like a welcome mat down for attackers. But it's knowing that there's always going to be activity on the perimeters and you're kind of – The AI is kind of like these sort of like CCTV cameras or something that are like seeing rustles in the bushes.

[00:23:08] FC: I think you're right. And it's really easy to oversell AI, because there's millions of people out there that are talking about AI and overselling it. And that's fine. But if you understand what it's good at and what it's not good at, it's really good at finding patterns in very, very large and high-dimensional data spaces. And this sort of matches that problem.

And I really love what you said. It's like if you had a CCTV that would just – If you got them everywhere, and they're good enough to just tell you when you need to know something and you can trust them to have your back the rest of the time, then you've solved a good part of the problem.

[00:23:56] CS: 99% of the cameras just show all quiet on the western front. And so, yeah.

[00:24:01] FC: Yeah. And I love the way you say 99%. I’ll tell you why. Because this gets to another thing that I think is very important about the cyber security mindset. We all, as cyber sec practitioners, have come up with, "Well, the bad guy only needs to be right once. I need to be right 100% of the time." It's the wrong mindset. You're never going to be 100% ever. And if you try, you're just going to spin your wheels. But if you find 98% of them with 20% or 10% of the effort and cost, you've done a lot of good for yourself.

[00:24:36] CS: Yeah. And also, I think that cuts down the ability or the chance that you're going to just burn yourself out because you're constantly like waiting for the inevitable thing to happen. And yeah, have to smash every bug, and have to get through everything.

Yeah. I understand that that money and budgets are always a factor. But what are some of the – We talked about the AI aspect. What are some of the logistical issues in implementing these kind of mass security upgrades? As you said, the moonshot, or the magic apple, or whatever?

[00:25:04] FC: I’ll tell you what. I think the biggest issue logistically, and that also is a really well-framed question, because that gets to what people need to plan for. Network implementation. I think that the bad guys hide – They're incredibly good at hiding what they're doing. That's their whole stock and trade. If they do obvious things, you'll find them, and you'll clean them out, okay?

You're really good at hiding their recon activity so that it looks normal. All right? And the only way to get better at that, to spot that, is with much deeper network instrumentation. That's putting sensors. Not just the so-called north-south links, where the firewalls are. But also, the east-west.

Okay. Logistically, that's a big challenge because it requires management effort. It's potentially unsafe, especially in IoT environments, if you don't do it right. We spent a lot of time, years, engineering out network sensors that would be easy to install very, very widely and very pervasively. They do edge analytics. So, what they send up to the analytics cloud platform is very small and very digested, all right?

They've done some of the work before you. And that saves on network logistics, right? Network costs. And I think that's really it. If you improve on the network instrumentation, then your sim solutions and your analytics just have a have much more to work with. And if the goal is to be AI-driven, AI likes more data. The more data you can give it, the smarter it's going to be. And so, that's really the logistic challenge, is just to see more. Just put more eyeballs automated. Your CCTV cameras, that's the right way to think of it.

[00:27:02] CS: And again, we don't often have moonshots anymore, and we don't often have these big unifying independence day style. Like, we're all going to solve this problem together because everyone – There's just so many little things. Well, we'll have to shut our thing down for six hours. And that's going to cause – We're going to lose X number of profit or whatever. Like, how do you sort of like bring it all together to sort of like implement this stuff and make people realize that it's important and doable and is not going to kill your shareholders or whatever?

[00:27:35] FC: Well, you couldn't be more right. Industrial environments, industrial production, or IoT environments, they can't go down. They're not allowed to go down. You just can't. I mean, yeah, there's a lot of stories we can tell about that. It's just out of the question.

The way to approach it is to touch those environments as little as possible. Ideally, just putting sensors in, okay? When we work with clients, we start off with just a handful of environments. Electric power company, for example, we'll pick two or three generators, or substations, or transmission lines. And we'll put some extra sensors in there. And all of a sudden, the asset inventory comes back, and it's much more accurate than they've seen before. And the rogue devices show up; the stuff that's not supposed to be there, the applications. And those just appear almost immediately. And it's safe and easy to do.

And at that point then, we've built some credibility in the customer, because they have to manage this whole thing internally with their own various stakeholders –

[00:28:45] CS: Yeah, yeah, you're just you're just putting it in place and then sending them to hoping that they're going to use their new gadget correctly, right?

[00:28:52] FC: But we work – Our job is to deliver better cyber security. Not just one more product, right? To work with them closely to say, "This is what you have. And this is what you've been missing." This is what you need to do about it. Okay. So, those are the questions we answer for them. And at that point we, build up trust rapidly.

One of the things that's always been a challenge with OT, with cyber physical, is a degree of mistrust between IT security people and the plant guys, okay? They sort of come from different worlds. And that's a much better problem than it used to be. It used to be substantial mistrust. But that's something you get good at in terms of corporate culture and working with clients to get them past it.

It's gradual. But if the tools and if the technology is non-invasive and safe, it goes pretty fast. That's basically a solvable problem.

[00:29:52] CS: Interesting. Yeah. Yeah, you're sort of casting things out here. And you're finding out immediately that – And it sounds like there could be like access issues. Like you said, there's like rogue systems or there's aspects of the network that like, "Oh, I didn't realize that was still connected to this thing." There are a lot of that kind of sort of like cleanup that should have been done years ago, right?

[00:30:16] FC: Well, the joke I – Well, it's not a joke. The reality we have, every time we go to a new environment – And it's wonderful because the plant managers are hungry for this information. And they just – It's very difficult to have really, really clear and accurate asset inventory and not just – Behavioral inventory as well. What are all those things doing? Including things I might not have suspected? And it's like we'll give people an accurate list. And we always say, "So, what's in that plan?" What do you got on your shop floor? Well X-number of robots, X-number of PLCs and all the other stuff, right? And they'll give you a network diagram that's probably five years old. All right? We'll give them back a list and we'll say, "Okay, what's this? What's this? What's this? What's this?"

And in some other cases it's like, "Oh, yeah. I forgot about that. I forgot to tell you about that. Yeah, that's out there." Other cases, it's like, "I thought we retired that two years ago?"

[00:31:23] CS: Yeah. Right. Right.

[00:31:25] FC: Well, it's still on your network, and it's still talking.

[00:31:27] CS: We fired Barney five years ago, and his account is still open. Yeah.

[00:31:32] FC: And the third kind is, "What on earth is that? Get the hard hat, get on the floor, find – You fix it right now."

[00:31:42] CS: Have you ever found sort of invasive hardware like that? Have you found like the equivalent of like a bug or something like that?

[00:31:49] FC: We very, very rarely. Like, single digit percent of the time, we will find rogue hardware like that. People are good at that. People know that trick. What we always find is rogue software, okay? And people are good at what they do. It's just really, really hard in a complicated environment to stay on top of everything. And you really need to know when someone has done something, made a config change, that left a little hole, okay? That happens every day. And so, it needs continuous monitoring. It's not enough just to do a sweep every year and close all the holes. You got to know continuously what's going on. We find all kinds of crazy things.

And you know what it comes down to, Chris? The vast majority? It comes down to unknown malware. And malware is the problem, okay? Because people get into your network and they do things you don't want them doing. And everybody invests heavily in sweeping their systems for malware. Well, that's great for finding malware everybody's seen before. That's already got a signature, right? So, you're sweeping all the software on your computer.

[00:33:08] CS: All the big names. Yup.

[00:33:09] FC: There's a DLL in it. It's got a little smudge in it. Well, that's not petty, or that's this, or that, or the other malware. What we find, kind of the hallmark of the approach of looking for behavior, looking for attacks in progress, rather than signatures or vulnerabilities.

Generally speaking, far more often than not, the majority of cases, we find malwares – We find signs of malware that have no signatures. Just this morning, we were working with a client and we told them, "Hey, you know what? I think you've got malware on this one particular computer." And we gave them the IP address. You know, I was wondering about that one. We just installed that in the front so that one of the interns could have some connectivity. And I was wondering about it. And we swept it. There was no malware on it. I told them, "You got malware on that machine, okay? Your students aren't going to find it. And I’ll tell you how – I know, it's because it's doing – Well, obviously, we can't give too much detail. But, of course. But it was doing a very standard kind of a diagnostic operation. It was an SNMP sweep, okay? But it was doing it in a way that was quite invasive. Not normal, okay?

We found another case in a production environment over in Europe. This is last week. A place that has generated electricity, okay? There's turbulence and generators in there, all right? And we found nine machines in that environment that we're doing ping sweeps. Not ping sweeps. Port scans. Port scans. You sweep around the network looking for open ports. And it's a tool. A lot of people use those. You just don't know what you got, right? That's a way of doing asset inventory, okay?

And our AI said eight of those are – You shouldn't be doing that in an OT environment, because it kind of destabilizes the switches. But that's a low-level alert. You shouldn't be doing port scans, all right? That an IoT tool. Not an OT tool. One out of the nine, RAI kicked out a high severity alert, because it was doing a port scan. But there were more than – Not one, but two different aspects of how it did it that were obviously designed to evade detection.

[00:35:41] CS: Wow! Man, this is –

[00:35:42] FC: Smoking gun. That is smoking gun malware. Okay? Obviously, they swept it. And they did the antivirus. These are good guys. They spent a lot of money on security and a lot of money on developing practices and expertise. But there's no signature for this malware. They didn't find it. But we found it through the behavior, all right? And they were like, "Oh my God, that's just –" Your jaw drops. Because, yeah, it's obvious it shouldn't be doing that, all right?

And again, the bad guys are good at what they do. So, the behavior was hidden. Yeah. They hide their tracks so you won't detect them. And AI, if trained properly, is good at finding those little signals that are going to go right past your eyes.

[00:36:31] CS: Interesting. Now, I want to get into some career stuff and some learning stuff. But before that, I guess I didn't really put it in the questions here. But can you talk a little bit about how sort of the IoT component of this varies from the OT? I know you said OT, it goes back 100 years. It goes back free computers. IoT is obviously a much newer thing. But are there any wrinkles in the way that you keep IoT secure that differentiate it from operating systems?

[00:37:00] FC: There's a lot of wrinkles. And the way I like to talk about it is, well, IT is traditional computers on traditional networks. OKI is non-traditional compute devices on non-traditional networks. But they're generally owned by somebody not IT.

IoT is the hybrid. IoT is non-traditional compute devices on traditional networks, okay? And usually, there's 10,000, at least, different kinds of IoT devices with different functional profiles, okay?

I’ve got a really great example for you. I’m not going to name the maker. Household name – You go to Home Depot to buy a home lighting switch. Those are wonderful. I mean, you can set up the colors in your house, mood lighting different times a day. Turn your lights on and off when you're not home. All that great stuff. And those are IoT devices. And they are very restricted functionality, okay? We think of them as having a job description, okay? A light switch has a job description. Its job is to turn your lights on and off and maybe phone home so that your smartphone app can talk to it.

Why did we find – Our AI found a lighting switch in one of our own networks that was doing those things and was also looking around for windows file shares on that network to connect to?

[00:38:36] CS: Oh, man. Don't tell me I have to get rid of my – I like my light function. I’ve done it. I push three buttons on my phone and I’m done for the night. It's great.

[00:38:48] FC: It's like, "Come on, guys." That's the kind of thing you want to know about. So, you either pick a different vendor for your light switch or whatever it is. But that's hygiene, right? And whether or not the maker designed that, that's outside of its job description.

What we look for with IoT devices, they are things that are designed to do very specific narrow tasks, unlike computers, okay? Computers are highly general, right? But anytime we spot an IoT – Whether we've seen it or not, okay? That's important. Because every single day, somebody's programming a Raspberry Pi to do something brand new, okay? Right.

But the great thing about IoT, behaviorally, is that they're there to do a job. And anytime you see it doing something that probably isn't part of the same job, you flag it. That's how we approach those. And that's turned out to be very, very fruitful. And I think a lot of the cases with IoT, I mean, it's all about convenience. We want the world to be densely connected, because there's so much value to that, right?

But we see things like – All right, here's another example. A company that had a ransomware attack, right? And they wanted us to figure out how it got in. The electric car charger in front of the building, that looks a lot like an IoT device, and somehow acquired their network, okay?

And so, that's like an IoT device that's in the network and possibly exploitable, possibly attackable by all the attackers that are in your network already, right? And we've seen – And this is a little bit more scary. Building management, commercial building management. The HVAC systems, okay? And the elevators and the closed-circuit televisions, right? Those are – And the economics of building management is extremely critical. It's so difficult to spend any kind of money on cyber security. That makes it difficult to be good at it. And very often you will find networks that just aren't locked down enough. And you'll see the HVAC equipment on the same network where people are walking into the building lobby with their iPads and iPhones just automatically acquiring the network, okay? See?

Somebody walks into a building where the air conditioning controls are literally reachable from the guy – Whatever malware is on somebody's iPhone? Okay. I’m not sure I want to be in that building, okay? And so, that's the kind of stuff you really just want to know about. Because the economics are so challenging, we need to come up with – And this is part of what we think we can do. We're trying to get to it. Make it so that that building manager has a cost effective and easily manageable way to spot those problems and knock them out before they turn into trouble.

[00:42:03] CS: Right. Yeah, that makes perfect sense. Yeah. I mean, because these are things that are not intrinsically securable as an object or whatever. Like, you need to sort of – Yeah.

[00:42:14] FC: I used to do this, right? I used to pull out of my pocket when I did presentations, like a Type K Thermocouple, right? It's on the yellow coiled cord you're dangling out of your pocket. Or it's a four-dollar part, okay? You're going to add a $35 firmware chipset so that it can be on the network. And now you've got a connected sensor. Okay. Are you now going to spend a few hundred dollars to put proper security management on it?

[00:42:45] CS: Add a firewall to it or whatever. Yeah, yeah.

[00:42:47] FC: I don't think. It doesn't work economically. It's not because you couldn't do it if we wanted to. See? That's where the challenge comes from.

[00:42:56] CS: All right. Yeah, now I definitely want to get to how we address the challenge and how the students and professionals of the future address this challenge. I realize that cyber security professionals all get in the industry for different reasons, whether it's for love of chasing the bad guy, or keeping a company safe, or just to prospect to a decent paycheck. I wonder if there's a way to get the word out that cyber security profession should consider aiming their job search at? Like, state and local government? Local utilities? Infrastructures? I mean, is there a way to let cyber security professionals in the US know that they could potentially be keeping their own town or city or municipality safe in a real tangible way?

[00:43:32] FC: I think there is. And of course, the challenge – I just did a conference last week up in Portland. And I said, "Bar none, the biggest challenge in cyber defense, is finding enough qualified people." Every head is like bobbing up and down. This is a room full of seasonals.

And so, I think that there's a lot of scope to partner between private organizations which invest heavily in expertise. And as you say, the public sector entities who own or are responsible for the critical infrastructure that we all run our lives on. And I think that a really important way of getting better – No. There's a lot of good people in cyber security. We don't want to get better people. And we want to get more people and that we want to retain them, and we want them to enjoy their work, okay?

[00:44:33] CS: Yup. Level them up and – Yeah.

[00:44:35] FC: I think a lot of the job is just, forgive me, it's scut work, all right? Because the tools and techniques that we use don't give enough highly actionable information and don't give enough depth of visibility. What you will find is very expensive and well-trained young people, all right? They spend their time writing scripts against SIM logs, okay? And your security login will give you terabytes of data every day from your firewalls and all the other things that generate logging. And it's just not good enough information. And that's why we think, if you can apply AI intelligently to take all that huge raw data and boil it out to the handful of events, a dozen maybe, a dozen a day. Not 500 million a day, all right? But take the events and turn them into, "This is really going on. This is suspicious. It's not normal. It's happening right exactly here." Okay? Now, the cyber security practitioners have a lot more to chew on, all right? And they become threat hunters and strategists, as opposed to just script monkeys.

And I think that's going to be a big help to getting younger people and smart people, because they'll make much more of a difference. See? I think that's a big part of what we can do to make that better for people.

[00:46:08] CS: And I think it's also worth reiterating that sometimes when people hear ai as it's going to automate all these low-level processes, that there's that kind of John Henry versus the cutting machine. Like, it's going to automate me out of a job. But just like you said, the scale of it, it's not like, "Oh, if I was more hard working, I would have found this," or whatever. Like, we're working at just a different scale of data. And I imagine, there's still going to be plenty of work for people who sort of interpret these processes.

[00:46:42] FC: I think that that could not be more true. And frankly, the way I like to think – I’ve been thinking about AI for years and years. And there's a really, really great book by a Japanese author. And it's in English. It's called The Stories of Ibis. And the author is Hiroshi Yamamoto. And I recommend that to you. You will love. It it's 20 years old, okay? It's a mash up of short stories. But he says some things in there about AI.

He presciently, 20 years ago, anticipated the technique we call deep learning, okay? And it says so many intelligent things about AI. And I think of an AI as a person, okay? A person is somebody who starts with basic knowledge, basic intelligence, and you give him training. And he has experience. And he learns things. To me, a SOC consists of a suite of different ais. And we use different AI techniques, not just deep learning or one of the other ones. We use a bunch of different ones, and we train them differently, all right?

Now you've got a room with some AIs in it. And they're like people. And then we have some humans in there right. And they're all looking at the same stuff. And I think it's a very, very important principle that there are things AI is good at and things that humans are good at. And they are not the same things. We have the AIs do the high dimensionality reduction of raw events. And the humans can do the low-level pattern recognition and low dimensionality space, plus the semantic interpretation. Now, you've got a really solid result.

[00:48:21] CS: Yeah. When that AI told you that there were nine different things running port scans, like, it didn't necessarily know what the solution to that was. It took people with years of sort of cognitive problem-solving abilities to sort of do the next to carry that information to something actionable, right?

[00:48:38] FC: Exactly what happened. Yeah, that's right.

[00:48:40] CS: For people who are just getting into this industry or maybe are listing this video and are saying like, "Oh, my God. I really want to do that kind of stuff." Like, what type of learning do you think they need to do right now? Are there particular learning paths or things they should be tinkering around with at home? Or what should they be sort of prepping themselves for over the next however many months? What should they be excited about? What should they be poking around in?

[00:49:08] FC: Well, I think that, certainly, artificial intelligence is got to be part of every young person's – If you're technical, or you're interested in the creative in business, information at scale transforms literally everything. And there's a lot of resources to that. But I think it's almost like a game, right? And I think the young people are really good at figuring out what they're attracted to in terms of studies and learning.

But I certainly think that the more you get knowledgeable about and – Pay attention in math class, all right? Because AI is linear algebra, okay? It's discrete algebra in a lot of ways, all right? Learn all that stuff. It's not that easy. But that's certainly a thing to do.

As far as cyber security, we really are needing to move beyond the traditional patterns and the traditional methodologies. And a lot of that is baked in very deeply with tools, techniques and practices. That's going to take a little bit of time to change. But again, young people are good at that. Again, young people don't start out without the preconceived notions, and we've already done it this way. That's what's good about it. I think they've got a shot. I think it's less straightforward to find good resources on cyber, because it's evolving so fast. But AI is just no end of fun.

[00:50:40] CS: Yeah. Yeah, it sounds like, "Boy, you've hooked me in." As long as there's not a lot of calculus involved. I can do algebra fine. But –

[00:50:47] FC: Well, a little bit. I mean, derivatives, right? That's not differential calculus. That's easy.

[00:50:54] CS: I would always understand the concept in calculus about three weeks after I failed the test. So, yeah.

[00:51:00] FC: Well, now, you don't have to pass the test.

[00:51:02] CS: Now I just do it – Yeah. Do it to do it. Yeah. Well, this has been great. As we wrap up today, Francis, feel free, tell me more about Insight Cyber and some of the big projects and exciting developments you have in store for the second half of 2022.

[00:51:16] FC: Well, again, we are busy getting our tech out to customers. And so, we've got some launch events going on later in the year that will be – Watch for the publicity on that. But there's just an incredible hunger from the customers that we're seeing to just know more about what's going on in their environments. And so, we're just expanding that rapidly on a daily basis. That's what we're mostly doing.

[00:51:46] CS: Nice. What's that?

[00:51:49] FC: It's a lot of fun.

[00:51:50] CS: Oh, yeah. Yeah, it's clear. Your excitement is palpable about all of these activities.

[00:51:56] FC: We love to show people our stuff. Anybody who wants to get a little bit – See what you've been missing. Just get in touch, and we'll show you.

[00:52:07] CS: Well, that was my last question. For all the beans here, if our listeners want to learn more about Francis Cianfrocca and Insight Cyber, where should they go online?

[00:52:14] FC: Well, our web presence, of course, is Insight Cyber, Insight Cyber Group. So, that's a great place to start. And we do LinkedIn and we do blog posts a lot. That's another place. But company is Insight Cyber. And thank you for the plug, Chris. Love to have anybody come and pay attention and ask questions. Just get in touch with us.

[00:52:35] CS: Beautiful. Francis, thank you for all your time and thoughts today. This has been terrifying, but also a blast. So, thank you.

[00:52:41] FC: Look at it. There's a way forward, okay? We will get ahead of the bad guys. That's what the game is all about.

[00:52:49] CS: Yeah, the beam of light that keeps one foot in front of the other.

[00:52:53] FC: Absolutely.

[00:52:54] CS: And, as always, I’d like to thank everyone listening to and supporting Cyber Work the podcast. New episodes of Cyber Work the podcast are available every Monday at 1pm Central both on video at our YouTube page and on audio wherever find podcasts are downloaded. And I want to make sure you all know that we have a lot more than weekly interviews to offer. You can actually learn cyber security for free on a portion of our InfoSec skills platform. Just go to infosecinstitute.com/free, create an account. You can start learning right now. We have 10 free cyber security foundation courses from Keatron Evans, our superstar teacher. Six cybersecurity leadership courses from Cicero Chimbonda. 11 courses on digital forensics, 11 on incident response, 7 on security architecture, DevSecOps, Python for cyber security, JavaScript security, ICS and stata. Ding! Ding! Ding! And plenty more. Just go to infosecinstitute.com/free and get your learning started today.

Thank you once again to Francis Cianfrocca. And thank you all for watching and listening. We will speak to you next week.

[END]