Security in the Age of AI Agents

Transcript

Intro: You’re tuned in to IoT and AI Leaders, your go-to show for insights, predictions, and big ideas on how IoT is reshaping the world of AI.

Ep 66 Jasson Casey, Beyond Identity

Nick Earle: Hi, I’m Nick Earle and welcome to another edition of IoT and AI Leaders. This week we’re going deep on security and not just device security, which we’ve talked about a little bit about in the past. But what we are going on is security in a world where IoT and AI come together at very, very rapid speed.

And, and my guest, is a real expert, in the field here. he is, Jasson Casey. He’s the CEO and co-founder of Beyond Identity. He’s got a deep security background, and he talks, he talks very nicely building it up as to where we came from, where we are, what’s gonna happen in the future, and what we need to look out for.

as everything gets smarter, everything gets ai, enabled. And also as we’re all buying products that are smart, enabled and contain agents, what are the pros and cons and what could go wrong? We’ve got a few good case studies here. and I give him one as well during, this of, in the, in the payments field of what can go wrong.

It’s a very, very informative podcast. Um, he knows his subject matter inside out. And, uh, you’ll hear that he is a really good marketeer because he takes a subject like security, which can be full of acronyms. He doesn’t use very many of them, uh, but he actually talks about monkeys locked in cages with pens, uh, which really, uh, make it easy for everybody to understand what model, uh, we need to go forward on.

So, with that, I’m gonna hand you over to my episode of IOT and AI leaders with Jasson Casey, the CEO and co-founder of Beyond Identity, and Enjoy. Jasson, welcome to the IOT and AI Leaders podcast. 

Jasson Casey: Thank you for having me. 

Nick Earle: Thank you for being here. And those of you who are watching can tell that Jasson not only is an avid book reader, but has a set of golf clubs in the background, which tells you a little bit about him.

What we’re gonna be talking about here is, the theme of our podcast for 2026 have been the collision, the merger of, IOT and AI with a particular emphasis on AI. But, before we do that, I just wanted to, and we spend most of the first part of the podcast on the AI side and then talk about how it’s morphing and those issues on security into the IOT world.

So to start off with maybe Jasson, for our listeners and viewers, indeed, maybe just a quick intro, to yourself. 

Jasson Casey: So my name’s Jasson Casey. I’m the CEO and Co-founder here at Beyond Identity. I have been in the infrastructure security space since the late nineties. Before Beyond Identity, I was the CTO of a company called Security Scorecard, which was a global threat intelligence and risk ratings organisation.

Before that, I ran engineering for an organisation named IronNet run by the former director of ENSA. And, before, there was a, before that, and a lot of it had to do with, think of it as like hyperscale and telco, data infrastructure. So I grew up in the world as an engineer, morphed into product management and then from there went into, you know, the job you see me in now, 

Nick Earle: And security, as we said at the intro, has always been important. You’ve got a deep background in a variety of, by the way, agencies, areas and I’m sure there are maybe a few listeners who dunno what NNSA is, but, maybe you can just explain that acronym for our non-American listeners.

Jasson Casey: So the NSA is the National Security Agency. It is one of the, the largest, government organisations focused on signals, intelligence and, kind of protecting and defending, the US’ interest in the cyber domain. 

Nick Earle: I know you have some strong views on this, and indeed, your company is built around this, which is how AI is changing the economics of cyber attacks. And one of the things that you’ve said is the fact that speed now matters more than sophistication. Maybe you could start off by just explaining that point of view.

Jasson Casey: So there’s kind of this famous cartoon that I’m sure a lot of your visitors or listeners would’ve seen before, and it has kind of like the evolution of man, right? Yeah. So on the left side of the cartoon, you kind of see, an early stage, ape of some sort. Yeah.

And as he walks from the left to the right, he becomes more upright and eventually becomes kind of a modern human. 

Nick Earle: Yeah. 

Jasson Casey: there’s a version of this in the cyber world, which basically is kind of showing the inversion. So it starts off as a sophisticated human on the left and devolves into kind of a less sophisticated animal on the right.

And the story they’re trying to tell is when this all got started in the seventies and the eighties, the sophistication of the cyber adversary was actually quite high. By definition. They had to kind of understand and create all of the tools that they used to penetrate your organisation.

Back then, it was a mainframe over a dial up, but still penetrate your organisation and steal the data they were after. As time has gone on, because of things like the proliferation of software, because of things like open source, because of the fact that there’s just a lot more people in this space today than before, you don’t have to be as sophisticated as you used to be to pull off a sort of , cyber attack.

AI doesn’t just take the next step. It takes the next leap. And so whether you’re someone who doesn’t necessarily know how the tools work or even how to construct the tools, because these AI models have been trained over so much open source software that does these things and already has these patterns, it brings these sophisticated tools essentially to the bear of maybe an unsophisticated user.

Or another way of looking at it is for the sophisticated user, it gives them an army. It gives them an army for something that they would’ve had to have done themselves or farmed out to kind of more low level analysts before. So for instance, if I’m planning an attack on someone, the first thing that I want to do is I wanna run reconnaissance.

I wanna understand what goes on in that organisation, who are the key individuals? What are their key relationships? What does their pattern of life look like? That would be an operation that would be run by a series of people kind of directing passive and active collection techniques. And in a world of, in the agentic world or the AI world, I don’t really need that team of people in the, across the, the span of a morning.

I can either direct an agent to kind of help me do that initial assessment, or I can construct a custom agent to do that assessment in a constant way so it can kind of compress my attack cycle. Let’s say I’m a reverse engineer and I’m trying to understand the vulnerabilities of software to create an exploit, or I’m trying to understand how malware actually works to create a defence.

Historically, I would set up an environment. I would detonate the malware. I would have the software and the environment and, and I would do both something called static and dynamic analysis. I would look at what I could tell from it, not doing anything, and then I would poke it with a stick.

All right? I would tell it to run, and I would see how it behaves as it actually runs. And I would build an understanding with the introduction of some of the modern AI tools. I don’t have to be present for some of those initial assessments, and like the 80 to 90% of the drudgery of the work I can actually kind of automate away.

So AI is the natural kind of evolution that we’ve already seen, which is, increasingly sophisticated tools in the hands of more and more actors that aren’t necessarily nation states or even organised crime anymore. And AI is even accelerating it more so, 

Nick Earle: world of very, very rapid change. And also we have got, now a, a lot more sophistication at the edge.

And, I know, when we were chatting prior to recording this, we were talking about the, the root cause and the, and this issue of really having to understand that a lot of issues also came from the fact that 70 to 80% of your view of issues were actually rooted in the fact that the credentials themselves were movable.

maybe , you could just talk a little bit about that and what are the consequences of movable credentials? Sure. 

Jasson Casey: What we mean is we mean something that’s used to either authenticate a human or a device or a service.

When we say movable, what we’re really saying is it’s a piece of data, right? So for instance, a password is a piece of data. A, a biometric technically is a piece of data. a these things called bar tokens, which are how you actually, traditionally deposit on authentication tech, credentials inside of like offline devices.

like drones, it’s a piece of data. And so if it’s data, it can get copied and if it can get copied, it can get stolen. When we say most cyber incidents are the cause of movable credentials, I’ll back it up with a little bit of data. There’s three organisations that track the evolution of cyber incidents year over year, that we pay attention to and kind of really reference their reporting.

So Mandiant, CrowdStrike, and Verizon. And Verizon has a reporting service called DBIR, the database of incident response in all three of those port reports, consistently, I wanna say for the last six to seven years. and the number dances between like 80 and 83%, or 80 and 85% depending on the source. The source of all of the, of all incidents tracked and reported on across those three organisations is an identity system failure and an access system failure.

And our argument is that that is describing a symptom that is not actually getting at the root cause. The root cause is what we believe is movable credentials. When I’m accessing a system, I authenticate myself. The way I authenticate myself is I share this piece of data in some way, shape, or form.

If that data can move, it can be stolen. and we clearly see that, through all of these incidents. So what we argue is it’s actually now possible to create a world of immovable credentials for authentication. And as the world shifts to that, it drastically changes the landscape of cyber attacks and it actually eliminates the root cause of 80 plus percent of all of the incidents every listener’s organisation has to deal with on a day in and day out basis.

Nick Earle: And if that’s the case, does that mean that in your view, when it talks about an immovable credential? I was thinking about, for example, something that, that. We use every day, you know, chip + pin. In fact, over here in Europe, chip + pin has been standard for many, many years.

is, is that what you would call, an immovable credential in that it’s immovable in the sense that the, the physical device is movable, it’s the credit card in your pocket, but the credential is bound up in a secure enclave in the chip. Is that sort of one of the key differences, is it kind of embedded in the electronics?

Jasson Casey: this is a perfect example. So before the modern credit card, we added a magnetic stripe that basically encoded a number, right? And this number was the data was a reflection of what was on the physical card. The number could be stolen right from online orders. The number could be stolen from skimmers.

And, I believe it came out of Europe, but chip and pen was this idea of, well, wait a minute. We don’t need to have like this symmetric secret that we shared to prove that we’re the right person to pay this bill. We had modern cryptography for a while.

It’s possible to create something called an asymmetric key pair and do a cryptographic signing. And it’s, furthermore, it’s possible to create a little specialised chip, right? A secure enclave or trusted execution environment, right? Whatever term you, you favour, where that signing key is locked in that chip.

And it’s never in a main processor. It’s never in memory. So maybe another analogy is imagine a jail and there’s a monkey in the jail, and the jail doesn’t have a door, but the monkey has a pen. So in the old world, you would come to the jail and you would ask the monkey to essentially give you the pen, and then you would go off and sign things.

So that pen was stealable, right? In the new world with this new mathematics or the, with cryptography, we could basically pass the bill through the bars to the monkey and we could say, sign this, and the monkey would sign it and hand it back to us. And that signature on that bill was our receipt of payment.

In fact, that’s very similar to how chip + pin actually works. There is a tiny monkey in your credit card, and that monkey is a little bit smarter than the example I gave. when you put your pin in that pin is the monkey’s way of knowing that you not only possess the monkey, but you are the same person.

Or at least you have enough of a relationship with the same person who set that monkey up in the car to where that monkey should, in fact, go ahead and sign that bill for you. Now, there’s another interesting part of this story, which is like the US and our kind of classic style. We rejected chip and pin forever, right?

and then the funniest thing happened.apple came onto the scene with the I iPhone. Google followed shortly with Android. they quickly realised that, wait a minute for the argument of convenience, wouldn’t it be nice to pay with your phone?

And so we ended up getting actually chip and pin in the US essentially through mobile payments. 

Nick Earle: Yeah. 

Jasson Casey: So the hardware that’s inside of your phone is actually identical to the hardware that’s inside of your modern credit card.

And, and in fact it’s, it’s a slightly more advanced version of chip and pin. It’s chip and pin or biometric, but the pen or the biometric essentially do the same thing. So when you pay for coffee or tea at that coffee shop with your phone, you basically have one of two experiences. You tap your phone and your phone says, smile or swipe your finger, or it, you haven’t set up biometrics or it doesn’t recognise you ’cause you have a mask on or something, and it says, Hey, give me your pin code.

In both of those cases, it’s basically they’re bringing your biometric or your pin code to the monkey and the phone. And when the monkey is satisfied, it, you also bring it your bill, it will sign the bill, give it back to you, and you give it to the merchant. And that’s how that works. That is, in fact, a perfect example of an immovable credential.

The only way of stealing your form of payment is physical theft. And while I’ve learned through my trips to London recently, that physical theft is certainly very easy, especially for phones. 

Nick Earle: Yeah. 

Jasson Casey: That is, it does not scale like online criminal theft does, which is many, many orders of magnitude more rampant than physical theft.

Nick Earle: You introduced yourself as having a big engineering background, but I think actually you’re a closet marketeer, because your analogies are really good and easy for people to understand. And I want to come back to the fact that the most of the world, you know, we are obviously, IOT company with, with, six, 7 million devices under management for our clients.

And, the issue of security on the device is huge for us. And I’m gonna come back to that. But, before we do that, the. That’s great. If you control the credit card, if you control the phone, I mean, the monkey is locked inside the, the, chip on the credit card, the monkey, in terms of, you know, face recognition, I guess the, the, the secure side of it is that you, your, your face is locked to your head, which is locked to your body.

For most devices that we’re buying nowadays that are all being marketed as smart devices, that we’re buying devices with AI in, I mean, just before I came on this podcast, my wife came up to me and said, what on earth does this mean? We have an app that can turn the lights on and off remotely in our house and set timers and whatever.

Mm-hmm. And suddenly they’ve released an AI version of it. I don’t know why I would want an AI version of it. 

Jasson Casey: Mm-hmm.

Nick Earle: But, but now they’re asking for my location and they’re asking for my electricity usage, and I suspect they’re trying to help me in some way, but it feels creepy, because these people are now asking for access to a lot of information.

I just bought a smart plug that can be programmed with on and off times so we can turn the lights on and off while we’re outside the house. So that’s just an analogy of how, you know, it’s almost impossible now to buy a device that’s not smart. Mm-hmm. 

Do you have any, examples or, or well-known ones or crazy examples of, devices, like real world, world failures of, or maybe consumer devices that bring this to life? 

Jasson Casey: Yeah, so, so there’s a recent one that actually kind of combines a lot of the stories that we’re talking about.

so, there’s a Chinese company called DGI, they’re very famous for their drones, but they make more than, more than that. They make, they make household vacuums, robotic vacuums. And a researcher recently, when I say a researcher, I mean like a security researcher, was, using one of these, modern AI agents called Claude Code, to reverse engineer the robotic vacuum.

I don’t know what his actual objective was. I think he was just a natural tinker and just wanted to know how did it work and could he take control of it and could he make it like, vacuum his initials in his house or something like that. 

Nick Earle: do.

Jasson Casey: Yeah. The first step in his process was he had to figure out what kind of credential, where it was located and how he could steal it or copy it, in order to issue control instructions to the vacuum. And so he turned Claude loose and Claude figured it out for him very quickly. And, and then he built up his controller and he is like, all right, now I want to actually start controlling my vacuum and make it do things.

And, it worked. It worked so well. It also took advantage of a standard, faux pas committed by DGI. He had taken control of 7,000 or so robotic vacuums across the world, not just his vacuum. and so this was all in the news, I wanna say, in mid to early April of this year.

So you can, if you’re interested, you could definitely pull that story and get more details. But it illustrates a couple things. like clearly this individual probably could have done it without the help of ai, right? He’s a professional security researcher. However, he was able to do it quickly with ai.

Yes. And it also illustrates that this root cause of most of our problems is still being violated by large companies that know better, right? This shareable credential seeded across all of these iot devices. So that’s kind of a humorous story. There’s another one I’ll give you. It’s a little bit, we’ll call it theoretical.

but it ties to what you were talking about with your electric switches and light bulbs a minute ago. And this one is a little bit more scary, but like somebody might say, Hey, what’s the harm of someone hacking my light bulb? What’s really the harm of someone hacking my dishwasher?

Is it really a problem? Do I really need to worry about it? you know, there’s a simple thing where if I could control enough IO OT devices at scale that actually were power devices of some sort that actually did sync a material amount of current a material wattage, then effectively my adversary now has a big on off switch.

Nick Earle: Yes. 

Jasson Casey: And the way power distribution works is power distribution is tuned not to the max capacity of everything attached to the network. It’s tuned to what the network’s typical usage is. So if I could hack enough IOT devices and control enough iot devices at scale, and I could flip them on and flip them off and flip them on and flip them off in the right way, I could actually create pathological loads on power distribution, blowing breakers.

And, when you start to actually look at the electrical systems across various countries, these breakers are not necessarily commodities. They’re not easy to replace. And if you break enough of them, 

Nick Earle: yes, 

Jasson Casey: you’re outage that lasts a material amount of time. So like there are in fact very real consequences that I would argue maybe they’re knock on or secondary effect, but there’s still civilization impacting 

Nick Earle: The great slash scary thing about security is that the case studies are really interesting, but they also frighten you, at the same time. And, we’re moving now into the conversation of how all of this overlaps with, with our world in IoT at Eseye And I’ll give you I’ll trade you, I’ll give you a, an Eseye story and I’ll try and make it, not be specific as to who it was.

We have customers, about 800 customers, and our customers have devices, 190 countries. And in one particular country we power something like 35% of all of the credit card authorization devices that are used in shops and restaurants, chip and pin.

Although in this country, they don’t use chip and pin, they just use a standard SIM card to connect to the telephone network. And because the telephone network is very, fragmented, a mobile network is very fragmented. They use our system because we can hop between any, any operator. So we’re basically selling the promise of a hundred percent connectivity because, you know, you have a chain of 30 restaurants. It’d be pretty common that at least 20 of them couldn’t use the same sim card because the network didn’t exist.

Well, the purpose of the story was that we were selling them a package of, I don’t know, two megabytes usage per month. It’s not much, taking payments in a restaurant, or a shop. And we got a complaint from a customer saying, are you guys ripping me off?

You know, you’re charging me four meg for four mega a month? No. Well, you’re using four meg. No, I’m not, you know, here’s the engineering proof that I’m only using two meg. So it turns out that they bought cheap Chinese payment machines. And, the, when we did a big trace, ’cause we are device people, and we did a big trace and we traced, you know, the APNs and, and the IP addresses.

And so there was something in the firmware of the device. But basically it was replicating all the payments for these devices back to China, Shenzhen. 

Jasson Casey: Yep. 

Nick Earle: So every single credit card payment that went through these devices, the data on everything, it did.

To the local bank in Shenzhen, there was no reason other than the device was manufactured there. and so when we pointed that out to them, I mean, A, they had to pay their bill, but, but b, it, it was kind of, we, we were saying, look, we told you, you know, it’s all about the firmware, the device, and the firmware and, and, the security codes and a lot of stuff is now gonna be device resident.

So this brings us into, sort of the next area, which is, what I call the, the collision between IOT and ai. Because I said earlier, you know, everything’s moving to the edge. Every device you buy now is smart.

And we have a third component that we haven’t yet talked about because now devices, we have software. They’re not devices, but we have agents. So let’s just park devices for the moment. And let’s talk about software agents. Sure. you, you can develop your, if you’re a big company, you can develop your own agents and all of the big platforms, and you mentioned Claude, but all, all of the big platforms will let you develop agents.

But not everybody wants to buy agents, but often they’re buying software. And the software contains agents, agents because they’re so intelligent and they’re exponentially growing in capability every year. Presumably agents are also a security threat. You don’t know really what you’re buying and its capabilities, how can we protect ourselves?

and what are the best practices, at least to try and protect yourself from the import of agents, which your employees could be doing just by downloading, capabilities, which, software capabilities, which include agents. I mean, they could be, you could be using thousands of agents without knowing it.

Jasson Casey: So agents are very powerful. Agents are the easy way of thinking about an agent, is an agent is really, really just a simple loop. And I’m sure most people have used a chat bot agent where they, they give the chat bot a prompt or a search or it is kind of like a search term but not, and the agent takes that and sends it off to an LLM and says, Hey, what do you wanna do about this?

And the LLM knows how to, based on that sentence, predict what the answer likely ought to be and sends it back down. 

Nick Earle: Yeah. 

Jasson Casey: And you know, slightly more advanced agents will add in what’s called tool calls. So when you ask for a thing, the agent actually may have a series of tools, like it may have the ability to search the web, it may have the ability to read websites, it may have the ability to write a file, it may have the ability to do X, Y, and Z.

And so when it sends your query up to the LLM, it also says, and by the way. Here are some tools I have the, I have in my facilities. If any information from these tools might help you, the LLM respond stronger or better to the user’s prompt. And so the LLM, rather than sending just a, a final answer back down, it might say, well, you know what?

I want you to go do these three internet searches and tell me what you find. And so it’ll loop, it’ll tell it what it finds, and it’ll say, all right, now I want you to go do these additional searches based on what I found. And it may loop multiple times. And then eventually, when it produces a result that it thinks is final, it then kind of dumps that back out to you.

That’s the chat agent. People are building versions of that, that aren’t necessarily driven by users, but driven in a more autonomous way, right? Like, maybe you suffer, like I do from too many emails, from too many people, to really be as responsive as you truly need to be. So the agents very, very simply is there to kind of, be an assistant and sort the email and figure out what’s junk, what needs a response, but not your attention versus what actually needs your attention.

But what’s to stop a clever person from emailing me something with a prompt injection that then convinces my agent to use a tool call to divulge my financials back out in the email response to my competitor? It is actually important for organisations to know what’s going on in their environment right now.

The large tent pole is called AI governance. I’m sure people have heard of this before. The specific area or the specific use case that we’re actually now subscribing, I would say is shadow AI. 

It’s called Ceros, C-E-R-O-S. Sh, you can take a look, at, at it online. There’s a, there’s a way for you to try it out, but it is a tool that helps organisations, essentially discover not just what agents exist in their environment, but what do those agents have access to? What tools do they have access to?

What providers are they, do they have access to? Maybe it’s okay if they’re talking to an Amazon bedrock instance of cl of Anthropics models. Maybe it’s not okay if they’re doing your, your, phone home to China, for some sort of, Chinese hosted model, right? Mm-hmm. Number one, understanding what’s going on in your environment and then building a plan around what you discover to essentially control the data flow.

It’s really, really hard to control an agent and it’s not possible to really control what an agent is going to do in itself. And the reason it’s not possible is because while you contributed, say an agent can’t make certain tool calls, it’s difficult if you let an agent write, start writing programmes.

It’s actually difficult to analyse a programme to truly understand what that programme is actually going to do without letting that programme execute. And, this actually tracks back to a very classic problem that Alan Turing, defined, back in the 1940s, right? Like, is this programme going to halt?

It turns out that it’s provable that like in many cases you can’t answer the question. So an easier problem to solve isn’t what is the agent going to do? An easier problem to solve is how do I track the data that flows in and out of the agent? How do I restrict the mingling of data and the understanding that if I let privileged data into an agent and I let that agent have access to public services, information xFi is possible without actually analysing that flow.

At a high level, establish an AI governance programme, understand what’s going on in your environment with some sort of shadow AI use tool, and then build some sort of policy around how you are going to monitor and enforce data access and data controls for agents.

Nick Earle: So let’s go a bit deeper on that because I think it really is the next train that’s coming down the tunnel or whatever analogy we want to use. It’s a big one. I think it was yesterday or even on TV this morning,

Somebody saying, this year, 2026, there will be more agents deployed than there are people on earth. Now that sounds a crazy statement because there are eight, 8 billion, and maybe they’re referring to people who use computers or whatever.

But it goes back to this issue that, you know, there are companies that are now paying their employees for how many tokens that they are using. Which is driving, use as many agents as possible. And, your performance evaluation. But the adoption of agents is proliferating agents, not just separate. They’re talking to each other. They’re making decisions, they’re sending data between each other.

So all of this raises the one, Entity that we haven’t talked about yet, or the one thing we haven’t talked about yet, which is the human being. 

Jasson Casey: Mm-hmm.

Nick Earle: So you talked about, you know, you’ve got a tool for measuring, looking at the inputs and the outputs and I guess pattern recognition, and, that’s unusual behaviour flag it.

You may wanna look here a bit more, which is a, it sounds like an extremely valuable tool, but what about the human in the loop? and the context of this question is one I ask all my guests, which is the, the thing I read in the popular press, and tv, is, you know, AI’s gonna take all the jobs away.

and I think the answer to that is a little bit more nuanced. One is that it’s going to be an augmented, collaborative environment between the human and the technology. But even then, it’s not like a sort of stack model, which is, here’s a stack of Lego bricks and AI can currently go up to Brick six, but by next year it’ll go up to Brick eight.

Mm-hmm. I think it’s, you’ve got a really good view on, it’s a more nuanced, division than simply task-based. And it’s to do with what the AI is doing, that it’s good at and what the human being can do that human beings will always be better at. And I know that you have a view on that.

I think that’s definitely worth sharing here and going into as we’re getting towards the end of the pod, 

Jasson Casey: Ai, ai is not magic. And, you know, we can get into the gory details of how it works, but we could also safely just treat it like a black box where the black box is a really good guesser, right?

You give it a question and it’s gonna give you an answer. And because of how that AI was built, the answer is gonna sound plausible. It doesn’t mean it’s gonna be true, but again, it’s a good guesser. So most of the time, maybe it is true and it’s gonna follow a probabilistic distribution, right?

In domains that matter, people typically pair up. This AI guesser with something called a deterministic verifier. So, for instance, there’s a lot of problems in the world where we already know whether a solution to that problem is in fact true or not, right?

And so in that case, we would say we have a deterministic verifier. So for instance, let’s talk about the world of math, right? if I give you an answer and I give you an equation, it’s fairly trivial for you to essentially verify if that answer is in fact an answer of the equation. Now there are other types of math problems where if I ask you, slightly differently, is this the most optimal solution for this problem?

All of a sudden you struggle and like the problem is not, not that tractable. So in a world of, of, of consequence or where things matter, successful folks are figuring out how do they actually partition problems, to the probabilistic portion and the deterministic portion. How do they bring deterministic verifiers to bear?

And how do they kind of wrap all of this together? The, it’s really, really important to just remember that at the end of the day, these AI tools, they’re trained, they’re trained after us, and I wouldn’t even say they’re trained after the best of us, right? They’re trained on the corpus of scientific literature which is kind of good.

They’re trained on the corpus of Reddit, which I think we could probably argue is, uh, you know, 

Nick Earle: yeah, exactly.

Jasson Casey: they’re trained on stack exchange, which used to be the number one go-to for programmers to figure out why their program’s not compiling. and you can see it when you interact with the AI on some of these problems, you see it basically adopt sometimes the lazy engineer approach, where you give it a programme and you ask it a problem about what’s going on, and it comes back and it says, Hey, your question’s not even relevant.

Here’s a solution that tweaks this thing and just solves the problem easily. And that’s generally, you know, that it’s kind of like it’s human behaviour, right? It’s trained off of human behaviour. And if you’re trying to understand something, principally, if you’re trying to actually do a root cause analysis, it’s frustrating.

Also when you take a step back and think about, well, how do you interact with an intern? How do you interact with a fresh graduate? These individuals are also going to have more distribution style behaviours. They’re going to need oversight. You can’t trust everything that comes out of them, but they’re generally effective at covering ground and being enumerative and getting work done.

And so like this human agent partnership or, or supervisory ship relationship, at least it’s where I see us being most effective. Certainly within beyond identity of how we’ve kind of done our AI transformation, for harder problems splitting the problem into. 

A lot of you have probably used chatbots. A lot of you may have even used some of these more powerful agents like Claude Cowork or Claude Code. And you know, the first instinct you have is you just dump a big problem in the prompt and then you see what comes back. And then as you have these sort of problems before maybe you realise, Ooh, I can take that prompt and I can maybe make it a template so I don’t have to type the whole thing anymore.

I just say, Hey, use that reasoning style or that skill that I talked about before and solve this problem. And then your next level of maturity is you realise, well, wait a minute. This probabilistic guesser is trained off the, the, the typical human interacting on Reddit and, and stack exchange. And I can see that I notice in its results, like I’ll say, go do this over this list, and it’ll like give up after eight, the eighth item.

it won’t actually verify the answers, for the entire list that I sent it. And you realise, well wait a minute, checking like enumeration is not a probabilistic guess. Enumeration is a very well-known deterministic action, for each one of these items do x for each one of these answers call the verifier.

And so your next journey is you realise, well, wait, maybe I can decompose this skill to be a series of prompts in a series of verifier calls or a series of function calls where the function calls are basically doing the probabilistic part, right? For each call the verifier and the prompt parts are doing the probabilistic parts.

Like I, I need actual reasoning here. Give me your best guess under these constraints. And so all of a sudden you’ve built something that looks a lot like a classical computer programme, but rather than a collection of functions that are getting called in a very structured way where each function is kind of deterministic.

Some of those functions are LLM calls where they’re basically probabilistic reasoning. More towards the mature or the, the slightly more advanced usage and pairing of AI that I see be effective both in the organisation, but I think it also mirrors itself in, in, maybe someone doesn’t, like, consciously think about their steps in that way, but they do evolve to that style of behaviour over time if they really are kind of working with the ai.

Nick Earle: would your prediction be as we finish here that, Given the advancement of AI and adoption, which is still in the early stages, that this issue of probabilistic deterministic interaction and breaking it into pieces and, you know, ask a question, verify, ask a question, verify is gonna become more and more important going forward?

And secondly, does this in effect, define the future skills requirement for success in the workplace? Yes. 

Jasson Casey: So earlier you said something like, is AI gonna kill all the jobs? And, and you think there’s a nuanced response there. And I would agree, there are certain jobs, and maybe a better way of putting it is there are certain behaviours that I think are not gonna be rewarded with full-time jobs.

Now that we have ai, the person who just wants to be given a punch list, that the knowledge worker who just wants a punch list to execute and does not want to consider the business context and does not want to consider the system context, who just want, thinks of themselves as like a, a software artisan, if you will.

the person who enjoys systems thinking, the person who enjoys the business context, the person who’s questioning the assumptions of like, well, wait a minute. As an organisation, why do we even behave in this way to begin with, right? We’re trying to solve this problem for the customer.

We did X, Y, and Z because that was the best technology afforded us at the time. But now with an ag agentic approach, I could maybe do things in a slightly different way. That’s the person who’s gonna get rewarded 

Nick Earle: Finding those people, before you hire them and checking they’ve got those skills, is a big issue because, that’s not what you get taught in the university environments except for early in career, et cetera.

And for a lot of people over the age of, I’m gonna say 40, I’m, I’m, I’m a lot older than 40, but I’ll, I’ll just pick 40. For a lot of people over the age of 40, they’re not touching AI right now. They’re, they’re not doing that because they think the younger people in the organisation do that.

The younger people in the organisation don’t have those skills either, because this is coming at them so quick. This has been extremely informative.

I’ve certainly learned a lot. I compliment you on your analogies. I will remember the monkey analogy. I wanna give you the opportunity, just to mention your offering again, have I got it right. Is it called Ceros? Did I get that? 

Jasson Casey: It’s called Ceros. C-E-R-O-S. Dot s sh 

Nick Earle: Okay. And, that can be found on your website. 

Jasson Casey: So just go to the website, Sarah, CER os sh It’s basically an, an ai, governance solution. it helps you, regardless of where you are in your AI journey, if you’re just getting started, it’ll help show you what already exists in your environment.

You may find it surprising, and if you’re a little later in your journey, it helps you actually, apply some of those controls and governance techniques that I’ve been talking about. 

Nick Earle: Before you get too far in, Jasson, this has been great. Thank you very much. Thanks. Thanks for joining the IOT and ai, leaders, podcast.

It’s the first time we’ve gone deep on, on security, and, and as those two areas are coming together, not just optimising the firmware as we mentioned, but now the AI layer and the agents. so I’m sure a lot of people would’ve found that very, very interesting. So, thanks again for being my guest.

Jasson Casey: Thanks for having me.

Outro: You’ve been listening to IoT and AI Leaders. We hope today’s insights help you drive smarter, faster business innovation with IoT and AI at the centre. Thanks for listening. Until next time.