How to Scale AI at the Edge

Nick Earle: [00:00:00] Every now and then, a leap in technology happens which at first seems just like technology, and people don’t see or understand its relevance. But then suddenly you realize it’s actually what it enables, and that enablement is something really, really significant. And that’s what we’re going to be talking about today in this episode of IoT and AI Leaders.

I’m talking to Neil Cresswell, who’s the co-founder and the CEO of a company which is actually based in New Zealand, but he’s over in Phoenix, Arizona. It’s called Portainer, and they allow companies to distribute software to thousands and thousands of devices, in the format of containers, Dockers, Kubernetes containers.

And that sounds like technology, but, but it is. But what it really is, is the ability to put intelligence and application processing at the edge and enable brand-new case [00:01:00] studies at massive scale. And that’s what I meant by a disruptive change because, most of these applications are one layer above, uh, in, in server farms or in the cloud, and we’re talking now about the applications being resident at the edge and distributed and maintained at the edge, maintained from a software update point of view, a security point of view, consistency of firmware, governance,

I think you’ll really enjoy this conversation, between Neil and myself. We talk about the disruption there’s been in the software, uh, uh, creation business and distribution business. And then we talk about what that means for certain industries, including IoT. Uh, and, and, and therefore why…

We finish with why and how is it now possible on a tiny budget, a departmental budget, your personal credit card, just a few dollars or pounds or whatever it is, how you can actually create something very rapidly and with vibe [00:02:00] coding that you couldn’t do previously to take to your boss and say, “Look what I’ve built.

Shall we make this into a project?” Neil gives a, a lot of, advice. He has a lot of passion about that subject, but he also gives a lot of advice on how to start small and grow gradually, and also has some great, uh, case studies, uh, from, John Deere in agriculture and, uh, the favorite one, uh, which you’ll hear, uh, to do with, um, distinguishing between a potato and a hand grenade.

Uh, all of that’s in this episode. Um, I think, as I say, you’re really gonna enjoy it, so let me hand over now, uh, to my podcast with Neil Cresswell, the CEO of Portainer.io. Enjoy.

So welcome to IoT and AI Leaders, your regular source of advice and insight for the coming confluence of AI and IoT at the edge. So my name is Nick Earle, and, I’m the host of IoT [00:03:00] AI Leaders. And today I’m joined, um, by Neil Cresswell, who is the co-founder and CEO of a company in Phoenix, Arizona, called Portainer.

And before we get into what they do and, um, case studies and all the usual things, uh, the overall theme for this podcast is similar to the one that we’ve been talking about quite a lot lately, which is, what does it actually take to deploy and scale AI at the edge? That’s where AI and IoT is gonna come together.

Uh, and, Portainer have got a lot of experience. In fact, they were set up to do this, to explore the technical, the operational, and the challenges of just doing that. But not just for a few devices, and not just if you’re a big company, but to hundreds, thousands, if not millions of devices and for companies of all sizes.

Uh, and so, uh, this is really a very specialized company performing a very important task to enable, AI, to scale at the edge. So [00:04:00] that’s, that’s, that’s the subject of, uh, this week’s episode. And with that, Neil, welcome to the, IoT and AI Leaders podcast.

Neil Cresswell: Thank you. Thanks for having me. We, we, we’re actually a Kiwi company, by the way.

So even though I’m crazy enough to be in Phoenix, Arizona, the company is actually based out of Auckland, New Zealand.

Nick Earle: I did not know that. I knew you were a Kiwi. Yeah. And I can hear that you’re a ki- Kiwi, but I kind of assumed ’cause you’re, you’re over in Arizona that, that, that was HQ. But it’s not, you’re a New Zealand company.

Neil Cresswell: Absolutely. Excellent. Which makes this even more fun.

Nick Earle: So, so for listeners who, who won’t be familiar with Portainer, can you tell us a little bit about the company, how it started, and the problem, that, that, you are trying, and in fact, indeed are solving for your customers?

Neil Cresswell: Yeah, so we, we are a, a, a management control layer for a piece of technology, uh, called, uh, containerization. It’s been popularized by, Docker and Kubernetes that is a, a piece of management [00:05:00] technology that itself spins up applications in software containers.

Uh, we sit o- on top of that again to allow you to manage Docker and Kubernetes at massive scale. It’s a 10-year-old company now. Six years commercial, four years, open source. And so we’re an open core company, so started with open source roots, but now commercial.

Nick Earle: With regard to IoT, my understanding is, um, you weren’t originally targeting, uh, industrial and IoT deployments, but now, um, a lot of your customers are using it for exactly that, and we’ll get into that.

But what was it about those environments that, given what you’ve said, uh, made, uh, Portainer a, such a good, such a good fit?

Neil Cresswell: So if you think about the technology, um, software containers, right? So software containers are an absolutely game-changing way of shipping and supporting software. So as a software vendor, [00:06:00] if you want to produce software, distribute that software to your customer base, have that customer base deploy the software or, and with the use of containers, run that software on devices, that, that is basically an absolute no-brainer for the use of containers now.

The, the, the days of installable software is, is done. Everybody now runs a container, and for the prime reason that a container is infinitely portable. So when I, as a software vendor, create my, my software application, package it in a container, and test it on, in my environment, what runs is exactly the same as what will run on a piece of hardware in your customer infrastructure.

And so it radically reduces software defects because I can, I can test it and certify it and then distribute it. Yeah. And so [00:07:00] software containers have, one- In regards to the, the distribution of software from software vendors. Okay, so now, now with that knowledge that, that every software vendor out there is now producing software in containers, well, now, now you have the problem of, well, now how do I run these software containers?

Then how do I manage those software containers at scale? Now, you’re right, Portainer was initially designed for the data center, because in the, in the early days, software containers were predominantly adopted by enterprise IT professionals sitting in a data center. But it, it kind of cut its teeth there.

It got proven, both the technology and Portainer got proven there. And obviously when something works, other industries and other sectors follow suit, too. So, you know, Portainer as a product has always been designed to make the use of this technology easy. You know, we, we, we really do humanize Docker [00:08:00] and Kubernetes and managing them at scale But they are still first and foremost an IT technology.

But through Portainer, we really did enable OT users to, or, or, you know, non-traditional IT professionals to leverage the same technology at scale, so.

Nick Earle: Yeah. And on that last point, so I, what I’m getting from that is that, as a lot of technology does, it starts off with the, with not just the big companies, but the big companies that have the big IT departments, they have the most engineers, they have the most specialists, they have the most money, frankly.

But really, the idea is to democratize it or, or scale it down. And, and so I think, uh, as you are saying, you’re making the, that, the power of containers, and the new model for software a- accessible, you, you’re sort of making it accessible to everybody, as I understand, without… They don’t have to have the big IT department to take advantage of containerization of software.

Neil Cresswell: Yeah. In the early days, and [00:09:00] if I’m completely honest, even still very much so today, you really do have to have quite a depth of IT technical background and knowledge. You have to understand Linux quite deeply. You have to understand how networking and storage works and you stitch all this together, and that is how a software container runs.

It runs in this bubble, but the bubble encompasses all of these Linux technologies- Yeah … under the covers. So really it was the realm of a Linux expert, a Linux specialist, which then got expanded more into an IT infrastructure specialist. And these skills are pretty hard to come by. You know, they’re not dime a dozen.

So it, it, it was a complicated technology, and whenever you have a complicated technology, adoption stalls or adoption lags because you- Yeah … have to wait for the market skill to pick up in order to, to adopt the technology. Yes. Well, what happens if you could take away experience and skill as a precursor to [00:10:00] adoption?

And that, that was the challenge I set ourselves as a product. How do we, how do we take skills and experience out of the equation so now anybody who wants to deploy containers can deploy containers in a safe and governed way? That, that was kind of the, the ethos. And so how do, how do you, how do you absolutely democratize access to this technology because the, the technology is, is game changing?

Nick Earle: And that’s essentially, um, what you do for your, your, um, uh, uh, uh, customers. Do they need to, uh, who themselves create solutions like IoT solutions that themselves get deployed by, by other, other companies. Do, do they… Do, do those, um… The move to containerization, for people not familiar with it, was, you know, our demographic in this podcast is often a lot of IoT, people who aren’t create, or haven’t at least up until now, we’ll get back to this, been creating their own applications, although the [00:11:00] democratization does open that door for them.

We’ll talk about that. But do they, how much do they need to know about the device itself? ‘Cause it used to be, you know, in client server or, or cloud or, or whatever, you had to know a lot about the, the device target environment. Does that become a lot less once you define it, as you said, once you define it in a, in a container, it, it then can be deployed across multiple bit- different bits of hardware without all these software ports that we used to have to do, and that’s, you know, exactly what equipment you had, and therefore you had to do all this.

There were different versions for different, hardware from different vendors. Does that essentially, with the open, open approach, and containerization approach, does that essentially become a lot less of a problem once you’ve defined your software, as I sa- in, in this container, uh, structure?

Neil Cresswell: Yeah. It, it does technically, as long as the person creating the container in the first place, the software vendor-

yeah … has, has made that available. So, so the [00:12:00] actual container runtime, Docker or Kubernetes, uh, it supports multi-architecture. Um, and so it can run on ARM or X86 or X64 or even, even ARM64, whatever you want, right? So it’ll understand all of the different architectures. It’ll then go and look for a variant of the application’s image for that architecture.

So when you are running, it’ll automatically hunt for a, hunt for an image that works for the architecture and run it. When you are building your image, there, there, there are, there are multi-architecture build images where you say, “Build, build me an image of my application and make an image available for all these architectures,” tick, tick, tick, tick, tick, go.

And so it’s very easy to build a multi-architecture image, and it’s very easy to run an image on any architecture. So it definitely, bridges the gap. But, it, there are, there are still limitations in regards to physical devices, you know, for, like, physical hardware that you need to expose.

So a different camera or a different card reader or something else ha- it obviously can’t be completely abstracted. [00:13:00]

Nick Earle: Given, given the promise, uh, of, of, of containers, with software, two questions really here. Well, it’s more the promises to the new model, as you pointed out. But, but two, two questions here.

One is, I presumably, uh, it makes scaling to hundreds, thousands, potentially millions of devices so much easier because, because it’s more consistent. So, so I assume one of the benefits is, is scaling and deploy- scaling deployment, uh, security patching, making sure you’ve got consistent compliance, um, uh, device lifecycle management.

Is, are they the sorts of things that, that re- are benefits, but they really become benefits when you… ‘Cause when we’re talking IoT, we’re talking millions, potentially in the future, billions of devices. So is that one of the reasons why these two technologies, you know, IoT as a use case at the edge, and containers for the software at, at the edge, are natural bedfellows, so to [00:14:00] speak, because of the, the, the fact that you can set a framework for all of those things and then do mass deployment, scaling?

Neil Cresswell: Yeah. When you, when you learn the inner workings of a software container and how an image is constructed, an image is basically just a zip of files and a bunch of configs on top of it, right? And so, the vendor or an internal developer, creates the software package, creates an image which bundles it up into an image file, which is then hosted centrally somewhere, and then devices out in the field, when they are commanded to go and run the container, they seek the image, pull the image, and then extract it and run it, right?

And so every single device in the field, hundreds, thousands, millions of them, all refer back to the one base image. Right. So they’re all running that exact image. Yeah. Now, what’s really cool is the software vendor says, “Oh, my goodness, there’s a bug. I’m gonna push a new version.” They push a new version of the image.

Yeah. Assuming these devices have, the right control software, they will then detect there’s a new version of the image and [00:15:00] then immediately update themselves over the air by pulling the updates or the changes to the image- Yeah, yeah … and then deploy and go. So- Yeah

that, that, that serves two benefits. It, it helps you fix bugs with security and software defects. That in a data center is useful and interesting ‘Cause you might have 100 or 1,000 servers, and let’s say you’re a large enterprise and you’ve got 50,000 servers. Well, 50,000 devices is a pretty small IoT project.

Exactly. And so, the benefits of this mechanism in a data center is X, but the benefits of the same technology deployed at the far edge is probably 100 times X. Right. So yes, absolutely. That’s what I think has rapidly accelerated the use of containers at-

the edge, the far edge, is that-

Nick Earle: And it’s a great, it’s a good example of actually why we r- renamed this podcast, around both IoT and AI, coming together, because, deploying software to a series of servers in your branches or campus, uh, [00:16:00] is one, uh, is one level of problem. But when you’re talking, IoT, anything with power, uh, anything with electronics and power, access to a power source, um, uh, will create data, and we’re talking about the problems of deploying something consistently and securely, and being able to update it, and gather data from it consistently.

Uh, as we said, between tens of millions and who knows in the future as these things get smaller and cheaper, um, I mean, literally billions of things, the potential. And I guess all of that is really the, as I understand it, uh, that is where you sit in the middle of that. You, you say to people, “We can, we can help you do that.

You don’t have to, do it all yourself. We can help you. We’ve done it before. We know what you need to do. We know what the framework is. We know what the guardrails need to be. We know what the audit trail or whatever needs to be. Uh, so we know how to do that, and we’re your partner to help you do that.”

Am I… I- Is that right?

Neil Cresswell: Yeah. Uh, so [00:17:00] if, if you think about the mechanics of creating an image o- of a container, and then getting that deployed out at the field- That is, it, it could be oversimplified to say, “Oh, no problems, I can ju- I can just, just get onto a machine, I can SSH and I can, I, I can run some commands and we’re done.”

And maybe I can, I can write a script to do that 100 times, maybe. Um, but when you get into the, the hundreds of thousands of devices or above, you have a real operational management problem there. It’s like, okay, well now how do I ensure that these devices out in the field here, some might be offline at this point in time, some online, some, some might be, might be, might have been offline for six months, so they’re out of date.

How do I get them all running a consistent version of the software at all times and if it’s offline and I trigger an update and it comes back online ag- it gets a new version? All of this historically has been a solved problem with, like, the [00:18:00] ESP32 version of the world, but that was all based on firmware and firmware management and we had some really complicated tooling.

Now with containers, it’s kind of industry standard. And so yes, our software is the control mechanism that says we will connect to all of these remote devices and we will help you, you know, facilitate application deployments and updates and consistency and governance and security and all that stuff.

So we, we’re this, this control mechanism, command and control mechanism to say, “Okay, well, let’s help you get containers running at massive scale out in the field on, on sparsely connected devices, satellite, 4G, whatever.”

Nick Earle: Yeah. Great, and, and that’s, um, obviously very familiar to us at Eseye.

I mean, just the other side of the coin from the IoT perspective, we do multi-operator, multi-RAT. So we, we can connect, and localize onto, um, about 27 different operators from a sing- single SIM, not just roam. We can roam onto 850 or so, but we can localize, uh, [00:19:00] onto 28 of them, which is important for, latency and per- getting around permanent roaming restrictions.

But we also do multi-RAT, um, so we ha- we can do the satellite, as you say, so toggle within the device between cellular and satellite.

Neil Cresswell: Mm-hmm.

Nick Earle: Which is gonna get more important going forward. And at the same time, we make the router which, which of course, um, uh, does, uh, Wi-Fi, ethernet, a lot of other, protocols for connectivity. So ubiquitous connectivity, access, is really important because as you say, if you can’t communicate with the device, you can’t get those containers consistently updated and whatever.

So the two things, do sit close together. Let, let’s move on to, now we’ve had that explanation, I spent some, uh, time on it because I know that certainly our, listeners- And viewers, of course we’re on YouTube. Um, but our listeners, come primarily from the IoT, perspective.

So let’s move, now that we’ve got that understanding, into IoT use cases. I mentioned earlier, about ed- edge employment, and it, it’s getting deployed across a wide range of industries. [00:20:00] Um, so, um, what are some of the most impactful use cases that you’ve seen? And, and not just s- a list of them, but, what separates the successful use cases, in your experience, from the ones that never make it past the, um, uh, pilot stage?

Are there any lessons that can be learned- … on how to take advantage of this? I’m sure there are.

Neil Cresswell: There are. And it, it’s actually a, a, a kind of personal bugbear of mine because, you know, we, we see… Yeah, we, we, we don’t sell the IoT software, right? Right. The software that actually- Yeah … actually does something out on a device.

Yeah. Right? We’re, we’re, we’re this comms layer. Yeah. And so a- as a result, we, we get to see all sorts of use cases come through us, right? Where they say, “Hey, we’re working with this particular software vendor on this solution, or this particular SI on this solution, and we need a way to take their software and distribute it,” right?

And so, so we, we get exposure to a whole bunch of really cool use cases. And the ones where my eyes roll back in my head a little bit and it’s like, “Oh my God, this thing is never gonna [00:21:00]exist beyond a, an initial trial,” and it’s because they’re trying to boil the ocean. You know, they, they, they want to have a business case that looks absolutely compelling, amazingly beautiful, and will never see the light of day because the chance of going from zero to the business case is pretty slim.

The ones that I see successful are trying to solve just one problem Just, just do one thing. Get, get this piece of data or, or stop this particular issue that hits the production line, or do something. Tho- those one, those ones are pretty, pretty quick turnaround. You can, you can deploy the use case.

You can prove its success. You can go back to the business and say, “Look, we, we said we were gonna do this for this amount of money. We’ve now delivered it on time, on budget, and here are the results. Now we’ve got another idea. We wanna go and do this next thing.” At that point, you’ve actually earned the trust of the business, and so now they’re like, “Okay, cool.

Well, you did the first one well, so yeah, by all means, go, go do the second, [00:22:00] go do the third, go do the fourth.” So the thing that, trips up people too much is they try and boil the ocean. They want to modernize everything all at once because that makes the most sense. Mm, not really.

Nick Earle: That is a very familiar problem in all walks of life. I mean, I’ve been in IT over 40 years. I won’t say the exact number, but well over 40 years, and, I’ve always seen that problem in every architecture change or deployment or whatever, um, uh, uh, starting with mainframes.

But I can understand why that happens because although the project team may understand what you say, and you talk about it as, you know, you’ve got a very deep technical background, and you talk about it in terms of your bugbear, and I understand that. But, you know, having been the CEO of a few companies and, um, been, been at management level, often the, the…

And you’re a CEO of a, of, of, of a company as well. But often in a, in a, in a bigger company or a non-technical company- The project can only get signed off if it’s gonna [00:23:00] solve a big problem. And if somebody says, “Well, we’re not gonna solve the big problem. Well, we are ultimately gonna solve the big problem, but we’re first of all going to prove that we can solve a little bit of the big problem, and then we’ll go to a little bit bigger of the big problem, and then ultimately we’ll solve the big problem.”

It’s a tough message for people to get across when the, when the CEO or the management team think they’re under some sort of competitive pressure to reinvent themselves, you know, all the management consulting speak.

Yeah. Disrupt yourself, reinvent yourself, shoot for the moon, moonshot project, do something. AI is, can do everything, so do as much as you can. So that, that tension must play out, um, when people, uh, try to follow that advice.

Neil Cresswell: Ab- absolutely. And there’s nothing stopping you having, having a grand vision, right?

Nick Earle: It’s a journey.

Neil Cresswell: So here’s the grand vision, but we wanna start here, and, and we’ll only go to here when we get to this, this step. But e- even so, if, if, if you think about a production line, right, wastage is, is, is, is a big cost [00:24:00] for a, for a business, right?

They’re wasting materials that are going through the production line. Yeah. Rejects occur, and so wa- waste, waste is real money. If you, if you can say, “Look, I’m gonna put in a, a, a new system that’s gonna have computer vision and smart sensors and whatever else, and the goal is to try and detect, abnormal or, or abnormal deviations in our production line so we can start to get an early warning when failures and rejects are going to, are gonna start occurring so we can therefore make changes to reduce waste.

And by putting this system in, we will reduce waste by 70%, and that’ll have a cost of X.” As a component of a much larger project, that is, relatively standalone. You know, the overall project might say, “And I also want to, want to improve this and improve this and improve this and do this.”

Like e- even in a production line, when you look at the plastic molding machines, you know, they are pumping molten plastic through a very expensive nozzle in a machine to go [00:25:00] and form something out of plastic. If something happens to that machine and it instantaneously turns off, that molten plastic becomes hard plastic.

You’ve now got a $100,000- Yeah … $500,000 nozzle

Nick Earle: that- Yeah, with, with solid plastic …

Neil Cresswell: you now need to strip and repair. And so even just saying, we need to be able to predict the future for this machine, and so if, if certain things occur, we need to, to purge the nozzle so we can shut it down and therefore save the, the half a million dollar- Yeah

repair job on the nozzle. Now, that, that might also be rolled in as part of the first use case, but you know, phase two, phase three, phase four. So it, it’s about putting things in the right sequence where you can actually learn and improve, cut your teeth, and then move on, move on, move on, and take on slightly more grandiose, you know, pieces of the overall project.

So there’s nothing stopping you, and everybody should write a relatively grandiose aspirational dream, but then break it into manageable chunks that will deliver real business benefits and be able to close them off. [00:26:00]

Nick Earle: Yeah. It, it’s good advice, and it, and it, um, as I said, it is, it’s hard to follow but it, but it is essential that people, um, refer to it.

And there’s lots of case studies in we, in our own business. It’s not the same as the plastic nozzle, but the… We, we, we have a global, um, took a little bit by this side, but we, we have this SIM, as I say, that is the most global SIM. But what the telecom operators keep doing is, they keep on, they keep on killing, uh, the radio access, uh, uh, type, uh, like the, you know, they, they end of life 2G.

Mm-hmm. And then, you know, everyone goes, “Well, I’m gonna put a 3G module in,” and then, you know, you haven’t done your deployment, they start end of life-ing 3G. And, and of course, e- the modules are linked to the, to the G number. And, and so often we’ve seen customers, one customer in particular, I can’t name them, but it was a US customer with 80,000 vending machines that had, done the big grand rollout.

Uh, and, um, uh, they’d, they’d put it on 2G [00:27:00] Uh, this is when I first, came to Eseye. They put it on 2G and refused to, to look at the end of life for 2G in the area that they were in. A- and then they suddenly re- uh, got the notification that they had, um, six months left. And of course- Yeah … as to your point about cost, a truck roll or, or physically touching the device, you know, a tr- a truck roll back then, this was about eight years ago, it was $200.

It’s probably $400, $500 now.

Neil Cresswell: Ti- times

Nick Earle: 80,000. But eight, 80,000 times that was- Yeah … you know, a hundred times bigger than the entire budget, budget for the, for automating the, the vending machines. And, and it, it, but if someone had said, “Let’s just prove it, and then-” expand it, but also look at the technologies changing so fast, so also feather in the fact that we actually need fu- to future-proof it, ’cause by the time you roll these things out, at least in our side of the business, IoT dep- container, uh, pushing containers is probably [00:28:00] much, much quicker than IoT deployment, which can take years to roll out.

80,000 devices is a multi-year, multi-year project Yeah. Okay, let’s move on, about this whole concept of, um, bigger models and, uh, small models and, and, and address it from a, AI perspective. Um, we, we’ve had this conversation on some of the previous podcasts. R- regular listeners will know that, that, that, you know, the concept of everything starts off s- big and then becomes small.

And the m- the AI models so far, have been, you know, very large models. But when we talk about AI at the edge, there’s, there’s a couple of things that seem to me that are fundamentally different. There, it, it’s the fact that … Well, there’s several things. One is the software itself is gonna be resident at the edge, and there are multiple reasons for that.

But, but also the model, the model will become smaller. The AI model will become smaller and, and, and resident within the edge, so it doesn’t have to backhaul, to the, uh, [00:29:00] uh, uh, cen- center. But, but, but that’s going, in, in tension against the fact that the amount of data is exploding, you know?

Mm-hmm. So when you, when you actually look at what you can connect, collect, and now AI’s ability to analyze enormous data sets extremely quickly, you get ever, ever requirement for ever, ever larger models, but also a distribution. And maybe the answer to that is what, what containers is re- are, are really good at.

Um, so you, you, you, you have this ability to collect information from 80,000 vending machines, lots of rich information about what, what buttons people press, how long they hesitate, like the old e-commerce stuff we used to do on, on, on, on laptops. But at the same time, the vast majority of that processing is actually being done at the edge, and you’re reacting at the edge, and then you’re backhauling occasionally, maybe, maybe only every few weeks for, or, or it could be overnight when, when things are cheaper.

Neil Cresswell: [00:30:00] Mm-hmm.

Nick Earle: Are, are you seeing, organizations, deploying, and, and, and wanting to deploy that model of smaller specialist models, but at the same time saying, “But I want to analyze bigger and bigger and bigger, data sets”?

Neil Cresswell: So using containers for IoT projects was, and still remains interesting when you talk about just traditional IoT projects.

What’s made, I think, a pretty rapid acceleration in IoT projects now and containers in, in IoT projects is actually AI, right? Because if, if you have an, like a traditional ESP32 device that had incredibly limited CPU, incredibly limited memory, and realistically, it, it, it was, it was, it was firing relays and latches and, and doing some relatively simplistic stuff, right?

Now, all of a sudden, you can run actual software. There’s products like, like Node, like [00:31:00]Node-RED out there now. You’ve got time series databases. These are real software products you can now run on, in containers on devices at the edge.

Now, you can only do that if the devices on the edge are actually computers. You know, mini computers or microcomputers or- Yeah … whatever you wanna call them, but they’re actually computers with a real CPU and real amounts of RAM and real disk. You can now start to do cool things on them.

And so we started to see, you know, full fat applications being run at the edge, not only controlling sensors and pulling data and doing other things, but actually now starting to store and synthesize data and, and, you know, summarize and send that back But that’s, that was done in software.

And now with the next generation again saying AI, well, now we can actually start to provide real intelligence at the edge and, and you can start to make sub-second decisions based on incoming data feeds and also historical trends on the edge as well without having- On the- … to go back to a brain.

Nick Earle: Right. [00:32:00] So the decision and the logic and the ability to extrapolate the insight, and create the inference model was always at the next level up from the device. The devices were, were-

Neil Cresswell: Collecting …

Nick Earle: they were, they were doing the job, but they weren’t really…

But, but they had to send data back for the analysis, and then the analysis has to come back. And what you’re, you’re saying is as AI scales and as the hardware scales and everything becomes cheaper and more deployment, more capabilities, it, it is moving down and being distributed to the edge. And, and I guess the thing that really…

A- and, and the edge device itself can do most of the work is what I’m hearing. And then just sort of update head office for the next layer up later on. Let’s bring that to life with some case studies, ’cause when we, we did the prep call for this, you had some really cool case studies. And these are not your case studies, they’re case studies of ones that either you’ve seen your customers use or, or at conferences you’ve, you’ve seen people, talk about.

I know there’s a really, really good one to do with hand grenades, which I just wanna keep, [00:33:00]pardon the pun, the powder dry on that for the moment. Mm-hmm. but one that I thought really illustrates the point you just made is from John Deere, who have always been at the forefront of using technology and, you know, they satellite communication to their, um, uh, combine harvesters to make sure they go in- Yeah

a straight line or plant seeds or whatever. But you told me about, a something that is now possible using that model that was, uh, uh, quite amazing. So maybe you could just- Yeah, I- … recap that John Deere story.

Neil Cresswell: This one, this one has, has nothing to do with Portainer. I just thought this was a really, really cool use case, and highlights how the, how the power of technology is now really down to, to the far edge in field, and this one literally is in field.

Yeah. Now, I, I heard this from, from John Deere themselves at, at a, at a conference that was put on by Gartner. And what they, what they said is, “We are polluting the world with, with chemicals, and we, and we’re doing that to kill bugs and to kill weeds,” right? ‘Cause that, that, that’s how we produce food.

Yes, there [00:34:00] is organic farming, but everyone knows organic farming yield is nowhere near as good as traditional farming. It’s not really traditional. Organic is more traditional, but you know what I mean. Yeah. Spraying chemicals on stuff and fertilizers. But they said, “Well, how, how can we, how can we try and remove pesticides and chemicals from a farm without impacting yield?”

And what they said was, “Well, why don’t we put a boom on the front of a tractor with a whole bunch of cameras, and these cameras will do two things. They will look for bugs and they’ll look for weeds. And then on the back of the tractor, there’ll be another boom with a bunch of lasers, and these are high-power lasers.”

And so as the tractor is driving down the farmer’s field, the cameras in the front are looking in front of the tractor. They’re looking for pests, weeds, or animals. They will then make a, a determination, okay, laser seven needs to fire and kill this weed or this bug. And so then as the tractor passes over that, it shoots the laser and kills, you know, kills the weed or [00:35:00] the bug.

Okay, well, that’s pretty cool, but if you’re getting a camera feed, you know, video is m- massive amount of data.

Nick Earle: Yeah.

Neil Cresswell: Video, a video feed is a-

Nick Earle: can’t, well, you can’t upload it to make that decision, do you?

Neil Cresswell: Tractor’s driven past it. You can’t take that from the tractor- Yeah

send it over satellite or radio-

Nick Earle: Yeah …

Neil Cresswell: you can’t make the decision. The, the tractor’s already driven on. It’s, it’s, it’s gone. Yeah. So they, they needed to be able to make, make that, that decision on the tractor itself in the time it took, it took for the tractor to pass that same point.

And so they, they actually had an AI engine on the tractor able to make that decision. And so as it’s driving along, it takes a f- it, it takes the, the hundreds of photos, go- goes to the, the onboard AI engine. AI engine makes a decision, and then sends the command to fire the laser. And I thought, that is amazing.

And that exact technology, that exact architecture you could take and replicate into so many use cases around the world. So many of them. And you see this in retail, you see it in, in other, other farming sectors. It’s phenomenal. [00:36:00] But the only way that made that possible was being able to run that AI inference, the photo inferencing or the video inferencing on the device in near real time.

Nick Earle: I was just thinking as you said it, I’m a really keen gardener, at both vegetables and, uh, and, and flowers and plants. Um, uh, and I was thinking of, you know, robotic Hoovers, inside houses. But if there was… If, if that technology scaled down to a, a, a, something, really tiny like the size of a coin one day- Yeah

and you could just let them loose in your garden, and every now and then you saw a little blue flash- Yeah … like the, the flash you get when y- your fly catcher catches a- Yeah … a fly in the house and it goes on the grill. But, but the idea of not being able… not having to do weeding, and it would just id- So long as it identified the right things and it didn’t kill the wrong things, um, uh, like, like, like the, the pet cat or whatever.

The idea of weeding just from a personal perspective, or some of the bugs, like slugs, just ranting of my personal frustration- Yeah … with what I have to [00:37:00] do, would be such an incredible capability if that technology scaled down to a consumer device. Uh- Oh, totally

and most things do start off and scale down, so maybe one day. Maybe one day I will, I will be able to zap, zap the weeds and kill the slugs. That would be pretty cool for me.

Neil Cresswell: Well, that, that use case is actually available up on the John Deere website too, so you can just, you can go get it and read it and enjoy the actual, um, uh, uh, tech details of it.

It’s pretty phenomenal.

Nick Earle: You told me about another one, and I’ve already sort of given a glimpse to it, which is the hand grenade one. I’m sure people are saying, “Well, what? What on earth is the hand grenade one?” But it’s well worth telling again, a- a- as, we get towards the end, and we’ve got a couple more questions to go, but can you just repeat the, the Belgian, the Belgian hand grenade story?

Neil Cresswell: Yeah. We, we, we had a, we had a customer that is, potato processing, and they happen to be in the area that was heavily, uh, foot-trafficked in World War I, and so there is a lot of unexploded ordinances all over [00:38:00] the area. That’s cool, but when you’re processing potatoes that are in the ground alongside unexploded ordinances, it’s sometimes pretty dangerous to dump truckloads of soil and potatoes in a production line,

Nick Earle: Yeah

Neil Cresswell: with, alongside an unexploded ordinance.

Nick Earle: Especially as they clean the dirt off the potatoes- As

Neil Cresswell: they- … by shaking

Nick Earle: them.

Neil Cresswell: And and the ordinances are, you know, th- this is World War I era stuff, right? So these are old and quite unstable now. So how, how do you, when, when, when a pile of dirt and potatoes and rocks and other things are put into production line, how do you, how do you scan that in real time so that you can actually eject the dangerous things before they enter the production line?

Now, sure, may- maybe you could, you could use a metal detector and some things will be metal and some things aren’t, so you know, there’s, there’s all sorts of various traditional ways of trying to find that. But there’s also now AI vision, you know, computer vision that’s able to detect things and respond quite quickly.

And so they, you know, th- this customer does just [00:39:00] this. I’ve visited them a couple of times ’cause I find the use case particularly interesting, and last time I was there they gave me a bag of cool stuff their machines had found, and it’s, you know, bullets and all sorts of shrapnel and other, other interesting pieces.

Wow … and so it, it, it really is quite, quite interesting how you can… You know, that, that one is actually a safety- feature. You know, they’re, they’re, they’re scanning this thing, you know, to make sure that they don’t hurt anybody in their factory because you definitely do not want something that goes bang going into a factory.

Nick Earle: Wow …

Neil Cresswell: so yeah, it was, that, I, I saw it as a- Yeah … as a really interesting use case

Nick Earle: for the same thing. Or, or in, or in your bag of potatoes from the supermarket. Um, um, you know, and I think, um, what all of this does is- You know, I, I take a step back and think about w- what’s the lesson we can learn from all of this?

Well, one is, you know, technology moves very, very quickly. New models emerge. Mass scalability, happens, which is what we’re talking about. The confluence of, uh, uh, AI and IoT at the edge is [00:40:00] creating the opportunity now to actually build and implement, and your distribution capabilities, the software side, to build and implement, use cases that previously you could, you could say, “Wouldn’t it be cool if,” and now a, a, a huge increase in, in, in, in, in the number of times your, your dream of what would be cool is now possible.

But there are more and more things that are possible, which will cause an explosion of, innovation. So there’s going to be a lot of people, uh, who are doing this. There are already, but, but, just, you know, it’s probably at the 1% or 2% compared to what it will be in the future as these deployments, are, these capability all get hardened, and more people adopt them, and the hardware moves on, the software moves on.

Um, so, so the… That leads me to the, the, the final question, which is if people are listening to this and they’re in an organization that thinks, “Well, I’ve got a few use cases like that that could be pretty cool, and [00:41:00] I, I understand your advice of start small and, and- Mm-hmm … and, and go slowly,” so, so what would you tell them to do?

If people wanted to get going on this and try it, what would your, advice be on how to get started, other than pick something small and start there, prove a small thing first?

Neil Cresswell: I don’t think there has been a time in my career in IT now, which is I would say approaching 30 years now, where we have had the ability to go from ideation to something practically implemented in such a short space of time.

Like, that use case that I just talked about, about before, where you can use vision to make a decision. We’ve had that for a while in a production line, right? But what was there was very, very expensive, proprietary hardware doing that. Yeah. There were, there were specialist cameras, specialist infrastructure, very, very expensive, and only incredibly large production lines could afford it.

Well, now with AI, and [00:42:00] AI inferencing, and other things like that around that, that’s now commoditized. You can use regular old cameras that you can pick up from Amazon or, or w- whatever, whatever you like. You can go get a, an Nvidia Jetson for relatively inexpensive, or a Raspberry Pi, or Orange Pi, whatever else.

You can put a relatively lightweight AI model on top of it- And you can start to do really interesting things, right? So you can now, all of these, these crazy ideas that you’ve h- that you’ve had that historically might have been, oh my God, these are awesome, but there’s no way. It’s, this’ll be a multimillion dollar, multi-year project to do this.

Now is probably gonna be a few thousand dollars in infrastructure and some time probably, probably Vibe coding. You can really ideate and get something usable in a POC pretty damn quickly. So my advice is just do it. Just do it.

Nick Earle: Do it.

Neil Cresswell: Don’t, don’t be- Uh- … don’t get scared. You know, you’re not trying to go and ask the business for a million dollar piece of hardware With the [00:43:00] hopes and dreams that it’s gonna have monumental impact to the production line. You’re talking about probably something that you can buy on a credit card, spend a few weeks experimenting, and get something usable to say, “Hey, look, I’ve built this thing.

It looks pretty cool. What do you reckon we now roll it out three or four or 10 times and, just see what happens?” Your throwaway for failure is now dollars and cents, not millions of dollars. So leverage that opportunity that we currently have in front of us.

Nick Earle: The, by the way, I, and I think there’s historical precedent for that. Uh, when I was at Cisco and, I, I went through the period when, you know, Cisco was selling… They had 50 million, I used to run the sport business, and 50 million installed base devices with a sport contract. It was a very nice business.

But then suddenly it became clear that the applications weren’t going on a box anymore, they were going, they were going in the cloud. And- Yeah … but it wasn’t just that, it was, it was, it was the cost model disruption of e- exactly as you said, because, you know, a Cisco box and, and the iOS software, Cisco iOS, that [00:44:00] would go on it, and everything that you needed to do, and storage and whatever, would cost you tens and tens of thousands.

So you had to ask for a budget, it was the project. And then what we found was that people were innovating based on exactly what you said, credit cards. Because AWS-

Mitch Thorpe: Mm-hmm …

Nick Earle: aWS allowed you to actually build applications and, and, and do your own thing.

And, you know, when we found out that the average person was spending something like, I don’t know what it was, the average AWS bill at one point, I think it was something like $12 per, per month. Yeah. I mean, how the hell can somebody build a business on $12 a month when we, we have to send, spend, sell something that’s $50,000?

And of course, that then took off and disrupted and, and the net effect was an explosion of innovation and, and we all move forward. And I think it’s happening again, but it’s happening- Mm-hmm … at the edge, to your point. Right. I totally agree with you. It’s happening at the edge. It enables IoT, which has b- showing so much promise for so long and not meeting it.

We talked also about 50 billion IoT things will be, [00:45:00] will be connected by 2020. we got to 8 or 11, but, but we never got to 50 billion, and it should really be 500 billion. So, so I think, uh, uh, uh, all of these coming together with then the economic model, as you say, where you can do it on a departmental credit card or just do it personally- Mm-hmm

yourself with vibe coding, is gonna cause an explosion of innovation and an explosion of building and an explosion of entrepreneurs, which is a very, exciting future, and we make another big leap forward. Um, uh-

Neil Cresswell: And, and, and by the way, those, those enterprises that don’t embrace this innovation- Will absolutely be out-innovated.

Yes. And, you know, margins in, in production is, is pretty slim, right? Everyone knows that. You know, it, like, like, it’s, it’s, it’s tough times now. You know, the cost to put, to put food on the table is, is at an all-time high. And so everyone’s being squeezed for margins to try and keep costs low.

The person who bears the brunt of it the most is the person at the very, very [00:46:00] start of that line who’s actually producing the food or whatever else. So, so they, they need to absolutely get as, to get as much efficiency as they can. And so being scared of innovation, thinking, “Oh, this is not for me,” or it, “it’s too big of a project,” they- then they, they will simply be, be out-innovated by their neighbors who will then win all the contracts ’cause they can now deliver- Yeah

the products- Yeah … or, or the produce or whatever it is for, for- Yeah … slightly less, and therefore win, win the deals. So you have to, you have to innovate. You have to. You, you can’t be scared. You, you should have, even if it’s just one person who’s, who’s, you know, h- head, head of R&D or skunkworks or whatever you wanna call it, to go and experiment with this stuff.

Go, go, go and take some crazy ideas and, and just see what happens. And again-

Nick Earle: Yeah …

Neil Cresswell: failure , is not failure. Failure is somebody says, “Well, that didn’t work. Let’s try the next idea.” Nine out of 10 ideas won’t work, but one idea will, and that idea will save the company a lot of money.

So it’s, worth trying.

Nick Earle: Yeah. And it’s always been the case, and the, the disruption has always come that way, and the companies that [00:47:00] don’t react sit on their laurels and, and just take the maintenance from the current business. It, it, mathematically it’s been proven, you know, in business schools it’s been proven that, that, you know, every wave of innovation, innov- capability that, that’s really big like this, typically if you take the top 10 companies by every industry, I think this is McKinsey research or could’ve been Gartner, as you say.

The top 10 countries by every industry, and you look at them on a sort of 15-year cycle only two typically on average by, across all industries stayed in the top 10-

Neil Cresswell: Yeah …

Nick Earle: and the 15 years becomes 10 years, becomes five years, becomes three years as technology grows, and they’re the ones that, that, that managed to innovate and, and, and, the others didn’t.

So it’s an opportunity for everybody, and it’s a very exciting area, and we have run out of time. I, I really enjoyed, the conversation. I think we probably, and I hope, I think you have, Neil, opened people’s [00:48:00] eyes, if they’re not familiar with this, of what the opportunity is, with a great call to action at the end.

Just do it. Innovate. And the costs are, are not big at all, and you may well be pleasantly surprised as to what you can do, and show to your boss and say, “Look what I’ve done.”

Neil Cresswell: And, and, and also for, for, for the bosses that are listening to this or watching this, encourage innovation.

Encourage it. Yes. Yeah. Don’t- Yeah … don’t, don’t shut it down, encourage it.

Nick Earle: And don’t see it as a threat. Yeah … but actually encourage it and maybe a skunkworks, as you say, is the right way of, uh… Take the rebels in your organization and put them in a different building or, or whatever, and give them some autonomy and a little bit of budget and just encourage them to, to dream- Experiment

and come up with crazy ideas and- Yeah … experiment. Yeah. Great. All right. Well, let’s leave it there. Thank you. Thank you so much, uh, for the, for the podcast, and, uh, the advice, and good luck with Portainer and what you’re doing. I think based on everything that you’ve said and, I think, you’re gonna be pretty busy, uh, going forward, to enable people, uh, [00:49:00] to do this, and I think it’s also good for IoT as well, IoT deployment, which is something I’m, passionate about.

Um, and, if people want to know more about your company, I… as I understand it,

Neil Cresswell: portainer.io. Portainer.io. Absolutely.

Nick Earle: Portainer.io, and, get all the information there, and, they can, uh, follow up. Neil, thanks for being my guest, uh, on, on the podcast today.

It was a great conversation. Thank you.

Neil Cresswell: Thanks again.