Why AI Must Move to the Edge

Transcript

Intro: You’re tuned in to IoT and AI Leaders, your go-to show for insights, predictions, and big ideas on how IoT is reshaping the world of AI.

Nick Earle: Hi, this is Nick Earle, Chairman of Eseye, and this episode is with David Linthicum. David, you’ll hear, has a deep and very relevant background and knowledge with regard to all things AI. In fact, I would say all things tech.

We’ve just finished the interview now, I’ve stayed on to record this intro, but looking back, we had about a 30-minute-deep dive on the subject of where AI will go on next, and in particular, the need for AI to replicate — my words and other heirs — replicate a client-server architecture.

In that it’s going to have to be resident within the device, as opposed to what it is right now, which is resident in the host, or back in the day, the mainframe model with the dumb device. That’s where we are now with AI, and we need to flip it, and we need to get the intelligent edge device, and the back-end systems of these big data centres need to still be there, but the processing and the data and the intelligence has to be at the edge. And David goes into it in deep detail with his experience, both running an analyst company, being the head of cloud over at Deloitte, and with his 17 books that he’s written in general around the subject.

A very knowledgeable guy, very nice guy, a very good interviewer. I think you’re really going to like it. We also get into the role of device companies and how they need to take the lead on setting the standard for device-centric AI, and we also have our usual chat about where are the jobs going to come from, and are we all doomed?

Spoiler alert, he says, “no, we’re not” and he explains why. So, with all of that, let me hand you over now to my interview with David Lithicum, who is the founder of Linthicum Research for my IoT and AI Leaders Podcast. Enjoy.

Nick Earle: So, David, welcome to the IoT and AI Leaders Podcast.

David Linthicum: Oh, it’s great to be here, thanks for inviting me.

Nick Earle: You’re very, very welcome, and for our listeners — and indeed viewers, because we do have both — and we’ve got them in many countries around the world.

Just a little bit of what I know about David, and I’m going to ask him to introduce himself. But some interesting things about David before we get going. First of all, first person we’ve had on the pod in over 5 years who has a high-level message which says, don’t ask to be one of my contacts on LinkedIn because I ran out of slots several years ago.

So, you maxed out the LinkedIn slot several years ago — the first person I’ve ever met who’s achieved that. But it also says, globally recognised thought leader, innovator, and influencer in AI, cloud, computing, and cyber security, and the founder of Linthicum Research.

So, David, pretty impressive profile. I know you’ve got a lot of experience, also being in Deloitte, amongst other companies. Well, as I say, welcome again to the pod, and maybe you could give a better intro to yourself than I just attempted.

David Linthicum: Yeah, sure. A good way to do it in an elevator pitch would be a retired Deloitte managing director. I was at that job for 8 years. I was chief cloud strategy officer, worked on AI systems, IoT systems, and cloud-based systems. Prior to that, I started and sold a bunch of companies, consulting companies, technology companies, things like that, many of them publicly traded.

And prior to that, I was an engineer – a software engineer. During my career, I wrote 17 books, including the two behind me that I wrote recently.

And I have a YouTube channel now called Cloud Computing Insider, another one called Dave Is Not AI, and I’m absolutely having a ball right now, as the industry is inflecting, and just kind of watching the whole thing take off and being a pundit.

Nick Earle: Yeah, we were chatting before the pod, and last week or so on… I don’t have anywhere near as impressive a track record as you, but it is similar, and I was many, many years in HP, and in Cisco, and running startups, or early-stage companies, and now doing a lot of podcasts, and a lot of AI work.

And it is a very interesting time. There is no shortage of raw material and no shortage of announcements. With that, let’s kick this off.

The subject… we recently rebranded this pod, IoT and AI Leaders, because we fundamentally believe that those two areas, which have been distinct, are both growing — not only growing rapidly but merging and coming together — and that needs to happen. Maybe we can kick off by me asking you, what is your view on that overall hypothesis?

David Linthicum: AI is converging with everything right now, so I think it’s going to be systemic to IoT, systemic to cloud, systemic to core legacy systems that we’re building, middleware, security, governance. Pretty much everything out there is going to be touched by AI, because it’s a functional… a very value-added utility that we can kind of quickly add to these existing systems, and so you can’t really draw the line between the two.

In the olden days — back in the 80s when I started doing AI — AI was in one silo, and the systems, the data, the applications were in another, and it was very difficult to mix the two. And there was really not a lot of conversions, but also it cost $20 million to build a very basic system back in the 80s. Here we can do the same thing for $20.

Nick Earle: And they’re going to be integrated into every part of the systems out there, and the reason why is we’re able to use it as a force multiplier. In the case of IoT, obviously.

David Linthicum: Cameras and things like that, and the ability to recognise faces, the ability to recognise crimes, the ability to do lots of things. That was always there, really, in your ability to upload an image into an LLM and have it analysed but not necessarily innate to the device.

And I think AI is going to live in these devices, whether it’s going to be an agentic state or just kind of a common AI system, and it’s really going to add a lot of functionality, and so much so, I think that the customers out there are going to expect it to be part of it. In other words, whether it’s dealing with robot maintenance on a factory floor, or cameras, or automobiles, and mobile computing, and all these sorts of things. AI is going to live in a distributed heterogeneous state, where it’s going to be there at our fingertips, in our hands, in our cars, adding value when we need it.

Nick Earle: I agree with that explanation, and it reminds me of a conversation, certainly, when I was just as I was leaving HP. I was based over in the Valley, and was involved in the cloud programme at HP, and then subsequently also at Cisco.

I remember having exactly that conversation about the internet was going to be ubiquitous, and the internet was going to be everywhere. Back in the day, the analogy we were using is that, today… it was the classic: you stay in a hotel, and you had to putz around trying to get your phone cable into the socket in the wall to be able to use your laptop, and then there was a room in reception that was called the internet room — the business office. The internet was a place.

And we were saying it was going to become ubiquitous, and of course it did. So, your answer is essentially that AI is going to be everywhere, and inside everywhere. The great thing about a technology is pervasive when it’s invisible. But it isn’t right now, is it?

AI so far has been… obviously manufactured by the Big Seven — that’s another subject — created by the Big Seven. But in terms of into devices, we’ve only recently seen, with innovations like OpenClaw and whatever, people saying, “Oh, AI is in a device,” the small Apple edge device which people are experimenting with.

But it ultimately does have to get into the devices, and when you look at that from a perspective of IoT, there’s a zillion devices, and they don’t have much memory, they don’t have much processing power. And AI can at times be pretty beefy in terms of its resources. So any views on how that will progress, how AI will ultimately seep into the edge and actually become resident in the devices?

David Linthicum: Yeah, that’s a great point. It’s funny, I wrote an article over the weekend in terms of how AI agents aren’t really agents, and how they’re dependent on backend LLMs to function, so they’re really not autonomous unto themselves. In other words, if the ChatGPT backend API ceases to work, then the agents are going to cease to work. And so, if we’re going to use agents and devices, we’re going to have a similar problem.

And as people are constructing these IoT systems, whether it’s something in a car, or a cell phone, or on an airplane, whatever, they’re really not constructing them in an autonomous way. They’re looking at back-end services and back-end cloud services as providing the brainpower, because that’s the path of least resistance.

In other words, I have access to a LLM that costs $120 million to train. And it’s going to provide me with some pretty credible answers, analysing videos, analysing voice, things like that, and be able to do so on demand.

That provides a limitation, however. If I’m in a jet plane, it’s going to be very difficult for me to transmit something over the internet — even though it’s possible — to have a direct understanding of the fact that my engine’s on fire, which is a pretty important concept to have when you’re flying an airplane.

Or the ability to put AI into systems that are not necessarily going to be network-connected, and are going to, in essence, need to have autonomy unto itself to give you the performance that you’re going to need. So, there’s latency in sending something back or invoking an API that’s going to be 5,000 miles away, and receiving a message back, and we’re acting upon it.

However, in my research in building these systems, AI doesn’t necessarily need to have these big honking’ GPUs, and it doesn’t have to be an LLM. And you don’t have to have an LLM support it. So, as we’re looking at kind of the future of IoT, and I’m seeing this in my research now, is that we’re looking at smaller… I say dumbed down, but smaller systems. Small language models are going to be innate to these devices. They’re going to be purpose-built for these devices.

I don’t need access to all of the capabilities of an LLM. I don’t need my device to write a song about me and do all the cool things that generative AI needs to do. I just need to have a certain very limited amount of functionality, and it’s able to do that lightning-fast and reliable way without backing connectivity.

And the architecture is there, and we know how to do it, and I think the CPUs are out there. You obviously mentioned Apple is moving in this direction. I think they’re making some inroads as well. But it’s perfectly fine to look at these commodity processors and build these systems. They’re going to be a subset of what a large language model is, meaning a small language model is going to be very, very small, they’re going to operate in very narrow purposes, and I think that’s what IoT is going to be. It doesn’t have to have the back-end capabilities.

And I view the architects who are building these systems now with the back-end LLM connections as just lazy architecture, because I understand it’s more complex for me to build a small language model.

Nick Earle: Every memory system, operating system.

David Linthicum: Security and governance, all that which is innate to a small device that has a very limited and low-power processor. But in connecting it into some sort of a back-end system, whether it’s an LLM or a cloud-based system, I’m making that architecture much more fragile and much less useful to me.

And you’re absolutely right. Network’s everywhere, internet is everywhere, but people have gotten overly dependent on that, and in many cases, IoT is going to be more successful if we build these things as autonomous devices, and we’re not focused on that right now.

Nick Earle: I wanted to use another analogy. One of the benefits of having been in IT for well over 40 years — in fact, I’m much closer to the 50 than 40 — is that you get to see a lot of models, and sometimes things look really new, but then you take a step back and say, “Hang on, we kind of have seen this before in other areas.”

And the one I’d like to posit that potentially is the same is actually enterprise software and client-server. I remember in the 90s — so let’s go back 10 years from when the mass adoption of the internet — I remember in the 90s, we had the… in the late 80s, we had the explosion in the edge, but then it was called the PC. It obviously wasn’t the mobile. The PC could do a lot of local processing, but the software was principally mainframe or host-based, and it couldn’t really… the PC was basically just a really expensive terminal, and people were creating, if you remember, X terminals, or dumb, thin terminals.

And then along came client-server, which actually was a three-tier model, or sometimes it was a five-tier model, separating the data from the logic and the display and the whatever. But essentially, the architecture for the software, the intelligence, was distributed across different levels, and there was some edge intelligence increasingly in the device, using the processor in the device, because of the rise in the processing power of the CPU and the falling cost of the memory.

Is that a relevant analogy to say what you think needs to happen next when you talk about small LLMs? That actually you would predict that we will get sort of micro-AIs, or whatever phrase they’re going to be, which will be device-resident, but they won’t exist on their own. They’ll be a sort of AI equivalent of a client-server data logic processing type architecture?

David Linthicum: Yeah, you and I are from the same past, so I agree with it, and at the end of the day, we’re just reusing old ideas in new ways, and that’s what we’re talking about here, is a tiered-based approach.

We’ve always associated tiering with AI systems, and for some reason, we’ve lost the idea of tiering when we moved into different dimensions and different architectures, and I think we need to find that again. The great thing about client-server is I could distribute things and put them in their own functional.

Nick Earle: Platform where they’re going to be optimised for my use case. The data, the transaction processing, and the interface that’s talking to the users.

David Linthicum: In this case, it’s basically the same thing. In other words, these devices are going to be connected, or they’re going to be sometime connected, and so you can use that as well to update the small language models, to update the firmware, things like that.

But they really shouldn’t be built for an all-on connection, where all of these things are occurring on the back-end systems, because at the end of the day, that’s a sledgehammer hitting a thumbtack, and it’s going to be way more power and way more cost that you have associated with it.

And that’s why many of these devices, I’m seeing them out in the market, they’re charging $50 a month for subscription costs for cameras and different IoT devices, security devices, things like that. And you’re paying for the back-end LLMs, because these things are AI-enabled, and so they’re building the logic and the tiers in the LLM, not necessarily in the device.

And eventually, you’re overpaying for that particular functionality. If you’re dealing with a security system, you’re worried about a door being open or not, and if it’s not open, you evaluate why it’s not open. Did a human walk through it, things like that. Those are extraordinarily easy things for AI systems to figure out, be it with base images and things like that, and do so in a fast way.

And your ability to couple it to some sort of an overly obese back-end LLM is going to slow down that processing. So, it’s okay to have the tiering here, and I think that doesn’t necessarily go away. You’re going to have a back-end connection, and IoT should always have that. WiFi and Bluetooth is extraordinarily cheap. We’re able to build into all these devices now for no heat, no power. But build your core functionality, which is at the edge of the technology, closest to the data, closest to the humans, closest to the devices where it’s going to be interfacing. Because if you’re sending everything over some sort of a back-end system, that’s not going to be acceptable longer.

Nick Earle: That’s the latency.

David Linthicum: Latency, latency, security, all these sorts of things, and I just think it’s lazy architecture. And so, when I was involved in some of these edge-based systems, the architects who were working on building these things were doing that each and every time. They were connecting it not always to AI, but some sort of a back-end cloud database or back-end cloud transaction processing.

And the device was doing nothing more but a dumb terminal, to your point, where it’s just sending information and requests back to the back-end system. It couldn’t do anything without connectivity. And to me, that was worthless, because if I’m losing connectivity, I’m losing the device. And we’re all familiar with losing connectivity to our phones and cameras and things like that and not working until we reset them.

But the reality is, all these things, these edge devices, they have CPUs, they have memory, they have data storage, they’re fully capable of running these small language AI models. We just, for some reason, got out of the habit of using the devices and the power of these things.

A Raspberry Pi, for example — I got one running in the other room — it does an amazing amount of things, and I can have a small language model that… I have DeepSeek running on one of them, that’s running on an instance of it that’s able to carry out an activity, and the thing is, I could care less if it’s hooked up to the internet or not. It needs some of the backend connectivity to update its model, update its firmware, update its operating system, but it’s completely autonomous and completely more usable, and that’s got to be the direction that we go.

Because I think that everything being, including the Agentic AI movement, everything’s connected to some sort of a back-end LLM. If you look at the open claw stuff that’s out there now, all it is is basically an LLM orchestrator. Where it’s binding these things into pieces of software that’s calling agents, and so the genius occurs at the LLM. It doesn’t occur within OpenClaw, or within the agent. And that widely distributed host-based model won’t scale.

It’s over-engineering these capabilities, and it’s not getting to the architectural principles that I think we’re going to need. We’re going to be able to build the cheapest, fastest, best way to do it, versus the fastest way to get into the market, and that’s what many of these IoT guys are doing now. Whether it’s a thermostat on my wall, they’re not building the functionality, they’re not building intelligence into the device, they’re pushing it on the back-end systems, and that’s architecturally easy, because we do have this connectivity.

But at the end of the day, it’s not going to perform well, it’s not going to function well, and it’s over-engineering a very simple problem. These IoT devices are typically built to steer a car, to deal with security, to deal with cameras, to deal with very specific, narrow range of functionality. And if we do that really, really well, we live up to the objectives of that device. And I think we got to stop hooking these things up where they’re coupled to back-end systems, because that’s not going to be architecturally acceptable, it’s too fragile.

Nick Earle: And you could add to that by saying… it’s interesting you use the analogy of the Chinese, the DeepSeek, because if you… in a world where the LLMs are huge, and they run in these huge data centres with the zillions of NVIDIA processor chips and whatever, the power, the cooling, the cost of the physical infrastructure is all these billions of dollars that are being invested — and invested in a circular way between the partners and back again to buy the services — is all based on this premise of… it’s almost like the mainframe analogy, isn’t it?

It’s… you’ve just got to build a bigger mainframe, and now we need a supercomputer, and now everybody’s got to have a supercomputer. But then you get the Chinese who come out and say, “Well, my benchmarks are just as good as yours,” and guess what? First of all, I open-sourced it, but also, it’s a much smaller model.

And it’s almost as if people went, “Oh, I’ve been so obsessed in going in that direction — everything’s got to go into the big AI mainframe, the data centre — that we didn’t do that.”

And then OpenClaw was the first example that got a huge publicity of any sort of edge. But ultimately, that’s almost like the middle layer in the three-tier client-server analogy.

As you say, there are hundreds of millions of devices. And although the architecture of IoT devices is completely different — there’s only like 6 or 7 cell phones, and so you can create applications for them, you only have to do 7, and you’ve got 99% of the world’s cell phone market covered — with IoT, you can’t do that.

But the idea of micro-AI seems inevitable. But it also will require a different type of architecture at the back end, and that’s not where the money and the investment is going right now.

So all of that is a setup for the next question, which is: this all makes sense, and certainly in the context of IoT, it’s absolutely essential, because in… what we said with cloud, 80% of the applications and the data will be processed at the edge, so with AI, arguably even more than 80%.

But given where we’re starting from, with these big models and these huge data centres that are being built at rapid speed, with all the money that’s going behind it, how… what’s your view on how long it’s going to take before we start to see the emergence of these… I’m going to use the phrase micro‑AIs that could fit inside a device, even though there is some new form of client-server model for AI? How long do you think it’s going to take? And are there any indications that you see in your research that it’s starting to happen?

David Linthicum: Yeah, I think the market’s going to have to drive it, because right now, the market momentum, to your point, is building huge amounts of AI infrastructure on the back end. I don’t know how we’re going to power this stuff. I live in the States, there’s 100 data centres near me.

And it’s going to have to be driven by market demand and market needs. In other words, if people are entering the market using these small language models, they’re able to provide better price performance than many of the ones who are LLM-connected. And by the way, there’s no subscription services, because they don’t have any kind of an additional operation cost, other than updating the firmware every year, or something like that.

And it just works. I think people are going to migrate to that, because they’re going to see that as a better benefit to them. All they’re looking for this thing to do is to be good enough in terms of what it’s for and not necessarily having the bunch of information and behaviours that the LLMs are able to provide.

So, if someone’s building a better thermostat than the Nest thermostat — and my Nest thermostats are back-end connected still, and they do a lot of the processing on the back end, some stuff in the thermostats — and it’s completely autonomous, and it doesn’t necessarily need to be hooked up with the internet for it to do these amazing things, where it’s able to provide a small language model processing with a small CPU, things like that, and they’re going to sell it for 20 bucks or 30 bucks, because that’s pretty much going to be double the cost of the parts that go into it, I think that’s going to win the game.

And I think the autonomous model is something that people are seeking right now. I notice when people are rolling out agentic-based systems — and this is a different thing, but it has an analogy to the IoT stuff — that they’re surprised that there’s so much back-end dependencies on the systems, because they were told it was going to be autonomous. Agent means autonomous. And the reality is, is very much like the client service systems that you mentioned. They’re very thin clients; they just call back-end systems. And they produce, and they run it. They may have some rules and logic, but they still depend on the back-end systems to run its rules and logic.

People are not necessarily going to accept that longer term, because it has architectural fragility where it won’t be able to scale, it won’t be able to deal with BCDR scenarios, any kind of glitch in the network is going to kick it offline, and it’s not necessarily going to do the test of time in terms of what IoT systems need to do.

Gotta remember, if we’re getting an IoT system, we just want the damn thing to work. It does a narrow set of functionalities, whereas for my thermostat, I want it to turn the heat on if it’s too cold, I want it to turn it off if it’s too hot. That’s not something that’s going to take a rocket scientist or an AI engineer to figure out in a very short period of time, so I need very narrow sets of logic. I can use a very small language model, a low-powered processor. And the backend connectivity’s going to be nice, but it doesn’t necessarily need to drive it.

I don’t need an LLM to drive those sorts of things, because that’s just going to add cost, it’s going to put capacity that’s going to be needed on the back-end systems. They’re going to have to charge you for that.

I don’t know if you ever hooked up to the ChatGPT API. I’ve done so with a few of my Raspberry Pi things. It’s very expensive to get these things done if you’re making these constant calls. And one of the things I found out when I did it is that, well, this isn’t really needed. And so I put a copy of DeepSeek on the same device, and it was able to carry out many of the subsets of stuff — a lower parameter version of DeepSeek that’s running on a low-powered processor — but it was good enough for what it’s for, and I didn’t have to maintain a back-end subscription to the API, and I can pass that directly to the consumers, because it’s not necessarily needed.

It’s architecturally inelegant, it’s going to make something fairly fragile, and I think people are going to vote with their dollars. And so if someone says this is truly autonomous, and they’re going to show the power of that, we’re not going to need these back-end connectivity, that’s going to be something that I think people are going to pick, and that’s what’s going to send the market in that direction.

We just have to vote with our dollars in this space. And the technology providers are just fast followers out there. They’re going after everybody like lemmings running off a cliff, but they are going to make changes and adjustments based on what they feel the market is doing. If their customers are asking for that, and that’s a core capability, and having run technology companies before, they’re going to go ahead and build in that direction, and I think the first movers in that space are going to do very, very well.

Nick Earle: I get one more view on that before we pivot to something else. I think this is a really good deep dive on this subject, which I haven’t certainly had this conversation before on the pod.

Just a little bit of background to the question again. So, we at Eseye, and we talked about it on the pod, we’ve had them on the pod, we power AT&T’s global platform. Now, one of the reasons they chose us, they have Cisco — as it was called — Jasper control centre for their national coverage, but AT&T don’t have 100%, as you know, you’re in the US, AT&T don’t have 100% coverage in North America, because you’ve got Timo, and you’ve got Verizon, and you have not spots. So, they can’t even do national coverage 100%, which IoT needs, because IoT devices have a nasty habit of being in places where there aren’t cell towers owned by the operator.

But then, from a global point of view, they don’t have the infrastructure for local breakout. They don’t have a series of processes, for data sovereignty, compliance, and local breakout, all of which means that the promise of where I was going with that is that the promise of 5G which certainly we find in our business is people say, “Well, 5G, what’s the problem? 5G’s going to solve it, because I’m going to get 100 times more bandwidth with 5G.”

They say, “But yeah, but 5G isn’t going to suddenly break the laws of physics,” because if you have to trombone it from another country where your device… you want a truly global device, a single SKU that works anywhere… if you’re going to trombone it back to the US, and then back out to the device without local breakout, it doesn’t matter whether you’ve got 5G everywhere.

First of all, you’re going to be using a lot of different operators, and secondly, they won’t have all implemented 5G, so you won’t always be on 5G. And they’re also implementing it with standalone architecture by GSA. So, it’s not the same, the APIs aren’t the same.

So, there’s this… one of the reasons AT&T went with us is because we do have a global network. We have a… we run the network for AT&T’s global IoT, which is kind of weird for a small company to be able to say that.

But when you scale that to everybody, and everybody wants truly global devices, and the potential of IoT for there to be hundreds, if not billions, hundreds of billions of devices doing what you said, this issue of physics has a nasty habit of getting in the way, and unless you can do the 80% processing at the edge, our view is that you just won’t get the mass adoption, no matter how good the devices are, or how compelling the use case looks on paper, if you can’t implement a global network of devices with a single product SKU, because that’s what you need to do, because otherwise your manufacturing and your supply chain costs are so expensive, then 5G isn’t going to come to the rescue, and suddenly being able to change the laws of physics, and your latency — you may be processing data at the edge, but some of it will still be being backhauled.

Without the edge, the true edge application, the issue you’re going to have is an awful lot of devices with terrible latency problems. So, there’s an architectural reason why the intelligence — 80% of the intelligence — has to go to the edge, not just a device-centric reason why, and I think a lot of people miss that point, because they think 5G is going to solve everything.

David Linthicum: No. And I heard that a lot when 5G first rolled out in the States, besides the fact they thought it was going to crash airplanes. The reality is bandwidth doesn’t save you, and I understand connectivity. I can have very strong connectivity, and we have very strong connectivity here in the States and the metropolitan areas, not so much in the rural areas, things like that, but the fact of the matter is you can’t rely upon it as something that’s going to be capable of getting you out of the woods each and every time.

Nick Earle: It’s not going to make up for these architectural deficiencies that we’ve talked about.

David Linthicum: You got to have a good… you got to have the right architecture, and you got to go through… remember the first rule — I’m an architect by trade — the first rule of architecture is try to decouple as much as possible, because that’s going to make a successful architecture.

And the thing is, if we’re coupling all these things to back-end LLMs, and back-end data and back-end processors, then we’re not doing that. Then we’re overly dependent on the connection. People say, “Well, the connection’s very good now, we have 5G, everybody has this massive bandwidth that comes down to the devices.”

That’s still about architecture, because in a scenario where you’re going to have failover, network outages, things like that, we have to have the device running, and even if it can run for 5 minutes until it reconnects, that’s not necessarily helpful. The ability to have this thing that’s able to run in a disconnected state for a long period of time and then be able to reconnect and then have the sync whenever it’s done, is going to be an autonomous thing that’s going to drive reliability.

So, if you want performance, you want reliability, you want scalability, and also you want cost efficiency, because I’m able to build these things for a fraction of the cost, because I don’t have to pay for the back-end GPU usage, which is expensive, then you’re going to be better off in doing that.

And you’ve got to remember, IoT’s a great application for this, because it’s a very narrow focus thing. We’re building a device that’s going to do a certain thing. In other words, we’re not building a computer that does millions and iterations of applications, it does thermostats, it drives a car, it drives a robot, things like that. And so, now we have the unique capability of customising an architecture that’s going to be more optimised for that.

And so, why would you want to couple it to some sort of a back-end system? And they would say, “Well, it’s easier for me to do that, because I can use the facilities of the LLM to build my logic, to build my systems, and get very sophisticated about it.”

But you’re overdoing it. In other words, you’re over-engineering it. You don’t need the LLM backend systems, you don’t need these huge databases, you don’t need supercomputers that are running on cloud to drive a very simple device that does a very simple thing, and I don’t know why everybody just got enamoured with that.

When IoT first started, it wasn’t as much of an issue, because connectivity wasn’t as good, the back-end systems, the cloud systems weren’t as good. And so, when we used to build and architect these things, the decoupling of the back-end system was a rudimentary… was a core priority in the architecture.

For some reason, we lost that. Once connectivity showed up, I think the architects, the developers, the IoT manufacturers got lazy, and I think we’re going to have to push them back into efficiency, cost efficiency, and the market’s going to drive them.

In other words, if I’m selling an IoT device and it costs $30 for a subscription to the back-end services, because they’ve got to pay the LLM provider to actually function the IoT device, I’m not going to want it, because it’s not worth it to me, versus, based on the very narrow functions that it does.

I’m going to go for the device that has completely autonomous systems, able to run to itself, able to have knowledge, the model, which is innate and native to the device, as well as the data, as well as the operating system, things like that, and it just works, as long as I power it up.

Whether my WiFi is operating or not, whether the network is down or not, or some remote… if Amazon had an outage, suddenly your IoT device won’t work, like we had back in October.

It’s just a smarter way to do it, and I think we’ve gotten lazy, and I think the market’s going to have to kick the butts and get them back into line in terms of architectural realities, in terms of cost efficiency. Now this stuff’s way too expensive.

Everything requires a subscription, and the architects haven’t really thought through how this thing’s going to scale up and work moving forward.

Nick Earle: So, we’ve given this subject a good kicking, so let’s move sideways slightly. Let’s assume — and I think it’s more than an assumption, I think it’s a certainty — that this is going to happen, and it has to happen. So now let’s move into how and who.

There’s kind of two models. One model is, an independent software company… well, first of all, I’m going to assume that the big guys won’t do it, because they’re so busy making trillions of dollars building these backends, so there’s no commercial incentive for them to create these micro versions of the AI. So, the innovation has to come from elsewhere.

And broadly, it seems to me that there’s broadly two directions it could come from. There’s an independent third party who tries to create a device standard for AI, and says, “I’ve got a catalogue of these micro‑AIs, and I’m going to be the device standard,” and therefore, they get a fraction of a cent for billions of devices to have their software embedded as firmware inside the microprocessor or whatever.

The other way seems to be the manufacturers. This will start to emerge from… by… you talked about the camera manufacturers. Maybe the camera guys do it first, or people who sell a lot of devices. Maybe the smartphone guys will do it first.

What’s your view of where this innovation is most likely to come from? Would it be from device-centric people, or software people trying to create a standard?

David Linthicum: Not software, because they’re not going to have… this is not going to have a good outcome for them, because in essence, they’re going to be architected out of the equation, which I think is perfectly fine.

I think the device manufacturers, people who are focused on how to build the best DSLR, or the best mobile computing system, or the best automated driving system, and the ability to kind of drive a standard that comes out of there.

And I don’t think it’s going to be one of these things where we’re getting all these vendors together and trying to agree upon a standard in terms of a small language model they’re going to build. It’s going to be argument after argument. Everybody’s going to have their own agenda, and if they’re not able to sell something, they’re not going to participate.

Nick Earle: The market will determine who’s successful, yeah.

David Linthicum: Right. I think it’s a de facto standard that comes from a device manufacturer, and a big one. They’re going to come up with something that’s going to be reliable, and they’re going to open source it, and people are going to figure out that it’s the best mousetrap, and they’ll start moving in that direction, and the market is going to drive them in that direction.

If you think about the adoption of most of the stuff that has occurred over the last 20 years has kind of gone that way, whether it’s Hadoop, or operating systems, or the Linux system, things like that. The people who build the best innovative solution, and are able to give it away for free, and really kind of promote the standard, and are able to drive a community, are the ones that are going to win the game.

So, you’re going to see a bunch of small language models are able to run on devices. They exist today. You can use a version of Claude and a version of DeepSeek that’ll run on a Raspberry Pi, things like that, and does not take a lot of space, and only supports a few parameters. But there’s nothing standard. It’s hobbyists like myself who build these things in a workshop, and I’m sure there’s some experimentation going on with many of these device manufacturers that are trying to get away from the LLM bills.

But that’s the way this stuff works, and normally it comes from something you don’t see coming. In other words, there’s suddenly a small language model out there that’s built by Sony, or built by… where they’re able to put a couple of million dollars in the innovation of this system, put it out in the open source, everybody adopts it, and they end up… are the ones who created some sort of innovation thing that takes it to the next level.

And that’s where it’s going to come from, and all technologies have basically come out of that kind of financial…

Nick Earle: Absolutely. And then they become a standard retrospectively.

All right, we’ve got a few minutes left, but I wanted to pivot completely and ask you a question to do with AI and jobs, if that’s okay to finish. We always ask guests, I always ask the guests this question, and I’d like to ask you the same one. So, I think you know the question, but I’ll lay it out.

So, I watch a ton of podcasts, do a lot of research, and of course everyone’s talking about the cognitive ability of AI, the agents, how the amount of work that AI in general can do is getting bigger and bigger as a percentage of all the tasks.

And I saw some recent data from Peter Diamandis, who does a Moonshot podcast, I’m sure you’re familiar with it. And he showed the charts earlier this week or last week, as the slowdown in U.S. hiring… now, there could be a ton of reasons why U.S. hiring is slowing down, but the net job ads, really slowing down. And the issues of graduate unemployment.

So, there’s a ton of commentary about this. What I’ve never really seen is anybody saying, “And this is what the answer is.” I’m going to put you on the spot a little bit.

Every wave of technology has always created a bunch of jobs. But then the other side of that coin is that there’s a time gap between when it’s introduced and when the jobs are created, and this introduction and this adoption’s really fast right now. Graduates — job seekers in general — are suffering tremendously right now, so there is this valley of death, so to speak.

And then secondly, no one’s really saying… other than we’re going to collaborate with the agents, be they physical or software — robots or agentic — no one’s really saying, “This is how I think it’s going to play out.” No one seems to have an answer, is what I’m trying to say. A good answer.

I’d like to ask you that question. How do you think it’s going to play out? Are we going to have mass unemployment and all the consequences that go with that?

David Linthicum: Yeah, the short answer is no, and here’s why. And what you’re seeing right now, in terms of the reduction in number of job postings out there, is a bit of a misunderstanding, and the narrative out there that people are accepting as reality, when reality is very different than how people perceive it.

And so, there is going to be a change in who we hire and how we hire, because you’ve got to remember, AI’s very good at automating tasks. What it’s not very good at is replacing humans.

We’re just finding out now the whole coder thing. Altman stood up and said, “In 2 years, we’re not going to need coders anymore… for 100 bucks, you can basically simulate 100 engineers,” and that kind of stuff. That’s not happening.

A lot of companies that went off and did that with the cloud-based AI coding found out they built a tremendous amount of technical debt. In other words, it’s code they can’t maintain, they’re going to have to fix it down the line, burns too many resources, things like that. And it’s lack of architecture, lack of understanding, vibe coding, all that kind of stuff. So, we’re hitting the reset button on that.

The engineering jobs are suddenly spiking up, and I know that because I teach AI architecture, and my students are in there, and they’re seeing more opportunities out there. That’s fairly recent.

Nick Earle: Good.

David Linthicum: As far as the long-term stuff, we’re going to see a transition in the way in which we work, very much like the Industrial Revolution, and the introduction of the car, all these things that basically change the way that we move and drive.

Now, people are going to shift their roles. For example, if you’re a truck driver, that may be completely automated in 5 years. And you’re going to basically have to move over into something else, but I think the net benefit of that is going to be additional jobs, because human beings are great at doing what they do, they’re innovative, they’re creative.

They have to drive this process. There has to be a human in the loop in each and every AI system that I built, and I built a bunch of them. And I’m not worried about them reducing hiring.

There is going to be some disruption, and people are going to have to figure out how to pivot and get into additional career, just like if you were a buggy mechanic in the late 1800s, you have to shift over…

Nick Earle: A buggy whip manufacturer.

David Linthicum: Yeah, exactly. But ultimately, there’s going to be more jobs around, and I think we have a declining birth rate in the United States, and I think that’s going to be fewer people that are in the market.

I do see people getting hurt now are the college kids just getting out of school, because the companies are panicking, because they don’t know whether or not they should hire them or not. Because once you hire somebody, it takes typically 2 years to get rid of them. And so, if they’re not needed, they’re going to be a draw in the business.

And so, that’s why they’re just pushing everything off until the last minute, because they have this narrative that AI is going to take over and take over many of the jobs, but many of the companies are finding that’s not the case.

Much of the cheerleading that was done two years ago by NVIDIA and OpenAI and Microsoft and Google and AWS is not coming true. Because they are obviously promoting and pushing the industry and talking about these huge transformations.

If you do see layoffs that are occurring where they’re saying AI is the reason they’re doing it, normally that’s not the case. We just saw the block layoffs that released 4,000 people. I looked into it. AI had nothing to do with that. AWS’s layoffs, 30,000 in the last year, nothing to do with it. Microsoft’s layoffs, Salesforce’s layoffs, they just over-hired during the pandemic, and they’re making shifts in how and when.

Nick Earle: It’s a convenient way of rationalising it.

David Linthicum: Right, and in doing that, not only are they lying, but they’re scaring the hell out of everybody, because people who are out there in the industry trying to figure out how to get a job are worried about these jobs, in essence, drying up and going away.

And the reality is you’ve got a couple of companies that are normalising, and that’s going to be the current pace. We’re going to see a net new rise of jobs.

I’m training generative AI architects now. We didn’t need those 5 years ago, now we do. AI, IoT, that’s going to be a huge growth area, as you guys see, you’re in the middle of the market.

And the ability to build these micro… these micro models on these particular devices, we need engineers to do that. AI’s not going to be able to do that.

And so, it’s a shift, it’s an adjustment, and it’s going to occur over a longer period of time than people are thinking. When you say 2 years, typically that means 10. Everybody has a way to adjust, and I think everybody’s going to be fine for the most part. Some people are going to get hurt in the short term, but they’re going to figure it out.

At the end of the day, you’re going to have everybody making more money, doing more interesting things, because much of the stuff that was drudgery work can be automated by AI, and I’m…

Nick Earle: As it always was.

And that’s a great point, an upbeat point on which to end. And I know we’ve kind of got to the end of… I felt we could have gone on for a lot longer, but I want to respect your time.

David, I really enjoyed the conversation. That’s one of the deepest that we’ve gone, and most thoughtful, conversations that we’ve had on the pod in terms of these issues. And I really like the… I liked all of it, but the inevitability of a client-server type architecture to move AI to the edge.

In relation to AI in general, but also IoT in particular, because of the most devices are still not connected and feeding their data into AI, and this is one of the big things that has to happen, I think is bang on the money.

And it’s not there yet, but I do agree with you, and many other people do as well, that the device manufacturers are the guys who are going to do this, not the mobile network operators, not the telecom companies, not even the IoT companies, because we’re a global IoT company, but we can’t do it as a software company and define that open standard, as you said.

So, I do believe that’s going to happen, and I think it was a really good, thought-provoking discussion, and I’m sure our listeners really enjoy listening to it. So, I just want to wrap up by saying thank you. Thank you for being a guest on IoT and AI Leaders. Very, very thoughtful.

I am still thinking about something you said right at the very beginning. Did you say you’d done 28 books? What was it?

David Linthicum: No, 17.

Nick Earle: Oh, excuse me. Excuse me. Well, it’s still impressive.

David Linthicum: Thank you.

Nick Earle: And there’s two behind, two behind your head, and so I would encourage people to look into your LinkedIn profile and look at the papers and speeches and books, because I think you have a lot to add to this debate.

So, thank you for being a guest on IoT and AI Leaders.

David Linthicum: Thank you.

Outro: You’ve been listening to IoT and AI Leaders. We hope today’s insights help you drive smarter, faster business innovation with IoT and AI at the centre. Thanks for listening. Until next time.