Why AI Agents Keep Failing

Transcript

(0:02) I'm running a hundred agents. Okay, but but how could you possibly be running a hundred agents that are that does anything productive when really you just banked without any context? That doesn't mean anything. It doesn't make any sense, right? What about I went to the river bank? That implies that I'm at the river, right? On the bank. But then what about I'm going to the bank to deposit money. What I'm trying to get at here is the way that you know what I'm talking

(0:37) about is dependent on the context. Ultimately the context matters here in understanding the meaning of what is happening. This is not only prevalent in language but also for agents to understand operations. Agents need context and it's not like you can just open an open AI or a clawed product and then it starts doing all the little things. Understanding context is a really complicated problem and it's not enough to just show this file here and show this file there. The context needs

(1:13) to be worked into the actual workflows themselves. And this is made possible through stuff like MCPs or model context protocols. A model context protocol is a way that you can connect external tools or external data to your agent. An agent being like a cloud code or codeex, any of these tools that have a model on top of what we call a harness. A harness is basically a rule set of instructions that a model can use. We'll get into that stuff later in a different video.

(1:48) But the whole problem here, it's not how can we get agents deployed. It's not how can we get the agents to do these workflows and do all this crazy stuff. The first problem that I'm seeing that is not being addressed is how do we get the agents to understand context? And I'll give you an example. If you're opening an email, to understand that a email is spam or if it needs to get pushed into a certain folder actually requires a lot of thinking, right? That

(2:18) there's a cognitive process that occurs when you're doing this stuff. And you really just don't you don't internalize that you're doing all these computations. It's just second nature. You know what a spam message looks like. It's pattern recognition. You know what spam looks like. You know what an important message looks like, right? But how does the agent know that the person that you met in person on Tuesday said that they'll send you an email? And how

(2:44) does the agent know that you met that person? That's context. So, there's tons of workflows that structurally work well, but break down because the agent doesn't have the information to execute the decisions. We're past the point where the agents need to kind of figure out, okay, is this possible to even do? Can the agent even do these types of things? And I use the a the word agent very loosely here, but we can define agents as AI models with instructions.

(3:16) Okay, a model with instructions that can do stuff, not just chat in a chat box, but can actually do something. Okay, so this context issue can be heavily solved by connecting external data. And usually this can be done through like an MCP which I explained before or other types of APIs or connectors. The point here is the context problem once solved unlocks all the crazy automation stuff that an enterprise or small business or an individual can do. So another example of

(3:50) this is if you're going through your email and you're like, "Okay, well I want some agent that can look through and sort my emails." Right? The email sorting process actually is heavily reliant on context that we internalize. A way that you can think about this is if you offload this task to a human, how many times would they come back with questions, right? And how do you set up a system? The the goal here is how do you set up a system where every time a

(4:18) real human would come back with a question, oh, have you talked to this person? Do you know this person? Do you think this is spam? Those can be solved by skills and connections to context or data sources that have all this information about your company, business, etc. So, a lot of the dayto-day stuff that we see online is is about agents and I'm running a 100 agents. Okay, but but how could you possibly be running a hundred agents that are that does anything productive

(4:48) when really you just need one? You could do a lot of damage with one agent that understands just who is in your business pipeline, how many people are in your business pipeline, and you set up a skill that can look at the business pipeline, check the emails, and then give recommendation for stage updates, right? So, if you have someone that is in a prospecting, right? You just talked to them and you're prospecting and then you check the email and they replied

(5:12) back. They said, "Hey, yeah, let's set up a call." Bam. the agent can automatically infer the state change or the the the position of the deal, the state of the deal by looking at the context in the CR. So really there's tons of opportunity for context management systems. The the state what what is the state and how do you the the skill and the goal becomes how do you externalize the state so that the agents can look at the context run an automation and then eventually

(5:49) perform the action all on its own. And this is why I'm such a big proponent of MCPs because you don't necessarily have to build your own CRM. The savvy SAS companies that have spent tons of time doing design decisions like linear or or a CRM. A linear is a product management application. They put all this design philosophy and elbow grease into product management so that you don't have to. So in the future I see businesses are still using these SAS products. The SAS

(6:25) products become somewhat commodities because the user interface becomes less important. So you can kind of commoditize the services and and that is a problem that these service providers are going to have to play. The commoditization of services is is something and is a risk if you're running that that business line. But for the operator who's not distributing these types of businesses, a tool, then you really can just select, okay, I'm going to grab this tool. And you're not

(6:55) necessarily interfacing with the user interface to update the state. the agents or manual pushes from your your your coding agent can update the state of the CRM of the email or any type of database or a notion instead of you doing it. So this kind of opens up a new way of interacting with SAS products that I kind of see in the future, which is you have this orchestration command center that basically can touch all the different SAS products that you have

(7:29) subscriptions to and they spend the design decisions on, you know, what is the schema of a CRM look like? And the the structure the structure is what you're paying for. you're paying for the design structure and the rigorous testing that they put into like what the best product management tool is. Let's say like a linear and then what you're doing is orchestrating between all those different tools as context for agents to perform tasks. And if you think of it

(8:00) through this lens, then you can start seeing that you kind of don't need a 100 agents. the the benchmark doesn't become I need a 100 agents running around. The benchmark really becomes how much context have I pulled in and how much externalized information do we not have supported, right? Maybe and and then you set up workflows for these things like inerson calls. Maybe maybe you had an inerson call and you have some type of recording device so that you can

(8:31) immediately move that into your context system because the context will allow the agents to perform better because it understands the disposition or the positioning or the state of a business, an individual or an enterprise.