Separator

When AI Agents Start Making Decisions

Separator

A construction company is planning a complex of 100 buildings. As a trial, they get an AI agent to produce the first design, and the engineers put it through a series of tests. They take a good look at the structure, do simulations, double-check the math, and then approve it in the end.

The design does the job. The agent has performed remarkably well.

Encouraged by the result, the company asks the same agent to design the remaining 99 buildings. The designs arrive quickly, following the same principles as the first. There is little reason to expect anything to go wrong, and reviewing every structure with the same intensity would take considerable time.

So, the process becomes more relaxed. The engineers begin trusting the system that produced the designs, rather than independently verifying each one.

Somewhere down the line, a productivity tool ended up in charge of deciding structural safety. The change may have seemed gradual, but its implications are anything but small.

This is the bitter question surrounding the rise of agentic AI. We are discovering how much work these systems can perform independently. Are we actually making sure they can take on that much responsibility?

The (over) confidence problem

An AI agent does way more than just answer a single question. It can break down a goal into smaller tasks, use external tools, retrieve information, and act on the results of its previous steps. In many cases, the entire process proceeds with little human intervention.

That makes it an attractive proposition for organizations looking to automate complex workflows.

Picture an agent going over a job application. It goes through the candidate's experience, lines it up against the job requirements, points out gaps, and gives a suggestion. Another agent might review that recommendation, while a third prepares a report explaining why the candidate should progress.

Looking at it from the outside, the process seems accurate. Multiple stages have been completed, checks appear to have taken place, and the final recommendation is supported by a logical explanation.

But what happens if the first agent misunderstands the candidate's experience?

The second agent may evaluate the incorrect interpretation. The third may turn that evaluation into a convincing report. By the time the result reaches the hiring manager, the original mistake has passed through several stages and acquired the appearance of a carefully considered decision.

An agent can explain how it reached an outcome without that explanation establishing whether the outcome is correct. And when several agents are working together, the number of steps involved can make a flawed assessment appear even more credible.

This creates a particular problem for critical decisions. An organization may know what its AI system did without having sufficient evidence that what it did was appropriate.

When convincing becomes dangerous

Healthcare offers a useful illustration of why this distinction matters.

In 2018, an investigation by STAT revealed that IBM's Watson for Oncology had generated cancer treatment recommendations that medical specialists and customers considered unsafe or incorrect. Internal documents indicated that the system had been trained substantially on hypothetical patient cases rather than the breadth of real patient outcomes its public positioning had suggested.

The system could produce treatment recommendations. The problem was whether those recommendations had been established as sufficiently reliable for the clinical situations in which they might be used.

Watson for Oncology was not an autonomous agent in the contemporary sense. However, its experience demonstrates a failure that becomes particularly concerning when AI systems gain greater independence: the ability to produce sophisticated recommendations can develop much faster than the evidence needed to justify trusting them.

A similar distinction exists in finance. An AI agent can analyze the stock market, assemble evidence supporting an investment and present a persuasive argument for why it should succeed. Given the same information, it may also be able to construct a persuasive argument against that investment.

Both explanations can sound reasonable. Neither establishes that the underlying prediction will be correct.

For a person using AI to explore possibilities, this may be manageable. For a system permitted to act on those possibilities, the consequences are different.

The question of responsible AI therefore begins well before a model produces its ultimate answer. It concerns the conditions under which that answer is allowed to influence a decision.

Where the safeguards disappear

Responsible AI is not a new idea. Frameworks such as the OECD AI Principles and the US National Institute of Standards and Technology's AI Risk Management Framework emphasize reliability, transparency, accountability and the management of risks throughout a system's lifecycle.

They also recognize that human oversight and the ability to examine or challenge an AI-generated outcome are important safeguards.

Agentic systems make these requirements harder to satisfy because their actions can vary according to the information they encounter.

A conventional assessment may follow a defined sequence of questions and scoring criteria. An agentic system could change its questions, consult different sources or pursue an unexpected line of reasoning based on an earlier response.

Testing a few successful examples cannot establish how reliably it will behave across all the situations it may encounter.

The difficulty becomes greater when multiple language-model calls depend on one another. An error at one stage can influence the information used at the next, even when every individual output appears reasonable.

Developers can establish limits, introduce validation checks and restrict the actions an agent is permitted to take. But doing so requires considerably more than adding a final human approval step.

Consider the idea of evaluating hundreds of AI-generated recommendations each day. If each recommendation arrives with a polished explanation, the process of independently investigating the underlying evidence can become more demanding than the original task. The reviewer may eventually begin approving conclusions on the strength of how confidently they are presented.

The human remains involved in the process, but the system has effectively become the decision-maker.

The interview that judges its candidate

This problem becomes particularly difficult in recruitment, where an AI interviewer may be responsible for conducting a conversation and evaluating the person on the other side of it.

Consider a software engineer responding to a technical inquiry. The candidate describes an unconventional but valid approach to solving the problem. The AI interviewer, expecting a different line of reasoning, interprets the response as incomplete.

It asks a follow-up question based on that interpretation. The candidate attempts to clarify, but the conversation has already moved in a direction determined by the agent's initial misunderstanding.

Ultimately, the system generates a comprehensive report. It highlights areas of technical misunderstanding, labels the candidate's reasoning as inconsistent, and suggests additional assessment.

Everything about the report suggests that a proper assessment has taken place. What the hiring manager cannot immediately see is how much of the result depended on a misunderstanding early in the conversation.

Before today's agentic interviewers, automated hiring systems revealed how flawed evaluation criteria could infiltrate recruitment.

In 2018, Reuters revealed Amazon dropped an AI recruitment tool for bias against women. The system, trained on past technical job applications, learned patterns showing a male-dominated applicant pool. Among other flaws, it penalized résumés mentioning "women's."

The system had learned to reproduce patterns in past hiring data without establishing whether those patterns represented competence.

Another example emerged in 2021, when HireVue announced that it had removed facial analysis from its video-interview assessments. The company said its research showed that visual analysis no longer added significant value compared with the information contained in candidates' spoken responses.

Neither case featured the autonomous interviewing now seen with agentic AI. Both, however, expose a key flaw that more conversational skill can't fix.

An AI interviewer might conduct a remarkably natural conversation. Whether it measures the competencies required for the role is an entirely separate question.

If the system chooses questions, defines suitable answers, and scores candidates, the organization must validate those choices. Otherwise, it risks automating the appearance of an interview without adequately validating the assessment taking place within it.

Is bigger AI really the future?

The enormous interest in AI agents is partly driven by the expectation that increasingly powerful frontier models will eventually handle almost every kind of task.

Better models can certainly improve what AI systems can do. They can figure out complex directions, deal with new situations, and work in lots of different programs.

But the assumption that better language models will eventually solve every important decision-making problem overlooks how much of that work depends on knowledge outside the model itself.

Consider the task of evaluating a software engineer.

Before setting up an automated test, someone needs to figure out what skills the job requires. They need to figure out what counts as enough proof of those skills, how to balance different abilities, and what kind of assessment mistakes are okay.

These questions cover software engineering, hiring science, and measurement. Assessing competence demands specialized knowledge and research.

A language model cannot establish the validity of those decisions simply by becoming better at conducting conversations.

A reliable method starts with grasping the domain and tailoring the assessment accordingly. AI can be tasked with roles suited to its strengths, like interpreting natural-language replies or spotting skill-related evidence.

Those components can be evaluated against reliable datasets, while the wider system applies established criteria and includes appropriate safeguards.

In certain cases, a smaller language model focused on a specific task may be better than a cutting-edge model with broad autonomy. The key point is whether the whole system is designed and tested for its intended responsibility.

This is where the promise of AI may eventually look very different from the picture being presented today. Rather than endlessly expanding the independence of individual agents, progress in high-stakes applications may come from stronger domain models, better evaluation methods and AI components that operate within carefully established boundaries.

For the Institute of Hiring Sciences, these distinctions are central to the development of reliable AI-driven hiring assessments. This calls for a structured framework that helps validate how AI systems measure competence and whether their conclusions are supported by reliable evidence, rather than judging them by the sophistication of the conversations they conduct.

So, for the construction company we talked about, the engineer who approved the first building still has 99 more structures to examine. The success of one design cannot establish the safety of all the others.

The real progress will come when the systems we build can demonstrate that they deserve the authority we are beginning to give them.

In Print

Most Viewed



🍪 Do you like Cookies?

We use cookies to ensure you get the best experience. Read more…