As AI agents gain greater autonomy to make decisions and carry out tasks, a UNSW legal expert examines whether they should have legal personality, and why accountability matters more than human resemblance.
Science-fiction films such as Blade Runner and I, Robot imagined intelligent machines living alongside people and raised questions about whether they might deserve the same rights.
Today, AI agents can use digital tools, initiate transactions and carry out tasks with limited human intervention.
As these systems become more autonomous, should any of them be recognised as a legal person? And if an AI agent causes harm, who should be legally responsible?
For UNSW Law & Justice Professor Mimi Zou, a commercial lawyer and an AI governance expert, the starting point is to separate several concepts that are too often bundled together.
“Autonomy, intelligence, legal personality and moral status are distinct concepts,” she says. “None necessarily follows from the others.”
Legal personhood is not a test of human likeness
Legal personality is when the law recognises an entity capable of holding specified rights, owing duties and exercising legal powers, Prof. Zou explains. It does not necessarily imply the entity is conscious or has human characteristics.
Corporations are the most familiar example. A company is a legal person without being a human .
“A company can own property, enter contracts, sue and be sued, and incur liabilities in its own name,” Prof. Zou says. “Its legal personality tells us what it can do under the law, not what it is capable of experiencing.”
In 2017, New Zealand’s Te Awa Tupua legislation recognised the Whanganui River as a legal person. Human representatives can exercise its legal rights, powers, duties and liabilities on its behalf.
“The legislation gives legal expression to the relationship of Whanganui iwi and hapū with the river and establishes a framework for its governance and protection,” Prof. Zou says.
The example shows that granting legal personality is a legal choice for a specific reason. For Prof. Zou, “the real question is what legal problem it would solve.”
“Legal personality is not all or nothing. Children have rights even though their capacity to exercise them independently is limited,” she says.
“Companies have extensive property and contractual rights, but not the full range of rights held by human beings.”
The accountability risks
Recognising an AI agent as a separate legal person could create new ways to assign responsibility. But it could also make it harder for people harmed by it to seek compensation from those who develop or deploy a system.
“Separate personality would not automatically absolve developers or operators of liability. That depends on the rules we enact,” Prof. Zou says.
“But a poorly designed regime could encourage risk to be channelled into thinly capitalised AI entities, leaving victims without effective redress.
“Imagine an autonomous AI trading agent makes transactions outside its instructions, causing $10 million in losses. The company deploying it argues that the agent is a separate legal person and should answer for its own decisions.
“But suppose the agent holds only $10,000 in assets. Victims might win a judgement and still recover almost nothing. There is a profound difference between identifying a defendant and securing an effective remedy.”
Corporate law has spent generations grappling with limited liability, directors’ duties and undercapitalised companies. “Do we really want to reproduce those problems across millions of autonomous agents?” Prof. Zou says.
How penalising AI systems for any wrongdoing would look like also raises difficult questions.
“Fines imposed on companies can affect shareholders, managers and organisational incentives. Imprisonment deprives a human offender of liberty,” Prof. Zou says.
“But what would punishment mean for software? Would shutting down an AI agent be punishment or retraining it rehabilitation?”
“The identity of an AI legal person may be difficult to define. If an agent is updated, its memory migrated or its code forked into two copies, which version holds its assets, contracts and debts?”
Different legal approaches to AI personhood could also complicate questions of jurisdiction and applicable law, Prof. Zou adds.
“An AI agent might be developed in the US, operated by an Australian company, hosted in Singapore and authorised to transact with European customers,” she says.
“If one country recognises the agent as a legal person and another does not, how should courts determine its contractual capacity, liabilities and legal status? Existing private international law rules would still matter, but their application could become more complex.”
Digital identity is not legal personhood
For Prof. Zou, the immediate priority is to make AI activity traceable to identifiable people and organisations, with clear authorisation and enforceable duties.
Some governments are already experimenting with aspects of that approach. In January 2026, Beijing E-Town issued a virtual idol identity certification to Yuri, an AI-generated singer. In June, it issued her a Digital Artist Card under a regulatory sandbox for commercial performances by digital humans. The card is a compliance credential for the virtual performer’s commercial activity, not recognition of Yuri as a legal person. A digital performer is also not necessarily the same as an autonomous AI agent.
In a separate development in September, the companies operating Yuri and another digital human opened corporate digital-yuan wallets for commercial transactions. The accounts belong to the operating companies, not to the virtual characters as independent legal persons.
Prof. Zou says the Chinese example highlights the need to distinguish a regulated digital identity from legally independent status.
“China is formalising how digital performers operate in regulated markets. The entity conducting the business remains the company behind the character. Issuing an AI performer an identity credential does not, by itself, create a new legal person,” she says.
Estonia announced plans in June to develop digital identification and authorisation for AI agents acting on behalf of people and organisations. Officials later clarified that agents would not receive personal identity codes. The emerging approach instead links an agent’s actions to the person who authorised it, within specified limits.
“The aim is to establish who authorised an agent, what it may do and who ultimately answers for its actions. For example, authorisation might permit an agent to read data or prepare a payment, but only within a defined financial limit,” Prof. Zou says.
The proposal in Estonia does not make AI agents citizens, companies or legal persons. Its stated design preserves human responsibility, although the legal and technical details are still being developed.
“Estonia illustrates the difference between identifying an agent and giving it independent legal personality. Imagine an agent permitted to pay company invoices, but only up to $5,000,” Prof. Zou says.
“A verifiable record could show who authorised it, what it was allowed to do, when it acted and whether that permission was revoked. Much of the practical benefit comes from identity and delegation, not personhood.”
Argentina is considering another approach. It’s revising companies’ law that would recognise companies to carry out operations through autonomous algorithms or AI agents. The measure remains a contested proposal. Importantly, Prof. Zou points out the company, not the software, would be the legal person, and legislators have been debating requirements for identified human oversight.
“An AI-run company is not the same as AI becoming a legal person,” Prof. Zou says. “The real issues concern human duties and effective remedies when operational decisions are increasingly automated.”
Could AI systems eventually have legal rights?
Prof. Zou doesn’t rule out lawmakers giving AI systems limited legal rights in the future, where there is a clear practical need. But she says this is different from recognising AI systems having moral interests of their own.
Before creating new AI legal persons, she argues, lawmakers should prioritise reliable agent identification and traceability, clearly defined authority and responsibility and effective remedies for people harmed by their operation.
“None of these requires us to settle whether machines are conscious,” Prof Zou says. “It requires enforceable rules and systems that make responsibility legible.”
Prof. Zou says lawmakers should also ask four questions before considering AI personhood:
- What legal problem would recognising AI as a person solve?
- Could the system meaningfully hold rights, perform obligations and exercise legally recognised powers?
- Could courts enforce obligations against it, and could other parties obtain an effective remedy?
- Would we be recognising the system’s interests for their own sake, or creating a legal vehicle for human activity?
That final question brings the debate back to its underlying purpose: commercial convenience, accountability and moral status are different reasons for granting legal protections.
“If we confer legal rights on AI agents, whose interests are we advancing? Are we protecting an entity with interests of its own, facilitating transactions, or providing another layer through which corporations can reduce their exposure to liability?” Prof. Zou asks.
“If future AI systems were shown to have morally significant experiences or interests, their welfare would deserve serious consideration. But intelligence or human-like appearance alone cannot establish that. As machines become more autonomous, human responsibility must not become more elusive.”