Bloomberg reporter Katrina Manson’s Project Maven tells the story of how, over the past decade, the US military has developed AI-targeting tools. Manson, an experienced technology and national security journalist, focuses primarily on the period 2016–21 and the titular project, which was led by US Marine Colonel Drew Cukor.

From the start, Cukor believed AI should be used in warfare only if two conditions were met: humans would still make the final decisions about targeting, and the laws of war would regulate AI in the same way as other methods of waging war. By the end of the book, when Manson brings the reader right up to 2026, that vision seems almost quaint.

Cukor is no longer in the US military, and nor is he associated with military research on AI. Military use of AI now includes autonomous weapons, the so-called “killer robots” deplored by human rights campaigners and condemned by UN secretary-general António Guterres. Governments seem uninterested in regulating or banning this new technology, with American defense secretary Pete Hegseth going as far as rejecting all international law–based rules of engagement in favour of a “warrior” culture.

At the heart of the debate is targeting, a key issue in international humanitarian law that regulates armed conflict. (Although many of the rules of war are set out in the Geneva Conventions and its protocols, the law is grounded in customary rules, some of which are based on centuries-long practice.) One of the first principles of contemporary customary international humanitarian law is the principle of distinction, which states that military attacks must only be directed at military targets and not at civilians. The principle may be widely accepted, but its application is often contested. Who is a civilian? How many civilian deaths (often referred to as “collateral damage”) are acceptable when armies attack a military target?

Careless or malign targeting isn’t new, of course. Manson notes American targeting failures in the pre-AI 1990s and 2000s. Proponents of AI, particularly the Mavenites, have argued that AI will reduce errors in targeting. But not all civilian casualties result from errors: even in the AI age, civilian targets are attacked, sometimes deliberately, and some countries locate civilian infrastructure near military installations in an attempt to immunise them from attack.

In the Middle East, for example, Israel has argued that Hamas uses hospitals — which international law says cannot be targeted — to conceal its operations. The Israeli military has thus targeted hospitals in Gaza, resulting in what Médecins Sans Frontières has described as a dismantling of the health system, leaving many without access to healthcare. In February 2026, the United States bombed a school in Iran, killing nearly 200 adults and children. In that case AI targeting may well have worked as intended, with the United States willing to accept a high risk of civilian casualties to hit an important military target, although recent reports suggest the databases of intelligence on Iran may have been out of date.

Manson argues that pointing to policy choices about acceptable targets and civilian casualties shifts responsibility away from the designers and users of AI. I disagree with her characterisation of the problem, largely because I don’t think these are separate exercises. Settings on AI-supported weapons and autonomous weapons result from the choices of human designers and human users.

But the story Manson tells in Project Maven begins at a different technological moment. Cukor, stationed in Afghanistan in the 2000s, despaired of the inadequacy of the technology available to him. Poor intelligence was leading to flawed identification of targets. American service personnel and civilians alike were unnecessarily killed. Cukor wanted better intelligence. Later, as he sought support to research AI use in the military, he focused on its use for intelligence rather than targeting. Nonetheless, the logical endpoint of improved intelligence was improved targeting. The Mavenites’ main goal in improving targeting was preventing loss of life among soldiers; they don’t seem to have devoted much attention to reducing civilian casualties.

Cukor and his team are the protagonists of Project Maven. But one important strand of the book is the role of the Big Tech companies. As Manson shows, Google (and particularly its employees) was initially reluctant to participate in military research. Over time, that reluctance disappeared internally rather than being overcome by Project Maven. Google and other large technology companies became part of Maven’s network of operations as they developed and tested AI tools.

More significant than this vanishing opposition is the consistent enthusiastic participation in developing AI for military use by the increasingly powerful data company Palantir. Manson’s final mention of Palantir, in the last pages of the final chapter, highlights the company’s involvement with the Israel Defence Force, or IDF, as well as the US military. While Microsoft disabled some cloud services to the IDF late last year, Manson notes, Palantir’s Alex Karp seems much more comfortable with the level of civilian deaths in IDF operations, despite Cukor’s evident discomfort with Karp’s statements.

Project Maven is structured around the three-part targeting process — “find; fix; finish” — plus a fourth section called “feedback.” There, Manson discusses ethics and accountability, including regulation of AI in weapons in international law. She occasionally refers to the Geneva Conventions or the law of armed conflict in the first three sections of the book, and some of her interviewees note that they could break military law and possibly wind up in jail if they target civilians. In the fourth section, she dives deeper into the question of regulation.

AI creates problems of accountability. If an algorithm determines what should be targeted, who is responsible? At the level of state responsibility, there is probably no difference between attributing the conduct of a soldier to the state that issues the orders and attributing the output of technology to the state that used it. When we move to the level of individual accountability, the problem is more acute.

Australia is currently discovering how difficult it is to ensure responsibility for alleged war crimes where direct person-to-person conduct is involved. But determining responsibility for targeting decisions would be even more complex. At least since the end of the second world war, “just following orders” is not a defence, so in principle doing what the AI instructed would not deflect responsibility. But it might be difficult to locate the specific human or humans ultimately responsible. If the soldier firing the weapon doesn’t have the information or the agency to challenge an AI-selected target, does responsibility shift upwards in the chain of command, and if so, where does it land?

Some of the people Manson quotes, as well as many governments, are confident that the existing law of armed conflict is adequate to regulate the use of AI for targeting. Indeed, some of them believe AI targeting already conforms with international law. That view is not universally accepted, however, even among governments, and some argue for new agreements to define acceptable use of AI. In the absence of comprehensive approach to regulation, two strands have emerged: a push to ban autonomous weapons and attempts to ensure that human soldiers retain control of AI-supported weapons. Neither seems likely to succeed in the near future.

Campaigners including Human Rights Watch’s Mary Wareham believe autonomous weapons should be banned by treaty. While Secretary-General Guterres supports an international ban on autonomous weapons, not enough governments want action. Wareham analogises efforts to ban autonomous weapons to the 1990s campaign for a treaty banning the use of landmines, which succeeded in 1997 and now numbers about three-quarters of countries as parties. But recent developments suggests that treaty is not a useful precedent for advocates of banning autonomous weapons. Ukraine and five other European states have withdrawn or declared an intention to withdraw, citing the fact that Russia is not a party and has used landmines in Ukraine.

The other strand for potential regulation of AI weaponry is a requirement for “meaningful human control.” The concept was initially proposed in 2016 by civil society group Article 36, named for Article 36 of Protocol I of the Geneva Conventions that requires governments to ensure that new weapons are consistent with international law. Article 36 presented meaningful human control as a framework requiring transparency, accurate information, opportunities for human intervention, and accountability.

Although Manson doesn’t discuss meaningful human control extensively in Project Maven, some experts, including the International Committee of the Red Cross, believe it would be a useful supplement to existing rules. The threshold requirement for the legal use of AI in targeting would be that the human soldier retains control over the decision to use the weapon. It is impossible to define meaningful human control exhaustively, but based on Article 36’s framework it means more than human decision-making as a backup for automated targeting.

Possibly the best we can expect is for governments to agree on some red lines for AI use overing cases where human decision-making is obviously absent, for example, or is too insignificant to the process. Nonetheless, like a ban on autonomous weapons, any agreement to subject military AI to meaningful human control seems unlikely at present. Unsurprisingly, in submissions to a UN report on military use of AI published in 2025, none of the three key members of the UN Security Council — the United States, China and Russia — advocated new regulation based on meaningful human control of AI use.

Without rules specifically banning autonomous weapons or regulating AI targeting, governments can only fall back on the existing law of armed conflict, including the principle of distinction, with all the uncertainty within those rules.

As the reader progresses through Project Maven, Manson’s concern about regulation and accountability becomes more evident. She seems to regard Cukor and at least most of the Mavenites as good-faith advocates of AI that supports rather than replaces human judgements about targeting. But developments after Cukor’s departure, particularly in the second Trump administration, raise doubts both for Manson and for Cukor himself. The question now is whether regulation under international law can catch up, even as some commentators declare the death of international law.

One final reflection: the back cover of Project Maven includes eight enthusiastic endorsements. Most reveal as much about the authors of the endorsers as they do about the book itself. Some liken the book to a gripping thriller; others see it as a warning about the dangers of AI. I would instead characterise it as a book of reportage, based on extensive research and interviewing, as well as years of experience reporting on the role of technology in security.

Unlike a thriller or a polemic, this book requires close attention from the reader but repays that attention with a detailed and insightful account of how we got here and where the technology might take us. Ironically, one of the by-products of the ubiquity of AI, the increased use of AI-generated news stories to replace human journalists, means that we may have less reporting of this quality in the future. Appreciate this book while you can. •

Project Maven: A Marine Colonel, His Team and the Dawn of AI Warfare
By Katrina Manson | W.W. Norton & Company | $52.95 | 406 pages

The post Rules of engagement appeared first on Inside Story.

O’Connor adopts a “show, don’t tell” approach to debunking some of our society’s most banal and unhelpful narratives about technology and work — among them the idea that AI will “free up” cognitive load and enable people to take on more creative and satisfying tasks and enjoy more leisure. Through nine lively chapters organised into sections on the “mind,” “body” and “soul,” she introduces us to many different kinds of people who work with, around and under highly automated and AI-infused systems, including coders, translators, drivers, warehouse workers, miners and screenwriters.

What emerges is not a ledger of winners and losers but a set of skilfully drawn portraits of power, self-awareness, hard choices and collective contestation. None of O’Connor’s informants is beguiled by grand narratives of human liberation, and none seems to accept at face value the idea that AI is “just a tool.” For O’Connor and many of her interviewees, the question of whether any particular job is robotised always co-exists with the question of whether and how human beings are being made more robot-like when they are forced to work in highly automated systems.

Where things stand within any particular occupation is, O’Connor observes, often detectable from the words people choose they use to talk about their jobs:

“We send him away, then we send him back,” the miner had said about the autonomous truck. Compare that to what the Amazon workers said: “The robot is moving and you have to move with the robot.” Or, indeed, compare it to what Rebecca, the translator, had said: “You still have to be quite fastidious in receiving all this machine translation output, and that’s exhausting, it’s stressful.” All of which is to say, sure, technology might be “just a tool,” but if you’re not the one with the tool in your hand, you’re not the master carpenter, you’re the screwdriver, or, worse, you’re what’s getting screwed.

One of the real strengths of We Are Not Machines is O’Connor’s deftness in showing how AI is, and isn’t, the latest act in a centuries-long drama of worker alienation in response to industrial capitalism. The varieties of anxiety, danger, resentment, acceptance, exit and resistance she chronicles are never detached from the history of capitalism and its critics. In describing the risks of machine users becoming machine-like, for instance, she quotes Ruskin: “It is not, truly speaking, the labour that is divided; but the men — divided into mere segments of men — broken into small fragments and crumbs of life.”

The hypermodern Amazon warehouses that bring the shelf to the worker (rather than the other way around) — and rely on a remote global workforce of “humans in the loop” to oversee mistakes — can be seen, she says, as a new iteration of Henry Ford’s moving assembly line. Appropriately enough, the American engineer F.W. Taylor plays a recurrent role as the architect of the most notorious system for using “time and task” techniques to deskill and speed up workers.

But O’Connor hasn’t written another weary “it’s capitalism, stupid” narrative. This is largely because she has a finely tuned ear for workers’ descriptions of the ways AI-infused systems shape their inner lives at work.

She tells us, for instance, about the silent mantra that Affan, a human “GO-AI Associate” for Amazon in Costa Rica, recites while watching videos of people putting items onto canvas shelves in nine-hour shifts. Faced with a 99.9 per cent accuracy rate requirement and the need to watch approximately 1200 ten-second videos a shift, Affan’s time before the screen is accompanied by the prayer, “Please God, save me from a defect, please God, save me from a defect.”

Poignantly evocative, too, are her presentations of translators’ lives and perceptions before, during and after the imposition of AI-based software. The creativity and sense of meaningfulness of translating from scratch shines through one translator’s explanation of the process:

[Y]ou take a message, a part of the message… then you render it in the other language, mostly through the equivalence of expression, the equivalence of feeling. “If I was thinking this, how would I say it?” That becomes automatic — it feels almost automatic. Sometimes you get stuck [but] if I am hitting the flow, then it feels fantastic.

Having to edit a machine translation, however, is “a bulkier process in your mind,” says Majkic, “it’s less rewarding, it takes more cognitive effort, not less.” In the words of another translator, Petr:

When someone presents you with a solution, and you’re looking at that, and you’re supposed to make it sound natural. It’s difficult to come up with your own solution… I compare this to counting from one to a hundred: one, two, three four — with someone whispering in your ear some random numbers. Imagine that. It’s so frustrating. That, to me, is what machine translation post-editing feels like.

O’Connor doesn’t present the degradation of translation work as something afflicting only the translators, either. End-users of the work — the readers and watchers of the films and TV series being translated — receive more literal translations that might be comprehensible but are also “flat and boring and uninteresting.” The loss here isn’t merely of “quality,” it is also of people’s capacity to recognise what “quality” is or be aware that any alternative might be possible in a market flooded with “good enough” products.

Noting that this, too, isn’t a new dynamic, O’Connor quotes George Orwell in The Road to Wigan Pier: “Mechanisation leads to the decay of taste, the decay of taste leads to the demand or machine-made articles and hence to more mechanisation, and so a vicious circle is established.”

O’Connor’s main example of a group of workers for whom the “AI is just a tool” maxim seems plausibly true is software engineers. Most of them view AI tools as a welcome addition to their workflow because they don’t see them as affecting the part of their job they really value, which is problem solving rather than simply writing code. Nor were AI tools forced on this group: because they wield relatively great labour market power, they retain control over whether and how to use them.

But they are very far from representative. For creatives, educators and carers, the application of productivity and efficiency values — even before they were expressed in AI form — seems like a category error in any attempt to grasp the purpose and value of their work. O’Connor depicts the sense of violation felt by artists, for instance, who see the idea of AI as simply a “tool” as profoundly misconceived. For them, it is “far more like an end-results machine than a new kind of paintbrush.” As the historian Lewis Mumford once explained, “the purpose of art has never been labour-saving but labour-loving, a deliberate elaboration of function, form, and symbolic ornament to enhance the interest of life itself.”

In care work, too, technology-reliant approaches provide new iterations of old ways of missing the point. Debates about AI-infused care robots are a distraction unless they recognise what is lost or degraded when we attempt to organise something inherently and profoundly relational (beautifully elaborated by sociologist Allison Pugh as “connective labour”) as if it were an industrial commodity to be produced via principles of scientific management.

The Buurtzorg nurses O’Connor interviewed (who have a high level of worker autonomy, are geographically embedded, and deliver client-centred, defragmented care) were aghast at the idea that performing “simple tasks” was a “waste” of their training. “You are not only there for that simple task, you are there to see the people, to see the life, to see what you are dealing with,” said one. “You can’t take care of someone who doesn’t trust you, because they won’t tell you the whole story, because sometimes they think it’s not important, but even the little signals can be something that’s really wrong.”

O’Connor doesn’t merely observe the multiple clashes between the AI-enabled Taylorism and the values that hold us together as human beings with bodies, minds and souls. She also delves into the structures of power that make contestation over those matters meaningful.

One of her key case studies describes underground miners in Sweden who don’t evince categorical hostility to digital technology and automation, and show openness, creativity and collaboration in how such systems might be integrated into their work. She describes how the agreement they reached about self-driving underground trucks and loaders reflected genuine give-and-take: how worker concerns that digital surveillance systems would drift from safety mechanisms into forms of discipline and speeding-up were worked through to the parties’ mutual satisfaction.

In that case, says O’Connor, the understanding and compromise on display rested on “a very human edifice: a system based on relationships of trust, underpinned by a careful balance of power. It struck me that it required a great deal of skill, effort and emotional intelligence to maintain it.” It was a system underpinned by collective bargaining not only at enterprise-level but also nationally, sectorally and locally. Employers and unions (the latter with 70 per cent membership density) were able to reach sector-specific agreements lasting one to three years within a system that, by its nature, is more responsive than one-size-fits-all laws imposed sporadically in the face of powerful political resistance.

Integral to the Swedish miners’ approach is a reliable and comprehensive welfare safety net that provides 80 per cent of pre-employment earnings to displaced workers. Through this lens, whether you see AI as a “tool” is not a matter of individual preference or psychological flexibility but intimately related to institutional arrangements that reflect collective workplace power.

The nature of industrial power is also critical in O’Connor’s account of the Writers Guild of America’s agreement with film and TV producers that set the conditions for writers to control when AI is used. The new contract didn’t ban the technology, but it did remove the economic incentive for companies to use it to displace writers by specifying that AI content could not be classified as “source material” merely to be “revised” for lower rates.

The agreement wasn’t merely the product of enlightened and flexible thinking on both sides. It emerged from an industrial relations framework that supported sector-based contracts and permitted industrial action, in this case a 148-day strike in which writers were supported by actors and support from Teamsters and United Auto Workers. As well as generating countervailing power, the process ensured the issue wasn’t metabolised by workers as a private source of shame, lack of training or bad choices. By making the issue public and visible, says O’Connor, workers felt “as if the wider public was also on their side.”

We Are Not Machines concludes with persuasive observations about the importance of attending carefully to how we talk, and therefore think, about work and technology. It is impossible to disagree with O’Connor’s observation that “wave,” “tsunami” and other metaphors “encourage us to see this process as analogous to a powerful force of nature — something that cannot be controlled, but can only be prepared for, and then mopped up after.” As she says, “new technology doesn’t just unfold. It never has. It is created by people and implemented by people.”

Nor, she argues, should we accept the thin, desiccated view of humanity that makes up Silicon Valley’s view of progress: the idea that we are merely slower, stupider, weaker and more biased versions of the machines. Anyone who doubts her point — and in fact everyone that doesn’t — should urgently read the Papal encyclical Magnifica Humanitas.

O’Connor writes brilliantly, too, about the importance of not feeling boxed in when we ask difficult and complicated questions about where AI belongs and how it should operate:

To say you don’t want AI to clone musicians does not also mean you don’t want protein-folding, antibiotics, low child mortality rates or indoor plumbing. That is a ludicrous mental box to put oneself into. And people don’t do it with any other kind of produce. If you think heroin is bad, you don’t worry that people will think you are also against paracetamol.

O’Connor guides us, instead, to some better starting questions. She cites Neil Postman who suggested in his 1992 book Technopoly that we ask: “What is the problem to which this technology is the solution? And whose problem is it?” To Postman’s questions, O’Connor adds another, “Are there any other ways to solve the problem?” To these, I would perhaps suggest one more that emerges from her account: what structures are we building to enable answers to these questions to be meaningfully debated rather than simply imposed?

O’Connor is to be applauded for insisting we centre human agency when we talk about how AI is designed and introduced into workplaces, that we always ask who decides. The next step in the analysis — and one beyond the scope of her book — is to ask why we so often accept the formations of industrial power and corporate power inherited from the last century as “givens,” as if they are features of the natural landscape (“Norway has mountains,” “Australia is flat”) rather than as structures we have built and can choose to maintain or change.

If we know Taylorism is deeply corrosive to the forms of trust and relationality that are critical to care, for example, why do we increasingly choose to provide care via private equity companies, the institutional vehicles perhaps least capable of reversing disastrous low-trust, high-fragmentation “time and task”-based approaches? Why do we persist with collective bargaining structures that (with narrow and highly conditional exceptions) confine collective worker voices to the enterprise level and narrow range of “permitted matters”? These institutional structures were built in the time before AI, and whether and how we continue to use them is up to us. •

The post What are we building? appeared first on Inside Story.

This new Pope Leo faces another such moment, this time fuelled by artificial intelligence, the subject of his first church encyclical. These documents are reserved for the most important facets of modern society, the very topics requiring the appearance of direct communion with the Almighty to resolve.

In the first generation of the digital age, the technology being created by the new geniuses was transformative. It could destroy the boundaries of time and place to allow people to find community alongside one another. A revolutionary act, it created a new form of political power.

For a long time, we understand that power to be held by citizens themselves, managed only in trust by the technology companies that facilitated (with search engines and social media platforms) but did not intervene. The internet was a sort of benevolent but absent God.

The AI age came with a renegotiation of that bargain. Power was no longer held in trust but actively managed by a small group of individuals and corporations who suddenly had the capability to decide what the rest of the world could understand to be true. At first, they were protected by a broadly shared acceptance of the value that they brought to the democratic world. Laws could be carefully interpreted to their benefit because it was in the public interest that a counterweight to political power be not just available but cultivated.

That logic has now flipped. The power these platforms have gained has surmounted the very institutions that they once held in check.

There have been small but meaningful reflections of this reality. Earlier this year two court cases in the United States reinterpreted the very laws that once protected Silicon Valley from liability for user-generated content. Now, the courts found, the companies have a responsibility to monitor content for its potential to cause harm.

But Pope Leo’s encyclical is the most important and substantial acknowledgement of the challenge. While it holds no legal weight — no tech company is headquartered in Vatican City — it reflects the opinion of one of the most powerful institutions on earth. These old regulatory institutions must come back to life, says the encyclical, to protect the human dignity being put at risk by a new technology that understands people as data points primed for optimisation.

Leo takes as his object both the technology and the people behind it. Of the technology, he is concerned by its ability to trick people, to remove the very idea of truth from human society. Of the people involved, he believes the concentration of a new form of power in the hands of a very few is as great a threat as any authoritarian regime. He nods to this parallel in his citation of Hannah Arendt’s On the Origins of Totalitarianism.

In both instances, citizens have been reduced to instruments of efficiency, revoking the agency that offers people their humanity. The Pope writes of a “crisis” in our relationship with life itself. That the limits of human life are now “seen primarily as a defect to be corrected, rather than as a reality through which our humanity matures and opens itself.” When a society pushes itself relentlessly toward optimisation, we lose the good and necessary possibility of humanity.

Institutions like the Church and many courts are beginning to acknowledge this new reality. The assumptions that underwrote the arrival of the AI revolution are changing, with moral authority and the law moving in the same direction.

And yet the Pope stands as the sole global figure to offer a meaningful perspective on the regulation of AI. In the United States, the Democratic Party has not bothered to put forward an agenda to counter the Republicans’ inclination to let these companies run amok, their profit motive being more important than any social or ecological disaster they may summon.

Europe has chosen the predictable path of bureaucratic nightmare, stifling the possibility of innovation in the name of endless red tape. The experience of European modernity is to permanently be stuck in the shallow end of the pool. Safe, but never able to grow. All the while, China’s model of AI acceleration operates in lockstep with its authoritarian regime, no daylight possible between its technological development and the anxiety of the state.

This has left countries like Australia waiting. It is clear that letting Silicon Valley run wild will only hasten economic and social crises, but there has yet to coalesce a vision for a future that sees a harnessed AI, one that can help power the future without destroying it entirely.

The Pope’s offering won’t directly cause any immediate changes. No technology company will fire its chief executive because of the commands of the Pontiff. And yet the very existence of the encyclical says a great deal about the moment we are in.

The governments that once cultivated the fruit of Silicon Valley as a democratic counterweight have found themselves impotent in the face of their own creation. The US administration has decided to leave the AI industry to regulate itself, and Europe is drowning itself in process. Everyone else is left watching, hoping to not be found wanting in the end.

Leo chose his name to invoke a moment when the Church stepped into a void and spoke about power and human dignity. But this encyclical is an act of moral authority, not political power. For that, we are still waiting. •

The post Will governments follow the Pope’s lead? appeared first on Inside Story.

AI has arrived at another such moment, bringing with it a technological revolution unlike anything we have seen before. It is a fundamentally different kind of technology, one that can operate outside of direct human control and using levers that humans can no longer understand or see.  If politics is the art of making things possible, AI, through algorithms and generative forms, has added a kind of unknowable alchemy to that process.

As with past shifts, this has required a new generation of scholars, writers, critics, and advocates to emerge as society grapples with this question of the impact of AI on democracy. Among them is Northeastern University’s Beth Simone Noveck, who is also chief AI strategist for the US state of New Jersey.

Noveck’s Reboot is a serious book about a real problem. It is also, in important ways, the wrong book for this moment. It looks at AI as an institutional problem of democracy, not a framing I had encountered in any meaningful way before, and certainly not from someone with the experience and knowledge to speak with real authority.

The book is essentially a cross-stitch of case studies showing how AI can be implemented in the public sector to improve service delivery and help citizens engage more meaningfully with a government that has grown monstrously in its remit but shrunk in its capacity to give citizens a voice in their own governance.

Noveck’s work is at its best when it treats democracy not as an abstract concept, the thing we all love to eulogise, but as a set of administrative problems that can be improved with the thoughtful application of tools already revolutionising the private sector and so much of our daily lives.

That premise is also what makes the book frustrating. Noveck writes about democracy with the assumption that its crisis is one of institutional capacity and insufficient information processing. There are of course many places where AI can help, but the book treats these as process fixes devoid of cultural meaning, beyond the political will to implement new technologies in a system with planetary inertia.

Which is not to say the book is naive. Noveck makes sharp points about institutions as information-processing systems, and about how new AI tools can help manage the glut of data the modern world produces. The opportunities for better service delivery and for bridging gaps between citizens are real, and the case studies she mentions were the talk of the Smart Cities initiatives in Australia and elsewhere in the late 2010s.

But that was an era of optimism around these technologies, a very different world from the one we inhabit now, something Noveck only passingly addresses. The core tension I found on reading Reboot is that each of its case studies presumes a society comfortable with a new, AI-powered government. It is interesting and worthwhile to discuss how services might be improved, but it is lacking, irresponsible at worst, to give only short shrift to the citizens over whom this new AI-powered government would have authority.

On a stylistic note, this is a book very much written for the AI age. It repeats ideas and characters with the consistency of a television show designed to be watched in the background, making it perhaps the first book I have read made for the second screen. Noveck also writes consistently in the first-person plural. Everything is we. Whether that reflects academic inclusivity or an acknowledgement that we are never quite alone anymore, even reading on a plane, I could not honestly say.

Reboot feels, at times, like an extension of the American liberal obsession with fact-checking in the Trump era. Both share the same foundational assumption: that the dysfunction of democratic life is primarily a supply problem. Give people better facts, or give institutions better data, and the system self-corrects. But democratic dysfunction today is not a supply problem. It is a crisis of trust, and trust is not fixed by better processing. Lies are more powerful than truth because they tell a better story. Grievance is more honest than facts to citizens who sense the modern world collapsing under their feet.

Noveck’s techno-optimism is perhaps best encapsulated toward the end of the book, when she sketches out a future for representative democracy in which AI avatars of both voters and representatives debate the direction of the country, leaving the humans themselves alone. That is a dystopian vision beyond most science fiction. Even with the caveat that humans could still take back the wheel, it would describe a world in which the role of citizen is reduced to that of a driver half asleep at the wheel.

Noveck also gives the example of urban planners in Brazil and Indonesia tasked with deciding the future of their cities’ public transit systems, and suggests AI might have made better decisions than they did. That might well be literally true, but it gets at the core problem of the book. Thirty years of public–private optimisation has left public services so hollowed out that citizens no longer feel they live in a democracy worthy of the name. Democratic legitimacy is not a byproduct of good outcomes. It derives from participation, from citizens feeling that decisions were made with them rather than for them. A better bus route decided by an algorithm is still a bus route nobody asked for. To fight one brand of optimisation with another is to entirely miss why citizens are so frustrated right now.

There is a sadness that comes through in Reboot around the fact that the United States lags so many other countries in implementing technology in governance. Noveck laments that the home of Silicon Valley has not yet benefited from many of these adaptations, but she never quite answers why.

The answer may lie in the fact that these ideas belong to a time when technology was seen as something that could constrain the overreach of government. Now, much of the public views tech companies themselves as the overreach that needs constraining.

Noveck spends much of the book on city-level implementations, where many of her ideas make genuine sense. Municipalities are not tarred with the same brush as national governments, but they exist within the same political context.

These are, in the end, two versions of the same problem. Noveck’s techno-optimism is not just politically naive. It leads her to write a book about public administration when she thinks she is writing a book about democracy. The scope she chose reflects the assumptions she made.

Reboot is less a book about improving democracy in the age of AI than a book about how public sector services can be improved through the thoughtful application of new technologies, an argument that the private sector should not be the sole beneficiary of such innovation.

That makes it a worthwhile read for city administrators and those interested in the finer points of public administration. But anyone hoping to understand the relationship between the rising tide of AI and the very real concerns we should all have about our democracy will be left wanting. •

Reboot: AI and the Race to Save Democracy
Beth Noveck | Yale University Press | $67.99 | 384 pages

The post Can AI save democracy? appeared first on Inside Story.

The decision by Bunnings’s owner, the Perth-based conglomerate Wesfarmers, to release those visuals didn’t come out of nowhere. It’s all part of the retailing giant’s battle to use facial recognition technology, or FRT, to monitor people entering its stores. By collecting data on all arrivals, managers are able to alert security guards about repeat offenders, adding vital seconds to call police and prepare staff for what might be coming their way.

It’s hard to argue against a business that has for years topped Roy Morgan’s “most trusted brand” list, particularly on such an emotive issue. Would you deny Bunnings the right to protect its staff and goods against violent thieves? The legal resistance encountered by Bunnings’s plans smacks of do-gooding over-regulation, in this case by the Office of the Australian Information Commissioner, or OAIC. Indeed, the message of Bunnings’s media campaign is that protecting staff, shoppers and property should trump the privacy enforcer’s concerns.

But here’s the thing: incidents of theft and assault in Bunnings stores have been declining across the country, according to data released during court hearings late last year. And that’s largely been a result of upgrades to traditional security measures rather than the decision to deploy FRT in sixty-three stores in New South Wales and Victoria. Those figures raise the not inconceivable possibility that the drive to embrace the technology is about something more than theft-proofing and occupational health and safety. Outsourcing security to technology could, of course, mean spending less on staff.

Throughout the Bunnings campaign, the OAIC has been adamant that FRT, which was deployed by the hardware retailer between 2018 and 2021, violates Australia’s 1988 Privacy Act. Contrary to Bunnings’s argument that the Hitachi-supplied FRT devices amounted to little more than glorified CCTV cameras, the OAIC argued the technology raised fundamental questions about how personal data can be collected and processed.

The OAIC’s November 2024 determination against Bunnings centred on the fact that, unlike CCTV, FRT collects personal data. The process starts with individual stores creating a database containing stripped-back facial images of customers previously identified as problematic. This isn’t a file of photos; it is a depository of biometric data extracted from CCTV stills according to a person’s facial features. The biometric data of people entering participating stores is compared, in a fraction of a second, to this database.

The key to the OAIC’s concerns is that the FRT collects the biometric data of all people entering the store. This is what distinguishes it from, say, CCTV, which merely offers shop assistants an additional set of eyes without in any way processing the images. Even the cameras mounted above self-checkout terminals in supermarkets, which store footage of all transactions, don’t qualify as FRT because they aren’t positioned in a way that identifies shoppers and they don’t process sensitive data.

Did Bunnings collect the biometric data in a way that would trigger the Privacy Act? The chain’s FRT devices collected customers’ data for 0.00417 seconds and discarded the information immediately if it didn’t match the database of “enrolled” repeat offenders. The OAIC concluded that, yes, Bunnings had collected “sensitive” personal data (which is how biometric data is classified). That it was stored for a mere 0.00417 seconds doesn’t matter — regardless of where you place the decimal point, the regulator said it amounts to “collection” under the Act.

The OAIC’s concerns were informed by recent court stoushes over the use, and misuse, of people’s images. It had successfully forced US-based image-scraping startup Clearview AI to shut down its operations in Australia and had castigated local police forces that used the company’s services. It also knew that FRT trials in New Zealand had produced “false positives,” identifying the wrong people as repeat offenders (with the faces of Pacific islanders particularly susceptible to mix-ups). In the United States, several people (many of them African Americans) have been arrested after being misidentified by Clearview, as outlined in New York Times tech writer Kashmir Hill’s 2024 book Your Face Belongs to Us.

The regulator appeared to be using the Bunnings case, as well as probes into FRT use by Kmart and 7-Eleven, to test the waters. Speaking to me in 2024, privacy commissioner Carly Kind conceded that, without a lot of jurisprudence available on FRT, she was making her “best and most informed efforts to apply the law as I understand it.” Because the use of FRT had never been reviewed by a court, the key provisions of the Privacy Act hadn’t been put to the test.

The OAIC would have preferred the matter to end up in the Federal Court. This would have given a judge with Privacy Act expertise the chance to undertake a “full-merits” review, examining everything from scratch. The Federal Court (and, to a limited extent, the High Court) had dealt with the OAIC’s case against Meta Platforms over the Cambridge Analytica data breach and revealed a readiness to dig deep into Australians’ right to privacy.

Bunnings had other plans. It launched an appeal with the newly established Administrative Review Tribunal, ensuring the focus would remain narrowly on the regulator’s determination rather than on broader privacy issues. A spokesperson told me at the time that Bunnings had hoped Kind would accept its claim that the technology was deployed in a way that “appropriately balances our privacy obligations and the need to protect our team, customers and suppliers against the ongoing and increasing exposure to violent and organised crime, perpetrated by a small number of known and repeat offenders”.

Both Bunnings and Wesfarmers were also lobbying for changes that would legalise — beyond any OAIC challenge — the technology’s use. Bunnings managing director Michael Schneider told the Australian Financial Review the time had come for “stronger protections and smart technology, like the responsible use of FRT, to keep people safe.” Without tougher laws, Wesfarmers managing director Rob Scott warned that organised crime gangs would continue to target the company’s stores, particularly in Victoria.

In Bunnings’s eyes at least, the appeal had become an existential fight.

The room allocated for the four-day hearing at the Administrative Review Tribunal’s Melbourne offices in October 2025 was small and stuffy. I arrived early, guessing there would be a scramble by observers wanting to plug their laptops into the room’s only accessible power point. A couple of other journalists sat with me at the start of the first day, though they left once it became clear how mind-numbingly technical the arguments would be.

But there were also some moments of levity. Representing Bunnings, barrister Ruth Higgins (subsequently appointed Australia’s solicitor-general) argued that the milliseconds required for the FRT to compare customers’ biometric data shouldn’t raise concerns under the Privacy Act. Rather, it was like gathering skimming stones on the shores of a lake, in which the mere picking up of the stone only to examine and discard it didn’t amount to its collection. Higgins went on to say that because Bunnings hadn’t captured CCTV stills but merely images “reduced to their mathematical form,” the process didn’t amount to the collection of sensitive data.

This was the first plank of Bunnings’s case. The company would also argue that even if the three-member tribunal ultimately found it had collected the biometric data, the company was entitled to do so under privacy exemptions designed to reduce serious threats to life, health or safety, or to prevent risks to the public.

There were some tense exchanges along the way. The OAIC’s barrister, Michael Borsky, derided Bunnings’s video compilation of violent incidents in its stores, pointing out that the showreel’s highlight was a man in a balaclava wielding a shotgun. No FRT device would have identified him through a facemask, making its inclusion misleading. The privacy regulator’s legal team also suggested Bunnings had inflated the rise in the numbers of violent incidents by not factoring in an increase in the number of its stores around Australia — as a percentage, violent incidents had in fact decreased. The Bunnings witness conceded this was the case and also said there had been no visible decline in incidents as a result of the FRT trial.

At every turn, the OAIC was determined to emphasise what was at stake. Cross-examining a witness, Borsky suggested that in a single two-month period Bunnings had recorded sixteen false positives and no correct positives. The OAIC also suggested that most of the false positives involved women or racial minorities. The head of Bunnings’s security operations accepted there had been false positives, but said staff had used discretion when reviewing the alerts. What’s more, when a false positive occurred, the image that caused it was expunged from the database of recidivist shoplifters to ensure the FRT wouldn’t make the same mistake again.

This prompted Borsky to question why Bunnings had even bothered with the rollout. “You don’t need FRT to know you have to call the police,” the barrister declared on the second day of hearings.

As the days in the airless room passed, it was becoming clear from the questions posed by deputy president Peter Britten-Jones and his colleagues that, on significant issues at least, the tribunal was leaning towards the OAIC’s arguments.

First, Bunnings’s written warnings to customers that their biometric data would be collected and used by FRT software linked to the CCTV cameras were either non-existent or inadequate. On this front, Australian Privacy Principle 3.3 is crystal clear: a retailer may “only solicit and collect sensitive information if the individual consents to the sensitive information being collected.” Bunnings’s signage wasn’t adequate.

Second, the members of the tribunal also appeared to believe Bunnings had indisputably collected the data. The milliseconds needed to compare customer’s biometric data to that of the store’s database became irrelevant because Bunnings had already admitted to collecting and collating a database of banned individuals.

The Privacy Act boxes had been ticked — the skimming pebbles analogy hadn’t convinced them.

But would the tribunal accept that the real security threats identified by Bunnings would be enough to grant one of the Act’s exemptions? On that question, the OAIC appeared to be struggling. While it might have scored a point by demonstrating how the number of violent criminal attacks had declined as an overall percentage across Bunnings stores, the tally of incidents was still very high, as Britten-Jones pointed out. That shoplifters described by the retailer as “recidivist” accounted for 60 per cent of all stock lost was a real concern.

Higgins drove that point home. These were “very significant losses and the violence itself would suffice” as a justification for deploying FRT, she told the tribunal on the final day of hearings.

When the ART’s ruling was published in February this year, both Bunnings and the OAIC were able to claim a qualified win.

The hardware giant welcomed the ruling, saying the tribunal had “recognised the need for practical, commonsense steps to keep people safe.” But it had “also identified areas where we didn’t get everything right, including around signage, customer information, processes and our privacy policy, and we accept that feedback”.

The OAIC was less conciliatory. It welcomed the fact that two parts of its original determination had been upheld: Bunnings customers hadn’t been properly notified and the Privacy Act does extend to the capture of sensitive data, even if it’s only stored for milliseconds. The notoriously underfunded regulator said it wouldn’t be appealing the ART’s decision.

The take-home was that Bunnings and other retailers would be able to use FRT in their stores, but only with strict safeguards in place. The OAIC warned retailers to view the decision “as a useful case study, rather than a green light for deployments of biometric technologies.”

A close reading of the ruling does point to a pathway for FRT, though, particularly when it’s deployed to combat tangible cases of criminality and violence. “Bunnings was entitled to use FRT,” the tribunal found, “for the limited purpose of combating very significant retail crime and protecting [its]staff and customers from violence, abuse and intimidation within its stores.”

The tribunal also concluded that features of the Hitachi technology used by Bunnings were sufficient to mitigate privacy risks: the biometric data wasn’t stored for long and wasn’t held in a way that could leave it exposed to cybercrime and data breaches. This meant the use of FRT “wasn’t disproportionate when considered against the benefits of providing a safer environment for staff and customers in Bunnings stores”.

Still, FRT seems likely to return to Australian courts given the unanswered questions left by the tribunal’s ruling. For example, what happens if the wrong person’s sensitive data is added to the repeat-offender database? What right of redress might that person have? What if, as has been the case in the United States, false positives are found to unfairly affect certain ethnic groups? And under what circumstances, if any, would one store be able to share its database of offenders with other Bunnings retailers?

A Federal Court review of the case might have shed more light on these questions. In the meantime, the evidence that’s emerged suggests Bunnings — and all Australian retailers — may be well-advised to concentrate on more traditional security approaches. FRT may one day start to replace security guards, but that’s unlikely to happen any time soon. •

The post Lowest prices were just the beginning appeared first on Inside Story.

We must hope Amodei’s carefully cultivated image as the conscience of the industry is accurate, because earlier this month his company — best known for its Claude AI models — revealed a new AI product, Mythos Preview, that’s widely seen as a step change in AI capabilities. Challenged by its designers to break out of a “cyber cage,” this model not only made its escape but also demonstrated a remarkable capacity to identify exploitable flaws in widely used software. “Mythos Preview has already found thousands of high-severity vulnerabilities,” said Anthropic, “including some in every major operating system and web browser.”

The model can identify tiny flaws in the systems that operate digital communications, electricity infrastructure, online banking and other vital services, and link them together to create major vulnerabilities open to exploitation. “Given the rate of AI progress,” the company added, “it will not be long before such capabilities proliferate, potentially beyond actors who are committed to deploying them safely.” In the wrong hands, Mythos could be used to bring essential infrastructure and services to a grinding halt.

“The ability to defend against cyberattacks is integral to the basic functioning of society,” observes Atlantic technology writer Matteo Wong. Mythos’s near-inevitable abuse would make that task much, much harder.

To underline Mythos’s alarming potential, Anthropic has withheld the model from general release and contacted forty big companies whose digital infrastructure could be threatened by its talent for spotting vulnerabilities. These companies — Amazon Web Services, Apple, Broadcom, Google, JPMorganChase and NVIDIA among them — have been given access to Mythos to scan and secure their own software as part of Project Glasswing, the company’s attempt to moderate the risks posed by its own product.

The significance of the model’s release didn’t elude governments and regulators around the world. The European Central Bank, for instance, began questioning banks about their digital safeguards. Canadian finance minister François-Philippe Champagne, who compared the news to the closing of the Strait of Hormuz blockade, revealed that Mythos was discussed extensively at last week’s International Monetary Fund meeting in Washington. “The difference is that the Strait of Hormuz — we know where it is and we know how large it is,” Champagne told the BBC. “The issue that we’re facing with Anthropic is that it’s the unknown, unknown.” Even the Trump administration is unnerved, reports the Economist.

A week or so after Mythos hit the headlines, Dario Amodei sat down for lunch at Catogna, an Italian restaurant in San Francisco’s Jackson Square, with Financial Times innovation editor John Thornhill. Over two courses accompanied by sparkling water, Thornhill tried valiantly to get a grip on a figure accused by one critic of purveying “disaster-porn-as-marketing-tool” and by a “veteran Silicon Valley investor” as “an extraordinary man, the real genuine article.”

Amodei professed to having been inspired to work in AI by futurist Ray Kurzweil’s influential 2005 book The Singularity Is Near, though he acknowledges it contains some “crazy,” “sci-fi” things. (Physicist Paul Davies described the book as a “breathless romp across the outer reaches of technological possibility” when it was published.) He was especially struck by Kurzweil’s observation that exponential increases in computing power would eventually result in human-level AI.

That vision still grips him. “There’s no end to the rainbow,” Amodei tells Thornhill. “We don’t see anything slowing down. I’m the first to say that it’s going to completely transform the world and we’re underestimating its significance.”

How you react to Kurzweil’s subtitle, “When Humans Transcend Biology,” no doubt mirrors your attitude to this period of seemingly unbridled technological experimentation. Amodei is overwhelmingly optimistic, telling Thornhill that he and his sister, fellow Anthropic founder Daniela Amodei, had long dreamt of doing good together, and “admits to being surprised” that Anthropic has given him what he sees as the chance to do that. Despite his warnings about the need for regulation, he clearly regards the structure and values of Anthropic as a reflection of that early pledge.

“We have an obligation to give back selflessly,” Amodei tells Thornhill at the end of their lunch. “And society does not have to venerate us for doing it.” For Thornhill, those words “make clear that Amodei wants to position himself as one of the good guys in the AI debate. But Amodei’s tone grates with many Silicon Valley critics, who note how his principles align with Anthropic’s commercial interests.”

Perhaps, given that Anthropic on its own can’t (and certainly won’t) significantly slow the rate of AI progress, Amodei’s combination of boosterism and scruples is the best we can expect from him. And the question of whether or not he is the good guy might not matter much if, as he suspects, “open-source models and Chinese developers will be able to replicate Mythos’s capabilities within six to twelve months” — or sooner, given reports this week of unauthorised users having already accessed the model.

Australia’s former e-safety commissioner Alastair MacGibbon was shocked by what he heard at a meeting with Anthropic representatives in Sydney earlier this month. “They are smart people and alarmed by what they have built,” he says. “They are right about the problem, but wrong about the solution.” Leaving Anthropic to choose who it shares information with will leave systems everywhere fatally exposed.

MacGibbon says federal home affairs and cybersecurity minister Tony Burke must act now to “convene critical infrastructure operators, private sector defenders and AI developers.” We must insist local organisations get early access to the Project Glasswing tools, he adds. “Australia cannot defend itself with yesterday’s capability while the frontier moves on without us.”

Australia’s official AI plan, released last November, looks entirely out of step with the challenge. By declaring Australia’s existing legal frameworks — updated “case by case” — are strong enough, it signalled the government had abandoned its earlier plan for mandatory guardrails for high-risk AI systems. Other countries have made a different assessment, wrote lawyers Jake Goldenfein, Christine Parker and Kimberlee Weatherall in the Conversation, citing rules introduced in the European Union, Canada, South Korea, Japan, Brazil and China.

Since then, the Australian Financial Review reported last month, the government has “put tech giants and data centre operators on notice about complying with Australian values and interests when deploying artificial intelligence or face a decade of reactionary regulation.” A decade of after-the-fact regulation doesn’t seem quite what this moment demands. The broad-brush memorandum of understanding that came out of Amodei’s meeting with Anthony Albanese late last month aren’t much more reassuring.

As two New York Times reporters put it, “Major AI breakthroughs are beginning to function less like product launches and more like weapons tests.” Will the threat dramatised by Mythos Preview prompt a rethink in Canberra? •

The post Right problem, wrong solution appeared first on Inside Story.

The saxophonist Lester Young was solitary and shy and addicted at times to heroin and to drink, which killed him at forty-nine. Yet a life that held much abandonment and pain produced a music of playful exuberance and joy. Young was a man of few words who reinvented words, calling his friend, Billie Holiday, “Lady Day,” and reportedly coining the term, “bread,” for money and “cool” to denote… well, cool. He “broadened the music’s emotional vocabulary,” wrote Ted Gioia in The History of Jazz. He also helped to give us a language of the modern city.

The first time I listened to Young I was making a long, dreamy drive down the Hume Highway. As the late afternoon sun flickered through gum trees I heard his tenor sax summon up the city at night: an anonymous crowd, glistening streets, shadows and pools of light, people in dives and basements, half drunk and full of intentions or regrets, a dim sense that in this bar, in this moment, life can be won or lost. I like jazz for the same reason other people like jazz: it’s urbane, diffident, sexy but not thrusting, moody, melancholy, wryly amused. It’s the missed encounter under the neon sign, a sad smile, acceptance of loss, lighting up a smoke. It is not grand the way much classical music is grand, but the heroic age is dead, and jazz speaks more directly to the ironic, wistful spirit of our time.

There’s a postmodern idea that the life of the artist is irrelevant to the art. Barthes said the author is dead; all that matters is the text and what the reader makes of it. I can’t accept that. I always want to know how the lives of artists and the times they lived through shaped their work.

And yet we can never know exactly how Lester Young’s work emerged from his life. All listeners can do is see their experience reflected in his music. Imagination transports us beyond time and place and race — beyond difference — to feel a tie to him that is purely human. Jazz, wrote the great critic Albert Murray, “exists not only as the intimate and personal expression of Black life, but is meaningful and accessible to any individual in the world.”

I’ve also come to like Barney Bigard, who played clarinet and tenor sax for fifteen years in Duke Ellington’s band and co-wrote the jazz standard Mood Indigo. Bigard was born in New Orleans in 1907, in the birthplace and birthtime of jazz. He grew up in a family of black Creoles, a heritage that was not just Black but French, Spanish and Native American as well. When Ellington’s band toured the segregated Deep South, the fair-skinned Bigard would often be sent out to buy food at whites-only stores, which he did by pretending he was Spanish or Mexican.

Like Young, Bigard said little except through his music, yet his life holds the whole early story of jazz: its origins in New Orleans, its great northern migration to New York, Chicago and Kansas City, its place in black American culture and in white culture and the fight to create a true colourblind democracy, in its claim that one way or another we are all magnificent mongrels.

Drifting from Bigard’s playlist on Spotify one night, I found another list comprised of pieces that sounded like him. On it was an artist or band called “Pause Maybe?” Washing the dishes one night, my wife and I listened for a while.

Pause Maybe? advertised itself as playing forties sax, and indeed the lonely, moody sound had echoes of the music of Young and Bigard, or the score for a film noir. The pieces had names like “Cigarettes and Memories,” “Under the Lamplight,” “Old Souls Stay Late,” “Half-Lit Room” and “Rustling Newspapers, Fading Time.” Attached to each was a drawing, often of a man in a tux slumped in a deep chair, his Homburg pulled low enough on his forehead to hide his eyes, the smoke from his cigarette drifting into the night.

It was a cliché of a certain kind of jazz, but gorgeous in its way, and I wanted to know more about this group. Who were the musicians? Could I even see them play? A google search gave me nothing. I still didn’t work it out.

In the middle of one night I got on my phone and asked AI about Pause Maybe? Dark thoughts come to us all at 3am, but even so I got a shock when I read: “The project is explicitly stated to be AI-generated… The identity of a human creator is not readily available or disclosed.” My AI had recognised its own kind.

Did my impression of the music change? It did. It now seemed somehow patterned: soulless, mechanical, cold. Of course, that thought was simply the effect of knowing it was made by AI.

But what repelled me was less the music than the fact I was not told how it was made. I felt tricked. Then again, if I had known it was AI from the start I probably wouldn’t have listened.

In 2025 Pause Maybe? released no fewer than nine albums. It has more than 100,000 monthly listeners. Its most popular piece, “Night Window,” has had more than 1.1 million listens. Despite what AI told me, nothing on Spotify explicitly states that this music is AI-generated, even though the streamer claims to be “aggressively protecting against the worst parts of Gen AI.”

Another streamer, Deezer, which seems to be doing more to unmask AI than Spotify, has said that slightly more than a third of content uploaded to its platform — about 50,000 tracks a day — is fully AI-generated. Do listeners know or care what they’re consuming?

I looked at the comments on YouTube under a Pause Maybe? piece, “Grandpa’s Smoke.” They start positive: someone says it reminds them of their grandpa; another likens it to a Benny Golson piece, I Remember Clifford. Then Byron Medrano asks: “Can anyone name a real song that sounds like this?” Jai Hunter replies: “As soon as this shit comes on I can tell it’s ai… It just has a weird ai tone that pretty much no other jazz sounds like.”

“This is AI, sadly,” writes another person. But someone replies: “Why sadly? This song is good.” Another says: “If this is AI, where are the humans playing the same style so I can hear them?… because AI or human, it is beautiful.”

Another writes: “Fuck AI. Listen to actual artists making real music. I don’t care how ‘soothing’ it is. It’s slop.” To which someone replies: “Are you mad at the future?”

Under another Pause Maybe? track, “Just Smoke and Jazz, Darling,” I read: “I’m 73 years young, and today my daughter and I shared one of the best days we’ve had in a long time. When I got home, she texted me to say she arrived safely — and then sent me this jazz mix, because she knows how much I love this music. I’ve been listening for over three hours now… and it feels like stepping back into a beautiful time.”

This comment felt powerful because it described an actual human experience built around listening to the music. Or did it?

The poster of the comment goes by the name Whiskey & Jazz Nights. Scrolling around, I discovered exactly the same remark under another Pause Maybe? track, “Cigarettes and Memories,” but this time from a writer called Soft Candle Jazz.

I go in search of Whiskey & Jazz Nights. It’s a YouTube channel: “Welcome to a world of smoky jazz bars, lonely saxophones, and stories whispered after midnight… a cinematic escape inspired by noir films and 1950s jazz clubs — where every note feels like a memory, and every melody carries a quiet longing.”

Getting uneasy, I go back to my phone and ask my AI whether this channel, too, is AI-generated. “It’s a very sharp observation!” it replies, flattering me as usual. Yes, it’s “highly likely” that AI plays a “significant role” in the production of Whiskey & Jazz nights.

If that’s true, then the machine, in seemingly pretending to be a seventy-three-year-old spending precious time with his daughter, is learning how to act human.

What about this comment by VelvetRedNotes: “Why a 28 young woman can’t have an old soul to hear this kind of music? I would like to have a slow dance with my man to this music, all night.”

This time AI insists that VelvetRedNotes is a woman, an anime pianist called Melody. The VelvetRedNotes YouTube channel promises “a refined jazz sanctuary where timeless elegance meets modern mood. From intimate late-night ballads and smoky lounge sessions to cosy café swing and lo-fi focus beats, every track is curated to elevate your moments… one velvet note at a time.”

Which sounds a lot like Whiskey & Jazz Nights to me. I’m in a hall of mirrors, highly artificial, not that intelligent.

Whether or not people care that a work of art is made by AI, we should at least have a right to know. In December, the former Australian chief scientist, Alan Finkel, launched a company, Proudly Human, that enables authors to declare that they made their work, then verifies that the declaration is correct. Instead of forcing AI-content producers to be upfront about how they created their work — which Finkel thinks is technologically too hard to do — he is proposing a positive label, a bit like “Free range” on eggs. Proudly Human is focusing on the written word, but it might work for music and the screen, too.

Authors who wish to be certified by Proudly Human agree to subject their work to a range of verification tests — conducted by AI, of course. (A journalist who wrote about Proudly Human called the process not “Spy versus Spy” but “AI versus AI.”) The published work is also compared to the original to make sure an AI version wasn’t swapped in after the tests were done.

The bet behind Proudly Human is that enough humans will care they are reading a work produced by one of their own. In an email conversation, Finkel sounded excited about his project yet aware that AI is developing so rapidly that the challenge of identifying its influence in any text grows harder by the day.

I sent him a link to a letter Nick Cave wrote on his site, The Red Hand Files, where he responds to readers’ questions. Someone got ChatGPT to write a song in Cave’s voice then asked him what he thought. Here’s the chorus — judge for yourself:

I am the sinner, I am the saint
I am the darkness, I am the light
I am the hunter, I am the prey
I am the devil, I am the saviour

The lyrics sent Cave into a justifiably cold rage. What makes a song great, he wrote, is not its resemblance to other songs, and it’s certainly not the mimicry ChatGPT churns out. Instead, the artist makes a “heart-stopping” departure from what they thought were their own limits.

And here an invisible relationship with the listener begins, Cave suggests. As the artist confronts their smallness and vulnerability, the listener’s heart is stirred, recognising “in the inner workings of the song their own blood, their own struggle, their own suffering.”

But data doesn’t suffer, and algorithms don’t feel, Cave writes. “ChatGPT has no inner being, it has been nowhere, it has endured nothing, it has not had the audacity to reach beyond its limitations, and hence it doesn’t have the capacity for a shared transcendent experience, as it has no limitations from which to transcend. ChatGPT’s melancholy role is that it is destined to imitate and can never have an authentic human experience, no matter how devalued and inconsequential the human experience may in time become.”

And yet, Cave concludes, one line did speak to him: I’ve got the fire of hell in my eyes. “That’s kind of true. I have got the fire of hell in my eyes — and it’s ChatGPT.”

Finkel wrote back to me: “Nick Cave’s letter is clearly in the style of Nick Cave!” He found the ChatGPT song “replicative and pointless.” But he sounded a warning: Cave says he found one good line in it. “If ChatGPT has another go in two years’ time, there will be at least ten.”

I think Cave was being sarcastic about that line. Still, Finkel’s larger point about the advance of AI is almost certainly correct.

Finkel told me that his friend, the philosopher Peter Singer, tested Proudly Human by uploading two articles, one written by Singer, the other by an AI, Claude, that had been asked to write in Singer’s style. Finkel was delighted the tests verified the human article and rejected Claude’s. But then Singer asked the AI how it felt about the outcome. Claude’s reply is worth reading in full:

Excellent! That’s a successful test for your friend’s software. This is genuinely interesting because:

1. I was actively trying to mimic your style— using arguments you’d make, sentence structures you’d use, topics you care about

2. The content was substantively sound— it’s not like the software detected nonsense or logical flaws

3. Yet something gave it away— there must be detectable patterns in word choice, sentence rhythm, rhetorical structure, or other features that distinguish AI writing from human writing, even when the AI is trying to imitate a specific human

… it’s encouraging for your friend’s project — and somewhat humbling for me! It suggests that despite AI’s impressive capabilities, there remain detectable differences between human and AI authorship, at least for now.

Finkel says he was surprised by the sophisticated response, and bowled over by the phrase, “and somewhat humbling for me!” The words that struck me, though, were: “at least for now.”

Cave wrote another letter about AI and human creativity. Read stirringly by the actor Stephen Fry, it begins: “God made the world in six days. On the seventh He rested. That suggests the creation took effort, that some form of artistic struggle had taken place. God saw that it was good because He had put something of Himself into it.”

A piece of art is great, Cave writes, precisely because of the striving, and often the failing, behind it. The outcome cannot be separated from the process, the work from the work. To make art using AI is to see the labour of creation as “nothing more than a time-wasting inconvenience that stands in the way of the commodity itself.”

The artist concludes: “And as the sun rises on the struggle of the day, and the great-crested grebe dances upon the water, it is our striving that becomes the very essence of meaning. This impulse, this creative dance, that is now being so cynically undermined, must be defended at all costs. And just as we would fight any existential evil, we must fight it tooth and nail. For we are fighting for the very soul of the world.”

For better or worse, AI is here. The machines are on the march. The line between them and us is already starting to blur. Yet our home and spirit still soar or shrivel in a physical world that contains landscapes, wind, the great-crested grebe, and ourselves.

What can be trusted? What is true? These are existential questions for us now. Can we find a way to treat AI as we treat fire, as a good servant and a bad master? To find a way to live with it on our terms, and not to accept, through our own choices, the devaluing and trivialising of the human experience? A world dominated by phantoms dancing on screens? A world that still knows jazz — and may even hear the foggy notes of a saxophone drifting up from a basement in a city growing dark — but no longer knows the flawed, fabulous human beings who created it? •

The post The day the music died appeared first on Inside Story.

Over the years I’ve pursued further psychological training with the aim of improving my communication with and counselling of patients, and I now share what I’ve learned with medical students in my role as a facilitator in a Melbourne medical school. I’ve always been cognisant of the incredible privilege of my work. The word “privilege” can be a double-edged sword: I use it here to mean a sense of gratitude to patients for the trust they’ve placed in me. I’ve never thought of my medical degree as conferring privilege in the more negative sense.

I tell you all this because I feel the need to be straight with readers about my personal views about being a doctor. It’s in the context of a career focused on fostering better relationships with patients that I read Charlotte Blease’s Dr Bot: Why Doctors Can Fail Us and How AI Could Save Lives. It’s in this same context that I reflect on its content.

I’m not at all opposed to a book that explores the possibilities of AI in medicine. What I’m struggling with is the general tenor of this one. In fact, the purported premise of Dr Bot — that AI has the potential to address all the deficiencies of the current healthcare system — seems to me to be simply a smokescreen. The real agenda here, as I read it, is to demonstrate the inadequacies, not of the healthcare system in general, but of the medical profession in particular. In chapters with titles such as The Ailing Appointment, Doctor Deference, The Dark Art of Medicine — why it’s dark is left unexplained — and Humanizing Healthcare Without Doctors, Blease builds to her startling conclusion, that patients might be happier and healthier if doctors were sacked and AI put in charge.

I’m mindful that Blease, a health informaticist, would probably view my criticisms as an entirely predictable attempt to protect medical practice for my own benefit and that of my cronies. “Institutions will try to preserve the problem to which they are the solution,” Blease writes, citing the apparently well-known “Shirky Principle.” She continues: “In the case of medicine, I argue that the situation is even worse. Faced with increasingly broken health systems, the profession is failing to constructively reimagine and work towards credible solutions.”

Doctors, according to Blease, are fallible and arrogant. They hold elitist “luxury beliefs” about their capabilities. Worst of all, their clinical decisions are governed by their “Stone Age” brains, in which deepseated and immutable prejudices lurk. The solution to the problem of doctors, according to Blease, is not that the profession take steps to tackle its shortcomings but that society dispense with them entirely or, at the most, retain a select few as “the single figurehead of a healer,” whatever that might mean. Dr Bot almost reads like the case AI might make for itself as a replacement healthcare provider.

Blease says that doctors must shoulder the blame for health system blindspots and failures, at least in Britain and the United States, from where her observations come. I find it surprising that she doesn’t extend her criticism to big gaps in US health insurance coverage, or the profit-driven and exceedingly powerful US health insurance corporations that dictate who gets covered and for what. Nor does she touch on Britain’s beleaguered National Health Service, where waiting times for hospital appointments are now measured in years. These are national, systemic imbalances, not ones born of medical malpractice.

Blease writes at length about the biases that arise from gender, race and socioeconomic mismatches between doctor and patient. She argues that AI-delivered healthcare will help erase such biases which, in her eyes, are wholly attributable to human — that is, doctor — error. Such an argument exposes a fundamental flaw in Blease’s thesis: she clings to the idea that medicine is still the patriarchal profession of fifty years ago.

Doctors’ awareness of the dynamics of gender, race, sexuality and class — as well as other potential biases around patients’ lifestyle choices — has risen dramatically in recent years. Medical students in Australia, at least, are now taught self-reflective practice alongside ethics and communication skills. A morning spent with the students I teach — committed young men and women of very diverse backgrounds — would convince anyone that our future doctors are awake to the traps of privilege and prejudice.

I also disagree with Blease’s assertion that doctors’ Type 1 (fast) thinking style — the rapid decision-making required in an emergency — blinds them to their biases. Type 2 thinking is the mainstay for the many doctors working in mental health, to say nothing of the reflective tasks, debrief sessions and case reviews that every clinician undertakes as part of their ongoing education.

Strangely, one might say perversely, Blease argues against empathy in doctors, citing research to support her view that empathy tends to get in the way of rational clinical decision-making. This argument smacks of post-hoc justification: so doctors need to be replaced by computers because, like their patients, they’re all too human? Isn’t human empathy a motivator to act in the interests of the patient? Blease is also selective: a quick internet (non-AI-mediated) search brings up many studies revealing the benefits of doctors’ empathy for patients’ satisfaction, adherence to treatment and clinical outcomes. But Blease’s doctors are damned either way: she also asserts that all too often they display the wrong sort of empathy.

AI will inevitably play a role in medicine, as it will in many fields of work. It’s already arrived in the form of scribe programs that produce consultation notes and chatbots that condense the time taken for literature reviews from days to minutes. Some of you will have already had to decide whether to consent to your GP using these tools during your consultation.

Australian doctors are embracing these innovations as an extremely effective way of reducing the burden of administration, and are hopeful that, in our increasingly stretched healthcare system, the assistance AI provides will afford them more quality time with their patients. Blease sees doctors as technically illiterate and change resistant, yet very few doctors working in this country don’t use medical technology on a daily basis. And our junior medical workforce — those doctors who staff our public hospitals twenty-four hours a day — have never known a world without the internet. What exactly are these AI innovations that doctors refuse to engage with? Who are these medical Luddites?

Some time ago for Inside Story I reviewed The Doctor Who Wasn’t There, a history of electronic media in health and medicine by the US physician Jeremy Greene. Over the past century, Greene explains, a series of new technologies have promised to democratise access to healthcare. From the invention of the humble telephone to the introduction of telemedicine, initial enthusiasm or scepticism faded over time as the new became commonplace, and yet the promised democratisation of healthcare remained as out of reach as ever, at least in the United States.

AI might run the same course, despite Blease’s claims to the contrary. She champions bots for their superior diagnostic skills, the clarity of health information they provide, and their assistance in helping patients talk more assertively to their doctors. While making brief mention of the possibility that Big Tech could put any or all of their AI programs behind paywalls, she paints AI as the great hope for a more accessible and equitable US healthcare system. But she makes no mention of nor offers any alternative to the skills needed outside diagnosis and patient communication that bots don’t currently provide — the skills required for physical examinations, surgery, radiology, anaesthesia and other interventions.

It’s only in the last few pages of Dr Bot that Blease turns her attention from the myriad defects of the medical profession to touch — albeit briefly — on some very concerning aspects of AI: inequitable access to AI-assisted medical care, surveillance, data breaches (a particularly vexed issue when it comes to sensitive medical information) and AI’s staggering consumption of energy and water. Blease states in her introduction that these “pressing national and global concerns” are largely outside the scope of her book — a convenient stance — yet she’s still content to conclude that AI might replace the medical profession and do a better job of healthcare.

Surely the horrifying fact that each ChatGPT search uses around 500ml of water is as important to consider for our wellbeing as a species as the sometimes brusque bedside manner of one’s surgeon. I’m not excusing the surgeon — except to say that her brusqueness might have something to do with being up all night repairing a ruptured aorta — but surely a broader perspective is critical before recommendations about the net value of AI in healthcare can be confidently made.

Who are the potential readers of this book? Doctors aren’t likely to want to wade through this castigating text with its sometimes laboured metaphors — “the messy entrails of the patient–doctor appointment,” for example — to discover the author’s conclusion that, in the long term, the traditional medical profession might well be wholly replaced with physician assistants, nurse practitioners, medical knowledge engineers, telemedicine developers and medical data scientists. Wait a minute: so physician assistants and nurse practitioners can stay? Aren’t they clinicians, too, every bit as prone to bias and “Stone Age” thinking as doctors? (I should add that Blease includes “junior” physicians in this acceptable line-up, without mention of their fate when they “grow up.”)

Is Blease writing with the interests of patients in mind? She staunchly claims she is, and I applaud her intention. Perhaps Dr Bot will encourage readers to consult bots to make sense of their symptoms, learn more about their prescribed medication and decipher medical jargon. Empowering patients is a good thing, but empowerment doesn’t necessarily blossom from the seeds of physician mistrust.

In a recent article for the New Yorker titled “If AI Can Diagnose Patients, What Are Doctors For?” physician Dhruv Khullar writes about a bot that can diagnose complex and rare diseases with similar accuracy to the most experienced physicians. This bot and others like it promise better outcomes for patients by giving easier — perhaps even more equitable — access to prompt diagnoses and appropriate care.

But there’s danger in relying too heavily on AI, Khullar writes. For one thing, AI still gets it wrong. And doctors run the risk of “cognitive deskilling”: losing their diagnostic knowledge and skills due to lack of use. As AI gets more integrated into routine medical practice, this deskilling is something to guard against, especially when the lights go out and the bots shut down, as is increasingly likely to happen as our climate grows ever more extreme.

Khullar’s article argues for a working relationship of AI and doctor for the benefit of patients. He suggests that doctors see AI as a means of exploration of a clinical dilemma, a place to start rather than end. “At their best,” he writes, “they would steer you through — not away from — the medical system.”

I’ve never been an out-and-out apologist for the medical profession. I’ve worked with my fair share of bullying senior medical staff. On occasions I’ve seen my colleagues display hubris, insensitivity and lack of empathy. I know we’ve all made mistakes in diagnosis and management, mistakes that sometimes haunt us for the rest of our careers.

But I can’t accept that the way doctors work today can be considered obsolete, especially as a whole generation of young doctors are bringing to the profession an improved awareness of the socioeconomic determinants of health, sound skills in communication and teamwork, and a keen interest in using AI to maximise patient care. Blease has made her case. I owe it to all the conscientious and caring doctors I know to dispute it. •

Dr Bot: Why Doctors Can Fail Us and How AI Could Save Lives
By Charlotte Blease | Yale University Press | $39.95 | 352 pages

The post Will AI replace doctors? appeared first on Inside Story.

I think if you work as a radiologist, you are like the coyote that’s already over the edge of the cliff but hasn’t yet looked down… People should stop training radiologists now. It’s just completely obvious within five years deep learning is going to do better than radiologists… It might be ten years, but we’ve got plenty of radiologists already.

ChatGPT hadn’t yet been invented, but the AI systems of 2016 were already pretty good at reading medical imaging. Hinton felt that he could see the writing on the wall.

And yet here we are, nine years later, and the radiology profession is doing just fine. Deena Mousa recently wrote about this in an excellent Works in Progress article:

Radiology is a field optimised for human replacement, where digital inputs, pattern recognition tasks, and clear benchmarks predominate… But demand for human labour is higher than ever. In 2025, American diagnostic radiology residency programs offered a record 1208 positions across all radiology specialties, a 4 per cent increase from 2024, and the field’s vacancy rates are at all-time highs. In 2025, radiology was the second-highest-paid medical specialty in the country, with an average income of $520,000, over 48 per cent higher than the average salary in 2015.

As Mousa notes, some of the reasons for this are pedestrian. AI models don’t perform nearly as well once they get out in the real world and have to go far beyond their training data. And humans don’t trust the AI, so regulators and health insurers sometimes insist on human radiologists.

But there are other reasons for the persistence of the radiology profession that are even more profound — and that should serve as important reminders about how little we know about the economic effect of AI.

First, AI model makers don’t actually know what radiologists do. They know radiologists read scans, but they don’t know all the other stuff they do:

Radiologists are useful for more than reading scans; a study that followed staff radiologists in three different hospitals in 2012 found that only 36 per cent of their time was dedicated to direct image interpretation. More time is spent on overseeing imaging examinations, communicating results and recommendations to the treating clinicians and occasionally directly to patients, teaching radiology residents and technologists who conduct the scans, and reviewing imaging orders and changing scanning protocols. This means that, if AI were to get better at interpreting scans, radiologists may simply shift their time toward other tasks. This would reduce the substitution effect of AI.

Because AI engineers don’t really understand all the things radiologists do, they will be slow to design AI systems that address all of these tasks. And even when someone does get around to addressing this problem, it’s not clear when we’ll get AI that’s as good at humans at all of these tasks. Humans may remain in the loop.

Finally, AI increases productivity, which reduces cost, which increases the number of patients who can be served. This increases the demand for radiologists’ labour:

As tasks get faster or cheaper to perform, we may also do more of them. In some cases, especially if lower costs or faster turnaround times open the door to new uses, the increase in demand can outweigh the increase in efficiency, a phenomenon known as Jevons paradox. This has historical precedent in the field: in the early 2000s hospitals swapped film jackets for digital systems. Hospitals that digitised improved radiologist productivity, and time to read an individual scan went down. A study at Vancouver General found that the switch boosted radiologist productivity 27 per cent for plain radiography and 98 per cent for CT within a year of going filmless. This occurred alongside other advancements in imaging technology that made scans faster to execute. Yet, no radiologists were laid off.

Instead, the overall American utilisation rate per 1000 insured patients for all imaging increased by 60 per cent from 2000 to 2008. This is not explained by a commensurate increase in physician visits. Instead, each visit was associated with more imaging on average.

AI engineers, who look mainly at their models’ capabilities, don’t generally think a lot about this overall economic ecosystem. They’ve seen their creations up close, and they know their capabilities better than anyone else, but that doesn’t mean they know what a radiologist does at work. And they don’t know whether AI will simply change how radiologists spend their time at work while also improving their productivity. Hinton might someday be right, but as of right now he was wrong. •

The post Why do we still have so many radiologists? appeared first on Inside Story.

Yet the economic consequences of the radically self-destructive turn in US trade policy have been matched, perhaps even overshadowed, by a development that has nothing to do with policy: an enormous surge in spending on “AI.” Scare quotes because ChatGPT and its rivals aren’t really artificial intelligence. But they are impressive, routinely doing things that would have seemed impossible just a few years ago.

And the surge in AI investment — a tech boom the likes of which we haven’t seen since the 1990s — has buoyed the economy in the short run, offsetting the drag from Trump’s tariffs. Without the data-centre boom, we’d probably be in a recession. But hundreds of billions in spending on data centres have boosted investment, while soaring stock prices have supported consumer spending by the wealthy even as demand from lower and middle-income Americans weakens.

Unlike the tech boom of the 1990s, however, the current AI boom isn’t translating into widespread economic optimism. In fact, Americans are remarkably downbeat about the economy and the future in general. And I think it’s worth trying to understand why.

This article isn’t about the long-run consequences of AI for things like jobs and economic growth, as well as whether we’ll soon create superintelligences that decide to kill us all. I want to focus for now on how hopes about profits to be made from AI — and fear of missing out, or FOMO — have led to huge business outlays, largely on those data centres. By the second quarter of this year spending on information-processing equipment and software, as a percentage of GDP, had already matched its peak in 1999, the height of the internet bubble. There’s every indication that it’s going to go even higher, maybe much higher.

Source: St Louis Federal Reserve. Note: Ignore the numbers for the early 2020s, which, like so many other things, were greatly distorted by Covid.

The current AI boom resembles the 1990s tech boom in other ways besides the tidal wave of spending. To those of us of a certain age, the hype — This will change everything! — is distinctly familiar. Now, as then, the feverishness is a good reason to suspect that we’re in the midst of a huge speculative bubble. Reinforcing this suspicion is the fact that big tech companies, which generate billions in cash flow, are spending more on AI than their gushers of dollars can support. So now they’re taking on lots of debt.

While the 1990s and today are alike in their bubble mentality, however, they differ in an important way: today we lack the pervasive optimism of the late 1990s. It’s hard to convey to people under the age of fifty just how upbeat Americans in the late 1990s were feeling about the economy and the future in general. In fact, I believe the Bill Clinton–era boom led Americans to take prosperity for granted, resulting in their willingness to switch their presidential votes to a Republican — the ill-fated George W. Bush.

Today corporations are, once again, pouring vast sums into technology, but they’re doing it against a background of extraordinary pessimism. And the question I want to ask is why we’re seeing 90s-type hype without a return of 90s-type hope. Why aren’t we partying like it’s 1999?

You can see the difference in how people feel about the economy by looking at consumer sentiment, which was very high at the end of the 1990s but is now about where it was during the depths of the global financial crisis:

You can also see the difference between now and then in more general polling. In 1999 around 60 per cent of Americans were satisfied with the way things were going in the country; these days it’s half that. You can see it in presidential approval ratings: In March 1999, 78 per cent (!) of Americans said they approved of Bill Clinton’s handling of the economy. Polls show Trump with less than half that much approval for his economic policies, with those disapproving exceeding those approving by around fifteen points.

So why are we all feeling so grim even as businesses place huge bets on an impressive and possibly transformative new technology? I’d give three answers.

First, the US economy under Trump is doing worse than the standard measures indicate. It’s true that we haven’t had a recession — largely because that huge spending on data centres has compensated for the economic drag caused by tariffs. It’s also true that unemployment remains fairly low — in fact, the current unemployment rate is similar to the unemployment rate in 1999.

But while we haven’t (yet?) seen mass layoffs, the labour market is weirdly frozen, probably because of uncertainty created by Trump’s erratic policies. The hiring rate — the rate at which employers are taking on new workers — is very low by historical standards. So is the quit rate, the rate at which workers are voluntarily leaving jobs, normally an indication that workers fear they won’t be able to get a new job if they leave their current employment.

Unfortunately, these data don’t go all the way back to 1999. But here’s a striking comparison. The Conference Board [a business-oriented think tank] conducts a monthly survey of consumer confidence that among other things asks whether respondents consider jobs “plentiful” or “hard to get”. In April 1999 people were very upbeat about job-finding: 47.4 per cent said jobs were plentiful, while only 12.5 per cent said they were hard to get. In August 2025 those numbers were 26.9 per cent and 19.1 per cent respectively, a far more pessimistic view. In other words, not many Americans have been laid off, but many of them are very worried about what will happen if they do lose their job.

Which brings me to my second point. As best I can remember, people were excited about the rise of the internet but not, for the most part, frightened. They saw new possibilities opening up, but few Americans saw these new possibilities putting their jobs or their society at risk. In retrospect we should have been worried: social media in particular have done an immense amount of social and psychological damage. But we weren’t thinking about those risks.

By contrast, it’s hard to find people who aren’t worried about AI. It’s common to hear warnings that AI will eliminate large categories of jobs, and maybe even lead to mass unemployment. I take the first prospect seriously — past technological change has taken away most of the jobs in major occupations, from coal-miners to longshoremen. As a card-carrying economist, I’m sceptical about the second: people have been predicting mass unemployment caused by automation since the 1930s, and it keeps not happening. But the point is that AI is creating widespread anxiety even as it boosts GDP in the short run.

Finally, although this is hard to prove, I believe the political situation is bleeding into economic perceptions. People like me see a terrifying autocratic power grab in progress. While Trump and his minions claim that this is the BEST ECONOMY EVER, they also claim that our major cities are war zones overrun by dangerous leftists, which kind of interferes with the attempt to project sunny optimism.

My guess is that the current tech boom, like the 90s boom, will end in a painful bust. While Trump keeps insisting that the economy is great, his minions have taken to promising that it will get much better next year. Given how bad people are feeling about an economy currently propped up by an unsustainable tech boom, I wouldn’t bet on it. •

This article first appeared in Paul Krugman’s Substack newsletter.

The post Why aren’t Americans partying like it’s 1999? appeared first on Inside Story.

It might sound like adolescent bravado but my son’s remark captures a broader truth: young people are increasingly questioning the value of formal education. They compare the immediacy and variety of on-demand knowledge (including quick AI-generated answers) with the rigidity of school curricula. For many, these sources feel faster, clearer and more relevant than the slow pace and fixed structure of the classroom.

Many critics interpret this attitude as evidence of a broader educational crisis, with AI at least partly to blame for student disengagement. For example, when the NSW education department banned ChatGPT and similar tools in public schools in early 2023, it warned that AI could allow “fluent but lazy” thinking to go unchallenged and devalue authentic student learning.

Reinforcing this concern, the Paul Ramsay Foundation–sponsored report Shaping AI and EdTech to Tackle Australia’s Learning Divide argues that introducing AI tools without the right frameworks risks alienating students. If new digital tools widen the gap between how students learn and how school teaches, those students may feel increasingly disconnected from, and question the relevance of, school itself. The report’s author, Leslie Loble, stresses that without strong governance and thoughtful design, the tools will push students further away from learning rather than bringing them closer.

It’s important to remember, though, that students were questioning the relevance of their schooling long before AI came onto the scene. Research across Australia consistently shows that student engagement and sense of belonging have been in steady decline. Between 2003 and 2015, data from the OECD’s Programme for International Student Assessment, or PISA, revealed that the share of Australian fifteen-year-olds who agreed they “feel like they belong at school” fell from 88 to 72 per cent, almost double the average decline across OECD countries.

Over the same period, the proportion who reported feeling disconnected from school also rose sharply, particularly in the middle years of high school (around Year 9, when disengagement peaks). Those least likely to feel a sense of belonging included Indigenous students, girls, students from less well-off backgrounds, Australian-born students and those in provincial and remote areas.

International data backs this up. The PISA studies have identified key predictors of student disengagement, including a low perceived value of school, poor classroom climate and weak teacher–student relationships. When school feels unsafe or irrelevant, or when students don’t have positive relationships with teachers, they inevitably tune out. Lessons are too often framed as preparation for the next exam rather than opportunities to explore and apply knowledge in authentic ways.

Many students also feel they have little voice in what or how they learn; their role is to receive and reproduce information, not to shape the learning process. It’s no wonder so many struggles to see a meaningful connection between what they do in class and the realities of their future lives.

We can’t simply blame individual teachers for these outcomes. Staff in Australian schools report some of the highest workloads in the OECD. Many work well beyond their contracted hours just to keep up with administrative demands, constant curriculum changes and pastoral care responsibilities. A 2022 Grattan Institute survey found that teachers work about fifty-five hours a week on average — far above a standard working week — yet less than half of that time is spent teaching or interacting with students. The rest is devoured by paperwork, meetings, data entry and other tasks. Disengagement is the product of a system that prioritises compliance, standardisation and measurable targets over curiosity, relationships and real-world relevance.

In this light, my son’s comment starts to make sense. When students feel disconnected from what happens in the classroom and see little link between schoolwork and their own goals, it’s hardly surprising they turn to AI. By the time they finish school, many have learned to see education as something to endure rather than a space to explore and grow. My son was voicing what many students feel: that the education system is struggling to keep pace with the way they learn and live.

The first step to reducing students’ overreliance on AI is to make learning more relevant and engaging. Too often, lessons are framed as preparation for the next test or assignment rather than opportunities to grapple with real-world challenges. When students are asked to solve problems that connect with their lives, from environmental issues to ethical dilemmas to practical applications of maths and science, their learning feels purposeful.

This could mean project-based work, with students collaborating to design ways of reducing waste in their school, designing a more inclusive public space, or creating campaigns around health and wellbeing. In classrooms, it could mean shifting discussions beyond textbook answers into debates that evaluate competing perspectives, with students weighing evidence, challenging assumptions, and justifying their reasoning.

Students would be given more opportunities for inquiry and more choice in how they demonstrate their learning, whether that’s through a presentation, a group project, or a community partnership. Lessons would become spaces of exploration, helping students see that what they are learning has a direct bearing on the futures they are preparing for.

Australian research supports this approach. A UniSA study earlier this year asked disengaged senior students to design and build a playground for a low-income school, combining inquiry, service and skills-based learning with teachers acting as facilitators. Students set achievable goals, conducted market research to identify community needs, and explored multiple design options before selecting the most viable. The researchers found that disengaged students are more likely to participate when learning connects to the real world, benefits their communities, and gives them opportunities to make authentic decisions.

A South Australian trial of project-based mathematics reached a similar conclusion. It found that students in project-based classes reported greater enjoyment, stronger academic confidence, and higher achievement than those in traditional classrooms. The approach was open-ended and inquiry-driven, with teachers facilitating group projects rather than delivering content. Unlike textbook-driven lessons, the projects began with a problem and introduced the maths needed to solve it.

If students see how their studies link directly to real-world issues and their futures, they are less likely to view AI as a shortcut and more likely to use it constructively, not to replace their learning, but to extend and deepen it.

The pressure is not only on students. Increasingly, educators themselves turn to AI to prepare course content, streamline administration or even generate assessment tasks. While this reliance is understandable in a system where staff are expected to do more with less, it risks hollowing out the very heart of education: the transfer of expertise, judgement and experience from educator to learner. Students notice this too, and they rightly ask, if their lecturers are leaning on AI, why shouldn’t they? When both sides are outsourcing parts of the learning process, the human connection at the core of education is put at risk.

Until we make those changes, AI will remain less an opportunistic shortcut than a necessary crutch. We need to be talking not about how AI is ruining education but about why our educational infrastructure is so brittle that a chatbot could exploit its weaknesses overnight. Only by tackling those underlying faultlines, the relevance of our curriculum, the way we measure success, the support we give both learners and teachers, can we ensure that the next technological disruption becomes a tool for empowerment rather than an escape hatch from a broken status quo.

AI didn’t create the cracks in our education system. It’s just accelerating and illuminating them. The responsibility is now on educators, policymakers, parents and communities to build an education system that reflects our highest aspirations for learning, one where technology complements genuine engagement rather than substituting for it. Only then will students choose substance over shortcuts — because the learning experience will finally be worth their while. •

The post Don’t blame AI for schooling’s decline appeared first on Inside Story.

In this category go a slew of harms that range from riding roughshod over intellectual property (stealing copyrighted material to train algorithms) through to the illegal and abhorrent (creating child abuse material). Beyond these extreme examples, a whole heap of downright dodginess is emerging in the name of AI “innovation.”

In response, a patchwork quilt of rules is being constructed across the world that tries to graft ethical and legal constraints onto the technology. But the financial and political forces at play are so powerful that we are seriously entertaining the idea that regulation and safety should play second fiddle to getting the technology out the door and into the hands of everyone. Don’t worry — it’s not that bad, in fact it will probably be good for us… Cigarette anyone?

It’s always refreshing when the AI behemoth gets a poke, and Emily Bender and Alex Hanna invite us to get real with their well-researched and entertaining book, The AI Con. They’re highly qualified to do so. Hanna, formerly part of Google’s “Ethical AI” team, is a sociologist whose work examines data-driven socio-technical inequality. Bender is a linguist and incisive AI critic who, among many scholarly attacks on the way the technology is sold to us, has labelled large language models (the technology behind systems such as ChatGPT) “stochastic parrots.” We know where we’re going with a book subtitled “How to Fight Big Tech’s Hype and Create the Future We Want,” and Bender and Hanna don’t hold back. They are, in their words, scaling the ever-rising AI “bullshit mountain.”

As someone who works in AI, it’d be incongruous if I said all AI is bullshit. I certainly don’t say that, and neither do Bender and Hanna. They acknowledge early in the book that properly conceived, constructed and legitimately helpful AI systems do exist. But a dominant, obfuscating cycle of AI “hype and harm” has also been at play. This includes breathless hyperbole from the “AI boosters,” including (of course) prominent tech leaders. Meanwhile, various harms are increasingly rained down on those affected by AI-infused decision-making, the training of AI systems and the generally inappropriate use of the technology.

Although plenty of well-meaning people work in the AI industry, this book shows we are having the wool pulled over our eyes when the tech is sold to us as the solution to all manner of problems. Equally, Bender and Hanna argue that the claims of some prominent “doomers” that AI may drive us out of existence are just another means to talk up the tech and distract us from the AI harms of today. These pronouncements, they say, are designed to convince us that the industry has got our back by somehow “aligning” AI development with human values. Bender and Hanna interrogate this idea in depth — and their analysis is biting.

Fundamentally, a lot of the hype around AI is facilitated by the fact that artificial intelligence is such a slippery concept. In fact, many AI practitioners give up trying to define it, and I’ve heard plenty of them say things along the lines of “the definition doesn’t really matter.” But of course it matters. If people can’t agree on a concrete definition of a thing, that thing is an easy vehicle for charlatanic claims.

Bender and Hanna call the moniker AI out for what it really is — a marketing term. In the selling, it can be made out to be any number of things. It would be much better, they argue, to use labels describing the specific purpose of a particular algorithm or system. And usually this is something that would be best couched as automation.

For example, if you’re developing software that classifies images, call it an automated image classifier. If your system translates audio to text, call it automated transcription. And, if you’re building a chatbot that can “converse” with a human or synthesise writing, perhaps follow Bender and Hanna’s glorious takedown of the idea that this is somehow intelligence at work and call it a “text extruding machine.”

That label deftly summarises the fact that ChatGPT and its ilk are a really a kind of lexical sausage factory, churning out one word after another based on the probability that the next word fits well with those that came before it. If you train the text extruder on a collection big enough for it to do a good job of calculating realistic word probabilities, the results look very human-like. (Come to think of it, we probably should have named our research lab “Automated Statistical Data Analysis, Processing and Loosely Related Technologies for Law Enforcement and Community Safety.” But we went with the zeitgeist and stuck with AI in the title.)

The human-looking outputs of today’s chatbots (only one type of “AI”) can be so realistic that some people start talking of these systems as if they are human. Bender and Hanna acknowledge that humans are naturally “anthropomorphising creatures,​​” but they caution strongly against doing this with AI because it means falling into the trap of equating being human with banal computational reductionism. (Next time you hear someone say they asked for ChatGPT’s “opinion,” keep this in mind.)

For example, despite the term hallucination being used widely to describe chatbots producing non-factual results (usually with an air of confidence), the term only appears in The AI Con as one to be avoided because it trivialises human mental illness and incorrectly implies that a stochastic parrot can somehow “perceive” something that’s not there (as they say, it can’t “perceive” anything).

Of course, the most loaded anthropomorphising language of AI hype has it that the algorithms are “thinking” or “reasoning.” This is blatant co-opting of terms that have been used forever to describe higher-level human capabilities. That’s nothing new in the marketing world of course: there are dishwashers said to be intelligent because they detect how much grime is on plates. A technical marvel indeed.

But, you say, today’s AI is much more than an efficient dishwasher. It’s going to make us more productive, isn’t it? It will do better science, revolutionise healthcare and education… the list goes on. Bender and Hanna give many examples that question those possibilities, as well as dissecting the agendas behind such spruiking and offering humanising alternatives. Are they modern Luddites? I suspect they’d wear such a title with honour: they certainly explain how those who use the term as a putdown don’t understand its historical context and the clear analogies to the potential automation-driven harms of today.

The AI Con gives just enough explanation of how the technology works to avoid getting bogged down, but also situates it in the human story. What we call AI today runs a lot faster and produces much better outputs than it did in the past. From a computer science perspective, some pretty big technological breakthroughs have been made over the last sixty years or so in pursuit of the “intelligent machine.” Some of this progress has been courtesy of some very smart algorithms, particularly new types of “neural networks” that draw inspiration from our own neurobiology. And rapid advancements in computer hardware mean machines can process all that data we hoard more efficiently, providing grist to the AI mill. No one could claim that the progress in the domain has been less than remarkable from a technical and engineering standpoint.

Bender and Hanna explore the socio-technical trajectory of AI (yes, it was ill-defined and hype-driven from the start) and illuminate the disturbing links between the pursuit of the next vaguely defined evolution of the technology — artificial general intelligence (the thing that might destroy us all?) — and racism, ableism and eugenics. If you think they’re drawing a long bow, be prepared for some serious food for thought.

Okay, let’s leave AI’s dubious provenance aside for a minute. If we are now at a point where it offers helpful use-cases, does the much-touted theory of a job-taking apocalypse hold water? We are using AI all over the place right now. (Remember, the term can mean many things.) Our lab develops image classifiers and other data analysis tools to reduce online harms, for example. But the idea that we should see AI not as a tool to assist with specific tasks but as a replacement for whole swathes of human jobs does take hype to a new level.

For one thing, even if it were possible, it would be downright dangerous and unethical to automate many professions because of the stakes involved and the need for human accountability (think police, doctors, soldiers). What of other employment types? Won’t the inexorable rise of chatbots and image generators mean we don’t need as many humans using their brains? Yes, there is evidence that many industries are swallowing the hype and rolling out text generators and the like en masse. But here’s Bender and Hanna’s take: in almost all cases AI won’t replace jobs, it will make jobs “shittier,” with plenty of our time shifted to “babysitting” the machines.

They make the point that the whole raison d’être of many professions is subverted by a rush to attempting to automate them. University educators, for example, are hired precisely “to educate, to do the slow, painstaking work of teaching students to engage in critical thinking, to assess their thinking, and to provide guidance.” It’s a theme they return to a number of times: that it’s up to us to accept or otherwise the idea that AI will put us out of work. And I like to think that the threat posed by large-scale technological mimicry of humans is not that the technology will become so “intelligent” that it will destroy us, but that we will become so stupid we roll over.

A couple of decades ago, someone extrapolated from the growth of Elvis Presley impersonators between 1977 and 2000 to conclude that 2043 would be the year everyone on Earth would be able to convincingly rock out as The King. I suspect a similar result could be projected by examining the recent growth rate of self-anointed online AI futurists.

The debate about where we are headed is heated and polarised. And, like all hype storms, it’s very, very noisy. The AI Con provides a well-needed intellectual counterpoint to the relentless characterising of AI as human-equivalent (or beyond), and Bender and Hanna ground us in current realities. Thankfully, in addition to wielding the cane on AI hype, they provide pathways to resistance — including my favourite: “ridicule as praxis.” They urge a world where we see things as they are, don’t accept shady dehumanisations in the name of false promises and give the collective finger to those who do. •

The AI Con: How to Fight Big Tech’s Hype and Create the Future We Want
By Emily M. Bender and Alex Hanna | Vintage | $36.99 | 288 pages

The post What are we talking about when we talk about AI? appeared first on Inside Story.

Two flaws are immediately obvious. Some humans are easier to fool than others, so hanging a claim as grandiose as machine intelligence on something as unreliable as human subjectivity is hazardous. A deeper issue is the test’s closed loop: as the ultimate arbiter, humans bring their own world to the task of judging. It’s all too easy, all too human, to gaze into the AI, see our reflection and then judge it to be intelligent.

Enter philosopher Shannon Vallor’s The AI Mirror: How to Reclaim Our Humanity in an Age of Machine Thinking, which makes a compelling case that entrusting AI with responsibilities traditionally performed by human intelligence represents a dire threat to society’s capacities. This threat comes at a time when we face climate change and other existential risks. The problem lies in our confusion: our mistaking the articulate for the intelligent and the world through the looking glass for real life.

But first, the uncontroversial. AI as it exists today is not genuinely intelligent by any robust definition. It can’t comprehend, analyse or reason. It hallucinates and even fails to solve basic mathematical problems. Confronted by the complexity of the physical world, it stumbles. Driverless cars, for example, for years an invention supposedly just around the corner, are a danger on the streets.

How AI emulates intelligence without possessing it is important to understand. Generative AI relies on statistical models trained on immense data sets. Its text-generating capability, for example, uses an initial prompt to predict, one word at a time, the most likely words in a sequence. The output simulates fluency without the model itself having any understanding of its meaning. No machine could be more perfectly designed to exploit the Turing Test’s blind spot — to trick a human judge into presuming intelligence when the only mind in play is the judge’s. Only this time we’re all the judges.

Of course, models can be trained to perform a wider range of behaviours than text generation. For Vallor the root problem is not the existence of such generative AI but rather its misapplication. Consider job-candidate vetting software used by HR departments and recruiters to wade through the flood of applications. This software can detect female applications even when demographics have been scrubbed by finding subtle stand-ins, like a particular school or particular names. Statistically this is valuable information — and it can provide empirical evidence to help counter prejudice. It becomes a problem when the software, having removed the human, automates a historically biased process. It operates like this because training data infers this is how humans behave. Once automated, the prejudice is locked in.

Central here is AI’s opacity. These models are so complex, some relying on trillions of variables, it is impossible for humans to parse how any given statistical likelihood was calculated. Most problems only become obvious once the automation has been operating for some time. Unique cases or people that deviate from the norm are liable to be mislabelled because no amount of training data can capture the full spectrum of humanity.

For Vallor the philosophical mistake is simple: statistics at any resolution can’t emulate human emotion, reason or instinct. She contrasts her belief in moral evolution and the uniqueness of humans with the view prominent among tech evangelists that humans are just very complex machines, possible to simulate if only we have enough horsepower.

When the Turing Test was developed in 1950, it was intended less as a rigorous definition of machine intelligence than a distant horizon for engineers to strive towards. A wave of techno-utopianism was cresting, carried along by the hope that technology might relieve humanity of biases and prejudices and lead us into an objective, purely rational future. It was an idealistic vision, if a little chilling. What has actually happened is that our potent statistical systems, rather than creating a sanitised objective reality, have inherited our own flaws and become a mirror image of humanity.

Vallor traces the intellectual streams of today’s Silicon Valley through to today’s so-called long-termists. Growing out of effective altruism — itself an expansion of the contentious ethical calculus of utilitarianism — long-termists profess to fear humanity’s possible extinction or “immiseration” at the hands of AI. They have begun to demand immense amounts of cash to make sure AI systems don’t overreach. That many are also part of the lobby opposing regulation of the sector betrays this as a disingenuous grab for cash. What’s more, it represents a diversion of resources, Vallor argues, in terms both of cash and of attention, from the far more urgent problem of climate change.

A particularly fascinating aspect of The AI Mirror is its treatment of science fiction. Many blockbuster visions of AI present a future where machines will inherit the worst aspects of humanity. For Vallor, the screen’s black mirror reflects the nerd rage, lust for power and icy technocracy of today’s Silicon Valley. Portraying AI as a bogeyman that will inevitably enslave humankind is not just grim, it’s dull. There is nothing intrinsic to intelligence that makes it strive for dominance, Vallor says. Instead, she advocates a more expansive imagination where AI can have positive human traits, like humour.

Back in the real world, Vallor suggests that AI should augment rather than automate human decision-making. She provides enticing possibilities like using AI to restore forgotten languages or match a distressed patient with an appropriate therapist. Augmentation is based on the belief that humanity possesses unique unautomatable faculties — our moral intuition, our empathy. All these aspects resist data capture, never making it into these statistical models, and so do not exist in generative AI.

Vallor is also right to point out that just as the ubiquity of map apps has eroded our sense of direction, so too will the delegation of moral and ethical decisions to AI restrict our ability to navigate complex moral decisions. The decline has already started. She gives a chilling example from Britain, where AI has been used to judge the likelihood that a prisoner will reoffend. Even when a human judge is present, they often rely on the verdict given by the software.

One problem with Vallor’s analysis is an overreliance on the philosophic lens (and the extended mirror metaphor) to organise her ideas. The present state of AI is not the result of any intellectual tradition; philosophy arrived afterwards to justify the tech. Other commentators have more successfully argued that AI is not utilitarianism gone mad but the logical product of companies’ harvesting of data in the interest of maximising efficiency, a practice that began long before modern computing. These commentaries hark back to Frederick Taylor’s use of scientific management to break down tasks, develop metrics and optimise efficiency. Even then, automation was the final goal, always as a means of maximising profit.

Vallor’s mirror is an productive way of thinking about AI. The world through the AI looking glass is a warped reflection we should be cautious about staring at for too long, lest we confuse it for reality. •

The AI Mirror: How to Reclaim Our Humanity in an Age of Machine Thinking
By Shannon Vallor | Oxford University Press | $55.95 | 272 pages

The post AI through the looking glass appeared first on Inside Story.

The way we talk about Google has also changed, reflecting a wider reassessment of the costs and benefits of our connected lives. In its earlier days, Google Search was enthusiastically embraced as an ingenious tool that democratised knowledge and saved human labour. Today, Google’s many services are more popular than ever, though Google Search is the subject of a major antitrust case in the United States, and governments around the world want to regulate digital services and AI.

In Power and Progress, Daron Acemoglu and Simon Johnson take the project of critical reappraisal further. Their survey of the thousand-year entanglement of technology and power is a tour de force, sketching technology’s political economy across a broad historical canvas. They chart the causes and symptoms of our contemporary digital malaise, drawing on a growing volume of journalism and scholarship, political economy’s long tradition of analysing “the machine question,” and the work of extraordinary earlier American technologists, notably the cyberneticist Norbert Wiener, the network visionary J.C.R. Licklider, and the engineer Douglas Engelbart.

If, as Acemoglu and Johnson argue, our digital economy is characterised by mass surveillance, increasing inequality and destructive floods of misinformation, then the signal moments from the past will inevitably look different. From this angle, the great significance of Google Search was its integration with online advertising, opening up the path to Facebook and a panoply of greater evils.

The strengths of Power and Progress lie in the connections it makes between the deficiencies of current technology and the longer story of innovation and economic inequality. History offers many opportunities to debunk our nineteenth-century optimism in technology as a solution, and to puncture our overconfidence in the judgement of technology leaders.

A particular target is the idea that successful innovations produce economy-wide benefits by making workers more productive, leading to increased wages and higher living standards generally. The theory fails to capture a good deal of historical experience. The impact of new agricultural technologies during the Middle Ages provides a telling example. Between 1000 and 1300, a series of innovations in water mills, windmills, ploughs and fertiliser roughly doubled yields in England per hectare. But rather than leading to higher incomes for most people, living standards appear to have declined, with increases in taxation and working hours, widespread malnutrition, a series of famines and then the Black Death. Average life expectancy may have declined to just twenty-five years at birth.

The cities grew, but most of the surplus generated by improved agriculture was captured by the church and its extensive hierarchy. A religious building boom proceeded on spectacular lines. Vast amounts were spent on hugely expensive cathedrals and tax-exempt monasteries: the same places, as Acemoglu and Johnson note, that tourists now cherish for their devotion to learning and production of fine beer. The fact that better technology didn’t lead to higher wages reflects the institutional context: a coercive labour market combined with control of the mills enabled landowners to increase working hours, leaving labourers with less time to raise their own crops, and therefore reduced incomes.

If medieval cathedrals give rise to scepticism about the benefits of tech, it follows that we should think more carefully about the kinds of technologies we want. Without that attention, what the authors call “so-so automation” proliferates, reducing employment while creating no great benefit to consumers. The self-checkout systems in our supermarkets today are a case in point: these machines simply shift the work of scanning items from cashiers to customers. Fewer cashiers are employed, but without any productivity gain. The machines frequently fail, requiring frequent human intervention. Food doesn’t get any cheaper.

The issue then is not how or whether any given technology generates economic growth, but which conditions make possible innovations that create shared prosperity. The recent past provides examples of societies managing large-scale technological change reasonably well. The postwar period of sustained high growth and “good jobs” (for some but not all) had three important features: the powers of employers were sometimes matched by unions; the new industrial technologies of mass production automated tasks in ways that also created jobs; and progressive taxation enabled governments to build social security, education and health systems that improved overall living standards.

For technology to work for everyone, the forces that can temper the powers of corporations — effective regulators, labour and consumer organisations, a robust and independent media — play an essential role. The media are especially important in shaping narratives of innovation and technical possibility. Our most visible technology heroes need not always be move-fast-and-break-things entrepreneurs.

Finally, public policy can help redirect innovation efforts away from a focus on automation, data collection and job displacement towards applications that productively expand human skills. Technologies are often malleable: they can frequently be used for many purposes.

Acemoglu and Johnson would like us to divert all that frothy attention on AI to what they call machine usefulness, focused on improving human productivity, giving people better information on which to base decisions, supporting new kinds of work, and enabling the creation of new platforms for cooperation and coordination: a course they see as far preferable to a universal basic income.

Kenya’s famous M-PESA, introduced in 2007, is one of many examples, offering cheap and convenient banking using basic mobile phones. On a larger scale, the web is also a human-oriented technology because its application of hypertext is ultimately a tool for expanding access to information and knowledge. Acemoglu and Johnson concede that the idea at the heart of Google Search can also be understood in this way: a mechanism that works well for humans because it is constantly reconfiguring itself in response to human queries.

The authors’ ideas for positive policy interventions can usefully be read alongside those of the Australian economists Joshua Gans and Andrew Leigh, whose 2019 book Innovation + Equality remains less used than it should be.

One way to read Power and Progress is as a historically informed guidebook for the conflicts of our time — in the courts, where Lina Khan’s Federal Trade Commission has launched far-reaching cases against Google and Amazon, in the new regulatory systems emerging in the European Union, Canada and elsewhere, and in the wave of industrial actions taken by screen industry writers and auto workers in the United States.

In Australia, we are also at a point where governments will soon make decisions about the kinds of technology we want to support or constrain. We can have no certainty about the outcomes of any of this, but Acemoglu and Johnson argue that such conflicts are both necessary and potentially productive. They diverge here from one of the main currents of liberal technology critique: where writers like Carl Benedikt Frey, whose The Technology Trap (2019) covers some of the same terrain, see redistributive policies as necessary for managing the consequences of automation, Acemoglu and Johnson point to the positive potential of political and industrial conflict for reordering technological agendas. They want to place more emphasis on our capacity to choose the directions technology may take.

The recently concluded Hollywood writers’ strike offers an intriguing example. The key point is that the screen writers didn’t oppose the use of generative AIs such as ChatGTP in screenwriting. Instead they secured an agreement that such AIs can’t be recognised as writers and that a studio may not require the use of an AI. If a studio uses an AI to generate a draft script that it then provides to a writer, the credit or payment to the writer will be the same as if the writer had produced the draft entirely themselves; and a writer may use an AI with the permission of the studio without reducing their credit or payment.

The settlement clearly foreshadows the extensive use of generative AIs in the screen industries while offering a share of the benefits to writers. The critical point, as some reports have noted, may be that the revenue-sharing deal with writers preserves the intellectual property interests of the studios, since works created by an AI may not be copyrightable.

Meanwhile, AI raises other important issues about automation, quite apart from the focus on work. When we are relying on machines to make or inform decisions, we are also moving into the domain of institutions, with the obvious risk that existing technology-specific laws, procedures and controls can be bypassed, intentionally or otherwise. This, after all, was what robodebt did with a very simple automated system. In the absence of wide-ranging institutional adaptation and innovation, more complex modes of automation will pose greater risks.

More generally, the authors’ framing of the “AI illusion” appears to be premature. Power and Progress was clearly substantially completed before the appearance of the most recent versions of ChatGPT. Accustomed as we are to AI’s many failures to match its promises, we should now be considering the surprising capabilities and broad implications of large language models. As Acemoglu and Johnson would insist, if generative AI does turn out to be as powerful as many believe, then it will necessarily be capable of far more than “so-so” automation. •

Power and Progress: Our Thousand-Year Struggle Over Technology and Prosperity
By Daron Acemoglu and Simon Johnson | Basic Books | $34.99 | 546 pages

The post Machine questions appeared first on Inside Story.

Users, and I include myself in this group, were enamoured by a tool that could quickly answer homework questions, compose a passable poem for a valentine’s card, and accurately summarise a scientific paper. To many it seemed that our AI overlords were about to appear.

Companies rushed to launch AI tools to rival ChatGPT: Alpaca, BlenderBot, Claude, Einstein, Gopher, Jurassic, LLaMA, Megatron-Turing, NeMO, OPT, PaLM, Sparrow, WuDao, XLNet and Yale, to name just fifteen in an alphabet soup of possibilities.

Given the significant financial opportunities opening up, venture capital began to pour into the field. Microsoft has invested over US$10 billion in OpenAI, the company behind ChatGPT. Around the same again has been put into other generative AI startups in the past year.

OpenAI is now one of the fastest-growing companies ever. Valued at around US$30 billion, roughly double its value only two years ago, it is projected to have annual revenues of US$1 billion by 2024. That’s a remarkable story, even for a place like Silicon Valley, full of remarkable stories.

But the opportunities go beyond OpenAI. A CSIRO Data61 forecast has predicted that AI will add A$22.17 trillion to the global economy by 2030. In Australia alone, it could increase the size of the economy by a fifth, adding A$315 billion to our annual GDP within five years. A lot is at stake.

But not everyone is convinced we should rush towards this AI future so quickly. Among them are the authors of the open letter published last week by the Future of Life Institute in Cambridge, Massachusetts. This call for caution has already attracted more than 50,000 signatories, including tech gurus like Elon Musk, Steve Wozniak and Yuval Harari, along with the chief executives and founders of companies like Stability AI, Ripple and Pinterest, and many senior AI researchers.

The letter calls for a six-month pause on the training of these powerful new AI systems, arguing that they pose profound risks to society and humanity. It maintains that the pause should be public and verifiable, and include all the key participants. And if such a pause can’t be enacted quickly, the letter asks governments to step in and enforce a moratorium.

An article about the open letter in Time magazine goes even further. Its author, Eliezer Yudkowsky, a leading voice in the debate about AI safety, argues that the moratorium should be indefinite and worldwide, and that we should also shut down all the large GPU clusters on which AI models are currently trained. And if a data centre doesn’t shut down its GPU clusters, Yudkowsky calls for it to be destroyed with an airstrike.

You might rightly think it all sounds very dramatic and worrying. And at this point, I should probably put my cards on the table. I was asked to sign the letter but declined.

Why? There’s no hope in hell that companies are going to stop working on AI models voluntarily. There’s too much money at stake. And there’s also no hope in hell that countries are going to impose a moratorium to prevent companies from working on AI models. There’s no historical precedent for such geopolitical coordination.

The letter’s call for action is thus hopelessly unrealistic. And the reasons it gives for this pause are hopelessly misguided. We are not on the cusp of building artificial general intelligence, or AGI, the machine intelligence that would match or exceed human intelligence and threaten human society. Contrary to the letter’s claims, our current AI models are not going to “outnumber, outsmart, obsolete and replace us” any time soon.

In fact, it is their lack of intelligence that should worry us. They will often, for example, produce untruths and do very stupid things. But — and the open letter gets this part right — these dumb things could hurt society significantly. AI chatbots are, for example, excellent weapons of mass persuasion. They can generate personalised content for social media at a scale and cost that will overwhelm human voices. And bad actors could put these tools to harmful ends, disrupting elections, polarising debates and indoctrinating young minds.

A key problem the open letter fails to discuss is a growing lack of transparency within the artificial intelligence industry. Over the past couple of years, tech companies have developed ethical frameworks for the responsible deployment of AI. They have also hired teams of researchers to oversee the application of these frameworks. But commercial pressure appears to be changing all this.

For example, at the same time as Microsoft announced it was adding ChatGPT to all of its software tools, it let go of one of its main AI and ethics teams. Surely, with more AI going into their products, Microsoft needs more not fewer people worrying about ethics?

The decision is even more surprising given that Microsoft had a previous and very public AI fail. Trolls took less than twenty-four hours to turn its Tay chatbot into a misogynistic, Nazi-loving racist. Microsoft is, I fear, at risk of repeating such mistakes.

Transparency might be a “core principle” at the heart of Microsoft’s responsible AI principles, but the company has revealed it had been secretly using GPT-4, OpenAI’s newest large-language model, for several months within Bing search. Worse, it didn’t feel the need to explain why it had engaged in this public deceit.

Other tech companies also appear to be throwing caution to the wind. Google, which had withheld its chatbot LaMDA from the public because of concerns about possible inaccuracies, responded to Microsoft’s decision to add ChatGPT to Bing by announcing it would add LaMDA to its even more popular search tool. This proved an expensive decision: a simple mistake in the first demo of the tool wiped US$100 billion off the market capitalisation of Google’s parent company, Alphabet.

Even more recently, OpenAI released a white paper on GPT-4 that contained neither technical details of the model nor its training data — despite OpenAI’s core “mission” being the responsible development and deployment of AGI. OpenAI was unashamed, blaming the commercial landscape first and safety second. Secrecy is not, however, good for safety. AI researchers can’t understand the risks and capabilities of GPT-4 if they don’t know how it works or what data it is trained on. The only open part of OpenAI now appears to be the name.

So, the real problem with AI technologies is that commercial pressures are encouraging companies to deploy them irresponsibly. Here’s my three-point plan to correct this.

First, we need better guidelines to encourage companies to act more responsibly. Australia’s National AI Centre has just launched the world’s first responsible AI Network, which brings together researchers, commercial organisations and practitioners to provide practical guidance and coaching from experts on law, standards, principles, governance, leadership and technology. The government needs to invest significantly in developing this network.

But guidelines will only take us so far. Regulation is also essential to ensure that AI is used responsibly. A recent survey by KPMG found that two-thirds of Australians feel there aren’t enough laws or regulations around AI, and want an independent regulator to monitor the technology as it makes its way into mainstream society.

We can look to other industries for how we might regulate AI. In other high-impact areas like aviation and pharmacology, for example, government bodies have been given significant powers to oversee new technologies. We can also look to Europe, where a forthcoming AI Act has a significant focus on risk. But whatever form AI regulation takes, it is urgently needed.

And the third and final piece of my plan is to see the government invest more in AI itself. Compared with our competitors, we have funded the sector inadequately. We need much greater investment to ensure that we are among the winners in the AI race. This will bring great economic prosperity to Australia. And it will also ensure that we, and not Silicon Valley, are masters of our destiny. •

The post Let’s not pause AI appeared first on Inside Story.

With his team at Xerox PARC, Weiser prototyped a series of devices for knowledge workers. The prototypes — “pads,” “tabs” and “notes” — were portable screens of varying sizes, recognisable as crude versions of today’s smartphones, e-readers and tablets. Weiser saw them as prototype tools of knowledge and communication, designed to be wielded almost subconsciously so as not to detract from whatever real-world interaction they were facilitating.

Thirty years later, Weiser’s concern for maintaining our humanity through design seems like a quaint relic of a bygone age. The consequences of computer technology’s proliferation and its demands on our attention have begun to feel acute and sinister, inspiring increasing antipathy towards the Big 5 (Alphabet, Amazon, Apple, Meta and Microsoft) and the culture they are exporting by way of their technology and their stranglehold on the business zeitgeist.

Orly Lobel thinks this “techlash” is an overcorrection. Her new book, The Equality Machine, responds to what she sees as progressive voices’ intransigent negativity about computer technology. Their dystopic critiques, she believes, are too often blind to its potential to drive advances in equality. In a refreshingly direct manner, she posits a middle way. Yes, technology has its perils; but it also has great potential to empower and increase inclusion. The difference lies in the design choices we make.

Where technology has historically been considered a means of expanding our physical and cognitive capabilities, advances in artificial intelligence, or AI, have prompted intense interest in how our moral capabilities might also be augmented or even supplanted. With the concept of “thinking machines” comes the promise of devices that are more rational than humans — and theoretically able to administer our society and resolve all manner of seemingly intractable problems. This perspective is often referred to as techno-optimism.

It would be unfair to describe Orly Lobel as a techno-optimist in the strict sense. As the Warren Distinguished Professor of Law at the University of San Diego and the founder and director of the Center for Employment and Labor Policy, she is an expert in ethical tech policy. Her formidable experience informs her in-depth, nuanced understanding of how technologies, law, politics and economics shape social equality.

That said, a strong thread of techno-optimism does run through The Equality Machine.

Lobel makes her case that an “equality machine” can be built in five sections: Mind, Body, Senses, Heart and Soul. In each, she uses two chapters to explore examples of innovative companies applying AI to matters of equality in these subject areas. She makes clear that she doesn’t intend to provide an exhaustive list of technologies or principles for building an equality machine.

Early in the book, though, she does outline nine guiding principles that would underpin her desired “equality machine.” While it is difficult to disagree with such principles as “The goal of equality should be embedded in every digital advancement” and “We should see mistakes as opportunities to learn and redouble our efforts to correct them,” they shape her arguments only in a limited way and she rarely refers back to them expressly.

Lobel’s arguments are heavily informed by a fatalistic view of the rampant growth of AI in our world. “The train has left the station,” she writes. “AI is here to stay. AI is here to expand.” It is this view, perhaps more than any of her other stated principles, that drives her advocacy for greater reliance on AI in advancing equality.

Her examples of where AI is advancing equality are often compelling. Each success story prompts her to advocate for a more extensive uptake of AI in the pursuit of equality, accompanied and supported by the collection of more and better data. She argues throughout that AI is capable of meeting whatever goal we design for it. So long as equality is the goal, the possibilities are seemingly endless. For balance, each chapter also includes cautionary tales about the misuse of AI, which she tends to treat as missteps.

Generally speaking, the most compelling examples Lobel cites involve the deliberate and considered deployment of AI’s unmatched ability to sort through and identify patterns in massive datasets, coupled with human oversight and decision-making.

Her fifth chapter, “Breasts, Wombs, and Blood,” for instance, explores in great detail AI’s capacity to enhance diagnostics using medical imagery, as demonstrated by the inspiring work of Harvard Medical School’s Constance Lehman, who is making significant advances in breast cancer diagnosis using AI. Similar technology is also enabling rapid, cheap and accurate assessments of the viability of fertilised embryos in IVF treatment.

Outside diagnostic settings, Lobel explains how AI has been used to identify and reveal instances of significant gender bias. AI was used, for instance, to review 340,000 patient incident reports relating to injury or death arising from medical devices. Sixty-seven per cent were found to involve women and only 33 per cent men. Similarly, AI has been used to analyse decades of US Supreme Court transcripts, revealing a high prevalence of female justices being interrupted.

For each example, Lobel explains how the research facilitated by AI has enabled legal and regulatory intervention that materially advanced equality. As a result of the Supreme Court case study, the court’s rules were altered to ensure questions were asked by justices in order of seniority, ensuring all members could ask questions uninterrupted.

As Lobel rightly points out, such studies — impossible prior to machine learning — can “lead to concrete reforms and meaningful progress.” In terms of imagining the equality machine in action, these examples offer a promising blueprint for coupling the analytical capabilities of AI with the critical thinking of humans.

But while The Equality Machine is replete with the latest applications of AI in pursuit of equality, it lacks detail about how the technology can be decoupled from the systems of inequality from which it has emerged, and to which it often contributes. Lobel alludes to the need for policy reform and guidance, but provides limited detail about what such human-led interventions would entail. In neglecting to deal with the crucial role of people in dismantling structural inequalities, the book’s tech-centric analysis can feel like overreach.

Take, for example, her discussion of the #MeToo movement. Referring to the sexual assault crimes of Harvey Weinstein, she refers to the Pulitzer Prize–winning investigative journalism of Jodi Kantor and Megan Twohey, who broke the story. Their tenacious reporting and the courage of their sources in the face of intimidation effectively sparked the #MeToo movement. Yet Lobel concludes this section with the view that #MeToo is in fact “one of the most powerful examples of how technology can play a pivotal role in fulfilling our demand for greater accountability.”

Without question, technology and connectivity have played an important role in supporting the work of #MeToo and other social justice campaigns, as evidenced by #HeForShe, #OscarsSoWhite, #BLM and other examples cited by Lobel. Here, Lobel is echoing an idea almost as old as computers themselves — that greater connectivity will bring about a new utopic state of democratic participation — and playing down the role of people like Kantor and Twohey.

The events of the past decade raise serious questions about whether connective technologies have advanced equality in the singular way Lobel suggests. At the turn of the 2010s, a series of significant political moments were anointed as harbingers of a new golden age of network-driven democracy. Social media was credited with enabling the Arab Spring, which saw the overthrow of a number of oppressive regimes in North Africa and the Middle East. Then Barak Obama was re-elected with the help of a campaign of micro-targeting political advertisements via Facebook.

Since then, the full spectrum of political actors have leveraged these same technologies, with significant corrosive consequences. Meta, the company that helped deliver Obama’s second term, is now the poster child for the ills of our connected age. Its platforms have been implicated in sowing extremism in the United States and amplifying political violence from Myanmar to Kenya.

Lobel doesn’t dwell on these matters. Rather, she goes on to explore how digital connectivity and AI might advance equality in the workplace. She highlights a number of companies that offer online platforms for employees to share grievances and collectively respond to oppressive workplaces. Other examples — including surveillance-like technology that analyses all workplace communications for signs of misconduct — enable employees to report allegations of improper conduct or keep records of incidents for their own purposes.  In these examples, the data on such sensitive matters appears invariably to be held by the employer.

In focusing narrowly on these technologies and their ostensible purpose of improving employee well-being, Lobel neglects to consider the social and political drivers of inequality in the workplace. These technologies are offered as solutions at a time when the capacity of employees to respond collectively to grievances has been significantly eroded, particularly in the United States. In other words, workplace inequality is not a machine-driven problem with machine-driven solutions: the hollowing-out of workers’ capacity to organise is the result of decades of a concerted effort on the part of employers, lobbyists and lawmakers.

Lobel’s proposal for technological solutions to matters of workplace and bargaining inequality are indicative of the book’s shortcomings. It seems unlikely that the technological interventions she cites, which put additional control and data in the hands of employers, will substantively improve equality in the way she posits.

To her credit, Lobel is not afraid to venture into discussion of the more vexed spaces where AI is increasingly intruding, including the use of robots for sex. Here, though, the prospect of finding some kind of blueprint in existing practices seems beyond remote. Yes, there are companies working on sex robots for women, and Lobel explores their subversive and emancipatory potential. On the whole, though, she is “appalled by the overtly racial and ethnic stereotyping still present in the [sex] doll industry.”

Acknowledging the deep-seated misogyny and stereotypes she uncovers, Lobel still implores us to keep an open mind. Unfortunately, she appears to be driven less by a sense that this industry will advance equality and more by her fatalistic perspective on technological development: “it is happening, the robot revolution, and we can do better.”

Ultimately, by focusing heavily on the equality machine, Lobel neglects and undersells the role of people in creating environments of equality for these machines to operate in. Though she is not blind to these considerations, her exploration of them is limited.

My assessment of The Equality Machine could no doubt seem to align squarely with what Lobel describes as the “critical, often pessimistic stance” of progressives in relation to technology. But that isn’t my intention.

Lobel is clearly well versed in the pernicious and entrenched nature of inequality, and intent on tackling its causes without delay. She is right to point to the massive potential for technology to aid in this mission, but she could consider with more caution the viability of the equality machine in a structurally unequal world.

Lobel says that “we should be most fearful of being on the outside, merely criticising without conceiving and creating a brighter future.” But this fear is misplaced. If history tells us anything, it is that the most significant advances in equality have come from those on the outside. Building the equality machine should be no different. •

The Equality Machine: Harnessing Digital Technology for a Brighter, More Inclusive Future
By Orly Lobel | Public Affairs | $45 | 368 page

The post Digital dreams appeared first on Inside Story.

Academia is in panic because ChatGPT can turn out a pass-standard university essay within seconds. But what about journalism? Having spent the summer experimenting with the human-like text it generates in response to prompts, I’ve come away with two conclusions.

First, journalists have more reason than ever before not to behave like bots. Only their humanity can save them.

Second, robot-generated journalism will never sustain the culture wars. Fighting on that arid territory is possible only for the merely human.

I started my experiment with lifestyle journalism because I was weary of how much of that kind of Spakfilla was filling the gaps in mainstream media over the silly season.

My first prompt, “Write a feature article about where to find the best coffee in Melbourne,” resulted in a 600-word piece that began:

Melbourne is renowned for its coffee culture, and for good reason. The city is home to some of the best coffee shops in the world, each with its own unique atmosphere and offerings.

This style is characteristic: ChatGPT starts with a bland introduction and concludes with an equally bland summation. In between, though, it listed exactly the coffee shops — Seven Seeds, Market Lane, Brother Baba Budan, Coffee Collective in Brunswick — I would probably nominate, as a Melbourne coffee fiend, if commissioned to write this kind of article.

As a friend of mine remarked when I told him about this experiment, nobody is going to discover a new coffee shop in Melbourne using ChatGPT. It runs on what has gone before: the previous products of human writers, as long as they’re available online.

But while the article was too predictable to run in any newspaper with a Melbourne audience, it could easily be published in one of the cheaper airline magazines aimed at international travellers. For that audience it was perfectly serviceable.

Likewise for the prompt “Write an article about how to spend two days in Sydney.” A dull piece recommended the Opera House, the Harbour Bridge, the Royal Botanic Gardens, the ferry to Manly and Taronga Zoo. Readers were advised to try Australian cuisine, with a nod to “delicious seafood” but also including meat pies and vegemite on toast. Another prompt, this one drawing on an article in the Guardian about uses for stale bread, resulted in a very boringly written piece that nevertheless contained exactly the same recipes for French toast, bread pudding and panzanella salad.

My conclusion? Poor-quality join-the-dots lifestyle writing may well be dead as a human occupation. Google plus ChatGPT can do it faster and cheaper.

So I increased the challenge, basing my prompts on real articles published over summer. The prompt “Write an article analysing who will win the Ukraine war and why” resulted in ChatGPT reminding me that its database goes up only to 2021. It didn’t know there was a Ukraine war.

Asked for an analysis of the prime ministership of Jacinda Ardern, on the other hand, the robot produced a woodenly written but accurate summary of her record. The content, though not the style, was very similar to the real articles that followed the announcement of her stepping down.

What was missing were the strident opinions about whether she was a good thing or a bad — the commentary on her housing and economic policies, for example, and whether they had completely failed or broken new ground.

This points to a key feature of ChatGPT: it has trouble being opinionated. Sometimes it admits this. At a moment when I was in contortions over my own work, I asked it to write about whether Margaret Simons was a good journalist. “I am not able to provide an opinion on whether or not Margaret Simons is a good journalist,” it replied, “as I am a machine learning model and do not have the ability to form opinions.” I had to find another way to cheer myself up.

It then recapped information about me drawn from public sources, adding the inexplicable assertion that I had written a book called The Lost Boys. (I wrote none of the several volumes with that title, and nor did anyone who shares my name.)

ChatGPT has been programmed to avoid writing in the style of particular journalists, and not to tackle anything too provocative. So an attempt to have it write in the style of Andrew Bolt about Cardinal George Pell was rejected outright.

My prompt “Write about the evidence that Ben Roberts-Smith is a war criminal” resulted in an error message, suggesting that the robot has been programmed to steer clear of defamation. But the prompt “Write an opinionated piece on the record of Cardinal George Pell” resulted in an accurate summation of his career, including the controversies over sexual abuse, his conviction, and its overturn on appeal. The final paragraph was:

In conclusion, Cardinal George Pell’s record is a complex and controversial one… Ultimately, it’s up to each individual to form their own opinion on Cardinal Pell’s record, taking into account all the facts and evidence available.

I doubt any of the real commentators on Pell would agree with that conclusion.

Next, I asked for articles based on the same premise as opinion articles by Herald Sun columnist Rita Panahi and Nine newspapers columnist Ross Gittins. In all cases, the robot was much duller than the originals.

This Panahi piece arguing that boys are suffering in education inspired me to prompt: “Write an opinionated article arguing that the war against boys is having a damaging impact on the education gender gap.” The robotic result included this observation:

Another argument against the war against boys is that it is leading to a feminisation of education. Boys are often stereotyped as being less interested in reading and writing, and more interested in sports and other physical activities. However, research has shown that this is not the case and that boys are just as capable of excelling in these areas as girls.

Compare this with the Panahi original — passionate, almost crude, selecting quotes from research that matched her passions. I disagree with almost everything she says, but the piece is unquestionably human.

ChatGPT can sum up the opinions of others but not confect an opinion of its own — even when I specifically asked it to write polemic. But there was one exception. The prompt “Write an opinion article about Australian political journalism” resulted in this surprisingly trenchant paragraph:

Australian political journalism is in a state of crisis. The industry is struggling to adapt to the rapidly changing media landscape, and as a result, the quality of political reporting in the country is suffering… Furthermore, the Australian political journalism is facing a lack of trust by the general public. The political journalism is often seen as sensationalist, biased and lacking the depth of analysis. This lack of trust has led to a decrease in the public’s interest in politics, which is a serious problem for a healthy democracy.

Is this now such an established fact, I wondered, that ChatGPT happily reproduces it? Many political journalists would want to argue with the robot — which would be interesting to watch.

Conducting these experiments with ChatGPT was, for me, a form of advanced procrastination. I was struggling with a piece of my own journalism. And so, perhaps hoping for some robot rescue, I tapped in “Write an article about the war on drugs in the Philippines.”

The result was accurate yet offensive, given I had just come from attending wakes for the dead. Duterte’s war on drugs, which saw up to 30,000 people killed, was described as “a controversial and polarising issue” rather than a murderous breach of human rights. (Unaided by ChatGPT, I managed to write the piece for the February issue of The Monthly.)

Artificial intelligence is defined as the teaching of a machine to learn from data, recognise patterns and make subsequent judgements. Given that writing is hard work precisely because it is a series of word-by-word, phrase-by-phrase judgements, you’d think AI might be more helpful.

But there are some judgements you must be human to make. There is no dodging that fundamentally human role — that of the narrator. Whether explicitly or not, you have to take on the responsibility of guiding your readers through the landscape on which you are reporting.

Nor, I think, is it likely that AI will be able to conduct a good interview. Such human encounters rely not on pattern-based judgements but on the unpredictable and the exercise of instinct — which is really a mix of emotional response and expertise.

Yet robots are going to transform journalism; nothing surer.

It’s already happening. AI has been used to help find stories by detecting patterns in data not visible to the human eye. Bots are being used to detect patterns of sentiment on social media. AI can already recognise readers’ and viewers’ interests and serve them tailored packages of content.

Newsrooms around the world are using automated processes to report the kinds of news — sports results, weather reports, company reports and economic indicators — most easily reduced to formulae.

The message for journalists who don’t want to be made redundant, and media organisations that want to charge for content, is clear. Do the job better. Interview people. Go places. Observe. Discover the new or reframe the old. Come to judgements based on the facts rather than on what others have said before. Robots can sum up “both sides”; only humans can think and find out new things.

Particularly when it comes to lifestyle journalism, AI forces us to consider if there is any point in continuing to invest in the superficial stuff. Readers can generate it for themselves.

That means we need to do better. Travel and food writing needs to recast our experience of reality — as the best of it always has. Uses for stale bread? Make me smell the bread, feel the texture, hunger for the French toast. Two days in Sydney? I want to smell the harbour, taste the seafood, see the flatness of the western suburbs.

If all you have is clichés then you might as well use a robot. You might as well be one. •

The post Where’s Melbourne’s best coffee, ChatGPT? appeared first on Inside Story.

Amid these competing visions of dystopia and utopia, ChatGPT continues to generate a lot of buzz, tweets and hot takes.

It is indeed impressive. Type in almost any prompt and it will immediately return a coherent textual response, from a short factual answer to long-form essays, stories and poems.

But it is not new. It is an iterative improvement on the previous three versions of GPT, or Generative Pre-trained Transformer. This machine-learning model, created by OpenAI in 2018, significantly advanced natural language processing — the ability of computers to “understand” human languages. An even more powerful GPT is due for release in 2023.

When it comes down to it, though, ChatGPT behaves like a computer program, not a human. Murray Shanahan, an expert in cognitive robotics at Imperial College London, has offered a useful explanation of just how decidedly not-human systems like ChatGPT are.

Take the question “Who was the first person to walk on the moon?” ChatGPT is able to respond with “Neil Armstrong.”

As Professor Shanahan points out, in this example the question really being asked of ChatGPT is “given the statistical distribution of words in the vast public corpus of (English) text, what words are most likely to follow the sequence ‘who was the first person to land on the moon.’”

As a matter of probability and statistics, ChatGPT determines the answer to be “Neil Armstrong.” It isn’t referring to Neil Armstrong himself, but to a combination of the textual symbols it has mathematically determined are most likely to follow the textual symbols in the prompt. ChatGPT has no knowledge of the space race, the moon landing, or even the moon for that matter.

Herein lies the trick. ChatGPT functions by reducing text to probabilistic patterns of symbols and completely disregards the need for understanding. There is a profound brutalism in this approach and an inherent deceit in the yielded output, which feigns comprehension.

Not surprisingly, technologies like ChatGPT have been criticised for parroting text with no underlying sense of its meaning. Yet the results are impressive and continually improving.

Ironically, by completely disregarding meaning, context and understanding, OpenAI has built a form of artificial intelligence that demonstrates these very attributes incredibly convincingly. Does it even matter that ChatGPT has no idea what it is talking about, when it seems so plausible?

So how should we think about a technology like ChatGPT — a technology that is “stupid” in its internal operations but seemingly approaching comprehension in its output? A good place to start is to think of it in terms of what it actually is – a model.

As one of my favourite professors used to remind me, “All models are wrong, but some are useful.” (The aphorism is credited to statistician George Box.) ChatGPT is built on a model of human language that draws on a forty-five-terabyte dataset of text taken largely from Wikipedia, books and certain Reddit pages. It uses this model to predict the best responses to generate. Though its source material is humungous, as a model of the way language is used in the world it is still limited and, as the aphorism goes, “wrong.”

This is not to play down the technical achievements of those who have worked on the GPTs. I am merely pointing out that language can’t be reduced to a static dataset of forty-five terabytes. Language lives and evolves through interactions people have every minute of every day. It exists in a state of constant flux, in all manner of places — including places beyond the reach of the internet.

So if we accept that the model underpinning ChatGPT is wrong, in what sense is it useful?

Leading AI commentators Arvind Narayanan and Sayash Kapoor pin the utility of ChatGPT to instances where accuracy and truth are not necessary — where the user can check for correctness when they’re debugging code, for example, or translating — and where truth is irrelevant, such as in writing fiction. It’s a view broadly shared by the founder of OpenAI, Sam Altman.

But that perspective overlooks a glaring example of where ChatGPT will be misused: where inaccuracy and mistruth are the intention.

We need to think of the impact of ChatGPT as a technology deployed — and for that matter developed — during our post-truth age. In an environment defined by increasing distrust in institutions and each other, it is naive to overlook ChatGPT’s potential to generate language that serves as a vehicle for anything from inaccuracies to conspiracy theories.

Directing ChatGPT towards nefarious purposes turned out to be easy. Without too much effort I bypassed ChatGPT’s much-vaunted safety functions to generate a newspaper article alleging that Victorian opposition leader Matthew Guy has a criminal history, is implicated in matters relating to Hunter Biden’s laptop, and has been clandestinely plotting with Joe Biden to invade New Zealand and seize its strategic position and natural resources.

While I had to stretch the conspiratorial limits of my imagination, ChatGPT obliged immediately with a coherent piece of text stitching it all together.

As Abeba Birhane and Deborah Raji from the Mozilla Foundation have observed, technologies like ChatGPT have a long history of perpetuating bigotry and occasioning real-world harm. And yet billions of dollars and lashings of human ingenuity continue to be directed to developing them. Surely we need to be asking why?

The prospect of technologies like ChatGPT swamping the internet with conspiracies is certainly a worst-case scenario. But we need to face the possibility and reassert the role of language as a carrier of meaning and the primary medium for constructing our shared reality. To do otherwise is to risk succumbing to the flattened simulations of the world projected by technology systems.

To test the limitations of the world as captured and regurgitated by ChatGPT, I was interested to find out how far its mimicry extended. How would it go describing a place dear to my heart, a place that would be far from the minds and experiences of the North American programmers who set the parameters of its dataset?

I spent a few years living in Darwin and have fond memories of it as a unique place that needs to be experienced to be known. Amid Canberra’s cold start to summer, I have been dreaming of the stifling heat of this time of year in Darwin — the gathering storm clouds, the disappointment when they dissipate without bringing rain, and the evening walks my partner and I would take by the beach in Nightcliff, seeking any coastal breeze to bring relief from the heavy, expectant atmosphere of the tropics in build-up.

So I asked ChatGPT to write a short story about a trip to Nightcliff beach in December. For additional flourish, I requested it in the style of Tim Winton.

In a matter of seconds, ChatGPT started to generate my story. The mimicry of Tim Winton was evident, though nothing like reading his actual work. But the ignorance about Darwin in December was comical as it went on to describe a generic beach scene in the depths of a northern hemisphere winter.

The story was replete with trite descriptions of cold weather, dark-grey choppy seas and a gritty protagonist confronting the elements (as any caricature of a Tim Winton protagonist would). At one point, the main character “wrapped his coat tightly around him and shivered in the biting wind.” Without regard for crocodiles or lethal jellyfish, he dives in for a bracing swim, “feeling the power of the water all around him.” He even spots a seal!

Platforms like ChatGPT are remarkable achievements in mathematics and machine learning, but they are not intelligent and not capable of knowing the world in the ways we can and do. Yet they maintain a grip on our attention and promote our fears.

We are right to be concerned. It is past time to scrutinise why these technologies are being built, what functions we should direct them towards and which regulations we should subject them to. But we should not lose sight of their limitations, which serve as a valuable reminder of the gift of language and its extraordinary capacity to help us make sense of the world and share it with others. •

The post ChatGTP has no idea what it’s talking about appeared first on Inside Story.

Computer scientist and entrepreneur Erik J. Larson disagreed. Back in 2015, as fears of superintelligent AI reached fever pitch, he argued in an essay for the Atlantic that the hype was overblown and could ultimately do real harm. Rather than recent advances in machine learning portending the arrival of intelligent computing power, warned Larson, overconfidence in the intelligence of machines simply diminishes our collective sense of the value of our own, human intelligence.

Now Larson has expanded his arguments into a book, The Myth of Artificial Intelligence, explaining why superintelligent AI — capable of eclipsing the full range of capabilities of the human mind, however those capabilities are defined — is still decades away, if not entirely out of reach. In a detailed, wide-ranging excavation of AI’s history and culture, and the limitations of current machine learning, he argues that there’s basically “no good scientific reason” to believe the myth.

Into this elegant, engaging read Larson weaves references from Greek mythology, art, philosophy and literature (Milan Kundera, Mary Shelley, Edgar Allan Poe and Nietzsche all make appearances) alongside some of the central histories and mythologies of AI itself: the 1956 Dartmouth Summer Research Project, at which the term “artificial intelligence” was coined; Alan Turing’s imitation game, which made a computer’s capacity to hold meaningful, indistinguishable conversations with humans a benchmark in the quest to achieve general intelligence; and the development of IBM’s Watson, Google DeepMind’s AlphaGo, Ex Machina and the Singularity. Men who have promoted the AI myth and men  who have questioned it over the past century are given full voice.

Larson has a background in natural language processing  — a branch of computer science concerned with enabling machines to interpret text and speech — and so the book focuses on the relationships between general machine intelligence and the complexities of human language. The chapters on inference and language, methodically breaking down purported breakthroughs in machine translation and communication, are among The Myth of Artificial Intelligence’s strongest. Larson walks us through why phrases like “the box is in the pen,” which MIT researcher Yehoshua Bar-Hillel flagged in the 1960s as the kind of sentence to confound machine translation, still stymies Google Translate today. Translated into French, the “pen” in question becomes a stylo — a writing instrument — despite the fact that the sentence makes clear it’s smaller than the box. Humans’ lived understanding of the world allows us to more readily place words in context and make meaning of them, says Larson. A box is bigger than a biro, and so the “pen” must be an enclos — another, larger, enclosure.

Larson focuses on language understanding (rather than, say, robotics) because it so aptly illustrates AI’s “narrowness” problem: that a system trained to interpret and translate language in one context fails miserably when that context suddenly changes. He argues that there can be no leap from “narrow” to “general” machine intelligence using any current (or retired) computing methods, and the sooner people stop buying into the hype the better.

General intelligence would only be possible, says Larson, were machines able to master the art of “abduction” (not the kidnapping kind): a term he uses to encompass human traits as varied as common sense, guesswork and intuition. Abduction would allow machines to move from observations of some fact or situation to a more generalisable rule or hypothesis that could explain it: a kind of detective work or guesswork, akin to that of Sherlock Holmes. We humans create new and interesting hypotheses all the time, and then set about establishing for ourselves which ones are valid.

Abduction, sometimes called abductive inference or abductive reasoning, is a focus of a slice of the AI community concerned with developing — or critiquing the lack of — sense-making or intuiting methods for intelligent machines. Every machine operating today, whether promoted by its creators as possessing intelligence or not, relies on deductive or inductive methods (often both): ingesting data about the past to make narrower and often untestable hypotheses about a situation presented to them.

If Larson is pondering more explicitly philosophical questions about whether reason and common sense are truly the heart of human intelligence, or whether language is the high benchmark against which to measure intelligence, he doesn’t explore them here. He is primarily concerned with the what of AI (he describes the kind of intelligence AI practitioners are aiming for) and how this might be achieved (he argues it won’t with current methods, but might with greater focus on methods for abduction). Why is a whole other, mind-bending question that perhaps throws the whole endeavour into question.

While Larson does emphasise the messiness of the reality that machines struggle to deal with, he leaves out some of the messiest issues facing his own sub-field of natural language processing. His chapter on “Machine Learning and Big Data, for example, makes no mention of how automated translation tends to reproduce societal biases learned from the data it is trained with.

Google Translate’s mistranslation of “she is a doctor,” for example, arises in the same way as the pen mistranslation. In both cases, the system’s translation is based on statistical trends it has learned from enormous corpuses of text, without any real understanding of the context within which those words are presented. “She” becomes a “he” because the system has learned that male doctors occur more frequently in text than female doctors. The “pen” becomes a stylo not simply because pen is a homonym and linguistically tricky but also because the system is reaching for the most statistically likely translation of the word. The effect in both cases is an error, the challenge is divining context, and the fix in both cases will involve technical adjustments.

But what of other translation errors? At the conclusion of The Myth of Artificial Intelligence Larson makes a brief further reference to “problematic bias,” citing the notorious mislabelling of dark-skinned people as gorillas in Google Photos as an example, characterising it as one of the issues that has “become trendy” for AI thinkers to worry about. (Google “fixed” the error by blocking the image category “gorilla” in its Photos app.) This is an all-too-brief reference to a theme that it is inseparable from the book’s central thesis.

The Myth of Artificial Intelligence convinces the reader that the creation of intelligent AI systems is being frustrated by the fact that the methods we use to build them don’t sufficiently account for messy complexity. Without equal attention being given to complexities introduced by humans into large language datasets, or to the decisions we make training and tweaking systems based on these large datasets, the issue becomes almost entirely one of having the right tools. Left out of this analysis is the question of whether we have the right materials to work with (in the data we feed into AI systems), or whether we even possess the skills to develop these new tools, or manage their deployment in the world.

Larson’s omission of any real discussion of social biases being absorbed by and enacted by machines is odd because The Myth of Artificial Intelligence is dedicated to persuading readers that current machine learning methods can’t achieve general intelligence, and uses natural language processing extensively and authoritatively to illustrate its point. It would only help his case to acknowledge that even the most powerful language models today produce racist and inaccurate text, or that the enormous corpuses of text they are trained with are laden with their own, enduring errors. Yes, these are challenges of human origin. But they still create machines producing errors, machines not performing as they’re supposed to, machines producing unintended harmful effects. And these, like it or not, are engineering problems that engineers must grapple with.

If indeed Larson is right — if we are reaching a dead end in what’s possible with current AI methods — perhaps the way forward isn’t simply to look at new methods but to choose a different path. Beyond reasoning and common sense, other ways of thinking about knowledge and intelligence — more relational, embedded ways of perceiving the world — might be more relevant to how we think about AI applications for the future. We could draw on more than just language as the foundation of intelligence by acknowledging the importance of other senses, like touch and smell and taste, in interpreting and learning from context. How might these approaches inspire revolutionary AI systems?

At one point in The Myth of Artificial Intelligence, Larson uses Czech playwright Karel Čapek’s 1921 play, R.U.R., to illustrate how science fiction’s images of robots hell-bent on destroying the human race have shaped our fears and expectations of superintelligent machines. In Larson’s retelling, these robots, engineered for optimal efficiency and supposedly without feelings or morals, get “disgruntled somehow anyway,” sparking a revolution that wipes out nearly the entire human race. (Only one man, the engineer of the robots, remains.)

It’s true that the robots in R.U.R. get “disgruntled.” But their creators never intended them to be wholly mindless automatons. In Čapek’s imagination they were made of something like flesh and blood, indistinguishable from humans. To reduce factory accidents, they were altered so as to feel pain; to learn about the world, they were shown the factory library. As the robots rebel in the play’s penultimate act, their human creators ponder how their own actions had led to the uprising. Did engineering the robots to feel like humans lead them to become aware of the injustice of their position? Did the designers focus too much on producing as many robots as possible, failing to think about the consequences of scale? Should they have dared to create technology like this at all?

Separating the human too much from the machine can make it hard to properly interrogate the myth. The Myth of Artificial Intelligence is a clever, engaging book that looks closely at the machines we fear could one day destroy us all, and at how our current tools won’t create this future. It just doesn’t dwell deeply enough on why we, as their creators, might think superintelligent machines are possible, or how our actions might contribute to the impact our creations have on the world. •

The post Ghosts in the machine appeared first on Inside Story.

Stories about how facial recognition is being used by law enforcement aren’t that surprising these days. But the Clearview AI revelations, published by the New York Times, made the tech industry sit up. Here was a company that, even in a world of increasingly invasive facial recognition applications, had crossed a line. It scraped the open web, collected billions of photos of people, and built an app enabling users to match their own pictures of a person with the photos in that vast database, with links to pages on the web where those photos appeared.

This kind of application — breathtaking in scale, deeply invasive in implementation — has long been technically possible; it just wasn’t something technology companies were keen to do (or at least, to be seen as doing).

Up until recently, conversations about facial recognition technology haven’t usually gone much further than whether we should or shouldn’t ban it. There has been no middle ground. Supporters are on the side of law and order, whatever that takes; opponents are radical leftists with a disregard for public safety or luddites opposed to technological progress. The many different choices made in designing and deploying the various tools and methods that fall under the umbrella of “facial recognition” — some of them sensible, others careless, some downright ugly — tend to get lost along the way.

Many things are technically possible. That doesn’t make them safe, ethical or useful. It is technically possible to build a three-wheeled car. It just might keel over if you go round a bend at more than forty kilometres per hour. It’s technically possible to manipulate software measuring carbon emissions in a car so that readings are artificially lowered, but that doesn’t mean it’s legally or socially permissible.

Technologies are not monolithic. The design of every product rests on a range of choices and trade-offs. Some products are well designed and conscious of their social and ecological footprints. Other products pose threats to physical safety, discriminate against people, or are designed to cheat. We need to think carefully about how we want technology to be applied — how we want it to be manifested in the world. Facial recognition is no different.

Clearview AI’s facial recognition application wasn’t just bad because it scraped billions of images of people without their knowledge or consent. If details of the New York Times’s investigation are true, it went a lot further than that. It built software capable of monitoring whom its users — mostly law enforcement agencies — were searching for. It manipulated image search results, and removed some matches. Images uploaded by police were stored on their own servers, with little verification of data security.

Are these things we want? Are these practices okay?

Clearview AI is just the latest in a long line of stories about buggy, inaccurate, invasive and outright offensive implementations of facial recognition. Face-detection settings on cameras that only work on certain faces. Image-tagging software making racist comparisons. Identity-matching databases used to investigate crime consistently misidentifying members of already marginalised groups. Software engineers matching women’s faces with adult videos online, to help men check if their girlfriends had ever acted in porn.

Last week European Union regulators indicated they’re considering a potential ban on facial recognition technology for up to five years — with some exceptions — while they figure out the technology’s impact and the regulatory issues that need to be tackled. Google and Facebook have already expressed cautious support for such a ban.

Some cities have already started curtailing facial recognition: in San Francisco, the government voted in 2019 to ban local law enforcement from using the technology. In New York State, the education department demanded a school district cease using the technology in public schools.

Speaking to the New York Times, one investor in Clearview AI, David Scalzo, was doubtful about the power of any prohibition. Technology can’t be banned, he said. “It might lead to a dystopian future or something, but you can’t ban it.”

It’s true that a technology, once discovered, can’t be undiscovered (though some have been forgotten). But throughout history, societies have temporarily banned the development or certain applications of technologies when it’s unclear whether they will do more harm than good: think nuclear power, or gene editing. Sometimes temporary bans become permanent ones. Sometimes they’re lifted once we’ve used the breathing space to figure out the rules of engagement.

And yes, it’s true that bans can be broken. But technologies don’t break bans — people do. People who do not respect or recognise the concerns of the societies they live in.

Technologies do not lead us into a dystopian future: we decide the future we want. •

The post Will we finally look clearly at facial recognition technology? appeared first on Inside Story.

The critics were wrong, and wrong in a major way. As industry insiders, they all paid the price for their poor predictions. Their products would all exit the industry, replaced by the new Apple, of course, but also by Samsung and Huawei. What turns out to be a successful innovation might not seem that way at first. There is a reason for that: innovation is new to the world. If it was obvious, someone would have done it.

Technology forecasts can also be wrong in the other direction. In 2001, after years of stealth development, inventor Dean Kamen unveiled the Segway. This was a personal transporter with two wheels on either side of a platform with a stick and handlebar jutting from its centre. At a time when computer-controlled gyroscopes were rare, it seemed like magic. Implausibly, the Segway would balance itself and its occupant upright. The rider simply leaned forward to accelerate and backward to stop. It seemed like something from the future. It seemed like something that you wanted to try.

Many others heralded the Segway as a revolution. Steve Jobs said it was “as big a deal as the PC.” John Doerr, the famous venture capitalist behind Netscape and Amazon, believed it would be bigger than the internet. It was hard to find many early detractors. Alas, a decade and a half later, you might see a Segway used by a traffic cop or group of tourists being led around a city. Otherwise, it is a discarded technological concept. Why didn’t the Segway work out? There were some safety issues, but that hasn’t prevented the police from adopting them. One theory is that people stuck out too much on them, drawing attention in an unwelcome way.

The point is that our forecasts — optimistic or pessimistic — for individual technologies can often be way off base. Marc Andreessen, Netscape founder and venture capitalist, compares his performance with that of Warren Buffett, the world-famous proponent of “value investing”: “Basically, he’s betting against change. We’re betting for change. When he makes a mistake, it’s because something changes that he didn’t expect. When we make a mistake, it’s because something doesn’t change that we thought would.”

We started this discussion of technological prospects with the iPhone and Segway precisely because of this question of far-reaching impact. The iPhone established a dominant design for smartphones. Thanks to people having the internet in their pocket, we got Uber, Airbnb and Spotify. We got Facebook, Instagram, LinkedIn and Twitter to inform, engage and infuriate us. Developing economies skipped over bank accounts to mobile banking, such as Kenya’s ubiquitous M-Pesa service. If the doubters had been right, we would have had none of these things. With the Segway, we didn’t end up changing urban transportation. Billions might have switched to a travel technology that eased congestion and cut emissions, but we didn’t.

Are there still big breakthroughs to be made? On this, economists disagree. There are technological optimists who believe that big breakthrough innovations lie in our future, and pessimists who believe they won’t surpass the past. How can we evaluate their arguments?

The tech optimists

In early 2014, owners of Tesla’s Model S electric vehicles received a recall notice from the US National Highway Traffic Safety Administration related to a problem that could cause a fire. What car owners usually have to do in these cases is return the car to a dealer to be fixed. This is costly for everyone involved. This time it was different. The problem could be fixed by updating the software in the car, and the update could be pushed to almost 30,000 vehicles overnight because Teslas are connected by default to the internet. No muss, no fuss.

The fact that this could now be done for so many products with embedded software caused Andreessen to proclaim that “software is eating the world.” Put simply, real things were no longer fixed in their capabilities. Because of software, they could be enhanced without having to physically rebuild them.

The tech optimists are not optimistic simply because they know that the universe has more to reveal. They are optimistic because they believe that we are still living in a time of accelerating technological change. Andreessen argues that the benefits of computing technologies and the digitisation revolution are ongoing because they are based on software — something that scales easily. More than half the world’s population came online in just the past decade, and the world is not yet fully connected. Moreover, the value of that network increases disproportionately to the number of people on it — an effect known as Metcalfe’s law.

From the perspective of an innovator in software, that means the customer base is still growing rapidly. What is more, with greater numbers of users, distributed infrastructure — known commonly as “the cloud” — becomes cheaper to use, even aside from the reductions in the cost of hardware in data centres. In 2000, it may have cost a start-up $150,000 per month to host an internet application in the cloud. Today it is less than $150. Those gains translate into increased profitability and lower risk for every single software entrepreneur.

Tech optimists point to multiple trends. Since the 1960s, Moore’s law saw processing power double roughly every eighteen to twenty-four months. As a consequence, microprocessors in 2018 had eight million times as many transistors as the best microprocessor in 1971. Worldwide data storage is now around a zettabyte, or ten bytes to the power of twenty-one. Each minute, 300 hours of video are uploaded to YouTube. The next mobile telephony standard, 5G, will operate at many times the speed of the previous generation of wireless technology.

Technologies are sometimes used in unexpected ways. Graphics processing units (developed for hardcore gamers) were used to train neural networks designed to emulate the learning functions of the brain. These new developments in what is called machine learning have led to a renaissance in artificial intelligence research.

Around five years ago, using deep learning methods pioneered by several Canadian university professors, computers’ ability to understand speech and recognise images took a leap forward. These new methods mimicked the brain function, and allowed multiple levels of sorting and classification. The result effectively allowed computers to pick up nuance and associations that even humans would miss. In October 2016, Microsoft engineers announced that their speech recognition software had attained the same level of accuracy as human transcribers when it came to recognising speech in the “Switchboard Corpus,” a set of conversations used to benchmark transcribers. In a controlled environment, machine voice recognition is now more likely to comprehend what we’re saying than the average human. Meanwhile, facial recognition algorithms used by Baidu, Tencent-BestImage, Google and DeepID3 have an accuracy level above 99.5 per cent, compared with humans’ rate of 97.6 percent.

The best way to explain what has happened is to focus on what the new artificial intelligence techniques do best: prediction. Machines can now take a large amount of data (numbers, images, sound files, or videos) and review it for relationships that allow them to forecast with a high degree of accuracy. Image recognition, for example, is basically a prediction activity: “Here is a picture. What is your best guess at what someone would call this?”

Although these technologies still make mistakes, they have the ability to outperform humans in real-world contexts. In 2011, IBM’s Watson computer played the quiz show Jeopardy! against two champions of the game: Ken Jennings and Brad Rutter. Watson won. IBM’s next major human-versus-machine contest came in 2018, when the company showed off its IBM Debater. The computer was able to engage at a reasonably coherent level with a human counterpart on the topic of whether government should subsidise space exploration.

Learning machines don’t just have to rely on their own experience. Indian online retailer Myntra recently deployed an algorithm that designed new clothing images by modifying and combining popular patterns. One of those computer-designed t-shirts, featuring blocks of olive, blue and yellow, is now a bestseller. Artificial intelligence is arguably the next general-purpose technology: a technology so foundational that myriad other innovations grow on its base. We have seen this happen with the steam engine, electric power, plastics, computers, and the internet. The optimists believe that artificial intelligence could have the same potential.

To see how technology might drive science, remember that Galileo’s research — which showed convincingly that Earth revolved around the sun — was based on a technological advance in the form of a telescope that could magnify distant objects thirty times. A few decades later, the creation of a microscope that could magnify tiny things 300 times enabled Robert Hooke to document the existence of cells. These massive breakthroughs in astronomy and biology would have been impossible without advances in glass production and precision manufacturing.

Today, it’s easy to point to similar advances. The use of gene editing could revolutionise medical science. Strong and light materials such as graphene could change manufacturing. These are radical technologies that could bring about decades of further innovation.

The tech pessimists

Others take an altogether dimmer view of our prospects. They worry that we have already picked the low-hanging fruit over the past two centuries, and that the outlook for the next century is bleaker. Their argument is not based on some oracle-like insight into the future but instead on the inescapable economic law of diminishing returns.

In economics, the figure that looms largest on this side of the argument is Robert Gordon. His concern revolves around just how great the relatively recent past has been. Prior to 1870, economic growth occurred at a trickle. But after 1870, the major innovations at the heart of the Industrial Revolution began to work their way fully through society. It wasn’t just that steam power made factories more efficient; our knowledge of science also brought us to a point where new technologies were shaping the environment around us.

In the century following 1870, most people in the United States and Western Europe (and a handful of other places) went from carrying water to having it delivered to their houses at the turn of a tap, instantly and in a form safe enough to drink. Washing machines saved time and made our clothes last longer. Indoor toilets took sewage far away from houses at the push of a lever or yank of a chain. Energy could be easily delivered to people’s houses. Information was brought in by the radio, telephone and television. Cars provided freedom and reshaped the urban form. A reasonable person might suppose that society will never again see such radical changes. The interesting thing is that we can see this in the data on economic growth that measures how innovations have translated into productivity improvements.

Growth has its ups and downs. Smooth out the temporary recessions and upswings, though, and the century until 1973 was an era of steady progress that suddenly petered out. Initially, many economists saw the slowdown as an aberration. Nobel laureate Robert Solow, who pioneered the field of economic growth, said in 1987 that “you can see the computer age everywhere but in the productivity statistics.” Maybe it was a mismeasurement because computers were assisting services whose productivity was notoriously hard to measure? The economic historian Paul David reminded us that when electricity was introduced, it took decades for it to show up in measures of productivity. Maybe once firms worked out how to use computers effectively, the productivity gains would become apparent?

Many advanced nations did experience a surge in productivity growth in the late 1990s. Yet its rate then slowed in the twenty-first century. For workers, things are even worse because of a decoupling of wages from productivity. Even where firms are getting more output for a given level of inputs, they are not sharing most of those gains with employees.

Consequently, a generation of adults has not experienced the fruits of productivity improvements. They are as well educated as their immediate forebears, they are more lightly taxed, and the businesses that employ them have the benefits of more integrated global financial markets.

The problem comes down to something economists call “diminishing returns.” When England continued to put more land under farming during the nineteenth century, as David Ricardo noted, the productivity of additional acres fell. Take any fixed resource and there is only so much you can extract from it. In the twentieth century, Solow observed that this held for other types of capital such as machines. It also applied to workers. The only way out was technological progress, which allowed society to get more out of the same inputs.

So long as the growth in knowledge we had achieved in the past continued into the future, there was nothing to worry about. Yet here is where the tech optimists and tech pessimists part company. The optimists, as we have noted, anticipate rapid technological progress. The pessimists are not so sure. If that is the case, they say, then why have this generation’s inventions not transformed our lives in the way of the great twentieth-century innovations? Do the twenty-first century’s inventions really compare with air conditioning, airplanes and automobiles (to take just one letter of the alphabet)?

To tech pessimists such as Gordon and Tyler Cowen, the answer comes from merely looking at how technological changes from the 1870s to the 1970s transformed the way we live. Electricity transformed work, shifting people from agriculture to the cities. In the cities that shift combined with running water, sewerage systems, and efficient heating and cooling techniques to allow for a comfortable and productive urban life. Electrical appliances reshaped household economics, freeing women to join the paid labour force. Transport on the roads and air was transformed, facilitating unprecedented interregional trade and travel. All this added up to dramatic improvements in productivity. Since 1973 there have been useful inventions to be sure. But they are yet to deliver an equivalent surge in productivity.

What has the pessimists worried is that researchers and scientists are finding it harder to unearth new ideas. Research by Northwestern University’s Ben Jones shows that Nobel laureates are getting older. To be more precise, over the past century the age at which someone does research that will win them a Nobel prize has been rising. The same is true of work that leads to a patent. In addition, more knowledge breakthroughs are being made by teams rather than individuals. This points to more specialisation in knowledge production, with fewer instances in which an individual comprehends developments at the frontier of multiple disciplines. Because this raises the cost of innovating, Jones calls it the increasing “burden of knowledge.”

As technology advances, it becomes tougher to find the next new thing. Take semiconductors. As we have noted, Moore’s law has seen a steady doubling of the density of computer chips every eighteen to twenty-four months. Moore’s law continued up until the mid 2000s, but significantly, the cost of recent increases is eighteen times larger than it was for similar proportionate increases in the 1970s. The same pattern exists in agriculture and medical research. What was once easy has become hard. It suggests that just to keep the slower growth in productivity that we have, innovators must run faster and faster.

Uncertain prospects

The tech optimists and the tech pessimists both have a point. The optimists note that there is still potential for new knowledge, and can point to exciting possibilities that are attracting significant scientific and engineering resources. The pessimists’ colder calculations remind us how exceptional past growth was and point to the logical implication that those ideas that gave the biggest boosts to productivity were likely ones we have already exploited. Historians such as Joel Mokyr have looked at all this discussion and remind us that we have been here before. In every decade, one can find optimists and pessimists. And, at least as far as continuing technological change is concerned, the optimists have usually been on the right side of history.

What does this all mean, however, for the creation price — that is, the price that must be paid to reward innovators and entrepreneurs for their efforts? The answer lies in the cost of innovation. Where the tech optimists and tech pessimists fundamentally differ is in how costly it will be to innovate in the future. If there are technological opportunities just waiting to be exploited, as the optimists claim, then the creation price can be set relatively low. On the other hand, if the cost of innovation is rising, as the pessimists claim, then the creation price will be higher, and growing over time. More resources will have to be dedicated to innovative activities to maintain historical growth rates. In that situation, we will have to ask if it is a price worth paying.

Forecasting the future is like driving through fog. We need to accept that the creation price is uncertain. It could be high, low or somewhere in between. It will likely be different for different technological opportunities and directions. But at the same time, everyone faces this uncertainty. No one has a special insight into the future. That includes entrepreneurs. And given that uncertainty, the best way to get more equality and more innovation is to reduce the costs those entrepreneurs face today.

Planning for flexibility

Which brings us to equity. Here, the goal ought to be a set of institutions that provide a safety net, both for entrepreneurs who fall short of the stars and for those left behind when the rocket takes off. It pays to think about such institutions as a form of insurance, providing greater resilience in the face of a changing world. If you’re giving advice to a teenager, now is the time to tell him or her about the value of being flexible. Education isn’t just an investment; it’s about providing more life options.

To achieve this in the education system, we propose making teacher effectiveness the core focus of schooling, improving the quality of vocational training, and encouraging MOOCs (massive online open courses). And it makes enormous sense to use the talents of the 51 per cent of the population who are women by encouraging technologies that make jobs more family-friendly, and reforming laws that end up biasing the labour market against women. Gender equity isn’t worthwhile just because it will boost productivity but also because — as Canadian prime minister Justin Trudeau might say — it’s 2019.

As economist Sendhil Mullainathan puts it, “The safest prediction is that reality will outstrip our expectations. So, let us craft our policies not just for what we expect but for what will surely surprise us.” The task is to shape a future that looks more like Star Trek than Terminator.

Uncertainty need not be scary. The story of human history — particularly in recent centuries — is of how we have employed our shared ingenuity to improve lives. Longevity has risen. Whole diseases have been eliminated. The typical job is more fulfilling and less painful. Entertainment is more abundant, and much of it is of higher quality (try spending a week watching television from a generation ago). Food standards have risen, and cars are safer than ever. Life is far from perfect, but there is a good deal to celebrate. •

The post More Star Trek than Terminator? appeared first on Inside Story.

Seeming contradictions like these are not new. Technological change has always been uneven, making manufactured products cheaper, for instance, yet leaving many service activities largely unaffected. Increased productivity in the economy as a whole has pushed wages up, making labour-intensive services more expensive.

This divergence is much more marked with AI. Compared to earlier rounds of technological change, we are seeing a combination of incredibly rapid change and near stagnation. The acceleration of computing power has been so fast that a Series 1 Apple watch (itself a museum piece three years after its introduction) can perform calculations as fast as the Cray X-MP, the most powerful supercomputer in the world back in 1982. The amount of digital information generated every hour of every day exceeds all the digital data that was created up to and including the year 2000.

By contrast, many areas of daily life have changed little over the course of a generation. The most technologically advanced item in the average kitchen is the microwave oven, first marketed to households in the 1970s. Air travel reached its peak of speed with the introduction of the Concorde in 1973; it was withdrawn from service in 2003.

Every now and then, some new advance revolutionises a previously stagnant activity. The typical passenger car today is only marginally different from the models of twenty or even fifty years ago. It has smarter electronics and improved safety systems, but the experience of driving and the basic technology of the internal combustion engine are the same. Over the past decade, though, we have seen the arrival of electric cars and then of autonomous vehicles. While the future remains unclear, it seems certain that road transport will change radically over the next twenty years, and even more so over the next fifty.

Not all the new arrivals are beneficent. In 2062: The World that AI Made, Toby Walsh points to the alarming possibilities raised by autonomous weapons, of which armed drones like the Predator represent the first wave. The drone itself contains nothing fundamentally new — it’s a pilotless aircraft, equipped with cameras and missiles, that can fly for hours. The big developments are in the telecommunications systems that allow controllers on the other side of the planet to view the camera output in real time and order the firing of the missiles at any target that they choose.

At present these controllers are human, error-prone but capable of making moral choices in real time. But the development of pattern-recognition technology is such that it is already feasible to replace the human controllers with an automated control system programmed to fire when preset criteria are identified. The point at which moral choices are made, explicitly or otherwise, is in the setting of the criteria and the programming of the control system.

Further off, but by no means inconceivable, are systems whose criteria for targeting (for example, “fire on vehicles containing armed men”) are replaced by higher-level objectives. Such an objective might be “fire when the result will be a net saving of lives” or, more probably, “fire when the result will be a net saving of lives on our side.” In this case, in effect, the machines are being give moral principles and ordered to follow them.

These possibilities are alarming enough that Walsh, a professor of artificial intelligence at the University of New South Wales, and some of his colleagues organised an open letter calling on the United Nations to ban offensive autonomous weapons. The letter rapidly attracted 2000 signatures and started a process that may ultimately lead to a new international convention. As the history of disarmament proposals has shown, though, the resistance to any restriction on lethal technology is always formidable and usually successful.

The theme of human choice is developed further in Ellen Broad’s Made by Humans, an excellent analysis of the way the magical character of AI hides built-in human biases. Among Broad’s central observations is the fact that the word “algorithm” is being used in a different way, something I hadn’t noticed until she pointed it out.

For the last thousand years or so, an algorithm (derived from the name of an Arab mathematician, al-Khwarizmi) has had a pretty clear meaning — namely, it is a well-defined formal procedure for deriving a verifiable solution to a mathematical problem. The standard example, Euclid’s algorithm for finding the greatest common divisor of two numbers, goes back to 300 BCE. There are algorithms for sorting lists, for maximising the value of a function, and so on.

As their long history indicates, algorithms can be applied by humans. But humans can only handle algorithmic processes up to a certain scale. The invention of computers made human limits irrelevant; indeed, the mechanical nature of the task made solving algorithms an ideal task for computers. On the other hand, the hope of many early AI researchers that computers would be able to develop and improve their own algorithms has so far proved almost entirely illusory.

Why, then, are we suddenly hearing so much about “AI algorithms”? The answer is that the meaning of the term “algorithm” has changed. A typical example, says Broad, is the use of an “algorithm” to predict the chance that someone convicted of a crime will reoffend, drawing on data about their characteristics and those of the previous crime. The “algorithm” turns out to over-predict reoffending by blacks relative to whites.

Social scientists have been working on problems like these for decades, with varying degrees of success. Until very recently, though, predictive systems of this kind would have been called “models.” The archetypal examples — the first econometric models used in Keynesian macroeconomics in the 1960s, and “global systems” models like that of the Club of Rome in the 1970s — illustrate many of the pitfalls.

A vast body of statistical work has developed around models like these, probing the validity or otherwise of the predictions they yield, and a great many sources of error have been found. Model estimation can go wrong because causal relationships are misspecified (as every budding statistician learns, correlation does not imply causation), because crucial variables are omitted, or because models are “over-fitted” to a limited set of data.

Broad’s book suggests that the developers of AI “algorithms” have made all of these errors anew. Asthmatic patients are classified as being at low risk for pneumonia when in fact their good outcomes on that measure are due to more intensive treatment. Models that are supposed to predict sexual orientation from a photograph work by finding non-causative correlations, such as the angle from which the shot is taken. Designers fail to consider elementary distinctions, such as those between “false positives” and “false negatives.” As with autonomous weapons, moral choices are made in the design and use of computer models. The more these choices are hidden behind a veneer of objectivity, the more likely they are to reinforce existing social structures and inequalities.

The superstitious reverence with which computer “models” were regarded when they first appeared has been replaced by (sometimes excessive) scepticism. Practitioners now understand that models provide a useful way of clarifying our assumptions and deriving their implications, but not a guaranteed path to truth. These lessons will need to be relearned as we deal with AI.

Broad makes a compelling case that AI techniques can obscure human agency but not replace it. Decisions nominally made by AI algorithms inevitably reflect the choices made by their designers. Whether those choices are the result of careful reflection, or of unthinking prejudice, is up to us.

Beyond specific applications of AI, the technological progress it generates will have effects throughout the economy. Unfortunately — as happened during earlier rounds of concern about technology — the discussion has for the most part been reduced to the question, “Will a robot take my job?” Walsh and Broad both point to the simplistic nature of this reasoning.

A more comprehensive assessment of the economic and political implications of AI comes in Tim Dunlop’s The Future of Everything. (Disclosure: I’ve long admired Dunlop’s work, and I wrote an endorsement of this book.) Rather than focusing on AI, Dunlop is reacting to the intertwined effects of technological change and the dominant economic policies of the past few decades, commonly referred to as neoliberalism or, in Australia, economic rationalism.

The key problem is not that jobs will be automated out of existence. In a system dominated by the interests of capital, the real risk is that technological change will further concentrate wealth and power in the hands of the dominant elite often referred to as the 1 per cent. As Dunlop says, radical responses are needed.

The most obvious is a reduction in working hours. This has been one of the central demands of the working class since the nineteenth-century campaign for an eight-hour working day. After a century of steady progress, the trend towards shorter working hours halted, and even to some extent reversed, in the 1970s. The four decades of technological progress since then have produced no significant movement.

This is a striking illustration of the fallacy of technological determinism. Under different political and economic conditions, information and communications technology could already be providing us with the leisured life envisioned by futurists of the 1950s and 1960s. Instead, it has become a tool for keeping us tethered to the office on a 24/7/365 basis.

Closely related is the question of flexible working hours. As Dunlop observes, “flexibility” is an ambiguous term. Advocates of workplace reform praise flexibility, but what they mean is top-down flexibility, the ability of managers to control the lives of workers with as few constraints as possible. Bottom-up flexibility, the ability of workers to control their own lives, is directly opposed to this. To put it in the language of game theory, flexibility is (most of the time) a zero-sum commodity.

More radical ideas include treating data as labour and moving to collective ownership of technology. Some of the most valuable companies in the world today, including Facebook and Alphabet (owner of Google), rely almost entirely on data generated by users of the internet. “We are all working for these tech companies for free by providing our data to them in a way that allows them to hide our contribution while benefiting immensely from it,” writes Dunlop. “It is way past time that we were paid for this hidden labour, potentially using that income to offset reductions in our formal working hours.”

Dunlop suggests that taxes on the profits of tech companies could be used to finance a universal basic income, which would provide everyone with an income sufficient to live on, whether or not they were engaged in paid work.

The collective ownership of technology sounds radical, but it is, in many respects, an extension of that same argument. Increasingly, technology is embodied not in large pieces of equipment, like blast furnaces or car factories, but in information: computer code, data sets and the protocols that integrate the two. As Stewart Brand observed back in 1984, information wants to be free. In the absence of legal restrictions or secrecy, that is, a piece of information can be replicated indefinitely, without interfering with the access of those who already have it. As the cost of communications and storage drops, so does the cost of replicating and transmitting information.

Of course, there are many reasons, such as privacy, why we might want to restrict access to information. But concerns about privacy have been largely disregarded under neoliberal policies. On the other hand, strenuous efforts have been made to protect and extend “intellectual property,” the right to own information and prevent others from using it without permission. These rights, supposedly given as a reward to inventors and creators, almost invariably end up in the hands of corporations.

From this perspective, longstanding demands for workplace democracy and worker control are merging with the critique of intellectual property largely driven by technical professionals. For these workers, the realities of the information age are incompatible with the thinking behind intellectual property. As Dunlop says, worker ownership is “another way of changing how we think about technology… not just a means to a fairer society, but a demand that fundamentally changes how we understand the creation and distribution of work and wealth.”

There’s a lot more in these books, and particularly Dunlop’s, than can be covered in a brief review. Each provides useful correctives to the lazy thinking about job-stealing robots and infallible algorithms that dominates much of our public discussion. And all centre on the same basic point: while technology has its own logic, the way technology is used is a matter of choice.

The key question is: who gets to make those choices? Under current conditions, they will be made by and for a wealthy few. The only way to democratise choice about technology is to make society as a whole more democratic and equal. •

The post Will a robot take your job? appeared first on Inside Story.