Writing — Ethics & Care AI

The Bone Ring

A steel ring cast as two rails of linked finger-bones joined by cross-struts, the oxidised recesses darkened, resting on a pale surface.

I wear a steel ring of bone on the little finger of my right hand, the hand I build with, and I put it there on purpose.

I am an engineer. Electrical first, then became a doctor of computer science, and 20 years of building systems. The part that shaped how I think was learning how information is given form. Bertin taught me that a mark on a page obeys a logic as strict as any circuit. Klee and the Bauhaus taught that line and structure carry moral weight. Engineering is about building: how you build a thing is what it does to the people who live inside it. A badly made building does not announce its cruelty. It makes lives quietly worse for decades, in ways no one traces back to the drawing.

Engineers learned that lesson at a cost, and then built an object so they would not forget it.

In 1907 the Quebec Bridge was the longest cantilever span ever attempted. The design was wrong. The errors were found during construction and judged not serious enough to stop the work. On the 29th of August the south arm fell into the St. Lawrence in about fifteen seconds and killed seventy-five men, thirty-three of them Mohawk ironworkers from Kahnawake. A second span fell in 1916 and killed thirteen more. The 1907 inquiry blamed the engineers. Canadian engineers now wear a ring on the little finger of the working hand, faceted so it catches as they write. The bridge is gone. The ring stays on the hand, and the lesson stays with it: what you build can kill people who trusted you to get it right, and you will feel that every day you work.

The wreckage of the Quebec Bridge after its collapse on 29 August 1907 — a tangle of twisted steel over the St. Lawrence.
The south arm of the Quebec Bridge after the collapse of 29 August 1907. Public domain, via Wikimedia Commons.

I build systems that make use of AI for care. The machinery that turns what a carer says at the end of a shift into a record someone else will act on. Call it efficiency if you like, the minutes saved and the notes written, but that is not what it is. Every output my systems make is about a body. A real one, usually old, often frightened, often unable to speak for itself. Strip away the care plan and the daily note and what remains is a person whose structure is failing, who has handed that structure to strangers, and now to software.

Bone carries symbolic weight. It is the structure everything else hangs on, and the last thing to go. Bone is also one of the oldest symbols in medicine. It is what endures. I wear the ring as a visual reminder of the humanity, and the potential for damage. I wear the ring as a sensory reminder that my clicks of the keyboard can cause harm.

Symbols are important. In my data visualisation work they are a foundational piece of the communication and understanding toolset. I teach this in the context of social care and medicine, where symbols and communication can change outcomes. I teach my students to pay real attention to symbols and the power they hold. Failure often hides behind the fact that a sign is there. Presence is often mistaken for real function. The function is the point. A reminder no one remembers or sees at the moment of risk is a useless decoration.

The pledge on the wall

I do not write this from the outside. I am building Coorie Well (audio-first care documentation and understanding) with Alyson McKechnie-Vale, a care home director and creative producer designing the path through Scotland's care sector. And I've joined Dedicate.life to build a new vision for it with Dr Caroline Emmer De Albuquerque Green, Director of Research at the Oxford Institute for Ethics in AI. Caroline runs a project on the responsible use of generative AI in adult social care, built by co-production: the people a tool affects help build it, rather than receiving it after the fact. Its Pledge for technology suppliers holds care above innovation and the dignity of the person as the thing not for sale.

A signature is a real commitment. It is also a single moment, and a commitment made in a moment has to be kept in all the moments after it or it means nothing. That is not a flaw in the pledge. It is the part the pledge cannot do for you. The signature says what you intend; daily practice is the only thing that proves it. The work I care about is the second part: building the commitment into how the day actually runs.

The ring catches on a corner. It reflects a fluorescent light. The meaning has to be internalised until the object becomes a symbol.

Move fast and break things

Popularised at Facebook, the phrase has become a meme of the tech industry. It is a fine creed when the thing that breaks is a feature, a release, a company that deserved to fail. It is indefensible when the thing that breaks is a person who cannot see the model, cannot read the log, and cannot speak. Care AI was born inside the culture that coined it and inherited its reflexes without asking whether they belong in a care home.

They do not. The disposition care AI needs is the one I was trained in and the industry threw away: the engineer's. An engineer assumes their work will be trusted by people who cannot inspect it, assumes it will fail, and treats that failure as theirs to foresee. The whole of engineering ethics is the refusal to ship a thing you have not imagined breaking, on someone you will never meet. That is the opposite of moving fast. It is the patience to ask, before the clever thing ships, who is underneath it.

It is also why co-production is not a courtesy.

Dropping a tool on the people it acts upon without involving them is the move-fast reflex in its purest form. It makes the cared-for a test surface. Building alongside the carer and the cared-for, holding their dignity as the thing not for sale, is the engineering instinct applied to people instead of bridges: you do not get to decide alone what is safe for someone who has to live inside your decision.

So I chose an object of my own, the way the engineers made theirs. Bone and steel. The object carries the commitment. A reminder of how to work, and why. I feel it when I type.

← All writing