There is a principle in the law that is as old as the law of the harm: the duty of care. The maker of the product that can harm owes a duty to the one who uses it — the care in the design, the warning of the risk and the responsibility for the harm. The principle is the product liability, and the liability is the law’s discipline.
That principle is arriving for the algorithms. The systems that decide — the loan, the hiring, the treatment and the pricing — are becoming products, and the duty of care is coming to them. The obligation to design them carefully is becoming legal.
The principle of the duty
The duty of care is the law’s principle, and the principle is the responsibility.
The maker owes the care to the user — the design that is safe, the warning that is clear and the responsibility for the harm. The duty is the principle, and the principle is the accountability. The duty has governed the physical products for the long history, and the governing is the law’s discipline. The duty is the care that the law requires.
This is why the duty matters: it is the principle, and the principle is the accountability.
The algorithms as the products
The algorithms are becoming the products, and the becoming is the subject.
The system that decides the loan, the hiring, the treatment and the pricing is the product in the market, and the product is the subject of the duty. The algorithm is not the abstract code; it is the product that is deployed, used and relied upon, and the reliance is the exposure. The algorithm as the product is the subject of the care, and the subject is the new.
This is why the products matter: they are the subject, and the subject is the new.
The harm of the algorithm
The algorithm can harm, and the harm is the trigger.
The decision that is wrong, the bias that is embedded and the failure that is consequential — the algorithm can harm, and the harm is the duty’s trigger. The harm is the condition that activates the care: the maker of the harmful algorithm owes the duty to the harmed. The harm is the law’s trigger, and the trigger is the liability.
This is why the harm matters: it is the trigger, and the trigger is the liability.
The care in the design
The duty requires the care in the design, and the care is the obligation.
The data that is tested for the bias, the system that is audited for the failure and the decision that is explained for the review — the duty requires the care in the design, and the care is the obligation. The maker of the algorithm must now design carefully — the testing, the monitoring and the accountability built in. The care is the duty’s content, and the content is the design.
This is why the care matters: it is the obligation, and the obligation is the design.
The warning of the risk
The duty requires the warning, and the warning is the transparency.
The user who relies on the algorithm must be warned of the limit — the system that can err, the decision that can be wrong and the use that is unsafe. The duty requires the warning, and the warning is the transparency. The transparent algorithm is the warned; the black box is the unwarned, and the unwarned is the exposure. The warning is the duty’s transparency.
This is why the warning matters: it is the transparency, and the transparency is the protection.
The responsibility for the harm
The duty carries the responsibility, and the responsibility is the consequence.
The maker of the harmful algorithm is responsible for the harm — the liability for the wrong decision, the compensation for the damage and the correction of the system. The responsibility is the duty’s consequence, and the consequence is the discipline. The maker who faces the responsibility designs carefully; the maker who does not pays the price. The responsibility is the duty’s enforcement.
This is why the responsibility matters: it is the consequence, and the consequence is the discipline.
The honest conclusion
The legal duty of care that is coming to the algorithms is the law’s new subject, and the new is the accountability.
The principle of the duty, the algorithms as the products, the harm, the care in the design, the warning and the responsibility — the algorithms that decide are becoming the products, and the duty of care is arriving.
The law that governed the physical products is extending to the algorithmic ones. The design must be careful, the risk must be warned and the harm must be answered. The duty of care is the law’s discipline, and the discipline is arriving for the algorithms. The makers of the deciding systems are becoming the makers of the products, and the makers owe the care. The duty of care is coming to the algorithms, and the coming is the new accountability — the care required, the warning given and the harm answered, as the law has always demanded of the makers.