But if you are in a full selfdriving car and it kills someone, is it on the user, or on the vendor of the car? Or does it depend on what is in the EULA and whether the user agreed to it?
This is a great rhetorical question. Can you believe that an LLM user is responsible for what the AI does but a self-driving car (SDC) user is not responsible for what the AI does? It doesn't seem very logically consistent.
In both cases, the company operating it promises they did lots of things to make it safe. SDC companies talk about testing, redundancy, simulations, and statistics. LLM companies talk about alignment, sandboxes, defense in depth, and restricting access to dangerous capabilities.
The one substantial difference I can see is that LLM chats are (nominally) passive but SDCs are active. In a chat, you're just having a conversation and then you're choosing how to act. When riding in a car, you the user are not in the loop between AI and taking action that affects the world around you. So they might be designed and engineered a bit differently. A SDC can't get you to agree to review the steering and braking decisions before they're put into action. Since you're not being asked to take that responsibility, it's more defensible to infer that the service has done whatever is necessary to make it safe.
The practical answer is you make sure damages are always covered by insurance, and as long as nobody acts maliciously or with extreme negligence there won't be any need for prison time.
Practical in what sense? This is exactly how you get EPA superfund sites while the executives that profited from the offending activity remain unaffected.
Practical in that we get to have widespread self driving cars. Real ones where you can look away from the road.
Do you have an "offending activity" in mind? The existence of car accidents isn't one. If you want the manufacturer to be liable for every mistake the car does, that's fine. But since that will increase the price of self-driving, likely as a subscription, it basically resolves into a type of insurance.
If the self-driving designers perform actual wrongdoing, that's a separate issue that can be handled like any other recallable hazard.
