> nobody should be deploying software in production that is capable of this kind of harm.
This "kind" of harm seems to hide a huge amount of complexity. Let's assume the algorithm starts off on day 1 just looking for purchases that tend to appear in clusters. There's a potentially unbounded number of clusters that could be regarded as harmful. The options are surely:
1. Be vigilant and special case items one by one. I presume they are doing this. Maybe they aren't putting enough resources behind it or not reacting quickly enough but even if they were then things would always slip through.
2. Never use any kind of bundling algorithm ever.
(1) is a forever ongoing task so it doesn't sound like you're referring to that as that's not something where you can "iron out the kinks".
However it also doesn't sound like you mean (2) as you seem to be saying there's a way to fix this once and for all.
Is there a third option I haven't thought of?
EDIT - I guess maybe all "bundles" could require human curation before they are allowed to be displayed. It would be a gargantuan task but it's a valid third option.
There's a hell of a difference between recommending pairing a cute shirt with some cute earrings and pairing an industrial chemical with an item from the pharmaceuticals category. There are extremely basic and straightforward safeguards that could have been put in place that simply do not exist.
Why is it amazon’s responsibility to safeguard your children? That’s your responsibility.
They can still figure out how to buy the items separately, and Amazon isn’t inducing suicidal ideation.
Perhaps your kids shouldn’t have unrestricted and unmonitored access to order things from Amazon? Start there, instead of insisting that Amazon do the parenting for you.
If you can't see the difference between things like "Amazon shouldn't sell dirty magazines to kids" and "Amazon should have basic protections in place to avoid selling lab grade chemicals along with drugs to facilitate a painless suicide as a bundle to whoever wants them with free shipping," I don't know how to have a reasonable conversation.
It's not about parents having oversight over kids. It's about mental illness: "kits" like this shouldn't exist at all for kids or otherwise. You use it once and you're dead because that's the whole point of the combination of goods. We as a society should be past the point where we need to discuss whether selling a literal suicide kit is okay or not. It's not, and we know from research that making suicide more accessible and convenient leads to more suicide. The only outcome of allowing Amazon to do what it's doing is additional deaths that could have been prevented.
> Why is it amazon’s responsibility to safeguard your children? That’s your responsibility.
Indeed. And we intend to exercise that responsibility to safeguard our children by stopping Amazon from mixing suicide kits in with shipments of innocuous household goods.
You intend to exercise your responsibility by abdicating it, and instead insisting that Amazon and other retailers do it for you, and society at large bear the cost?
I think we disagree on the definition of “responsibility”.
Yes. Society can bear the cost of having to jump through a few extra steps to buy pure concentrated chemicals that have been demonstrated to repeatedly kill teenagers.
So the people who have legitimate use for those chemicals can jump through a few hoops to get them.
And Amazon, a trillion dollar company, can do a better job with its algorithms and stop recommending them and selling them to teenagers.
I am super comfortable inflicting those costs on the world.
I’m sure you’re comfortable with whatever authoritarian bureaucratic nonsense would help calm your social media-induced panic attack if the hour, but that’s not a standard of evidence under which I’m willing to make the world a bit worse.
It sounds like we won’t agree on ethical fundamentals, much less this topic, so I wish you well, and hope you can source enough bubblewrap to feel safe existing in the world.
The world is worst because a trillion dollar company don't have the incentive to operate responsibly. Because they don't care, and shrug responsibility at every turn. Because they are not willing to uphold any standards. Nobody expects a bubble-wrap, just a recommendation engine that's safe around chemicals. What an expectations from a trillion dollar company. Please don't recommend dangerous chemicals to children or information that tells them how to harm themselves.
By the way your car is not a death-trap because of us nagging the industry to do better. You are welcome!
It’s not authoritarian nonsense to have and enforce basic principles of product safety in commerce.
For evidence refer to the fact that your family hasn’t died of botulism or a falling building facade.
This is pretty basic stuff. It’s not a social media panic attack it’s an active lawsuit backed by sworn statements and evidence, and the fact pattern is genuinely apalling. I get skepticism at the headline but read through the details.
To be clear, Amazon isn't just selling these things. They're actively promoting their sale, together. Their systems are literally encouraging the combined purchase of multiple products from different sellers in different categories to create a deadly suicide kit. Are you suggesting Amazon has no responsibility for what their systems—unprompted—promote?
It does not seem like they are doing 1 in any sufficient way: its pretty easy to argue that this is by far the most important special case (it’s literally life and death stakes), and they are failing at it even after a growing PR campaign and advocacy made them aware months (years?) ago.
Amazon is a $1T company. They have the resources. This is willful negligence.
This is a good example showing that all work is not removed by automating a task, such as selecting which products are related and displaying that on an online store’s product page. Some work may be removed, but new work is added to compensate for what the algorithms can not trivially do, such as considering whether the links between these products are ethical to present to users.
It seems to me that companies massively underestimate (or willingly don’t compensate for) the importance of this newly introduced work, only cashing in on the savings from the work removed. Other examples of similar behavior include automatic moderation, which also massively fails to remove actual dangerous content.
This kind of behavior looks like greed to me, but I guess it’s legal, and needs more regulation.
Exactly this, Amazon's recommendation engine doesn't have the domain knowledge about chemicals and such, what can be mixed with what. Their recommendation may mislead people.
How would that even address the problem? Someone buying these things to commit suicide will still buy these things even if they need to do a few more clicks and searches because the items aren't conveniently bundled or recommended.
> EDIT - I guess maybe all "bundles" could require human curation before they are allowed to be displayed. It would be a gargantuan task but it's a valid third option.
This is the correct way to handle it. It being a "gargantuan task" doesn't matter. Everybody would freak out if the local supermarket advertised this kind of product but somehow "tech" (read: online retailers) can get away with this because they are big?
Amazon posts billions in profits, it can certainly afford to control their algorithms. If it's impossible, then stop using algorithms and use human curation like literally every brick and mortar business.
Nothing slipped through. Amazon has been fully informed about it, is doing it on purpose, and is engaging in expensive legal battles to be allowed to keep doing it.
This story is so much worse than the headline suggests, despite that seeming impossible.
If they artificially limit their "AI" from outside pressure, then they will have admitted that they can do it, and they'll get pressured to do it more and more. They don't want to deal with that, and would rather wait until a law is passed "forcing" them to do so - coincidentally Amazon will be best positioned to comply with that law and for some reason it'll only ever be used to go after smaller competitors.
They don't want to establish the precedent that recommending a harmful item makes them liable for that harm at any cost even at the cost of a few dead children. Before this in a previous scandal they were promoting "literature" that suggested parents of autistic children cure them with bleach and recommended the bleach they could inadvertently murder their children with.
This "kind" of harm seems to hide a huge amount of complexity. Let's assume the algorithm starts off on day 1 just looking for purchases that tend to appear in clusters. There's a potentially unbounded number of clusters that could be regarded as harmful. The options are surely:
1. Be vigilant and special case items one by one. I presume they are doing this. Maybe they aren't putting enough resources behind it or not reacting quickly enough but even if they were then things would always slip through.
2. Never use any kind of bundling algorithm ever.
(1) is a forever ongoing task so it doesn't sound like you're referring to that as that's not something where you can "iron out the kinks".
However it also doesn't sound like you mean (2) as you seem to be saying there's a way to fix this once and for all.
Is there a third option I haven't thought of?
EDIT - I guess maybe all "bundles" could require human curation before they are allowed to be displayed. It would be a gargantuan task but it's a valid third option.