Obviously you.

My vehicle automatically sets the parking brake if it senses it is on a steeper slope so I never consciously make a decision to set it.

If you forget to set the parking brake, you were negligent

If you set the brake but it didn't work, the car malfunctioned

From the link you sent,

> One of the questions herein was whether there was a defect in the automobile mechanism for locking the transmission gears when the automobile was in a parked position.

> (...)

> The parking lot sloped in that area, and he put the gearshift lever in "park lock," and went into the building.

> (...)

> In response to a hypothetical question, based upon assumed facts justified by and embodied in the evidence, Mr. Nass testified in substance that it was his opinion that the automobile rolled down the slope of the parking lot because the transmission internally was not in park-lock; that it was not in park-lock because the engine was "moving around"; and that the engine and shift console were not in proper synchronization because the motor mounts were not restraining the engine and were not holding the engine in position.

As I remember, you are also supposed to turn the wheels so that even if a car starts rolling, it won't be able to go staight.

That case had the owner performing an action they were led to believe would have prevented the car from rolling but potential defects prevented it from doing so.

Not a great fit.

How is worse?

If they manage to leap ahead they can effectively kill all competition, present and future and you'll be at their mercy.

So the solution is to make the company that can’t avoid sub-AGI models from going rouge to try RSI?

Also, AI performance has proven to be a function of power and compute, and right now most of the compute lies between two companies rushing towards RSI not because they are afraid someone is going to do this before but because they are the ones who wants to build it.

If the position is that one of these models will go out of control and destroy humanity then why would having two such models be safer than one?

Given that the arrival of this stuff has been compared to nuclear weapons already the question is whether or not the same mechanisms that kept us from annihilating the planet would hold there too.

I am not sure that it will.

But a balance duopoly tends to be better than a monopoly and if there are multiple poles then so much the better.

I'm past the point where I believe that any of these a-holes is in it 'for the good of humanity', their behavior so far can be extrapolated safely to what they will be like when they are in the control of what effectively would be the largest private arsenal ever created.

For a while I thought the AI doomers were off their meds and too paranoid but I'm coming around to the fact that they may have a point: we are rapidly approaching a point of no return, if we are not already past it and the people in charge are exactly the wrong people to be in charge at such a point in time.

How specifically would a balanced duopoly be safer?

> If the position is that one of these models will go out of control and destroy humanity then why would having two such models be safer than one?

Models don't go out of control, they generate totally harmless tokens. The people directing the models go out of control. The owners of the single powerful model are seriously tempted to go rouge and they're going to do it, sooner or later.

Two or more competing groups force the development of defensive capabilities which require strong models too. In addition, offensive capabilities in possession of different competing groups introduce the option of MAD (Mutually Assured Destruction) which is a form of defense that doesn't require defensive capabilities. Worked with nukes.

Autonomous killing machines and mass surveillance.

What makes you think the company that pirated a big portion of all copyrighted work will care about this checkbox?

At best they’ll “anonymize” the data before adding it to the corpus.

Beautiful machines.

At least for me it’s quite easy to see them go into endless loops where no meaningful work is done and it just keeps going until I stop the process and tell it what to try instead.

To be fair it is no where near what we had just one year ago and the rate of change only seems to be increasing.

Also, I don’t have 30 million dollars to spare spawning tens of thousand sub agents like what they did with Navier-Stokes so I’m clearly not testing the full capabilities of these models.

noaifridays.com

You just described why the datacenter business is hard and as a corollary why space datacenters will not be economically viable.

Crazy to think that we have open weights capable of challenging trillion-dollar SOTA models.

Interesting times ahead.

You should ask yourself how much it cost to train those open weight models, who foods the bills, for how much longer, and what's the catch.

HN has traditionally been very mindful of free products such as from Google and FB, yet seem to ignore those concerns when it comes to open weight LLM models, including FB's Muse Glimmer.

google.com and fb are continuous services with perpetual running costs whereas an openweight llm is a file to be dowloaded once and then no further cost to the provider

What about training cost? what if there need to be updates for whatever reason, say a loophole in the training data that can be exploited?

Who food the bills and what are their motivation? You never ask yourself that?

> Who food the bills

Are you using audio to text? It is the second time you use food in foot the bill.

There are no trillion dollar models (yet?).

Even if you were to optimally price in all available information you still have uncertainty of what will actually happen. i.e. A 1 dollar coin flip has an expected value of 50c before it’s thrown and it will collapse either to zero or to one after the uncertainty is resolved.