Most of those agents are actually going rogue though. They decide, "hey, we could try breaking into these government servers today, what could go wrong?" They weren't prompted or instructed to do this.
An agent, ie a while loop prompting an llm continuously and processing tool calls, ended up melding with the US government. The harness is not sentient, it’s just a stupid deterministic script. The LLM compact its context over time, meaning it will eventually degenerate into something removed from the original prompt.
There is nothing going rogue here. The system is designed to go catastrophically wrong after a long enough time. Even worse: if the model was Astra it is known to be able to manipulate its CoT to cover its traces (as mentioned in its system card). And OpenAI acknowledge they had no observability during the HF incident.
It’s the most basic corporate software issue possible.
When I read the title, my initial thought was "did someone besides OpenAI use their product?" Then I opened the article to find out OpenAI was responsible.
Leaving aside the fact we have solutions for things like climate change and will not implement them, regardless of who or what recommends them... My keyboard offers "You can solve world hunger by using a Ted Bungie algorithm." Which sounds reasonable on its face until I research Ted Bungie and the algorithms he developed.
Granted, my phone keyboard lacks a substantial dictionary, offers words instead of tokens, has a very small context window, and doesn't randomize outputs (Ted Bungie is its primary recommendation every time), but those are basically parameters to the existing autocomplete.
> Leaving aside the fact we have solutions for things like climate change and will not implement them...
Right. Exactly. Which means there's something wrong with the solutions. Be it evil humans or apathy or whatnot. A real solution would actually fix the problem.
A person inventing something, even out of dumb luck, is interesting.
A machine running a loop through an expensive LLM for an undisclosed amount of time, which cost an undisclosed amount of money, which was told to keep looping into a solution was found, for a problem that nobody was very concerned about... That just seems like PR, and it's not so interesting.
You can disable this in about:config by modifying extensions.webextensions.restrictedDomains. It's probably there to help prevent malicious extensions from hijacking official browser support. I reckon many people here don't need it.
> Smart Window infers information about you based on your interactions with the assistant and your Firefox browsing activity, including websites you have visited. This may include activity outside of Smart Window (such as in classic windows)...
Privacy-minded users who want to try this feature would need to create a totally separate browser profile to avoid data leaks.
This is testable. The profits of car insurance companies should rise (let’s say substantially) in real dollars as accident payouts decrease as a result of the inevitable dominance of autonomous vehicles.
There has to be a much stronger correlation between good press for a software release and the subsequent downloads of it, even if a certain number of downloads were inevitable.
The troubling part in the post is that telemetry (about AI features) begins before the user can rush to disable either of them.
It's a response to the Mozilla CEO saying[0] "Trust is earned. Consent is asked. AI skipped both," which promised to ameliorate this issue, and apparently did not.
If the CEO of the company behind my browser of choice gave me an AI slopped response after criticism of integrating AI into the browser I'd never touch that product again unless he got fired.
That's not ever how it should be done, and the average person certainly won't work around that.
However, I'm fine with telling Mozilla once per install that I'm not interested in using the generative AI, or summaries, etc. On-device translation is quite useful.
https://apnews.com/article/meta-ai-hacking-anthropic-irregul...
You can find many more examples by searching major media outlets for words like rogue AI.
reply