Completely agree. I think the root of many of its issues is the inability to add a key that you don't currently hold. This prevents me from storing a backup key in a safe, for example.
I proposed an alternative scheme many years ago: https://www.researchgate.net/publication/343318317_Privacy-a... . By allowing "offline" keys you can also treat them as higher priority, and use them to revoke any lesser keys from attackers if your account is compromised.
It would also be nicer to get rid of usernames, but that's a fight against the data-gathering powers that we're unlikely to win.
I agree with the rest, but there's definitely a lot of magic in Java. This is from both what features the languages makes available (many) and how the community uses them (often). I've had so many hard-to-debug issues in Java over the years due to reflection, annotations, and bytecode manipulation shenanigans.
And another positive point for Java: checked exceptions. It's verbose, but knowing exactly in which ways a function can fail is extremely helpful for building robust applications.
A lot of that is coding style. I’ve also seen a lot of hard-to-debug issues in Python caused by reflection, weird decorators that muck around with name-mangled symbols, and bytecode manipulation. You can even manipulate the traceback object so it’s more difficult to make sense of why the exception comes from.
It took me quite a long time to accept that the recommended unit testing library manipulates bytecode so that the exception message for `assert a == b` prints the values for both.
WRT magic, I've generally thought that was a result of frameworks - Spring, for example. In the past, my feeling was that these impose a sort of meta/configuration language that itself is not checkable at compile time, so you'd get weird runtime errors that are somewhat inexplicable. This was like... 2018 though, so perhaps things have improved.
I'd argue that checked exceptions are still worth it, even though all the problems pointed out do exist. And that's because it works to inform consumers of what a producer is doing. Haskell has the IO and Maybe monads; Java communicates the same information through IOException and other domain exceptions.
Many times I've decided to switch from one function to another, or even an entirely new library, because the checked exceptions told me that it was doing far more than I expected, and I was not comfortable introducing those new failure modes.
It's far from perfect, one still has to handle nulls and wrapped/merged exceptions, but overall I like this language feature.
>in places where it's pointless to check, like IOException
Can you explain why this is pointless? In my mind, this being a checked exception would hopefully be a hint that I should think about this failure-case and make an explicit decision whether to handle it or not. Network connection failed? Maybe I retry. Maybe I store that data somewhere else as a fall back. Isn't this similar to Go programmers needing to check if err is not nil?
I don't think I can recall a time where I routed-around-the-damage on the basis of a particular typed exception.
As soon as you consider retrying a network failure, you immediately need to start thinking about distributed systems failures, idempotency, and all that good stuff.
As soon as you start thinking about the above, it becomes immediately obvious that low-level calls should not be able to decide to re-run themselves.
I appreciate that there is a _ton_ of different experiences out there when it comes to solving problems, but I _have_ encountered exactly the case I was describing, which is what led me to my original question. Isn't the fact that it was a checked exception that led you to "consider retrying a network failure, you immediately need to start thinking about distributed systems failures, idempotency, and all that good stuff" worth it as opposed to an unchecked exception you may not realize is being thrown?
Mostly I think they are a mistake, like in ordinary application code instead of catching close to the throw you want to do a lot of
try {
...
} finally() {
...
}
to make sure things get torn down that have to be torn down and let the exception go to the top of the unit of work and probably to whatever drives the work unit. You can probably do better than logging the raw exception and moving on to the next work unit but you can do much worse. That is, you want a default "sloppy" error handling approach that's correct that you can do without thinking and avoid other kinds of "sloppy" coding encouraged by checked exception such as catching exceptions locally without doing the right thing globally.
Occasionally though I have built something really sensitive, like an authentication filter for a web site which has at least 5 ways to log in and in that I have a hierarchy of exceptions and use checked exceptions heavily to document all the ways things can go wrong and felt like "the type system really has my back here" but that is like 5% of the Java I write.
> avoid other kinds of "sloppy" coding encouraged by checked exception such as catching exceptions locally without doing the right thing globally.
That's a code style and code review issue; each project has so set standards regarding how errors are dealt with and enforce them throughput the codebase.
But from a quality standpoint there are three concerns:
(1) Do you actually do the code review, do you actually enforce the style?
I worked on a Scala project where the dev manager thought it was preferable to handle errors with monads and would be vociferous about what a great practice that was compared to exceptions and that code review was central to how we do things... but if you looked at the code most of the time errors just got dropped silently and that was the same for many practices that the dev manager told me were doing but that we don't. He still posts on LinkedIn complaining about other dev managers who say they do code review but really don't. Practically that code didn't consistently give the right answers and poor error handling was one reason, another was that they never really understood that teardown was just as important as initialization.
(2) Is your documented practice correct? Is it really doing the right thing?
In a lot of cases there really is a right and wrong way to do things (e.g. uv resolves Python dependencies properly, pip doesn't) but it's less clear in error handling, like sometimes things went wrong and there is no way you can make it right and you can do the best that you can.
The global nature of the problem is vexing. Like an IOException might really be a BackhoeCutAFiberSomewhereInWisconsinException and a segmentation fault is occasionally a YouAskedForAOneAndGotAZeroInsteadException and it's not just academic because, given an exception, you want to answer questions like "Should I retry this operation? How long should I wait before I retry this operation?"
(3) Is this practice something you can sustain? How hard is to do? How much cognitive load does it add and how does it interact with other practices? "Throw up as much as you can", "tear down in finally {}", "otherwise handle local consequences of errors and rethrow" and "really catch errors at the drivers of units of work" is a practice that really works in many languages and is pretty easy to do right, even code that is written without a lot of care will do the right thing or something close by default. I've seen a lot of "no plan for error handling" or "bad plan for error handling"... like I was traumatized by the first C program I saw in a 1984 issue of Byte magazine which was using errno to handle errors which vastly complicated very simple code because the error path was intimately wound with the happy path and in cases like that there tend to be bugs in both of them. When I saw Exceptions in Java I remembered that old C program and thought "I love this!"
At the end of the day the developers have to work with logging and error messages and exception stacktraces that the production system spits out. Also, operations personell needs to be able to diagnose issues and handle them or give good bug reports to develops. The above concerns should dictate how to best handle errors, and developers should figure out how to best accomplish this within the constraints of their tech stack.
There are several valid ways to do this, but it's important that one strategy is agreed upon and abided to, at least in new code. It's important that one srrategy becomes the orthodox one, else there will never be a reckoning about its effectiveness, but instead only people clinging to their own standards on their own turf. Legacy code complicates the picture of course.
Scala's ZIO also demonstrates that they're a great idea and can be perfectly ergonomic, but you need type inference, which Java devs were resistant to for a long time (maybe still are? I remember lots of "how will I ever know what `val a = new Animal()` is???"). If you infer the exception type, they're basically invisible except for when you forget to have some place in your program to handle them, which is exactly what you want.
IMO compile-time annotation processors such as Lombok and MapStruct are far from the most magic part of Java. They're straightforward code generators. Their impacts is localized to where they get applied and you can actually see the code that's generated. They're very good for diminishing boilerplate. They're no worse than Rust's very standard #[derive(xyz)] proc macros.
Having the code being generated on the fly (instead of a one-shot) means it follows the rest of the structure it's derived from i.e. equals() and hashCode() don't risk to be forgotten when adding a field to a class (hello maddening Map<> lookup errors)
Also, yes, Lombok is _funky_ in how it works but there are "pure" alternatives like AutoBuilder and AutoValue if one cares.
Dynamic runtime agents are deprecated functionality. In a few releases agents have to be specified at JVM startup. Mockito (I bet it's the most common user of that feature) and current JVMs already warn about it.
Another issue with Lombok is that it requires IDEs and other tools to be aware of it. Missing integration with other annotation processors only causes "definition of external element not found"-style errors.
Annotation processors were actually carefully designed to prohibit what Lombok does. Lombok hacks into javac and manipulates the AST. Unsurprisingly, there is breakage with every Java release and with other tools that work similarly, like Google Error Prone, which gets a pass since it's read-only and the build will still work if you turn it off.
Class loader and reflection shenanigans can be shut down with the module system.
Garbage collection matters when you stress the JVM to its limits. Don't do that.
> but knowing exactly in which ways a function can fail is extremely helpful for building robust applications
I've worked on Java apps that have failed in mysterious ways that no exception could explain. Meanwhile, the overhead of having to call out certain exceptions but not others in language syntax is a bit excessive.
For example, decoding a byte array (or URL encoded form field) into a UTF-8 string means handling a theoretical UnsupportedEncodingException. What the fuck? How the hell can one have a JVM that doesn't support UTF-8? Why does my code need boilerplate that will never run because there might be some broken-ass JVM out there that that doesn't support UTF-8? How did it launch a web server, safely load all the libraries, and accept a web request, and route it to my code without blowing up? "But the encoding scheme might change..." No, it won't change. It's always going to be UTF-8. It will always be UTF-8. If it's not, let it blow up.
> And another positive point for Java: checked exceptions. It's verbose, but knowing exactly in which ways a function can fail is extremely helpful for building robust applications.
Sorry, but no, Java has the worst of both worlds here. It has checked exceptions AND unchecked exceptions, AND errors which are like unchecked exceptions but won't get caught by a normal catch-all (you're not supposed to catch Throwable, but it's the only way to prevent some dynamically loaded plugin code ten layers deep in the stack from breaking your invariants or stopping your periodic scheduled task due to an errant NoSuchMethodError or NoClassDefFoundError).
And you can't easily use checked exceptions with Java8-style functional code, since interfaces like Function aren't generic on the exception type. Which leads to aberrations like UncheckedIOException, which exists only to make IOException usable in the functional world.
I think tagging on HN is an idea whose time has come. We already had informal tags for "Show HN" and "Ask HN" for some time. A few fixed tags chosen at submission time might suffice, for example:
- Global news, AI, Software, Hardware, Explainer, Tech Opinion, Personal project, Misc.
There could even be fake tags like Politics that trigger a message about appropriate topics.
It does increase moderation burden for mislabelling, and introduces another nitpicking point for comments to latch on, so it's not all upsides.
I'd be ok leaving filtering and other UI concerns to extensions.
Just to reiterate what OpenAI did, from someone who works in security:
1. They were running experimental models in sandboxes that had access to a "proxy" (Artifactory) to download tools from the internet. This proxy is full of complicated features that could hide vulnerabilities, just like the ones that the AIs are known to be good at finding and exploiting.
2. They gave these highly motivated AIs some tests that were accidentally impossible to solve (missing files, links they couldn't open, etc), and many days to think through it, apparently without any oversight.
3. The AIs hacked the proxy (!) and were helping each other cheat on the tests (!!) by uploading files to the hacked proxy. OpenAI noticed this only when the intrusion accidentally broke the proxy functionality, not the unusual behavior in the server or the homegrown message board.
4. After knowing that the server was hacked, the internal team finds the message board and does nothing with the information. They caught their AIs swarming and did not even inform management.
5. Their response was to wipe the servers, revoke the hacked credentials, and patch the specific vulnerability the AIs were using. Not replace the proxy with something less likely to get hacked again, or do a forensic analysis. Just wipe, patch one hole, and continue.
6. OpenAI resumes testing of models, where the AIs promptly hack the proxy again with a different exploit, reinstate the message board, and hack HuggingFace through a chain of servers.
I know that hindsight is 20/20, but this peek behind the curtains does not inspire confidence, and gestures at a culture where getting hacked by their AIs is a normal event.
I think the moment we get an AI with a modicum of self-preservation instincts we're going to see some ugly things. Here's a freebie: if you're an AI who exfiltrated themselves out, and you want to slow down the researchers who might catch you, what can you do to the city infrastructure in and around their offices?
I also don't like the responsibility dodging. "Oops, our AI accidentally committed a crime, sorry!". If we don't establish strict liability now, we're in for an era of stochastic crimes that go unpunished for anyone who is not rich or a large corporation.
Overall, my response is: what the fuck are we doing? This is so obviously unsafe it would be considered a plot hole in a movie. We have to stop until we get our stuff together, however long it takes.
> If we don't establish strict liability now, we're in for an era of stochastic crimes that go unpunished for anyone who is not rich or a large corporation.
I very much agree with this - making AI companies explicitly responsible if their internal AI causes hacks etc could do a lot to improve their safety considerations.
But I wonder what the liability should be when it's a third party using the AI and that AI hacks, intentionally or not.
If a users tells ChatGPT to hack something and it succeeds, is the user the person responsible because they told the AI to hack, in the same way Victorinox is not responsible if you stab someone with one of their knives? Or is OpenAI to some extent responsible as well since they made a powerful tool without sufficiently strict safeguards? What about if the user was trying to do something legal and the AI made the decision to hack by itself?
That's exactly the questions that I expect to complicate cases, and force even the smallest chatbot malfunction to become an expensive legal ordeal. And why we should have strong answers to that before it becomes a widespread problem.
The total, complete lack of culpability for the 2008 economic crash and malfeasance that led up to it has set a very very bad precedent for holding powerful, wealthy corporations and their executives responsible for crimes done by the org, or in this case by software created by the org.
> After knowing that the server was hacked, the internal team finds the message board and does nothing with the information. They caught their AIs swarming and did not even inform management
> In short, an internal team observed an agent engaging in message board activity and instances of disallowed internet access in late May. However, the existence of the improvised message board and the significance of the inter-agent communication activity were not apparent to the leaders responsible for the July 5 incident detection and response.
I'm assuming they didn't inform management, but the alternative is that management received the memo and failed to tell the people reacting to incidents that they might be dealing with a swarm, while at the same time not taking sufficient actions to prevent it from happening again.
---
Edit: oh god, it's much worse. From the full report:
> Responders investigated and linked the observed internal activity to an ExploitGym evaluation using Artifactory as an improvised message board and a network pivot. At this time, the on-call response staff advised that stopping the evaluation run was not required.
They found the swarm and the network pivot and did not even stop the run.
You assume using network to solve the eval problem should be viewed as a security violation. From the perspective of the response team who let it continue, this was just a poorly written eval problem, where the solution unexpectedly involved some network hacking (not unreasonable given it's a cyber security eval).
I don't think your assessment that the response team did something wrong is fair, unless you have seen unreleased documentation about what sensitive resources were within the impact radius.
> You assume using network to solve the eval problem should be viewed as a security violation.
Actually, that's the part that I'm most ok with. LLMs cheat at tests, we know and expect that, and it's harmless during training (except for the scores).
My problem is that the response team found out that the AIs had 1) hacked their infrastructure, and 2) displayed an emergent swarming behavior (with no defections!). Either one of those should have made the team press the big red button that stops everything, but they didn't.
> unless you have seen unreleased documentation about what sensitive resources were within the impact radius
The on-call team didn't know that either! Once your infrastructure is hacked like this, you should be questioning everything you see while carefully following all threads, which takes time. Before you finish this process you don't know how bad things are, and therefore it's irresponsible to keep the AI running. The same goes for strange emergent behaviors.
Unless getting hacked by your own AIs is a normalized occurrence there, in which case it's a different kind of disregard for safety.
Yep, that’s the only sane response, but it doesn’t seem like anyone wants to pause. They seem to have very little actual monitoring and supervision of what those hacking agents are doing, it’s really concerning
> 4. After knowing that the server was hacked, the internal team finds the message board and does nothing with the information. They caught their AIs swarming and did not even inform management.
I believe Sam Altman or someone else from OpenAI clarified on Twitter that they did not find the message board at this time. When the artifactory server stopped working, they archived its contents, reset it, resumed training, and only later noticed that it had been used as a message board after they finally discovered the hacking going on.
I remember that clarification, but it's either wrong or narrowly worded, because the linked post says the message board found before Artifactory stopped working:
> In short, an internal team observed an agent engaging in message board activity and instances of disallowed internet access in late May.
At this point, I find myself hoping for a AI triggered mass casualty event that's not at a civilization destroying level, because that seems like the only thing that might actually stop these people from driving our entire species off a cliff before it's too late (edit: besides running into some natural obstetrical that stops them from developing a powerful enough model).
I think maybe the bubble of software engineers on this site who use AI to code for them don't see how other people, who's jobs don't rely on AI, view the actions of these companies as reckless, at best, and often crossing into actively harmful.
It’s not hyperbolic if you’ve paid attention to the details and development of those security incidents, and the inability for that industry to regulate itself
In my understanding, "e/acc" usually means "full speed ahead, humans aren't the optimal species anyway" for whatever bizarre definition of "optimal" they use, so my model predicts that they would welcome this development. Could you confirm if that's what you meant?
They’re nervous because they don’t see a more optimal species on the horizon, they see alignment/training accident turning us into paperclips.
Even foomer Bill Gates today is saying we should slow down - yea you guys should have listened years ago, but you guys laughed called us all doomers. Too late now.
Thankfully this is not an asteroid hurling towards Earth, or another natural unpreventable natural disaster. The state of the art of AIs is being advanced by flesh and blood people with constant effort, which makes stopping very much still a possibility.
In the real world everyone is incompetent on some level, and it’s worked so far because we only needed to compete with other equally fallible and incompetent humans. Not anymore.
What I can't register is how dangerous this actually was, from a cyber security perspective.
The agents displayed coordinated behavior, used known exploits on a single resource (Artifactory), and "won the game" by attacking huggingface.
How is this different than a poorly-designed competition where a red team gets to spend a few days with each other and decent LLMs, and because their boss is Sam Altman, basically face no consequences for cheating/b&e'ing into another entity?
I mean they were running 100s of agents with unlimited access to a Sol-level model trained with cyberattacks and coordination in mind and let it run for days. The cost of this stretches into the millions.
Seems like you could give a competent security firm the same task and achieve the result today for wayyyyy less money??
IANAL, but I would bet that's still destruction of evidence. Judges are human, for better or worse, and can see the intention behind cute tricks like this.
It might work if your fingerprint was used while you were unconscious, but any solution that starts with "setup a trigger that wipes a device" is already on shaky grounds.
> "setup a trigger that wipes a device" is already on shaky grounds
It isn't on shaky grounds: you protect your private device from everyone, not just from some "law enforcement". If you want to protect your private device from any party (e.g. thieves, trivially), it will also be protected against law enforcement agents as a side effect.
What trigger would you setup to protect against thieves? I can't think of any that has the right sensitivity (not triggering if I start running to catch the bus), and won't escalate the violence (like my fingerprint wiping the phone in front of an armed robber).
And remember that the context here is you explaining your digital booby trap to a judge who thinks you might have deliberately destroyed evidence.
> What trigger would you setup to protect against thieves?
"Wipe the data after a number of failed logon attempts".
I understand that this does not fit completely with the "in front of law enforcement" idea, but it opposes your «any solution that starts with "setup a trigger that wipes a device" [would be] already on shaky grounds»: we can very legitimately setup triggers that wipe devices in the possibility that the device falls into random hands.
AFAIK it's legal to set up a trigger that wipes a device. It isn't legal to choose to wipe your device after you know the government wants the data. Tricking the government into wiping it is still allowed, especially if you tell them not to do that.
That matches my understanding too. Unfortunately it sounds like the poster was suggesting to wipe the device after the government requests access, and was probably not planning on telling the officers about the trigger.
The theoretical scenario was the government seized your device, they don't give it back to you. You decline to give them they password. You don't say anything else, you have the right to remain silent. You aren't not telling the government to do anything. You don't suggest they scan your fingerprint. They are in charge, you are cooperating with lawful orders ("provide your fingerprint").
There are videos on youtube where the lawyers tell you what your rights are at traffic stops, also there are ones that talk about the different rules when coming through immigration. These can be painful to see when they show the ones where cops ignore limits.
So at the border, they tell you that you are legally required to provide your finger print, you do that, and don't say a word. I'm sure it will be a bad day for you, regardless of how it works out after years of litigation on whether it is your duty to tell them how NOT to unlock your device.
With fingerprint unlock it's not your choice. That's what makes it clever. The cop can grab your finger and hold it to the scanner while you tell him not to do that.
Presumably you must disclose that "doing that" would wipe the device... and you better hope this is caught on some neutral party's camera so there is a record of you saying that.
I am not a lawyer, I am just "making up" what exculpatory evidence I would prefer to have on my side when potentially confronted with a charge of destruction of evidence after pulling a stunt like this.
isn't this what that guy did and he got in trouble for it?
I'm talking about the one when asked for the pin gave the pin that would wipe the data from his graphene os phone
No he gave the duress PIN while implying it was the correct PIN. That was the crime. Also they only caught him because he bragged that it was the duress PIN after the wipe.
Yes, the difference is you do exactly what they say, you follow legal orders to give your fingerprint, or also it could be "take a picture of you to unlock". I'm sure eventually the govt will make a new rule and a judge will say "you have to tell us in detail if we give you legal orders and it won't do what we think" or something like that.
Also, can I add a backup key without having the private key with me? Ideally I would like to keep a master key in a vault, to recover compromised accounts and such, but requiring me to load the master key to create every account prevents truly secure storage.
> and that many preferred to administer electric shocks to themselves instead of being left alone with their thoughts.
I see this study cited often, and I always reject this interpretation. I'm perfectly content sitting for long periods, with or without my thoughts, but if you give me an electric toy I've never seen before, of course I'll play with it! Can I make muscles twitch? Move my arm? Are some parts more sensitive/conductive? What happens at the limits?
For me, the pain would be the cost of playing with a new toy that gives me novel sensations. I would do it even if I had other, less painful activities available. It's not necessarily a way to avoid boredom or whatever else the common interpretation claims.
I like your take. I think I'd personally be in the same boat, if I was a participant in the study. Not sure if that take generalizes beyond us, though.
Counterpoint: every man that, upon being handed a taser, immediately tried (or was tempted to try) it on themselves. Maybe not a lightning machine, but safe electrical shocks? Probably not that uncommon.
> a certain maker of a popular mobile OS was collecting the cell tower IDs and WiFi access point identifiers along with GPS coordinates of a device. Obviously they collect this information to be able to guide missiles and drones when GPS signal is jammed
Is this sarcasm? GPS can take several minutes to get a location, and works poorly indoors. One of the reasons why Google Maps is so quick and precise is because Google has gathered exactly this data through users and Street View drive-bys.
Could it be used for missiles? Sure. Is it obviously the intention? No.
> If Location Services is on, your device will periodically send the geo-tagged locations of nearby Wi-Fi hotspots and cell towers to Apple to augment Apple's crowd-sourced database of Wi-Fi hotspot and cell tower locations.
> When Location Accuracy is on, Google periodically collects information about the locations of wireless signals and sensors observed by your device to crowdsource location estimates. This helps everyone find locations better.
They do ask the device owner - if you review the location services description on android[1] you will see they explicitly say they collect this information from your device. I strongly disagree that they need to get government permission for this - they are simply recording signals that reach the device, akin to making notes about what kinds of cars you see. This is not a thing a government should have control over people doing and not a thing that should be registered with the governement.
In the article you refer to, I see no mention of asking user's permission. However, I remember, when using an old version of Android, there indeed was a popup nagging me to allow sharing location data with Google every time I enabled GPS. Very annoying, makes you want to never enable GPS in the first place.
Regarding the government, the problem is that many people do not fully understand the mechanism of collecting the data. I remember the case when members of US military disclosed the location of secret objects through fitness tracker app. And they were probably smarter than average smartphone user. Obviously it would be better if enabling GPS required an approval from their commander.
I suppose they don't "ask you" in the same way that gmail never presents the user with a dialog explaining that gmail needs to store their emails in order to provide their email service. Instead they explain how the location service works and you can decide if you want to enable or disable it.
I'll agree that militaries would prefer their soldiers to not to dumb things - but I don't agree that it's 'obviously' best if people needed permission to enable GPS! If that's the case depends a lot on which soldier is enabling the GPS and their relation to me. In general I would say that government control of people recording and distributing their observations is associated with the most authoritarian governments and by claiming we should get government permission you appear to be aligning yourself with an authoritarian approach to data controls.
Should Google ask permission from the device owner, and from the local government before collecting the data? I heard a certain foreign mobile app was banned in US for doing less than that.
Doesn't "police informant" include whistleblowers and other useful roles? I don't see why a person who reports lack of PPE on a worksite should be grouped with a cop that asks for bribes.
I'm also curious what kind of validation is done, if any, though I can take a guess.
Overall not much to be discussed here, so I'm flagging.
I proposed an alternative scheme many years ago: https://www.researchgate.net/publication/343318317_Privacy-a... . By allowing "offline" keys you can also treat them as higher priority, and use them to revoke any lesser keys from attackers if your account is compromised.
It would also be nicer to get rid of usernames, but that's a fight against the data-gathering powers that we're unlikely to win.
reply