It wouldn't be easy, but I'm imagining it would probably require identifying the other side of some historical transactions, and being able to prove that. Like if you had ever sent to/from any other address you held the key to, you could sign a transaction from that other address.