Would be interested in this too, I did some work in the past to make it work via Envoy proxy using HTTP CONNECT but that requires plugging in proxytunnel[0] or nc on client side.

  > $ nslookup abc.exe.xyz  
  > abc.exe.xyz canonical name = s001.exe.xyz.  
  > $ telnet s001.exe.xyz 22  
  > Trying 100.20.12.135...  
  > Connected to s001.exe.xyz.  
  > Escape character is '^]'.  
  > SSH-2.0-SSHPiper

Looks like it uses sshpiper[1]?

[0] https://github.com/proxytunnel/proxytunnel

[1] https://github.com/tg123/sshpiper

They seem to have a similar concept: https://www.youtube.com/watch?v=UqS3NxJ2L_I But I think very few think of Alexa as a personal assistant, most people are just happy playing music and toggling room lights.

Disclaimer: I work at Truefoundry

You can give us a shot at https://truefoundry.com We are a general purpose ML Deployments platform which works on top of your existing Kubernetes clusters (AWS EKS, GCP GKE or Azure AKS) abstracting away the complexity of dealing with cloud providers and Kubernetes. We support Services for ML web apps, APIs, Jobs for ML training jobs, Model Registry for storing models, Model Servers for no code model deployments. (Our platform can be partially or completely self hosted for privacy and compliance)

Adding one or more GPUs (V100, T4, A10, A100, etc) is simply one extra line https://docs.truefoundry.com/docs/gpus#adding-gpu-to-service...

Examples:

- Stable Diffusion with Gradio: https://github.com/truefoundry/truefoundry-examples/tree/mai...

- GPT-J 6B fp16 with FastAPI: https://github.com/truefoundry/truefoundry-examples/tree/mai...

Love TrueFoundry! We use it for Infra provisioning on our own cloud and deploying the ML Models behind a choice of a specific model server. Pricing model is also good for early start-ups :)

PS: (I am one of the founders) - you can write to us at founders@truefoundry.com We can help understand your use case and try to suggest whatever is best from whatever is available in the ML Serving ecosystem.