pstadler 12 days ago [-]
Use WireGuard[1] instead. It's way faster than Tinc and other userland VPN implementations. I've been using it for the same purpose as the author of the article and it has been rock solid - not a single issue during almost two years. Setup and configuration is a breeze[2].

[1] https://www.wireguard.com/ [2] https://github.com/hobby-kube/guide#wireguard-setup

Edit: Benchmarks on Hetzner Cloud (1vCPU, 2GB)

  $ iperf3 -c kube1
  Connecting to host kube1, port 5201
  [  4] local 10.0.1.2 port 57622 connected to 10.0.1.1 port 5201
  [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
  [  4]   0.00-1.00   sec  77.2 MBytes   647 Mbits/sec   79   1.37 MBytes
  [  4]   1.00-2.00   sec  78.8 MBytes   661 Mbits/sec    0   1.51 MBytes
  [  4]   2.00-3.00   sec  81.2 MBytes   681 Mbits/sec    0   1.62 MBytes
  [  4]   3.00-4.00   sec  85.0 MBytes   713 Mbits/sec  134   1.20 MBytes
  [  4]   4.00-5.00   sec  80.0 MBytes   671 Mbits/sec    0   1.28 MBytes
  [  4]   5.00-6.00   sec  77.5 MBytes   651 Mbits/sec    0   1.33 MBytes
  [  4]   6.00-7.00   sec  88.8 MBytes   745 Mbits/sec    0   1.37 MBytes
  [  4]   7.00-8.00   sec  73.8 MBytes   619 Mbits/sec    0   1.39 MBytes
  [  4]   8.00-9.00   sec  78.8 MBytes   661 Mbits/sec    0   1.41 MBytes
  [  4]   9.00-10.00  sec  80.0 MBytes   671 Mbits/sec    0   1.42 MBytes
gant 12 days ago [-]
Running Kube on their cloud servers? Well have fun with that, the "vCore" is a very inconsistent unit unless you get their dedicated core servers. I moved back to Hetzner Bare Metal because you can't have anything that will push the resource boundaries on these boxes.

Also regarding Wireguard, I really like how tinc will find a new path and allows you to route over other nodes as needed. Wireguard can't really do that out of the box, every link is 1:1. You can of course setup something on top of that, but I miss the ease with which tinc does this.

chrismeller 12 days ago [-]
I was actually surprised at the lackluster performance on the cloud products as well and recently spun up a dedicated box for a workload that actually required consistent performance. I never expected the performance to match a bare metal option of course, but coming from any of the other cloud providers I expected it to be more equivalent than it turned out to be.
subway 11 days ago [-]
along the lines of automatically re-routing, tinc also has some neat anycast-like capabilities -- you can assign the same ip to multiple nodes, and the lowest latency/shortest route node wins
jmngomes 12 days ago [-]
I was considering using autossh to create a private link between servers, because in my case it's only a handful of servers.

Can you comment on how stable a Wireguard tunnel is? Did you manage to get the link/VPN to stay up permanently with little to no maintenance?

amaccuish 12 days ago [-]
There isn't really a up/down of wireguard, once the interfaces are configured, you just start pumping packets through, it's pretty invisible.
pstadler 12 days ago [-]
Found it to be incredibly stable, plus the links are self-healing due to its design.
jamescun 12 days ago [-]
I second Wireguard, I've been using it recently instead of an overlay network in Kubernetes (configured as a kubenet). Incredibly easy to set up and very performant.
amq 12 days ago [-]
Could you describe how you did it?
cbluth 12 days ago [-]
Some information on how to do this would be awesome
zaarn 12 days ago [-]
I would use WireGuard tbh, but I use pfSense for Networking and there doesn't seem to be a userspace implementation available that runs on it (I did try some FreeBSD binary that I copied over but that didn't quite work out).
moviuro 12 days ago [-]
The -go version should work, though (if compiled correctly). https://git.zx2c4.com/wireguard-go/
ibotty 11 days ago [-]
I hope https://github.com/gsliepen/tinc/issues/179 becomes a reality: tinc ui and features on top of wireguard!
romantomjak 12 days ago [-]
I did know about this, but it looks very interesting! Will defo check it out, thanks!
mwest 12 days ago [-]
You can achieve something similar with Hetzner's recently introduced "vSwitch feature". Works across their different DCs, which is nice. Some docs here: https://wiki.hetzner.de/index.php/Vswitch/en

I've been using ZeroTier to give a common backplane to my Hetzner servers, DO droplets and AWS instances.

jmngomes 12 days ago [-]
I understand this may not be an issue in your case, but vSwitches won't encrypt your data in transit between servers, unlike a VPN or ssh tunnel.
gant 12 days ago [-]
It depends. I've seen some shit on cheap bare metal providers, including getting ARP poisoned on Online.net.

Hetzner has been great overall. They've been very very helpful in documenting me reacting to abuse emails too when I got into some user-generated-content related legal trouble.

fapjacks 11 days ago [-]
I really have to second the praise of Hetzner overall here. I have run a couple of their dedicated machines for several years and have nothing but good things to say about them and their service.
chrismeller 12 days ago [-]
vSwitch is only, AFAIK, available on their dedicated servers (well, anything in Robot... which also includes their legacy virtualized product). OP is using their new cloud offering, which doesn't have an equivalent option.
12 days ago [-]
danielh 12 days ago [-]
> Normally you only get one public IP and no private interfaces.

From my understanding, this statement is not quite correct, as Hetzner allows you to set up VLANs:

> With the vSwitch feature, you can connect your dedicated root servers in multiple locations to each other using VLAN via the administration interface Robot.

You probably still want to encrypt the traffic passing through those VLANs.

They also offer the option to install custom hardware, so you might even be able to get a second NIC connected to your own private switch.

chrismeller 12 days ago [-]
That only applies to their dedicated servers. OP is using their cloud offering, which doesn't support this feature or custom hardware.
TomMarius 12 days ago [-]
Isn't the point of DO's private networking that you don't need to encrypt the traffic? Or is it just internal, but not private?
jarym 12 days ago [-]
Well private just means it’s isolated from other networks - it doesn’t mean that your ‘private’ network can’t be snooped (by Hetzner, hackers, etc.)

We’re experimenting with Wireguard on all internal hosts and disabling SSL.

chrismeller 12 days ago [-]
Yeah, I found that an odd comparison to make as well. If you want encrypted traffic that's all well and good, but there's no reason to assume that the private network is going to be any different performance wise than the exact same encrypted solution over the public interface - a network is a network is a network in this case.

Since the goal was to have a private network between your own boxes, the encryption was only really "required" to protect private data because it had to transit the public network in Hetzner. Since DO provides a private network natively there's (in theory) no justification for the encryption, which means you'd get native performance, hence the advantage.

pstadler 12 days ago [-]
Are you sure DO's private network traffic is actually encrypted or even isolated? Back some time ago, any host within the same private network could be reached. I wasn't surprised to see connection attempts from random hosts on eth1.
TomMarius 12 days ago [-]
Other comment there talks about it, they changed it a while ago and now it's isolated, but not encrypted.
nsomaru 12 days ago [-]
I’ve heard DOs internal traffic is internal not private. You’ve got to lock your boxes down anyways.
nicolaslem 12 days ago [-]
This changed a few months ago. You still share the same private network with everyone else in the DC, but only machines on your account can communicate with each other.
therealmarv 12 days ago [-]
that's good to know! thanks for this information.
_Codemonkeyism 12 days ago [-]
What about Zerotier with Hetzner?
jbverschoor 12 days ago [-]
Yeah I'm not sure why zerotier is not getting enough credits here on HN. It works flawlessly, is super fast, easy, works on iphone, and they have a small hardware box now.
chrisper 12 days ago [-]
Zerotier doesn't do PFS Perfect Forward Secrecy... and somehow it is too easy to add new clients to the network without you noticing.
radiowave 12 days ago [-]
IIRC in Zerotier new clients are easily added, but traffic to and from them is blocked by default, until you approve them in the web interface.
manigandham 11 days ago [-]
New clients have to be approved before they can join. How would you not notice?
chrisper 11 days ago [-]
Where do you approve them?
manigandham 11 days ago [-]
The online control panel where you setup your private network in the first place. This is where you configure the IP range and other settings, and accept any devices that try to join.
jbverschoor 10 days ago [-]
It's too bad you mae these comments without actually having tried it.