Super User Asked by thistleknot on December 11, 2020
I setup a bridge, that’s fairly straightforward. But then it occurred to me that none of my nodes are ever going to pull more than their own 1Gb worth of bandwidth from the server and I have 2 nodes and 1 NAS that are all 1Gb. So I thought why not use these directly to the server (unfortunately the switch I bought is 10/100, I do have a Gb router with 4 ports that I’m utilizing at the moment, but that limits the # of connections I can uplink from the 4 nic server, [i.e. I’m only using 1 atm]).
So what I’d like to do is hook up my 2 nodes to 2 ports, and the NAS to 1 port, but all these devices (with the exception of the NAS) need to talk to each other. When they are on a switch, no big deal, but on the ports bridging doesn’t do the job. I read I need to enable routing.
However, all the examples I have for routing are for routing separate networks.
Right now this is what I have
Server (Node 1): 192.168.3.18
Node 2: 192.168.3.20
Node 3: 192.168.3.22
NAS: 192.168.3.19
How should I setup my networking if I want these devices to remain on the same network but I simply want the server’s NIC’s to act as as switch? I imagine I’d have eno1 dedicated with an ip 192.168.3.18, but am unsure if I need to set ip’s on eno2 eno3 and eno4. I’m thinking I need to, but again am unsure on how the routing commands should look like
/etc/network/interfaces
auto lo
iface lo inet loopback
iface eno1 inet manual
auto bond0
iface bond0 inet manual
slaves eno1 eno2 eno3 eno4
bond-mode 802.3ad
auto vmbr0
iface vmbr0 inet static
address 192.168.3.18
netmask 255.255.255.0
gateway 192.168.3.1
bridge_ports bond0
#bridge_ports eno1
bridge_stp off
bridge_fd 0
bridge_maxwait 0
#auto en02
iface eno2 inet manual
#address 192.168.3.14
#netmask 255.255.255.0
iface eno3 inet manual
iface eno4 inet manual
auto lo
iface lo inet loopback
iface eno1 inet manual
auto bond0
iface bond0 inet manual
#slaves eno1 eno2 eno3 eno4
bond-mode 802.3ad
auto vmbr0
iface vmbr0 inet static
address 192.168.3.18
netmask 255.255.255.0
gateway 192.168.3.1
bridge_ports bond0 eno1 eno2 eno3 eno4
#bridge_ports eno1 eno2 eno3 eno4
bridge_stp off
bridge_fd 0
bridge_maxwait 0
#auto en02
iface eno2 inet manual
#address 192.168.3.14
#netmask 255.255.255.0
iface eno3 inet manual
iface eno4 inet manual
Correct answer by thistleknot on December 11, 2020
If I bridge the adapters. I need to connect all 4 ports to a switch.
No, you don't. That's what
ip link add br0 type bridge
is for: It implements a "software switch", where you can have all 4 ports externally facing as part of the "switch", and br0 as internally facing port.
So you can achieved the same thing in software, and the throughput limitation is the speed of the NICs that serve as ports for this "switch".
Bridging means it treats all 4 nics as a single nic.
No. It means you have a software switch with one internally facing NIC. Just like if you'd only use one NIC, connect it to a switch, and connect the other devices to the switch.
But when I set this mode (i.e. I enslave the other eno adapters in /etc/network/interfaces), if I connect another machine directly to an enslaved port. I can't ping it.
I don't know what you've done in /etc/network/interfaces, and why it didn't work for you.
It should, and the way I do it usually is to first use ip to set up everything, and once it works, I make it permanent in /etc/network/interfaces, or otherwise.
If you cannot ping, then debug: Make sure everyone got a proper IP address (i.e., login to one of your nodes and verify), use tcpdump whereever you can (including on your node) to see where packets get lost, use ip route get to verify routing, etc.
You are bonding your ports, you are not bridging them.
These are very different things. In particular, for bonding to work, you need a corresponding bonding construction on the other end of the pair (usually it's only a pair) of connections. Which you don't have, so it's no surprise nothing works.
Remove the bond, add all NICs as port to the bridge.
Answered by dirkt on December 11, 2020
Get help from others!
Recent Answers
Recent Questions
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP