Experimental: Advertise container network with BGP in ipvlan l3

[YujiOshima]

In ipvlan l3 mode, ipvlan driver advertise container network to another host.
Containers can communicate each other inter host easily in ipvlan l3mode.
See also #1014
network create command like
docker network create -d ipvlan --subnet=192.168.1.0/24 --ip-range=192.168.1.32/28 -o ipvlan_mode=l3 -o bgp-neighbor=10.0.1.1 -o vrf=100 -o parent=eth0 -o asnum=65001 -o rasnum=65002 vrf100
and docker run --net=vrf100 busybox,
ipvlan advertise container address(now 192.168.1.33) as /32.

Options

vrf : vrf ID

If you set vrf ID, ipvlan driver advertise container route as VPN network.
Advertise only same vrf ID network.

bgp-neighbor : IP address of BGP neighbor

For a test between two hosts, set other host ip address in bgp-neighbor, and vice versa.
And you don't need to set asnum and rasnum (use default 65000).
More than three hosts, you must use route reflector of eBGP.

asnum, rasnum : AS number of BGP speaker

asnum is self as, rasnum is neighbor as.

Signed-off-by: YujiOshima [email protected]

[mavenugo]

@YujiOshima thanks for the awesome contribution. Exciting to see the L3-to-the-edge case shaping up very well.

Github is not able to render your commit properly due to the size of the changes (mostly due to vendor-in I believe). Can you please use the same PR, but split the changes into few commits ?

1. 1 commit for each Godep Package
2. separate commit for libnetwork specific changes.

Also as we discussed earlier, a few functionality comments :

• I suggested the vrf keyword only to find a common value that can be used to distribute routes between multiple networks across the hosts (or even within the same host). But if that conflicts with the existing understanding of vrf we should reconsider this. WDYT ?
• What is the expected behavior if the above vrf field is not specified ?
• what do you think about adding another option to the driver to let the user decide if they like to advertise /32 host routes or the entire subnet range ? we can keep /32 as the default. But, I think there is a use-case for both.

[YujiOshima]

@mavenugo Thank you for commnet!

split the changes into few commits ?

OK, I'll do it right now.

vrf keyword?

Yes, definitely. It use vrf function of BGP speaker, this is one of vrf use case I think.
But the concept of this network is not equal vrf, similar to vpn.
Use other word? e.g. vpn-id,

expected behavior if the above vrf field is not specified ?

no vrf network behavior is almost same with vrf network. But one difference in expected behavior, a route of vrf net is advertised only host that have same vrfID network, and no vrf network advertise every host.

what do you think about adding another option

I agree. I add -o subnet-advertise option that enable advertise --subnet or --ip-range subnet.

[fredhsu]

For the BGP neighbor on the ToR switch, it seems to me the most logical thing would be to use eBGP peering with either unique AS #'s or a single AS but allow the AS to be repeated. Are there any advantages to using iBGP and RRs?

[nerdalert]

@fredhsu I think both eBGP and iBGP should be an option since its just design decisions of the network infra. wdyt?

[fredhsu]

@nerdalert True, I guess at the end of the day BGP client doesn't need to care how its implemented in the core.

[YujiOshima]

Sorry closed by mistake...

[YujiOshima]

@mavenugo now I split the commits.
@fredhsu @nerdalert Thank you for comments.
My opinion is using eBGP in leaf/spine But using iBGP in leaf/host.
So it's enough for iBGP.

use eBGP peering with either unique AS #'s or a single AS

I think if you use same AS in eBGP, there is some limit in topology for avoid loop. So different AS should be used for eBGP.
How about default is using iBGP, and when -o rasnum is set using eBGP?
I wouldn't want to make more options...

[mavenugo]

@YujiOshima as discussed offline, for a specific functionality of exchanging the routes, this PR brings in huge dependencies which is hard to justify. Since the BGP requirements for ipvlan-l3 mode is quite narrow, is it possible to choose only the relevant dependencies instead of pulling in so many packages ?
Since you are the gobgp expect, I will let you be the judge on how this can be achieved.

[mavenugo]

@YujiOshima reviewing the code a bit deeper, it is clear that l3_route_manager.go is the only place the gobgp package is used & the complexity stems from the fact that gobgp uses other dependencies such as grpc, etc.. which is not essential for the ipvlan driver since the server and client both are embedded in the driver and hence there is no strong reason to use such dependencies. Can you think of ways to use the gobgp package without the other dependencies ? If not, do you think we can modularize gobgp for leaner deployment cases like this one ?

[YujiOshima]

@mavenugo yeah It would probably be able to reduce dependencies.
I import gobgp-server package since it is the fastest way. But gobgp-server package has many not essential function in this case. So I will pick up minimal bgp server function from gobgp-server package.

[YujiOshima]

@mavenugo I reduce dependencies and refactor api.
For using gobgp without grpc, GoBGP must be also refactored.
This modification is not merged in GoBGP upstream, But it will be.

Failed Ci because of go get tools/cmd/vet.
It will fix by #1190

[GordonTheTurtle]

@YujiOshima It has been detected that this issue has not received any activity in over 6 months. Can you please let us know if it is still relevant:

• For a bug: do you still experience the issue with the latest version?
• For a feature request: was your request appropriately answered in a later version?

Thank you!
This issue will be automatically closed in 1 week unless it is commented on.
For more information please refer to #1926