Add docs for running the project and more info on it

tariromukute · May 14, 2024 · 34c6561 · 34c6561
1 parent ece0765
commit 34c6561
Show file tree

Hide file tree

Showing 5 changed files with 164 additions and 32 deletions.
diff --git a/.DS_Store b/.DS_Store
diff --git a/NOTES.md b/NOTES.md
@@ -41,6 +41,8 @@ docker run \
     -it \
     --cap-add=NET_ADMIN \
     --cap-add=SYS_ADMIN \
+    --cap-add=CAP_SYS_ADMIN \
+    --security-opt apparmor=unconfined \
     -v /sys/:/sys/ \
     --device /dev/net/tun \
     tariromukute/tc-gtpu:latest
@@ -194,9 +196,28 @@ netstat -s
 sysctl net.ipv4.tcp_timestamps
 
 sysctl -w net.ipv4.tcp_timestamps=0
-
 ```
 
+Collect on the relavant interfaces with tcpdump
+```bash
+tcpdump -i demo-n3 -A -w demo-n3.pcap
+tcpdump -i demo-n4 -A -w demo-n4.pcap
+tcpdump -i demo-n6 -A -w demo-n6.pcap
+tcpdump -i demo-dn -A -w demo-dn.pcap
+
+(sudo timeout 10 tcpdump -i demo-n3 -A -w demo-n3.pcap & \
+ sudo timeout 10 tcpdump -i demo-n4 -A -w demo-n4.pcap & \
+ sudo timeout 10 tcpdump -i demo-n6 -A -w demo-n6.pcap & \
+ sudo timeout 10 tcpdump -i demo-dn -A -w demo-dn.pcap & \
+ sleep 15) && sudo pkill -P $$
+
+(sudo tcpdump -i demo-n3 -A -w demo-n3.pcap & \
+ sudo tcpdump -i demo-n6 -A -w demo-n6.pcap & \
+ sudo tcpdump -i demo-dn -A -w demo-dn.pcap & \
+ sleep 30) && sudo pkill -P $$
+
+mergecap -w demo.pcap demo-n3.pcap demo-n4.pcap demo-n6.pcap demo-dn.pcap
+```
 ## Useful Resources
 
 - [Understanding tc “direct action” mode for BPF](https://qmonnet.github.io/whirl-offload/2020/04/11/tc-bpf-direct-action/)

diff --git a/README.md b/README.md
@@ -0,0 +1,98 @@
+# GTP-U library based on eBPF TC
+
+The project implements GTP-U encapusalation on traffic from network namespace. It can be used to emualte UE devices sending traffic to the UPF.
+
+**Why the implementation**
+
+The current support through libraries like [libgtpnl](http://git.osmocom.org/libgtpnl/) and [libgtp5gnl](https://github.com/free5gc/libgtp5gnl) require kernel modules, which in some environments, like Docker without control on the underlying host, can be difficult to get running. The docker environments make development and sharing of work easier. With eBPF, the GTP-U functionality can be supported without the need for kernel modules.
+
+## Get started
+
+This setup utilises Docker and Docker Compose for deployment. Ensure you have them installed before proceeding.
+
+### Usage
+
+| Option              | Short Option | Description                                                                                | Required |
+|---------------------|--------------|--------------------------------------------------------------------------------------------|----------|
+| --help              | -h           | Displays help message and available options                                                | No       |
+| --gtpu-interface    | -g           | Name of the interface used for GTP-U tunnel encapsulation                                  | Yes      |
+| --tnl-interface     | -i           | Prefix of the UE interfaces and UE/namespaces                                              | Yes      |
+| --src-ip            | -s           | Source IP address of the GTP-U tunnel                                                      | Yes      |
+| --dest-ip           | -d           | Destination IP address of the GTP-U tunnel                                                 | Yes      |
+| --ue-ip             | -u           | IP address to be assigned to first User Equipment (UE). Incremented for subsequent UEs     | Yes      |
+| --bridge-address    | -b           | Bridge interface address (Optional)                                                        | Yes      |
+| --ul-teid           | -p           | Uplink TEID (Tunnel Endpoint Identifier)                                                   | Yes      |
+| --dl-teid           | -l           | Downlink TEID (Tunnel Endpoint Identifier)                                                 | Yes      |
+| --qfi               | -q           | Quality of Flow Identifier (QFI)                                                           | Yes      |
+| --num-ues           | -n           | Number of simulated UEs (for testing purposes) (Optional)                                  |          |
+| --pcap-file         | -f           | Path to a pcap file for recorded or captured traffic (only when verbose is vvv)            |          |
+| --verbose           | -v           | Enable verbose output (Optional)                                                           |          |
+
+### Test with docker compose (eUPF, pfcp-kitchen-sink, OpenN6LAN)
+
+You can test the project with eUPF, using docker compose to intialise the setup. For more details on the setup see the [OpenN6LAN](https://github.com/tariromukute/OpenN6LAN) project.
+
+```bash
+# Set up
+docker-compose -f docker-compose/docker-compose-pfcp-eupf.yaml up -d
+
+# Run ping on one of the namespace (UE), eugtp0
+docker exec -it ue-sim \
+    ip netns exec uegtp0 ping -c 4 8.8.8.8
+
+# Display traffic sent by UEs and on the GTP-U interface
+docker exec -it ue-sim \
+    tcpdump -ttttnnr /home/tu-gtpu.pcap
+```
+
+### On Docker
+
+On Terminal 1:
+```bash
+docker run -d --name tc-gtpu \
+    --cap-add=NET_ADMIN \
+    --cap-add=SYS_ADMIN \
+    --cap-add=CAP_SYS_ADMIN \
+    --security-opt apparmor=unconfined \
+    -v /sys/:/sys/ \
+    --device /dev/net/tun \
+    tariromukute/tc-gtpu:latest \
+    tail -f /dev/null
+
+docker exec -it tc-gtpu \
+    ./tc-gtpu -g eth0 -i uegtp -s 192.168.71.130 -d 192.168.71.134 \
+    -u 12.1.1.2 -b 12.1.1.1 --ul-teid 1234 --dl-teid 1234 --qfi 9 \
+    -n 2 -f /home/tu-gtpu.pcap -vvv
+```
+
+On Terminal 2: Generate traffic
+```bash
+# Run ping on one of the namespace (UE), eugtp0
+docker exec -it tc-gtpu \
+    ip netns exec uegtp0 ping -c 4 8.8.8.8
+
+# Get the pcap
+docker exec -it tc-gtpu \
+    tcpdump -ttttnnr /home/tu-gtpu.pcap
+```
+
+### Build docker image
+
+Get project
+
+```bash
+# Clone repo
+git clone --recurse-submodules https://github.com/tariromukute/tc-gtpu.git
+# Navigate to project directory
+cd tc-gtpu
+```
+
+Run project
+
+```bash
+docker buildx build --platform=linux/amd64 -t local/tc-gtpu:latest -f Dockerfile .
+```
+
+## Contribution
+
+Please create an issue to report a bug or share an idea.
diff --git a/docker-compose/docker-compose-pfcp-eupf.yaml b/docker-compose/docker-compose-pfcp-eupf.yaml
@@ -19,14 +19,22 @@ services:
         - /bin/bash
         - -c
         - |
+            apt update -y;
+            apt install curl -y;
+
+            mkdir -p /etc/netns/uegtp0/
+            echo 'nameserver 8.8.8.8' > /etc/netns/uegtp0/resolv.conf
+            mkdir -p /etc/netns/uegtp1/
+            echo 'nameserver 8.8.8.8' > /etc/netns/uegtp1/resolv.conf
+
             ./tc-gtpu -g eth0 -i uegtp -s 192.168.71.130 -d 192.168.71.134 \
             -u 12.1.1.2 -b 12.1.1.1 --ul-teid 1234 --dl-teid 1234 --qfi 9 \
             -n 2 -f /home/tu-gtpu.pcap -vvv
-        healthcheck:
-            test: ip netns exec uegtp0 ping -c 4 192.168.73.129 || exit 1
-            interval: 10s
-            timeout: 5s
-            retries: 5
+        # healthcheck:
+        #     test: ip netns exec uegtp0 ping -c 4 192.168.73.129 || exit 1
+        #     interval: 10s
+        #     timeout: 5s
+        #     retries: 5
         volumes:
             - /sys/kernel/debug/:/sys/kernel/debug/
             - /sys/fs/bpf:/sys/fs/bpf
@@ -42,11 +50,13 @@ services:
     edgecomllc-eupf:
         platform: linux/amd64
         container_name: "edgecomllc-eupf"
-        image: ghcr.io/edgecomllc/eupf:main
+        image: tariromukute/edgecomllc-eupf:sfc-latest
         entrypoint:
         - /bin/sh
         - -c
         - |
+            sysctl -w net.ipv4.conf.eth2.send_redirects=0;
+            sysctl -w net.ipv4.conf.all.send_redirects=0;
             ip route del default;
             ip route add default via 192.168.72.138 dev eth2 &&
             sh /app/bin/entrypoint.sh
@@ -80,26 +90,50 @@ services:
                 ipv4_address: 192.168.71.134
             n6_net:
                 ipv4_address: 192.168.72.134
+                mac_address: 02:42:ac:11:65:43
     n6-lan:
         platform: linux/amd64
         privileged: true
         init: true
         container_name: "n6-lan"
-        image: tariromukute/n6-lan-simple:latest
+        image: tariromukute/openn6lan-ovs:latest
         command: 
         - /bin/bash
         - -c
         - |
             sh testovs.sh
+            # ovs-vsctl add-br brovs1
             ovs-vsctl add-port brovs1 eth1
             ip addr flush dev eth1 && ip addr add 192.168.72.138/26 dev brovs1 && ip link set brovs1 up
-            iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
             ip route add 12.1.1.0/24 via 192.168.72.134 dev brovs1
+
+
+            ovs-vsctl add-port brovs1 eth0
+            ip addr flush dev eth0 && ip addr add 192.168.73.138/26 dev brovs1 && ip link set brovs1 up
+            iptables -t nat -A POSTROUTING -o brovs1 -j MASQUERADE
+
+            ip route del default
+            ip route add default via 192.168.73.129 dev brovs1
+
+            ip link set dev eth1 xdpgeneric obj /app/nsh-decap.bpf.o sec xdp_nsh_decap
+
+            # Without this setup packets are not forwarded
+            sh /app/ovs/install-static-rules.sh
+            
+            sysctl -w net.ipv4.conf.all.send_redirects=0
+            sysctl -w net.ipv4.conf.brovs1.send_redirects=0
+            sysctl -w net.ipv4.conf.eth0.send_redirects=0
+            sysctl -w net.ipv4.conf.eth1.send_redirects=0
+
             tail -f /dev/null
         devices:
             - /dev/net/tun:/dev/net/tun # https://docs.openvswitch.org/en/stable/intro/install/userspace/#building-and-installing
         volumes:
             - /lib/modules:/lib/modules
+        deploy:
+            resources:
+                reservations:
+                    memory: 2G
         networks:
             n6_net:
                 ipv4_address: 192.168.72.138

diff --git a/docker-compose/sessions-eupf.yaml b/docker-compose/sessions-eupf.yaml
@@ -67,7 +67,7 @@
     forwardingParameters:
       destinationInterface: SGiLAN
       networkInstance: core.oai.org
-      forwardingPolicy: 0x00000000000000000000000000000000000000000112233
+      forwardingPolicy: 000000000000000000000000000000000000000000011223F
   - farID: 15
     applyAction: Forward
     forwardingParameters:
@@ -76,25 +76,4 @@
       outerHeaderCreation:
         desc: OUTER_HEADER_CREATION_GTPU_UDP_IPV4
         teid: 1235
-        ip: 192.168.71.130
-# - seid: 1
-#   pdrs:
-#   - pdrID: 1
-#     precedence: 0
-#     pdi:
-#       sourceInterface: Core
-#       networkInstance: internet.oai.org
-#       ueIPAddress:
-#         isDestination: true
-#         ip4: 12.1.1.2
-#     farID: 13
-#   fars:
-#   - farID: 13
-#     applyAction: Forward
-#     forwardingParameters:
-#       destinationInterface: Access
-#       networkInstance: access.oai.org
-#       outerHeaderCreation:
-#         desc: OUTER_HEADER_CREATION_GTPU_UDP_IPV4
-#         teid: 1234
-#         ip: 192.168.70.130
+        ip: 192.168.71.130