Difference between revisions of "Quantar Linking"

From W9CR
Jump to navigation Jump to search
 
(11 intermediate revisions by the same user not shown)
Line 58: Line 58:
 
I use IP space in the 172.16.0.0/12 space with a /20 for the tunnel interfaces and different space for each routers loopback.  At some sites and the hub it's handy to have a local interface which can do dhcp so you can use a local laptop to connect to the network elements directly.  This should be given security considerations if enabled.  
 
I use IP space in the 172.16.0.0/12 space with a /20 for the tunnel interfaces and different space for each routers loopback.  At some sites and the hub it's handy to have a local interface which can do dhcp so you can use a local laptop to connect to the network elements directly.  This should be given security considerations if enabled.  
  
−
I will also setup remote access VPN into the network at the HUB as well.  This needs it's own subnet for routing of connected clients; a /27 can be used here as well, but it must be separate from any other space used on Ethernet interfaces at the hub.  
+
I will also setup remote access VPN into the network at the HUB as well.  This needs it's own subnet for routing of connected clients; a /27 can be used here as well, but it must be separate from any other space used on Ethernet interfaces at the hub.
 +
 
 +
{| class="wikitable"
 +
! style="font-family: "Roboto Mono", monospace" colspan="8" |3845 Ports
 +
|- style="font-family: "Roboto Mono", monospace"
 +
! Voice Port !! Serial Port !! ATAC !! Site !! # Local !! # Remote !! STUN !! Color
 +
|- font-family: "Roboto Mono", monospace
 +
| 0/0/0 || s1/0 || 1 || TPA || 1100 || 1101 || 10 || White/Blue
 +
|- font-family: "Roboto Mono", monospace
 +
| 0/0/1 || s1/1 || 2 || MIA || 1200 || 1201 || 20 || White/Orange
 +
|-
 +
| 0/1/0 || s1/2 || 3 || JAX || 1300 || 1301 || 30 || White/Green
 +
|-
 +
| 0/1/1 || s1/3 || 4 || MCO || 1400 || 1401 || 40 || White/Brown
 +
|-
 +
| 0/2/0 || s1/4 || 5 || FLL || 1500 || 1501 || 50 || White/Slate
 +
|-
 +
| 0/2/1 || s1/5 || 6 || EYW || 1600 || 1601 || 60 || Red/Blue
 +
|-
 +
| 0/3/0 || s1/6 || 7 || RSW || 1700 || 1701 || 70 || Red/Orange
 +
|-
 +
| 0/3/1 || s1/7 || 8 || SPG || 1800 || 1801 || 80 || Red/Green
 +
|-
 +
| 3/0/0 || s1/8 || 9 || BOW || 1900 || 1901 || 90 || Red/Brown
 +
|-
 +
| 3/0/1 || s1/9 || 10 || MLB || 2000 || 2001 || 100 || Red/Slate
 +
|-
 +
| 3/1/0 || s1/10 || 11 || SQR || 2100 || 2101 || 110 || Black/Blue
 +
|-
 +
| 3/1/1 || s1/11 || 12 || TLH || 2200 || 2201 || 120 || Black/Orange
 +
|-
 +
| 4/0/0 || s1/12 || 13 || PBI || 2300 || 2301 || 130 || Black/Green
 +
|-
 +
| 4/0/1 || s1/13 || 14 || GNV || 2400 || 2401 || 140 || Black/Brown
 +
|-
 +
| 4/1/0 || s1/14 || 15 || PNS || 2500 || 2501 || 150 || Black/Slate
 +
|-
 +
| 4/1/0 || s1/15 || 16 || PIE || 2600 || 2601 || 160 || Yellow/Blue
 +
|}
 +
 
 +
{| class="wikitable"
 +
! Router !! Tunnel IP !! Subnet /24 !! LoopBack !! gi2/0 /31 !! gi2/0.1 /30 !! gi2/0.172 /27 !! ESW Vlan1 /30
 +
|-
 +
| HUB || 172.18.1.1/22 || 172.19.0.0 || 10.2.3.0 || 172.19.0.254 || 172.19.0.249 || 172.19.0.1 || 172.19.0.250
 +
|-
 +
| TPA || 172.18.1.10 || 172.19.1.0 || 10.2.3.1 || 172.19.1.254 || 172.19.1.249 || 172.19.1.1 || 172.19.1.250
 +
|-
 +
| MIA || 172.18.1.20 || 172.19.2.0 || 10.2.3.2 || 172.19.2.254 || 172.19.2.249 || 172.19.2.1 || 172.19.2.250
 +
|-
 +
| JAX || 172.18.1.30 || 172.19.3.0 || 10.2.3.3 || 172.19.3.254 || 172.19.3.249 || 172.19.3.1 || 172.19.3.250
 +
|-
 +
| MCO || 172.18.1.40 || 172.19.4.0 || 10.2.3.4 || 172.19.4.254 || 172.19.4.249 || 172.19.4.1 || 172.19.4.250
 +
|-
 +
| FLL || 172.18.1.50 || 172.19.5.0 || 10.2.3.5 || 172.19.5.254 || 172.19.5.249 || 172.19.5.1 || 172.19.5.250
 +
|-
 +
| EYW || 172.18.1.60 || 172.19.6.0 || 10.2.3.6 || 172.19.6.254 || 172.19.6.249 || 172.19.6.1 || 172.19.6.250
 +
|-
 +
| RSW || 172.18.1.70 || 172.19.7.0 || 10.2.3.7 || 172.19.7.254 || 172.19.7.249 || 172.19.7.1 || 172.19.7.250
 +
|-
 +
| SPG || 172.18.1.80 || 172.19.8.0 || 10.2.3.8 || 172.19.8.254 || 172.19.8.249 || 172.19.8.1 || 172.19.8.250
 +
|-
 +
| BOW || 172.18.1.90 || 172.19.9.0 || 10.2.3.9 || 172.19.9.254 || 172.19.9.249 || 172.19.9.1 || 172.19.9.250
 +
|-
 +
| MLB || 172.18.1.100 || 172.19.10.0 || 10.2.3.10 || 172.19.10.254 || 172.19.10.249 || 172.19.10.1 || 172.19.10.250
 +
|-
 +
| SQR || 172.18.1.110 || 172.19.11.0 || 10.2.3.11 || 172.19.11.254 || 172.19.11.249 || 172.19.11.1 || 172.19.11.250
 +
|-
 +
| TLH || 172.18.1.120 || 172.19.12.0 || 10.2.4.12 || 172.19.12.254 || 172.19.12.249 || 172.19.12.1 || 172.19.12.250
 +
|-
 +
| PBI || 172.18.1.130 || 172.19.13.0 || 10.2.4.13 || 172.19.13.254 || 172.19.13.249 || 172.19.13.1 || 172.19.13.250
 +
|-
 +
| GNV || 172.18.1.140 || 172.19.14.0 || 10.2.4.14 || 172.19.14.254 || 172.19.14.249 || 172.19.14.1 || 172.19.14.250
 +
|-
 +
| PNS || 172.18.1.150 || 172.19.15.0 || 10.2.4.15 || 172.19.15.254 || 172.19.15.249 || 172.19.15.1 || 172.19.15.250
 +
|-
 +
| PIE || 172.18.1.160 || 172.19.16.0 || 10.2.4.16 || 172.19.16.254 || 172.19.16.249 || 172.19.16.1 || 172.19.16.250
 +
|}
 +
 
 +
 
  
 
== circuits over this ==  
 
== circuits over this ==  
Line 64: Line 142:
  
 
== Interface naming ==
 
== Interface naming ==
 +
It's very helpful to have an interface name on each interface on the router.  These should be in ALL CAPS and follow a description pattern.
 +
 +
Any IP address should be in DNS and reverse DNS.
 +
 +
Description "<circuit-ID> <source host>-<src cross connect> to <destination host>-<dest port>-<dest cross connect>"
 +
 +
Examples:
 +
interface serial 1/0
 +
  description "HUB-ATAC-pt1 to TPA-s0/0/0-Quantar-KP1UG-pt1"
 +
Meaning Hub connected to S1 going to Tampa s0/0/0 and connecting to Quantar KP1UG Port 1
 +
 +
interface Tunnel0
 +
  description "HUB DMVPN Multipoint"
 +
Hub router multipoint interface for DMVPN
 +
 +
interface GigabitEthernet0/0
 +
  description "HUB to upstream ISP <CIRCUIT ID>
  
 
== DNS ==  
 
== DNS ==  
Line 73: Line 168:
 
== network monitoring ==  
 
== network monitoring ==  
  
−
A server with LibreNMS or Observium
+
=== RPI4 4gb ===
 +
 
 +
I use a RPI-4 4gb with an external SSD for this. 
 +
 
 +
Software for RPI4
 +
 
 +
apt-get install ntp ntpdate python vim screen ipsec-tools strongswan fail2ban snmp haveged libacl1-dev python3-dev libssl-dev gcc g++ fio pbzip2 ncdu sudo
 +
vim /etc/dhcpcd.conf
 +
172.31.7.65 DNS server
 +
 
 +
 
 +
vim /etc/screenrc
 +
uncomment "termcapinfo xterm|xterms|xs|rxvt ti@:te@"
 +
 
 +
vim /etc/bash.bashrc
 +
  uncomment the stuff below
 +
  # enable bash completion in interactive shells
 +
 
 +
 
 +
update-alternatives --config editor
 +
  Then select #3 vim.basic
 +
 
 +
sudo timedatectl set-timezone UTC
 +
 
 +
=== Configure Postfix ===
 +
 
 +
Postfix is installed to forward mail for root to a smtp host.
 +
 
 +
<code>apt-get install postfix mailutils</code>
 +
 
 +
This will run an installer with a curses interface and you must select '''Satallite System'''.  Check the '''System mail''' name is the hostname of the server, and the '''SMTP relay host''' is ''morty.keekles.org''. '''Root and postmaster mail''' should be ''me@myemail.com''.
 +
 
 +
Should you need to reconfigure this use:
 +
 
 +
<code> dpkg-reconfigure postfix </code>
 +
 
 +
other aliases are setup in /etc/aliases.  You must run ''newaliases'' after this is updated for them to take effect.
 +
 
 +
=== Setup LibreNMS ===
 +
 
 +
The RPI-4 is configured as follows
 +
 
 +
https://docs.librenms.org/Installation/Install-LibreNMS/
 +
 
 +
==== Install Software ====
 +
apt install software-properties-common
 +
apt update
 +
apt install acl curl apache2 composer fping git graphviz imagemagick libapache2-mod-fcgid mariadb-client mariadb-server mariadb-common  mtr-tiny nmap php7.3-cli php7.3-curl php7.3-fpm php7.3-gd php7.3-json php7.3-mbstring php7.3-mysql php7.3-snmp php7.3-xml php7.3-zip rrdtool snmp snmpd whois python3-pymysql python3-dotenv python3-redis python3-setuptools
 +
 
 +
==== Add LibreNMS user ====
 +
useradd librenms -d /opt/librenms -M -r -s /bin/bash
 +
 
 +
==== Download LibreNMS =====
 +
cd /opt
 +
git clone https://github.com/librenms/librenms.git
 +
 
 +
Set Permissions
 +
 
 +
chown -R librenms:librenms /opt/librenms
 +
chmod 771 /opt/librenms
 +
setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
 +
setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
 +
 
 +
Install PHP deps
 +
su - librenms
 +
./scripts/composer_wrapper.php install --no-dev
 +
exit
 +
 
 +
Set timezone
 +
 
 +
vim /etc/php/7.3/fpm/php.ini
 +
date.timezone = Etc/UTC
 +
 
 +
vi /etc/php/7.3/cli/php.ini
 +
 
 +
==== Config the mariadb ====
 +
vi /etc/mysql/mariadb.conf.d/50-server.cnf
 +
innodb_file_per_table=1
 +
lower_case_table_names=0
 +
 
 +
systemctl restart mysql.service
 +
 
 +
 
 +
==== config apache ====
 +
 
 +
cp /etc/php/7.3/fpm/pool.d/www.conf /etc/php/7.3/fpm/pool.d/librenms.conf
 +
vim /etc/php/7.3/fpm/pool.d/librenms.conf
 +
 
 +
 
 +
==== config Rancid ====
 +
sudo apt-get install rancid subversion
 +
https://docs.librenms.org/Extensions/Rancid/
 +
 
 +
 
 +
 
  
 
SNMP  
 
SNMP  
  
−
SSH for management  
+
SSH for management
  
 
= Hardware for linking =
 
= Hardware for linking =
Line 122: Line 311:
 
= Making it work =  
 
= Making it work =  
  
−
== configuring the hub ==
+
== Configuring the hub ==
 +
 
 +
=== IP Interface Config ===
 +
 
 +
* Loopback 0
 +
interface Loopback0
 +
  description "HUB loop0"
 +
  ip address 10.2.1.1 255.255.255.255
 +
 
 +
 
 +
* Gi0/0
 +
interface GigabitEthernet0/0
 +
  description "Outboound interface to AS3356"
 +
  !whatever your IP is.  This can't be DHCP on the hub
 +
  ip address 4.5.6.1 255.255.255.254
 +
  !turn on nat and tag this as an outside interface
 +
  ip nat outside
 +
  !normally you don't want packets being re-assembled, but this might be needed
 +
  !depending on the craptacular network you have end points on
 +
  ip virtual-reassembly in
 +
  ! next three setup the media type
 +
  duplex auto
 +
  speed auto
 +
  media-type rj45
 +
  ! disable CDP and DECnet MOP
 +
  no cdp enable
 +
  no mop enabled
  
 
=== DMVPN config ===
 
=== DMVPN config ===
 +
 +
This sets up the DMVPN config on the tunnel interface. 
 +
 +
 +
* Tunnel 0
 +
interface Tunnel0
 +
  description HUB DMVPN multipoint to clients
 +
  !set the bandwidth, needed for routing protocols
 +
  bandwidth 10000
 +
  ip address
 +
  no ip redirects
 +
  !set the MTU of the tunnel
 +
  ip mtu 1400
 +
  ip pim nbma-mode
 +
  ip pim sparse-mode
 +
  !this is an auth string used to uniquely id this DMVPN session. 
 +
  ip nhrp authentication HUBKey3
 +
  !Allows NHRP to automatically add spoke routers to the multicast NHRP mappings.
 +
  ip nhrp map multicast dynamic
 +
  !this number is globally uniquie 32 bit number.
 +
  ip nhrp network-id 737373
 +
  ip nhrp holdtime 600
 +
  ip tcp adjust-mss 1350
 +
  ip ospf network broadcast
 +
  ip ospf priority 255
 +
  delay 1000
 +
  cdp enable
 +
  tunnel source GigabitEthernet0/0
 +
  tunnel mode gre multipoint
 +
  tunnel key 737373
 +
  tunnel protection ipsec profile DMVPN_PROFILE shared
 +
  hold-queue 50 out
 +
 +
 +
 +
=== Routing Config ===
 +
 +
* OSPF
 +
router ospf
 +
 +
=== Serial/Stun Config ===
 +
 +
 +
 +
 +
=== Voice Port Config ===
 +
 +
* voice class permanent
 +
voice class permanent 1811
 +
  signal timing oos timeout disabled
 +
  signal keepalive disabled
 +
  signal sequence oos no-action
 +
 +
 +
* voice-port
 +
voice-port 0/0/0
 +
 +
* dial-peer voice 1601 voip
 +
 +
* dial-peer voice 1600 pots
 +
 +
=== NAT Config ===
 +
 +
=== IPSEC Remote Access Config ===
 +
 +
This will give you remote access to the server for remote access using a PSK and XAUTH
 +
 +
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_conn_esyvpn/configuration/15-sy/sec-conn-esy-15-sy-book/sec-easy-vpn-srvr.html
 +
 +
You will need to select a pool of addresses for this to use.  As this is on the hub, we're using 172.19.0.128/28
 +
 +
! need this for the VPN client
 +
aaa authentication login vpnclient local
 +
aaa authorization network localgroups local
 +
 +
! need a username
 +
username bryan privilege 5 secret changeme
 +
 +
crypto logging session
 +
crypto logging session
 +
!
 +
crypto logging ezvpn group VPN
 +
!
 +
! define the isakmp policy priority
 +
crypto isakmp policy 10
 +
  authentication pre-share
 +
  group 2
 +
  lifetime 3600
 +
!
 +
crypto isakmp policy 200
 +
  authentication pre-share
 +
  lifetime 3600
 +
 +
! This is a setup of the ISAKMP Config group, it will be referred to in the ISAKMP
 +
! profile VPN-DIALER-USERS  This is the group ID for the pre shared key.
 +
crypto isakmp client configuration group VPN-DIALER-GROUP
 +
  ! This is the pre shared key for the VPN
 +
  key SEKRET-KEY-HERE 
 +
  ! users will use this router as a DNS server
 +
  dns 10.2.3.0
 +
  ! the pool of IP addresses used by the VPN
 +
  pool VPN-POOL
 +
  ! access list to split traffic locally, this will allow on listed subnets to go into
 +
  ! the VPN.  Remove this if you want all traffic to go via the VPN
 +
  acl VPN-DIALER-SPLIT
 +
  ! allow the client to access it's local lan segment when connected to the VPN
 +
  include-local-lan
 +
  ! banner to display at login on the client (optional)
 +
  banner ^C         
 +
Quantar Link User Access
 +
Licensed users only
 +
         
 +
This is HUB1
 +
All access is logged.
 +
This is a private system.
 +
Disconnect _NOW_, you are not welcome here
 +
                      ^C
 +
 +
! this setups a Phase 1 ISAKMP profile of VPN-DIALER-USERS
 +
crypto isakmp profile VPN-DIALER-USERS
 +
    ! This ties it back to the VPN-DIALER-GROUP and
 +
    match identity group VPN-DIALER-GROUP
 +
    !these next two tie it to the AAA configured for auth
 +
    client authentication list vpnclient
 +
    isakmp authorization list localgroups
 +
    ! this makes the server listen, not initiate
 +
    client configuration address respond
 +
    ! the isakmp client configuration group that this profile uses
 +
    client configuration group VPN-DIALER-GROUP
 +
    ! finally the virtual template interface that will be used to make a virtual-access
 +
    ! interface when the user connects
 +
    virtual-template 199
 +
!
 +
!
 +
! this is the IPSEC hash and algo used and is referred to in the ipsec profile
 +
crypto ipsec transform-set VPN-DIALER-TRANSFORM-SET esp-aes esp-sha-hmac
 +
!
 +
! This ipsec profile is referenced in the Virtual template 199 to specify which hash
 +
! and algo will be permitted.  You can define multiple transform-sets
 +
crypto ipsec profile VPN-DIALER-PROFILE
 +
  set transform-set VPN-DIALER-TRANSFORM-SET
 +
!
 +
! This is the virtual template that will be cloned into a virtual-access tunnel when a
 +
! Client Connects.  This makes the client in their own tunnel interface.
 +
interface Virtual-Template199 type tunnel
 +
  description VPN-DIALER-USERS
 +
  ! this borrows the loopback 0 IP for the router side of the tunnel.
 +
  ip unnumbered Loopback0
 +
  ! cut the MTU
 +
  ip mtu 1400
 +
  ! turn on NAT (optional)
 +
  ip nat inside
 +
  ip virtual-reassembly in
 +
  ! this will adjust the IPv4 tcp MSS to avoid MTU issues
 +
  ip tcp adjust-mss 1200
 +
  tunnel mode ipsec ipv4
 +
  ! tie the tunnel config back to a crypto ipsec profile
 +
  tunnel protection ipsec profile VPN-DIALER-PROFILE
 +
 +
! define the local Pool of IP's
 +
ip local pool VPN-POOL 172.19.0.128 172.19.0.143
 +
 +
! put in a default null route for this subnet.  When a client connects it will be a /32
 +
! and longer prefix
 +
ip route 172.19.0.128 255.255.255.240 Null0 tag 69
 +
 +
! this next part will prevent an OSPF LSA going out every time a user connects.  Normally
 +
! when the user connects they would have a /32 route and it woould be announced to all
 +
! routers in OSPF.
 +
router ospf 1
 +
  ! make a summary address for this
 +
  summary-address 172.19.0.128 255.255.255.240
 +
  ! and if it matches tag 69, redistribuite it.
 +
  redistribute static subnets tag 69
 +
 +
 +
! this is the IP space allowed in the tunnel for split routing. 
 +
ip access-list extended VPN-DIALER-SPLIT
 +
  permit ip 10.0.0.0 0.255.255.255 any
 +
  permit ip 172.16.0.0 0.15.255.255 any
 +
 +
=== DNS Config ===
 +
 +
=== SNMP Config ===
 +
* SNMP ACL
 +
 +
=== SSH Config ===
 +
 +
=== NTP Config ===
 +
* NTP ACL
 +
 +
=== User config ===
 +
aaa new-model
 +
!
 +
!
 +
aaa authentication login default local
 +
aaa authentication login vpnclient local
 +
aaa authorization console
 +
aaa authorization config-commands
 +
aaa authorization exec default local
 +
aaa authorization network localgroups local
 +
 +
=== VTY ACL ===
 +
 +
 +
=== DHCP ===
 +
ip dhcp pool P25NX-local-80
 +
  network 172.31.7.80 255.255.255.240
 +
  default-router 172.31.7.81
 +
  dns-server 172.31.7.81
 +
  lease 0 0 15
 +
 +
 +
=== ESW module config ===
 +
 +
* IP management
 +
* SNMP
 +
* rancid
  
 
== Configuring a client ==
 
== Configuring a client ==

Latest revision as of 10:09, 17 March 2021

THIS IS A WORK IN PROGRESS

There are a few ways to link Quantars, using p25nx, mmvdm, and the method presented here using astro tacs.

Quantars were designed to be "linked" only as a simulcast or linked receiver network for a single system. This is accomplished by connecting the wireline for analog and the v.24 for digital. Quantars may be linked back-to-back by using a v.24 cross over and connecting the wirelines between them. While this is the simplest way, it doesn't scale. Using an ASTROTAC comparator to sit between the Quantars as a network switch function.

Each Quantar needs a two links to the ASTRO-TAC, a v.24 and a wireline link. If all Quantars are at the same site, this is really easy to do, just connect the wirelines and v.24's using cross overs. In the early 1990's the v.24 and wireline interfaces made quite a bit of sense, the v.24 and wireline would plug directly into a channel bank, and the DS1 or fractional DS1's between sites was easy. As of today, T1's are hard to find and very expensive. Channel banks are thousands of dollars on the used market now.

Today the internet is ubiquitous and linking via IP is the preferred method. This means we must transport synchronous serial and analog voice for each Quantar back to a central site. There are a few ways to do this via IP or MPLS, and presented here will be using cheap and easily available cisco routers in a VPN.


Architecture overview

There are several parts of this network, but the fundamental precept is creating link of rs232 and wireline voice between each port on the AstroTAC and each Quantar. This is no different than if all the equipment was in the same room, or linked into channel banks on T1 lines between sites. We're using IP multi-point VPN overlay network to simulate a circuit based network.

Our network stack is : 

Internet
--------
DMVPN
--------
STUN and G.711 VOIP circuits
--------
ATAC and Quantars

What's presented here assumes each site has a stable IPv4 internet connection and the router has a globally routeable IPv4 address. The overlay network is based on Dynamic Multi-Point VPN (DMVPN) and allows us to configure a single hub router and have the same or similar config on all the endpoints. Traffic from router to router builds a connection on the fly encrypted between each router, even if the IP address of the routers change.

That last concept is important as anything that prevents routers from forwarding IP inbound (ie if the router is behind NAT) will prevent site to site connectivity. Generally this isn't needed as we only talk hub to site for the circuits, but can present it self during troubleshooting from one site to the other.


Logical connections

Template:Required image



Network Planning

Lots of routers, lots of ports, lots of circuits. You will need to name and keep interfaces described or you will be unable to troubleshoot this network.

Reliable bandwidth and pps is required. The analog links will require about 60 kbit/s at 33 PPS at all times , stun is another 20 kbit/s when it's operating. A fully loaded 16 port ATAC3000 using all remote ports will be 1.280 Mbit/s at 1220 PPS or about a fully loaded T1.


The first


overlay network

Our first step will be designing the overlay network. This consists of diagraming out sites and what equipment will go else where. As part of this I like to fully populate out and cross connect the central hub to the ATAC. This way it's all cabled up and I don't need to make changes at the hub to add an additional site.

Template:Required image

IP addressing

IP addressing needs to be through out for the overlay network. As this will not interconnect with any other network, you're free to use your own addressing scheme.

I use IP space in the 172.16.0.0/12 space with a /20 for the tunnel interfaces and different space for each routers loopback. At some sites and the hub it's handy to have a local interface which can do dhcp so you can use a local laptop to connect to the network elements directly. This should be given security considerations if enabled.

I will also setup remote access VPN into the network at the HUB as well. This needs it's own subnet for routing of connected clients; a /27 can be used here as well, but it must be separate from any other space used on Ethernet interfaces at the hub.

3845 Ports
Voice Port Serial Port ATAC Site # Local # Remote STUN Color
0/0/0 s1/0 1 TPA 1100 1101 10 White/Blue
0/0/1 s1/1 2 MIA 1200 1201 20 White/Orange
0/1/0 s1/2 3 JAX 1300 1301 30 White/Green
0/1/1 s1/3 4 MCO 1400 1401 40 White/Brown
0/2/0 s1/4 5 FLL 1500 1501 50 White/Slate
0/2/1 s1/5 6 EYW 1600 1601 60 Red/Blue
0/3/0 s1/6 7 RSW 1700 1701 70 Red/Orange
0/3/1 s1/7 8 SPG 1800 1801 80 Red/Green
3/0/0 s1/8 9 BOW 1900 1901 90 Red/Brown
3/0/1 s1/9 10 MLB 2000 2001 100 Red/Slate
3/1/0 s1/10 11 SQR 2100 2101 110 Black/Blue
3/1/1 s1/11 12 TLH 2200 2201 120 Black/Orange
4/0/0 s1/12 13 PBI 2300 2301 130 Black/Green
4/0/1 s1/13 14 GNV 2400 2401 140 Black/Brown
4/1/0 s1/14 15 PNS 2500 2501 150 Black/Slate
4/1/0 s1/15 16 PIE 2600 2601 160 Yellow/Blue
Router Tunnel IP Subnet /24 LoopBack gi2/0 /31 gi2/0.1 /30 gi2/0.172 /27 ESW Vlan1 /30
HUB 172.18.1.1/22 172.19.0.0 10.2.3.0 172.19.0.254 172.19.0.249 172.19.0.1 172.19.0.250
TPA 172.18.1.10 172.19.1.0 10.2.3.1 172.19.1.254 172.19.1.249 172.19.1.1 172.19.1.250
MIA 172.18.1.20 172.19.2.0 10.2.3.2 172.19.2.254 172.19.2.249 172.19.2.1 172.19.2.250
JAX 172.18.1.30 172.19.3.0 10.2.3.3 172.19.3.254 172.19.3.249 172.19.3.1 172.19.3.250
MCO 172.18.1.40 172.19.4.0 10.2.3.4 172.19.4.254 172.19.4.249 172.19.4.1 172.19.4.250
FLL 172.18.1.50 172.19.5.0 10.2.3.5 172.19.5.254 172.19.5.249 172.19.5.1 172.19.5.250
EYW 172.18.1.60 172.19.6.0 10.2.3.6 172.19.6.254 172.19.6.249 172.19.6.1 172.19.6.250
RSW 172.18.1.70 172.19.7.0 10.2.3.7 172.19.7.254 172.19.7.249 172.19.7.1 172.19.7.250
SPG 172.18.1.80 172.19.8.0 10.2.3.8 172.19.8.254 172.19.8.249 172.19.8.1 172.19.8.250
BOW 172.18.1.90 172.19.9.0 10.2.3.9 172.19.9.254 172.19.9.249 172.19.9.1 172.19.9.250
MLB 172.18.1.100 172.19.10.0 10.2.3.10 172.19.10.254 172.19.10.249 172.19.10.1 172.19.10.250
SQR 172.18.1.110 172.19.11.0 10.2.3.11 172.19.11.254 172.19.11.249 172.19.11.1 172.19.11.250
TLH 172.18.1.120 172.19.12.0 10.2.4.12 172.19.12.254 172.19.12.249 172.19.12.1 172.19.12.250
PBI 172.18.1.130 172.19.13.0 10.2.4.13 172.19.13.254 172.19.13.249 172.19.13.1 172.19.13.250
GNV 172.18.1.140 172.19.14.0 10.2.4.14 172.19.14.254 172.19.14.249 172.19.14.1 172.19.14.250
PNS 172.18.1.150 172.19.15.0 10.2.4.15 172.19.15.254 172.19.15.249 172.19.15.1 172.19.15.250
PIE 172.18.1.160 172.19.16.0 10.2.4.16 172.19.16.254 172.19.16.249 172.19.16.1 172.19.16.250


circuits over this

Interface naming

It's very helpful to have an interface name on each interface on the router. These should be in ALL CAPS and follow a description pattern.

Any IP address should be in DNS and reverse DNS. 

Description "<circuit-ID> <source host>-<src cross connect> to <destination host>-<dest port>-<dest cross connect>"

Examples: 

interface serial 1/0
 description "HUB-ATAC-pt1 to TPA-s0/0/0-Quantar-KP1UG-pt1"

Meaning Hub connected to S1 going to Tampa s0/0/0 and connecting to Quantar KP1UG Port 1 

interface Tunnel0
 description "HUB DMVPN Multipoint"

Hub router multipoint interface for DMVPN 

interface GigabitEthernet0/0
 description "HUB to upstream ISP <CIRCUIT ID>

DNS

Having all interfaces named in DNS is a good idea. This makes troubleshooting and tracerouting on the overlay network much easier. As the hub router has a bunch of extra CPU, it's easy to configure this on the hub and have it be primary DNS. This is well baked in IOS and other than some additional config it's not hard.


security

network monitoring

RPI4 4gb

I use a RPI-4 4gb with an external SSD for this.

Software for RPI4 

apt-get install ntp ntpdate python vim screen ipsec-tools strongswan fail2ban snmp haveged libacl1-dev python3-dev libssl-dev gcc g++ fio pbzip2 ncdu sudo

vim /etc/dhcpcd.conf 172.31.7.65 DNS server


vim /etc/screenrc 

uncomment "termcapinfo xterm|xterms|xs|rxvt ti@:te@"

vim /etc/bash.bashrc 

  uncomment the stuff below 
  # enable bash completion in interactive shells


update-alternatives --config editor 

  Then select #3 vim.basic

sudo timedatectl set-timezone UTC

Configure Postfix

Postfix is installed to forward mail for root to a smtp host.

apt-get install postfix mailutils

This will run an installer with a curses interface and you must select Satallite System. Check the System mail name is the hostname of the server, and the SMTP relay host is morty.keekles.org. Root and postmaster mail should be me@myemail.com.

Should you need to reconfigure this use:

dpkg-reconfigure postfix

other aliases are setup in /etc/aliases. You must run newaliases after this is updated for them to take effect.

Setup LibreNMS

The RPI-4 is configured as follows

https://docs.librenms.org/Installation/Install-LibreNMS/

Install Software

apt install software-properties-common
apt update
apt install acl curl apache2 composer fping git graphviz imagemagick libapache2-mod-fcgid mariadb-client mariadb-server mariadb-common  mtr-tiny nmap php7.3-cli php7.3-curl php7.3-fpm php7.3-gd php7.3-json php7.3-mbstring php7.3-mysql php7.3-snmp php7.3-xml php7.3-zip rrdtool snmp snmpd whois python3-pymysql python3-dotenv python3-redis python3-setuptools

Add LibreNMS user

useradd librenms -d /opt/librenms -M -r -s /bin/bash

Download LibreNMS =

cd /opt
git clone https://github.com/librenms/librenms.git

Set Permissions 

chown -R librenms:librenms /opt/librenms
chmod 771 /opt/librenms
setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/

Install PHP deps 

su - librenms 
./scripts/composer_wrapper.php install --no-dev
exit

Set timezone

vim /etc/php/7.3/fpm/php.ini date.timezone = Etc/UTC

vi /etc/php/7.3/cli/php.ini

Config the mariadb

vi /etc/mysql/mariadb.conf.d/50-server.cnf 

innodb_file_per_table=1
lower_case_table_names=0

systemctl restart mysql.service


config apache

cp /etc/php/7.3/fpm/pool.d/www.conf /etc/php/7.3/fpm/pool.d/librenms.conf vim /etc/php/7.3/fpm/pool.d/librenms.conf


config Rancid

sudo apt-get install rancid subversion
https://docs.librenms.org/Extensions/Rancid/



SNMP

SSH for management

Hardware for linking

Will you need analog? If no you can eliminate a bunch of configuration and half the circuits. The bandwidth requirements at the hub will be less as well.


Hub site

A hub site needs a router with at least the number of ports on it you plan to use remotely. A converted quantar might only have 5 ports, but a fully loaded ATAC could be 16. If they are all remote you will need that too. The hub router also need to process a number of things on the network. All in all it's not much, but given the cost of the cisco routers on the used market the following config has become my standard.

  • Cisco 3845 router
    • AIM-VPN/SSL-3 VPN Module for 3825/3845 Routers
    • 768M or 1GB of RAM
    • 2GB flash disk
    • PVDM2-64 (DSP module for analog lines)
    • Optional PSU PWR-3845-AC-IP, this will supply 48v for the switch card
  • 1, NM-16A/S - Cisco 800-20840-01D 16-Port Async/Sync Serial Network Module (you can use the NM-8A/S too, but it's got different connectors)
  • 8, Cisco VIC2-2E/M 2-Port Ear and Mouth Voice Interface Card (only for analog)
  • 2, Cisco NM-2V, carrier cards for VIC2 to put in the NM ports
  • 1, NME-16ES-1G-P 16 Port POE switch module. This is optional but it provides a 3750 switch in a network module with POE that can power RPi's or other local devices.
  • CAB-SS-232FC, RS-232 Cable, DCE Female to Smart Serial for all serial ports
  • OR CISCO CAB-232FC RS-232 Cable, DCE Female to Serial if using the 8 port card.


Edge site

An edge site can support one or two Quantars or cascaded (advanced) ATACs. If you don't need to support Voice, an 1841 or other 1800 series router can be used, but the cost delta is negligible. This entire 2811 setup is under $100 via eBay. I like to use the same cables and connectors for serial at the hub as at the edge sites, this determines the cards for serial. If you're doing voice, two ports of serial matches the 2 ports of E and M well.

  • Cisco 2811
    • AIM-VPN/SSL-2 VPN Module for 2811 Routers
    • 512M of RAM
    • 2GB flash disk
    • PVDM2-64 (DSP module for analog lines)
  • 1, WIC-2A/S 2 port Serial WAN Interface Card OR
  • 1, WIC-1T 1 port Serial Card uses the larger serial connector
  • 1, Cisco VIC2-2E/M 2-Port Ear and Mouth Voice Interface Card (only for analog)
  • CAB-SS-232FC, RS-232 Cable, DCE Female to Smart Serial for all serial ports
  • OR CISCO CAB-232FC RS-232 Cable, DCE Female to Serial if using the 8 port card.

You may want to pickup a serial adapter and FTDI serial to USB dongle for console access just to leave at the site. This can be handy.


Making it work

Configuring the hub

IP Interface Config

  • Loopback 0
interface Loopback0
 description "HUB loop0"
 ip address 10.2.1.1 255.255.255.255


  • Gi0/0
interface GigabitEthernet0/0
 description "Outboound interface to AS3356"
 !whatever your IP is.  This can't be DHCP on the hub
 ip address 4.5.6.1 255.255.255.254
 !turn on nat and tag this as an outside interface
 ip nat outside
 !normally you don't want packets being re-assembled, but this might be needed
 !depending on the craptacular network you have end points on
 ip virtual-reassembly in
 ! next three setup the media type
 duplex auto
 speed auto
 media-type rj45
 ! disable CDP and DECnet MOP
 no cdp enable
 no mop enabled

DMVPN config

This sets up the DMVPN config on the tunnel interface.


  • Tunnel 0
interface Tunnel0
 description HUB DMVPN multipoint to clients
 !set the bandwidth, needed for routing protocols
 bandwidth 10000
 ip address 
 no ip redirects
 !set the MTU of the tunnel
 ip mtu 1400
 ip pim nbma-mode
 ip pim sparse-mode
 !this is an auth string used to uniquely id this DMVPN session.  
 ip nhrp authentication HUBKey3
 !Allows NHRP to automatically add spoke routers to the multicast NHRP mappings. 
 ip nhrp map multicast dynamic
 !this number is globally uniquie 32 bit number. 
 ip nhrp network-id 737373
 ip nhrp holdtime 600
 ip tcp adjust-mss 1350
 ip ospf network broadcast
 ip ospf priority 255
 delay 1000
 cdp enable
 tunnel source GigabitEthernet0/0
 tunnel mode gre multipoint
 tunnel key 737373
 tunnel protection ipsec profile DMVPN_PROFILE shared
 hold-queue 50 out


Routing Config

  • OSPF
router ospf

Serial/Stun Config

Voice Port Config

  • voice class permanent
voice class permanent 1811
 signal timing oos timeout disabled
 signal keepalive disabled
 signal sequence oos no-action


  • voice-port
voice-port 0/0/0
  • dial-peer voice 1601 voip
  • dial-peer voice 1600 pots

NAT Config

IPSEC Remote Access Config

This will give you remote access to the server for remote access using a PSK and XAUTH

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_conn_esyvpn/configuration/15-sy/sec-conn-esy-15-sy-book/sec-easy-vpn-srvr.html

You will need to select a pool of addresses for this to use. As this is on the hub, we're using 172.19.0.128/28 

! need this for the VPN client
aaa authentication login vpnclient local
aaa authorization network localgroups local 

! need a username
username bryan privilege 5 secret changeme

crypto logging session
crypto logging session
!
crypto logging ezvpn group VPN
!
! define the isakmp policy priority 
crypto isakmp policy 10
 authentication pre-share
 group 2
 lifetime 3600
!
crypto isakmp policy 200
 authentication pre-share
 lifetime 3600

! This is a setup of the ISAKMP Config group, it will be referred to in the ISAKMP
! profile VPN-DIALER-USERS  This is the group ID for the pre shared key.
crypto isakmp client configuration group VPN-DIALER-GROUP
 ! This is the pre shared key for the VPN
 key SEKRET-KEY-HERE  
 ! users will use this router as a DNS server
 dns 10.2.3.0
 ! the pool of IP addresses used by the VPN
 pool VPN-POOL
 ! access list to split traffic locally, this will allow on listed subnets to go into 
 ! the VPN.  Remove this if you want all traffic to go via the VPN
 acl VPN-DIALER-SPLIT
 ! allow the client to access it's local lan segment when connected to the VPN
 include-local-lan
 ! banner to display at login on the client (optional)
 banner ^C          
Quantar Link User Access
Licensed users only
          
This is HUB1
All access is logged.
This is a private system.
Disconnect _NOW_, you are not welcome here 
                      ^C

! this setups a Phase 1 ISAKMP profile of VPN-DIALER-USERS
crypto isakmp profile VPN-DIALER-USERS
   ! This ties it back to the VPN-DIALER-GROUP and 
   match identity group VPN-DIALER-GROUP
   !these next two tie it to the AAA configured for auth
   client authentication list vpnclient
   isakmp authorization list localgroups
   ! this makes the server listen, not initiate 
   client configuration address respond
   ! the isakmp client configuration group that this profile uses 
   client configuration group VPN-DIALER-GROUP
   ! finally the virtual template interface that will be used to make a virtual-access
   ! interface when the user connects
   virtual-template 199
!
!
! this is the IPSEC hash and algo used and is referred to in the ipsec profile
crypto ipsec transform-set VPN-DIALER-TRANSFORM-SET esp-aes esp-sha-hmac 
!
! This ipsec profile is referenced in the Virtual template 199 to specify which hash
! and algo will be permitted.  You can define multiple transform-sets 
crypto ipsec profile VPN-DIALER-PROFILE
 set transform-set VPN-DIALER-TRANSFORM-SET 
!
! This is the virtual template that will be cloned into a virtual-access tunnel when a
! Client Connects.  This makes the client in their own tunnel interface.
interface Virtual-Template199 type tunnel
 description VPN-DIALER-USERS
 ! this borrows the loopback 0 IP for the router side of the tunnel.
 ip unnumbered Loopback0
 ! cut the MTU
 ip mtu 1400
 ! turn on NAT (optional)
 ip nat inside
 ip virtual-reassembly in
 ! this will adjust the IPv4 tcp MSS to avoid MTU issues
 ip tcp adjust-mss 1200
 tunnel mode ipsec ipv4
 ! tie the tunnel config back to a crypto ipsec profile
 tunnel protection ipsec profile VPN-DIALER-PROFILE

! define the local Pool of IP's
ip local pool VPN-POOL 172.19.0.128 172.19.0.143

! put in a default null route for this subnet.  When a client connects it will be a /32
! and longer prefix
ip route 172.19.0.128 255.255.255.240 Null0 tag 69

! this next part will prevent an OSPF LSA going out every time a user connects.  Normally 
! when the user connects they would have a /32 route and it woould be announced to all 
! routers in OSPF.
router ospf 1
 ! make a summary address for this
 summary-address 172.19.0.128 255.255.255.240
 ! and if it matches tag 69, redistribuite it.
 redistribute static subnets tag 69


! this is the IP space allowed in the tunnel for split routing.  
ip access-list extended VPN-DIALER-SPLIT
 permit ip 10.0.0.0 0.255.255.255 any
 permit ip 172.16.0.0 0.15.255.255 any

DNS Config

SNMP Config

  • SNMP ACL

SSH Config

NTP Config

  • NTP ACL

User config

aaa new-model
!
!
aaa authentication login default local
aaa authentication login vpnclient local
aaa authorization console
aaa authorization config-commands
aaa authorization exec default local 
aaa authorization network localgroups local

VTY ACL

DHCP

ip dhcp pool P25NX-local-80
 network 172.31.7.80 255.255.255.240
 default-router 172.31.7.81 
 dns-server 172.31.7.81 
 lease 0 0 15


ESW module config

  • IP management
  • SNMP
  • rancid

Configuring a client

Retrieved from "https://wiki.w9cr.net/index.php?title=Quantar_Linking&oldid=6024"