Difference between revisions of "Quantar Linking"

From W9CR
Jump to navigation Jump to search
Line 256: Line 256:
 
vim /etc/php/7.3/fpm/pool.d/librenms.conf
 
vim /etc/php/7.3/fpm/pool.d/librenms.conf
  
 +
 +
==== config Rancid ====
 +
sudo apt-get install rancid subversion
 +
https://docs.librenms.org/Extensions/Rancid/
  
  

Revision as of 07:16, 8 August 2020

THIS IS A WORK IN PROGRESS

There are a few ways to link Quantars, using p25nx, mmvdm, and the method presented here using astro tacs.

Quantars were designed to be "linked" only as a simulcast or linked receiver network for a single system. This is accomplished by connecting the wireline for analog and the v.24 for digital. Quantars may be linked back-to-back by using a v.24 cross over and connecting the wirelines between them. While this is the simplest way, it doesn't scale. Using an ASTROTAC comparator to sit between the Quantars as a network switch function.

Each Quantar needs a two links to the ASTRO-TAC, a v.24 and a wireline link. If all Quantars are at the same site, this is really easy to do, just connect the wirelines and v.24's using cross overs. In the early 1990's the v.24 and wireline interfaces made quite a bit of sense, the v.24 and wireline would plug directly into a channel bank, and the DS1 or fractional DS1's between sites was easy. As of today, T1's are hard to find and very expensive. Channel banks are thousands of dollars on the used market now.

Today the internet is ubiquitous and linking via IP is the preferred method. This means we must transport synchronous serial and analog voice for each Quantar back to a central site. There are a few ways to do this via IP or MPLS, and presented here will be using cheap and easily available cisco routers in a VPN.


Architecture overview

There are several parts of this network, but the fundamental precept is creating link of rs232 and wireline voice between each port on the AstroTAC and each Quantar. This is no different than if all the equipment was in the same room, or linked into channel banks on T1 lines between sites. We're using IP multi-point VPN overlay network to simulate a circuit based network.

Our network stack is :

Internet
--------
DMVPN
--------
STUN and G.711 VOIP circuits
--------
ATAC and Quantars

What's presented here assumes each site has a stable IPv4 internet connection and the router has a globally routeable IPv4 address. The overlay network is based on Dynamic Multi-Point VPN (DMVPN) and allows us to configure a single hub router and have the same or similar config on all the endpoints. Traffic from router to router builds a connection on the fly encrypted between each router, even if the IP address of the routers change.

That last concept is important as anything that prevents routers from forwarding IP inbound (ie if the router is behind NAT) will prevent site to site connectivity. Generally this isn't needed as we only talk hub to site for the circuits, but can present it self during troubleshooting from one site to the other.


Logical connections

Template:Required image



Network Planning

Lots of routers, lots of ports, lots of circuits. You will need to name and keep interfaces described or you will be unable to troubleshoot this network.

Reliable bandwidth and pps is required. The analog links will require about 60 kbit/s at 33 PPS at all times , stun is another 20 kbit/s when it's operating. A fully loaded 16 port ATAC3000 using all remote ports will be 1.280 Mbit/s at 1220 PPS or about a fully loaded T1.


The first


overlay network

Our first step will be designing the overlay network. This consists of diagraming out sites and what equipment will go else where. As part of this I like to fully populate out and cross connect the central hub to the ATAC. This way it's all cabled up and I don't need to make changes at the hub to add an additional site.

Template:Required image

IP addressing

IP addressing needs to be through out for the overlay network. As this will not interconnect with any other network, you're free to use your own addressing scheme.

I use IP space in the 172.16.0.0/12 space with a /20 for the tunnel interfaces and different space for each routers loopback. At some sites and the hub it's handy to have a local interface which can do dhcp so you can use a local laptop to connect to the network elements directly. This should be given security considerations if enabled.

I will also setup remote access VPN into the network at the HUB as well. This needs it's own subnet for routing of connected clients; a /27 can be used here as well, but it must be separate from any other space used on Ethernet interfaces at the hub.

3845 Ports
Voice Port Serial Port ATAC Site # Local # Remote STUN Color
0/0/0 s1/0 1 TPA 1100 1101 10 White/Blue
0/0/1 s1/1 2 MIA 1200 1201 20 White/Orange
0/1/0 s1/2 3 JAX 1300 1301 30 White/Green
0/1/1 s1/3 4 MCO 1400 1401 40 White/Brown
0/2/0 s1/4 5 FLL 1500 1501 50 White/Slate
0/2/1 s1/5 6 EYW 1600 1601 60 Red/Blue
0/3/0 s1/6 7 RSW 1700 1701 70 Red/Orange
0/3/1 s1/7 8 SPG 1800 1801 80 Red/Green
3/0/0 s1/8 9 BOW 1900 1901 90 Red/Brown
3/0/1 s1/9 10 MLB 2000 2001 100 Red/Slate
3/1/0 s1/10 11 SQR 2100 2101 110 Black/Blue
3/1/1 s1/11 12 TLH 2200 2201 120 Black/Orange
4/0/0 s1/12 13 PBI 2300 2301 130 Black/Green
4/0/1 s1/13 14 GNV 2400 2401 140 Black/Brown
4/1/0 s1/14 15 PNS 2500 2501 150 Black/Slate
4/1/0 s1/15 16 PIE 2600 2601 160 Yellow/Blue
Router Tunnel IP Subnet /24 LoopBack gi2/0 /31 gi2/0.1 /30 gi2/0.172 /27 ESW Vlan1 /30
HUB 172.18.1.1/22 172.19.0.0 10.2.3.0 172.19.0.254 172.19.0.249 172.19.0.1 172.19.0.250
TPA 172.18.1.10 172.19.1.0 10.2.3.1 172.19.1.254 172.19.1.249 172.19.1.1 172.19.1.250
MIA 172.18.1.20 172.19.2.0 10.2.3.2 172.19.2.254 172.19.2.249 172.19.2.1 172.19.2.250
JAX 172.18.1.30 172.19.3.0 10.2.3.3 172.19.3.254 172.19.3.249 172.19.3.1 172.19.3.250
MCO 172.18.1.40 172.19.4.0 10.2.3.4 172.19.4.254 172.19.4.249 172.19.4.1 172.19.4.250
FLL 172.18.1.50 172.19.5.0 10.2.3.5 172.19.5.254 172.19.5.249 172.19.5.1 172.19.5.250
EYW 172.18.1.60 172.19.6.0 10.2.3.6 172.19.6.254 172.19.6.249 172.19.6.1 172.19.6.250
RSW 172.18.1.70 172.19.7.0 10.2.3.7 172.19.7.254 172.19.7.249 172.19.7.1 172.19.7.250
SPG 172.18.1.80 172.19.8.0 10.2.3.8 172.19.8.254 172.19.8.249 172.19.8.1 172.19.8.250
BOW 172.18.1.90 172.19.9.0 10.2.3.9 172.19.9.254 172.19.9.249 172.19.9.1 172.19.9.250
MLB 172.18.1.100 172.19.10.0 10.2.3.10 172.19.10.254 172.19.10.249 172.19.10.1 172.19.10.250
SQR 172.18.1.110 172.19.11.0 10.2.3.11 172.19.11.254 172.19.11.249 172.19.11.1 172.19.11.250
TLH 172.18.1.120 172.19.12.0 10.2.4.12 172.19.12.254 172.19.12.249 172.19.12.1 172.19.12.250
PBI 172.18.1.130 172.19.13.0 10.2.4.13 172.19.13.254 172.19.13.249 172.19.13.1 172.19.13.250
GNV 172.18.1.140 172.19.14.0 10.2.4.14 172.19.14.254 172.19.14.249 172.19.14.1 172.19.14.250
PNS 172.18.1.150 172.19.15.0 10.2.4.15 172.19.15.254 172.19.15.249 172.19.15.1 172.19.15.250
PIE 172.18.1.160 172.19.16.0 10.2.4.16 172.19.16.254 172.19.16.249 172.19.16.1 172.19.16.250


circuits over this

Interface naming

It's very helpful to have an interface name on each interface on the router. These should be in ALL CAPS and follow a description pattern.

Any IP address should be in DNS and reverse DNS.

Description "<circuit-ID> <source host>-<src cross connect> to <destination host>-<dest port>-<dest cross connect>"

Examples:

interface serial 1/0
 description "HUB-ATAC-pt1 to TPA-s0/0/0-Quantar-KP1UG-pt1" 

Meaning Hub connected to S1 going to Tampa s0/0/0 and connecting to Quantar KP1UG Port 1

interface Tunnel0
 description "HUB DMVPN Multipoint"

Hub router multipoint interface for DMVPN

interface GigabitEthernet0/0
 description "HUB to upstream ISP <CIRCUIT ID>

DNS

Having all interfaces named in DNS is a good idea. This makes troubleshooting and tracerouting on the overlay network much easier. As the hub router has a bunch of extra CPU, it's easy to configure this on the hub and have it be primary DNS. This is well baked in IOS and other than some additional config it's not hard.


security

network monitoring

RPI4 4gb

I use a RPI-4 4gb with an external SSD for this.

Software for RPI4

apt-get install ntp ntpdate python vim screen ipsec-tools strongswan fail2ban snmp haveged libacl1-dev python3-dev libssl-dev gcc g++ fio pbzip2 ncdu sudo 

vim /etc/dhcpcd.conf 172.31.7.65 DNS server


vim /etc/screenrc

uncomment "termcapinfo xterm|xterms|xs|rxvt ti@:te@"

vim /etc/bash.bashrc

  uncomment the stuff below 
  # enable bash completion in interactive shells 


update-alternatives --config editor

  Then select #3 vim.basic

sudo timedatectl set-timezone UTC

Configure Postfix

Postfix is installed to forward mail for root to a smtp host.

apt-get install postfix mailutils

This will run an installer with a curses interface and you must select Satallite System. Check the System mail name is the hostname of the server, and the SMTP relay host is morty.keekles.org. Root and postmaster mail should be me@myemail.com.

Should you need to reconfigure this use:

dpkg-reconfigure postfix

other aliases are setup in /etc/aliases. You must run newaliases after this is updated for them to take effect.

Setup LibreNMS

The RPI-4 is configured as follows

https://docs.librenms.org/Installation/Install-LibreNMS/

Install Software

apt install software-properties-common
apt update
apt install acl curl apache2 composer fping git graphviz imagemagick libapache2-mod-fcgid mariadb-client mariadb-server mariadb-common  mtr-tiny nmap php7.3-cli php7.3-curl php7.3-fpm php7.3-gd php7.3-json php7.3-mbstring php7.3-mysql php7.3-snmp php7.3-xml php7.3-zip rrdtool snmp snmpd whois python3-pymysql python3-dotenv python3-redis python3-setuptools 

Add LibreNMS user

useradd librenms -d /opt/librenms -M -r -s /bin/bash

Download LibreNMS =

cd /opt
git clone https://github.com/librenms/librenms.git

Set Permissions

chown -R librenms:librenms /opt/librenms
chmod 771 /opt/librenms
setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/

Install PHP deps

su - librenms 
./scripts/composer_wrapper.php install --no-dev
exit

Set timezone

vim /etc/php/7.3/fpm/php.ini date.timezone = Etc/UTC

vi /etc/php/7.3/cli/php.ini

Config the mariadb

vi /etc/mysql/mariadb.conf.d/50-server.cnf

innodb_file_per_table=1
lower_case_table_names=0

systemctl restart mysql.service


config apache

cp /etc/php/7.3/fpm/pool.d/www.conf /etc/php/7.3/fpm/pool.d/librenms.conf vim /etc/php/7.3/fpm/pool.d/librenms.conf


config Rancid

sudo apt-get install rancid subversion
https://docs.librenms.org/Extensions/Rancid/



SNMP

SSH for management

Hardware for linking

Will you need analog? If no you can eliminate a bunch of configuration and half the circuits. The bandwidth requirements at the hub will be less as well.


Hub site

A hub site needs a router with at least the number of ports on it you plan to use remotely. A converted quantar might only have 5 ports, but a fully loaded ATAC could be 16. If they are all remote you will need that too. The hub router also need to process a number of things on the network. All in all it's not much, but given the cost of the cisco routers on the used market the following config has become my standard.

  • Cisco 3845 router
    • AIM-VPN/SSL-3 VPN Module for 3825/3845 Routers
    • 768M or 1GB of RAM
    • 2GB flash disk
    • PVDM2-64 (DSP module for analog lines)
    • Optional PSU PWR-3845-AC-IP, this will supply 48v for the switch card
  • 1, NM-16A/S - Cisco 800-20840-01D 16-Port Async/Sync Serial Network Module (you can use the NM-8A/S too, but it's got different connectors)
  • 8, Cisco VIC2-2E/M 2-Port Ear and Mouth Voice Interface Card (only for analog)
  • 2, Cisco NM-2V, carrier cards for VIC2 to put in the NM ports
  • 1, NME-16ES-1G-P 16 Port POE switch module. This is optional but it provides a 3750 switch in a network module with POE that can power RPi's or other local devices.
  • CAB-SS-232FC, RS-232 Cable, DCE Female to Smart Serial for all serial ports
  • OR CISCO CAB-232FC RS-232 Cable, DCE Female to Serial if using the 8 port card.


Edge site

An edge site can support one or two Quantars or cascaded (advanced) ATACs. If you don't need to support Voice, an 1841 or other 1800 series router can be used, but the cost delta is negligible. This entire 2811 setup is under $100 via eBay. I like to use the same cables and connectors for serial at the hub as at the edge sites, this determines the cards for serial. If you're doing voice, two ports of serial matches the 2 ports of E and M well.

  • Cisco 2811
    • AIM-VPN/SSL-2 VPN Module for 2811 Routers
    • 512M of RAM
    • 2GB flash disk
    • PVDM2-64 (DSP module for analog lines)
  • 1, WIC-2A/S 2 port Serial WAN Interface Card OR
  • 1, WIC-1T 1 port Serial Card uses the larger serial connector
  • 1, Cisco VIC2-2E/M 2-Port Ear and Mouth Voice Interface Card (only for analog)
  • CAB-SS-232FC, RS-232 Cable, DCE Female to Smart Serial for all serial ports
  • OR CISCO CAB-232FC RS-232 Cable, DCE Female to Serial if using the 8 port card.

You may want to pickup a serial adapter and FTDI serial to USB dongle for console access just to leave at the site. This can be handy.


Making it work

Configuring the hub

IP Interface Config

  • Loopback 0
interface Loopback0
 description "HUB loop0"
 ip address 10.2.1.1 255.255.255.255 


  • Gi0/0
interface GigabitEthernet0/0
 description "Outboound interface to AS3356"
 !whatever your IP is.  This can't be DHCP on the hub
 ip address 4.5.6.1 255.255.255.254
 !turn on nat and tag this as an outside interface
 ip nat outside
 !normally you don't want packets being re-assembled, but this might be needed
 !depending on the craptacular network you have end points on
 ip virtual-reassembly in
 ! next three setup the media type
 duplex auto
 speed auto
 media-type rj45
 ! disable CDP and DECnet MOP
 no cdp enable
 no mop enabled

DMVPN config

This sets up the DMVPN config on the tunnel interface.


  • Tunnel 0
interface Tunnel0
 description HUB DMVPN multipoint to clients
 !set the bandwidth, needed for routing protocols
 bandwidth 10000
 ip address 
 no ip redirects
 !set the MTU of the tunnel
 ip mtu 1400
 ip pim nbma-mode
 ip pim sparse-mode
 !this is an auth string used to uniquely id this DMVPN session.  
 ip nhrp authentication HUBKey3
 !Allows NHRP to automatically add spoke routers to the multicast NHRP mappings. 
 ip nhrp map multicast dynamic
 !this number is globally uniquie 32 bit number. 
 ip nhrp network-id 737373
 ip nhrp holdtime 600
 ip tcp adjust-mss 1350
 ip ospf network broadcast
 ip ospf priority 255
 delay 1000
 cdp enable
 tunnel source GigabitEthernet0/0
 tunnel mode gre multipoint
 tunnel key 737373
 tunnel protection ipsec profile DMVPN_PROFILE shared
 hold-queue 50 out


Routing Config

  • OSPF
router ospf

Serial/Stun Config

Voice Port Config

  • voice class permanent
voice class permanent 1811
 signal timing oos timeout disabled
 signal keepalive disabled
 signal sequence oos no-action


  • voice-port
voice-port 0/0/0
  • dial-peer voice 1601 voip
  • dial-peer voice 1600 pots

NAT Config

IPSEC Remote Access Config

DNS Config

SNMP Config

  • SNMP ACL

SSH Config

NTP Config

  • NTP ACL

User config

aaa new-model
!
!
aaa authentication login default local
aaa authentication login vpnclient local
aaa authorization console
aaa authorization config-commands
aaa authorization exec default local 
aaa authorization network localgroups local 

VTY ACL

DHCP

ip dhcp pool P25NX-local-80
 network 172.31.7.80 255.255.255.240
 default-router 172.31.7.81 
 dns-server 172.31.7.81 
 lease 0 0 15


ESW module config

  • IP management
  • SNMP
  • rancid

Configuring a client