+++
title = 'FireMon'
date = 2013-12-01
[extra]
years = '2013–Present'
+++

FireMon is a software development company based in Overland Park, KS.  As the
System Architect, I focus on building a scalable platform for delivering
FireMon software to customers that is easy to use.  FMOS, the FireMon Operating
System, is a mechanism for delivering the FireMon <abbr title="Security
Intelligence Platform">SIP</abbr> to customers and a collection of tools for
deploying and managing the software in a wide array of environments, ranging
from a single server to massive multi-node ecosystems.

<!-- more -->

# FMOS Generation II

* Based on CentOS 7
* Full-disk encryption using LUKS
* Anaconda installer with custom addon for generating machine-specific LUKS
  master key passphrase
* Kickstart script for fully-automated installation
* Used Koji to build RPM packages for first- and third-party software
* Distribution included Ansible for configuration management
* systemd units for controlling FireMon application services
* Configuration policy for deployment of all FireMon software and
  third-party dependencies
  * Support for single-server and distributed deployments
  * Automatically compute JVM heap sizes for each process based on availnable
    resources
  * Configures Elasticsearch in single-node or clustered mode
  * Configures PostgreSQL with optional replication to standby servers
  * Configures Kernel NFS server and client to share filesystem data between
    machines
  * Configures FireMon application server processes, including connection and
    authentication information for PostgreSQL, Elasticsearch
  * Configures strongSwan IPsec/IKEv2 key management daemon for opportunistic
    encryption of Elasticsearch communication
* …

# FMOS Generation III

* Based on CentOS 7, later CentOS 8 (Stream)
* Immutable SquashFS root filesystem image
* …


# DevOps Team Lead

* Deployed and maintained hundreds of internal and cloud systems
  * HashiCorp Vault
  * Elasticsearch
  * Atlassian Bitbucket
  * Jenkins
* Used PXE for provisioning on-premises virtual machines
* Ansible configuration management


# Internal Tools

## FMOS Web Tools

* Internal application used by software developers and support agents
* Multi-tiered architecture with multiple nodes at each tier to avoid any
  single point of failure
  * Application Server Tier: Python 3.6/FastAPI
  * Storage Tier: GlusterFS
  * Index Tier: Elasticsearch
  * Cache Tier: Redis
  * Message Tier: RabbitMQ
  * Worker Tier: Python 3.6/Celery
  * Ingress: HAProxy
  * User Interface: Typescript/Vue+Vuetify


## QEMU VM Log Socket Proxy

* Component of FMOS End-to-End tests running on-premises using QEMU/libvirt
* Uses kernel *inotify(7)* events to detect virtual machine log channel socket
  files appearing on the VM host
* Automatically connects to sockets as they appear
* Receives all data from channel sockets and writes them to a file in the
  libvirt storage pool
* Written in Rust


## FMOS ISO Writer

* Internal application used by development and QA teams to write FMOS installer
  images to USB disks attached to remote physical appliances
* Accessible via purpose-built, ultra-minimal Linux distribution (Kernel and
  Busybox only) delivered by network boot/PXE
* Written in Rust