awesome-sysadmin/README.md

Awesome Sysadmin

A curated list of amazingly awesome open source sysadmin resources inspired by Awesome PHP

• Awesome Sysadmin
  • Backups
  • Cloning
  • Cloud Computing
  • Cloud Storage
  • Code Review
  • Collaborative Software
  • Configuration Management Database
  • Configuration Management
  • Continuous Integration & Continuous Deployment
  • Distributed Filesystems
  • DNS
  • Hosting Control Panels
  • IMAP/POP3
  • IT Asset Management
  • LDAP
  • Log Management
  • Monitoring
  • Metric & Metric Collection
  • Network Configuration Management
  • Newsletter
  • NoSQL
  • Packaging
  • Queuing
  • RDBMS
  • Security
  • Service Discovery
  • SMTP
  • Software Containers
  • SSH
  • Statistics
  • Ticketing systems
  • Troubleshooting
  • Project Management
  • Version control
  • Virtualization
  • VPN
  • XMPP
  • Web
  • Webmails
  • Wikis
• Resources
  • Blogs
  • Books
  • Editors
  • Newsletters
  • Repositories
  • Websites
• Contributing

Backups

Backup software.

• Amanda - Client-server model backup tool.
• Bacula - Another Client-server model backup tool.
• Backupninja - Lightweight, extensible meta-backup system.
• Backuppc - Client-server model backup tool with file pooling scheme.
• Brebis - A fully automated backup checker
• Burp - Network backup and restore program.
• Duplicity - Encrypted bandwidth-efficient backup using the rsync algorithm.
• Lsyncd - Watches a local directory trees for changes, and then spawns a process to synchronize the changes. Uses rsync by default.
• Rsnapshot - Filesystem Snapshotting Utility.
• SafeKeep - Centralized pull-based backup using rdiff-backup.
• TarSnap - Secure backup service with an open-source client.
• UrBackup - Another client-server backup system.
• DREBS - AWS EBS backup script that supports strategies.

Cloning

Cloning software.

• Clonezilla - Partition and disk imaging/cloning program.
• Fog - Another computer cloning solution.
• Redo Backup - Easy Backup, Recovery and Restore.

Cloud Computing

• Archipel - Manage and supervise virtual machines using Libvirt.
• CloudStack - Cloud computing software for creating, managing, and deploying infrastructure cloud services.
• Cobbler - Cobbler is a Linux installation server that allows for rapid setup of network installation environments.
• Eucalyptus - Open source private cloud software with AWS compatibility.
• Mesos - Develop and run resource-efficient distributed systems.
• OpenNebula - An user-driven cloud management platform for sysadmins and devops.
• OpenStack - Open source software for building private and public clouds.
• The Foreman - Foreman is a complete lifecycle management tool for physical and virtual servers. FOSS.

Cloud Orchestration

• BOSH - IaaS orchestration platform originally written for deploying and managing Cloud Foundry PaaS, but also useful for general purpose distributed systems.
• Cloudify - Open source TOSCA-based cloud orchestration software platform written in Python and YAML.
• Juju - Cloud orechestration tool which manages services as charms, YAML configuration and deployment script bundles.
• MCollective - Ruby framework to manage server orchestration, developed by Puppet labs.
• Overcast - Deploy VMs across different cloud providers, and run commands and scripts across any or all of them in parallel via SSH.
• Rundeck - Simple orchestration tool.
• Salt - It's written in Python.

Cloud Storage

• git-annex assistant - A synchronised folder on each of your OSX and Linux computers, Android devices, removable drives, NAS appliances, and cloud services.
• ownCloud - Provides universal access to your files via the web, your computer or your mobile devices.
• Seafile - Another Open Source Cloud Storage solution.
• SparkleShare - Provides cloud storage and file synchronization services. By default, it uses Git as a storage backend.
• Swift - A highly available, distributed, eventually consistent object/blob store.
• Syncthing - Open Source system for private, encrypted and authenticated distrobution of data.

Code Review

Web Based collaborative code review system.

• Gerrit - Based on the Git version control, it facilitates software developers to review modifications to the source code and approve or reject those changes.
• Review Board - Available as free software uner the MIT License.

Collaborative Software

Collaborative software or groupware suites.

• Citadel/UX - Collaboration suite (messaging and groupware) that is descended from the Citadel family of programs.
• EGroupware - Groupware software written in PHP.
• Horde Groupware - PHP based collaborative software suite that includes email, calendars, wikis, time tracking and file management.
• Kolab - Another groupware suite.
• SOGo - Collaborative software server with a focus on simplicity and scalability.
• Zimbra - Collaborative software suite, that includes an email server and web client.

Configuration Management Database

Configuration management database (CMDB) software.

• i-doit - Open Source IT Documentation and CMDB.
• iTop - A complete open source, ITIL, web based service management tool.
• Ralph - Asset management, DCIM and CMDB system for large Data Centers as well as smaller LAN networks.
• Clusto - Helps you keep track of your inventory, where it is, how it's connected, and provides an abstracted interface for interacting with the elements of the infrastructure.

Configuration Management

Configuration management tools.

• Ansible - It's written in Python and manages the nodes over SSH.
• CFEngine - Lightweight agent system. Configuration state is specified via a declarative language.
• Chef - It's written in Ruby and Erlang and uses a pure-Ruby DSL.
• Fabric - Python library and cli tool for streamlining the use of SSH for application deployment or systems administration tasks.
• Pallet - Infrastructure definition, configuration and management via a Clojure DSL.
• Puppet - It's written in Ruby and uses Puppet's declarative language or a Ruby DSL.
• Salt - It's written in Python.
• Slaughter - It's written in Perl.

Continuous Integration & Continuous Deployment

Continuous integration/deployment software.

• Buildbot - Python-based toolkit for continuous integration.
• Drone - Continuous integration server built on Docker and configured using YAML files.
• GitLab CI - Based off of ruby. They also provide GitLab, which manages git repositories.
• Go - Open source continuous delivery server.
• Jenkins - An extendable open source continuous integration server.
• Vlad the Deployer - Deployment automation.

Distributed Filesystems

Network distributed filesystems.

• Ceph - Distributed object store and file system.
• DRBD - Disributed Replicated Block Device.
• LeoFS - Unstructured object/data storage and a highly available, distributed, eventually consistent storage system.
• GlusterFS - Scale-out network-attached storage file system.
• HDFS - Distributed, scalable, and portable file-system written in Java for the Hadoop framework.
• Lustre - A type of parallel distributed file system, generally used for large-scale cluster computing.
• MooseFS - Fault tolerant, network distributed file system.
• MogileFS - Application level, network distributed file system.
• OpenAFS - Distributed network file system with read-only replicas and multi-OS support.
• TahoeLAFS - secure, decentralized, fault-tolerant, peer-to-peer distributed data store and distributed file system.
• XtreemFS - XtreemFS is a fault-tolerant distributed file system for all storage needs.

DNS

DNS servers.

• Bind - The most widely used name server software.
• djbdns - A collection of DNS applications, including tinydns.
• Designate - DNS REST API that support several DNS servers as its backend.
• dnsmasq - A lightweight service providing DNS, DHCP and TFTP services to small-scale networks.
• Knot - High performance authoritative-only DNS server.
• NSD - Authoritative only, high performance, simple name server.
• PowerDNS - DNS server with a variety of data storage back-ends and load balancing features.
• Unbound - Validating, recursive, and caching DNS resolver.
• Yadifa - Lightweight authoritative Name Server with DNSSEC capabilities powering the .eu top-level domain.

Hosting Control Panels

Web hosting control panels

• Ajenti - Control panel for Linux and BSD.
• Feathur - VPS Provisioning and Management Software.
• ISPConfig - Hosting control panel for Linux.
• VestaCP - Hosting panel for Linux but with Nginx.
• Virtualmin - Control panel for Linux based on webmin.
• ZPanel - Control panel for Linux, BSD, and Windows.

IMAP/POP3

IMAP/POP3 mail servers.

• Courier IMAP/POP3 - Fast, scalable, enterprise IMAP and POP3 server.
• Cyrus IMAP/POP3 - Intended to be run on sealed servers, where normal users are not permitted to log in.
• Dovecot - IMAP and POP3 server written primarily with security in mind.
• Qpopper - One of the oldest and most popular server implementations of POP3.

IT Asset Management

IT Assets Management software.

• GLPI - Information Resource-Manager with an additional Administration Interface.
• OCS Inventory NG - Enables users to inventory their IT assets.
• RackTables - Datacenter and server room asset management like document hardware assets, network addresses, space in racks, networks configuration.
• Ralph - Asset management, DCIM and CMDB system for large Data Centers as well as smaller LAN networks.
• Snipe IT - Asset & license management software.

LDAP

LDAP servers.

• 389 Directory Server - Developed by Red Hat.
• Apache Directory Server - Apache Software Foundation project written in Java.
• Fusion Directory - Improve the Management of the services and the company directory based on OpenLDAP.
• OpenDJ - Fork of OpenDS.
• OpenDS - Another directory server written in Java.
• OpenLDAP - Developed by the OpenLDAP Project.

Log Management

Log management tools: collect, parse, visualize ...

• Elasticsearch - A Lucene Based Document store mainly used for log indexing, storage and analysis.
• Fluentd - Log Collector and Shipper.
• Flume - Distributed log collection and aggregation system.
• Graylog2 - Pluggable Log and Event Analysis Server with Alerting options.
• Heka - Stream processing system which may be used for log aggregation.
• Kibana - Visualize logs and time-stamped data.
• Logstash - Tool for managing events and logs.
• Octopussy - Log Management Solution (Visualize / Alert / Report).

Monitoring

Monitoring software.

• Cacti - Web-based network monitoring and graphing tool.
• Cabot - Monitoring and alerts, similar to PagerDuty.
• check_mk - Collection of extensions for Nagios.
• Dash - A low-overhead monitoring web dashboard for a GNU/Linux machine.
• Icinga - Fork of Nagios.
• LibreNMS - fork of Observium.
• Monit - Small Open Source utility for managing and monitoring Unix systems.
• Munin - Networked resource monitoring tool.
• Naemon - Network monitoring tool based on the Nagios 4 core with performance enhancements and new features.
• Nagios - Computer system, network and infrastructure monitoring software application.
• Observium - SNMP monitoring for servers and networking devices. Runs on linux.
• OMD - The Open Monitoring Distribution.
• Opsview - Based on Nagios 4, Opsview Core is ideal for small IT and test environments.
• Riemann - Flexible and fast events processor allowing complex events/metrics analysis.
• Sensu - Open source monitoring framework.
• Sentry - Application monitoring, event logging and aggregation.
• Shinken - Another monitoring framework.
• Thruk - Multibackend monitoring webinterface with support for Naemon, Nagios, Icinga and Shinken.
• Xymon - Network monitoring inspired by Big Brother.
• Zabbix - Enterprise-class software for monitoring of networks and applications.
• Zenoss - Application, server, and network management platform based on Zope.

Metric & Metric Collection

Metric gathering and display software.

• Collectd - System statistic collection daemon.
• Collectl - High precision system performance metrics collecting tool.
• Dashing - Ruby gem that allows for rapid statistical dashboard development. An all HTML5 approach allows for big screen displays in data centers or conference rooms.
• Diamond - Python based statistic collection daemon.
• Ganglia - High performance, scalable RRD based monitoring for grids and/or clusters of servers. Compatible with Graphite using a single collection process.
• Grafana - A Graphite & InfluxDB Dashboard and Graph Editor.
• Graphite - Open source scaleable graphing server.
• InfluxDB - Open source distributed time series database with no external dependencies.
• KairosDB - Fast distributed scalable time series database, fork of OpenTSDB 1.x.
• OpenTSDB - Store and server massive amounts of time series data without losing granularity.
• RRDtool - Open source industry standard, high performance data logging and graphing system for time series data.
• Statsd - Application statistic listener.

Network Configuration Management

Network configuration management tools.

• GestióIP - An automated web based IPv4/IPv6 IP Address Management tool.
• RANCID - Monitors network device's configurarion and maintain history of changes.
• rConfig - Another network device configuration management tool.

Newsletters

Newsletter software.

• DadaMail - Mailing List Manager, written in Perl.
• phpList - Newsletter manager written in PHP.

NoSQL

NoSQL databases.

• Column-Family
  • Apache HBase - Hadoop database, a distributed, big data store.
  • Cassandra - Distributed DBMS designed to handle large amounts of data across many servers.
  • Hypertable - C++ based BigTable-like DBMS, communicates through Thrift and runs either as stand-alone or on distributed FS such as Hadoop.
• Document Store
  • CouchDB - Ease of use, with multi-master replication document-oriented database system.
  • ElasticSearch - Java based database, popular with log aggregation, and email archiving projects.
  • MongoDB - Another document-oriented database system.
  • RavenDB - Document based database with ACID/Transactional features.
  • RethinkDB - Open source distributed document store database, focuses on JSON.
• Graph
  • FlockDB - Twitter's distributed, fault-tolerant graph database.
  • Neo4j - Open source graph database.
• Key-Value
  • LevelDB - Google's high performance key/value database.
  • Redis - Networked, in-memory, key-value data store with optional durability.
  • Riak - Another fault-tolerant key-value NoSQL database.

Comparison of NoSQL servers: http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

Packaging

• fpm - Versatile multi format package creator.
• omnibus-ruby - Full stack, cross distro packaging software (Ruby).
• packman - Full stack, cross distro packaging software (Python).
• tito - Builds RPMs for git-based projects.

Queuing

• BeanstalkD - A simple, fast work queue.
• Gearman - Fast multi-language queuing/job processing platform.
• NSQ - A realtime distributed messaging platform.
• RabbitMQ - Robust, fully featured, cross distro queuing system.
• ZeroMQ - Lightweight queuing system.

RDBMS

Relational DBMS.

• Firebird - True universal open source database.
• Galera - Galera Cluster for MySQL is an easy-to-use high-availability solution with high system up-time, no data loss, and scalability for future growth.
• MariaDB - Community-developed fork of the MySQL.
• MySQL - Most popular RDBMS server.
• Percona Server - Enhanced, drop-in MySQL replacement.
• PostgreSQL - Object-relational database management system (ORDBMS).
• PostgreSQL-XL - Scalable Open Source PostgreSQL-based database cluster.
• SQLite - Library that implements a self-contained, serverless, zero-configuration, transactional SQL DBS.

Security

Security tools.

• Denyhosts - Thwart SSH dictionary based attacks and brute force attacks.
• Fail2Ban - Scans log files and takes action on IPs that show malicious behavior.
• SpamAssassin - A powerful and popular email spam filter employing a variety of detection techniques.

Service Discovery

• Consul - Consul is a tool for service discovery, monitoring and configuration.
• Doozerd - Doozer is a highly-available, completely consistent store for small amounts of extremely important data.
• ZooKeeper - ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

SMTP

SMTP servers.

• Exim - Message transfer agent (MTA) developed at the University of Cambridge.
• Haraka - A high-performance, pluginable SMTP server written in JavaScript.
• MailCatcher - Ruby gem that deploys a simply SMTP MTA gateway that accepts all mail and displays in web interface. Useful for debugging or development.
• Maildrop - Open Source disposable email SMTP server, also useful for development.
• OpenSMTPD - Secure SMTP server implementation from the OpenBSD project.
• Postfix - Fast, easy to administer, and secure Sendmail replacement.
• Qmail - Secure Sendmail replacement.
• Sendmail - Message transfer agent (MTA).

Software Containers

Operating system–level virtualization.

• Bitnami - Produces open source installers or software packages for web applications and development stacks as well as virtual appliances.
• Docker - Open platform for developers and sysadmins to build, ship, and run distributed applications.
• OpenVZ - Container-based virtualization for Linux.

SSH

SSH tools.

• autossh - Automatically respawn ssh session after network interruption.
• Cluster SSH - Controls a number of xterm windows via a single graphical console.
• DSH - Dancer's shell / distributed shell - Wrapper for executing multiple remote shell commands from one command line.
• Mosh - The mobile shell.
• parallel-ssh - Provides parallel versions of OpenSSH and related tools.
• SSH Power Tool - Execute commands and upload files to many servers simultaneously without using pre-shared keys.

Statistics

Analytics software.

• Analog - The most popular logfile analyser in the world.
• GoAccess - Open source real-time web log analyzer and interactive viewer that runs in a terminal.
• Piwik - Free and open source web analytics application.
• Webalizer - Fast, free web server log file analysis program.

Ticketing systems

Web-based ticketing system.

• Bugzilla - General-purpose bugtracker and testing tool originally developed and used by the Mozilla project.
• Cerb - A group-based e-mail management project built with a commercial open source license.
• Flyspray - Web-based bug tracking system written in PHP.
• MantisBT - Another web-based bug tracking system.
• osTicket - Open source support ticket system.
• Otrs - A free and open-source trouble ticket system software package that a company, organization, or other entity can use to assign tickets to incoming queries and track further communications about them.
• Request Tracker - Ticket-tracking system written in Perl.
• TheBugGenie - Open source ticket system with extremely complete users rights granularity.

Troubleshooting

Troubleshooting Tools.

• mitmproxy - A Python tool used for intercepting, viewing and modifying network traffic. Invaluable in troubleshooting certain problems.
• Sysdig - Capture system state and activity from a running Linux instance, then save, filter and analyze.
• Wireshark - The world's foremost network protocol analyzer.

Project Management

Web-based project management and bug tracking systems.

• ChiliProject - Fork of Redmine.
• GitBucket Clone of GitHub written in Scala; single jar install.
• GitLab - Clone of GitHub written in Ruby.
• Gogs - A self-hosted Git service written in Go.
• Phabricator Written in PHP.
• Redmine - Written in ruby on rails.
• The Bug Genie - Written in PHP.
• Trac - Written in python.

Version control

Software versioning and revision control.

• Fossil - Distributed version control with built-in wiki and bug tracking.
• Git - Distributed revision control and source code management (SCM) with an emphasis on speed.
• GNU Bazaar - Distributed revision control system sponsored by Canonical.
• Mercurial - Another distributed revision control.
• Subversion - Client-server revision control system.

Virtualization

Virtualization software.

• Ganeti - Cluster virtual server management software tool built on top of KVM and Xen.
• KVM - Linux kernel virtualization infrastructure.
• oVirt - Manages virtual machines, storage and virtual networks.
• Packer - A tool for creating identical machine images for multiple platforms from a single source configuration.
• Vagrant - Tool for building complete development environments.
• VirtualBox - Virtualization product from Oracle Corporation.
• Xen - Virtual machine monitor for 32/64 bit Intel / AMD (IA 64) and PowerPC 970 architectures.

VPN

VPN software.

• OpenVPN - Uses a custom security protocol that utilizes SSL/TLS for key exchange.
• Pritunl - OpenVPN based solution. Easy to set up.
• SoftEther - Multi-protocol software VPN with advanced features
• sshuttle - Poor man's VPN.
• strongSwan - Complete IPsec implementation for Linux.
• tinc - Distributed p2p VPN.

XMPP

XMPP servers.

• ejabberd - XMPP instant messaging server written in Erlang/OTP.
• Metronome IM - Fork of Prosody IM.
• MongooseIM - Fork of ejabberd.
• Openfire - Real time collaboration (RTC) server.
• Prosody IM - XMPP server written in Lua.
• Tigase - XMPP server implementation in Java.

Webmails

Webmail applications.

• Mailpile - A modern, fast web-mail client with user-friendly encryption and privacy features.
• Roundcube - Browser-based IMAP client with an application-like user interface.

Web

Web servers.

• Apache - Most popular web server.
• Cherokee - Lightweight, high-performance web server/reverse proxy.
• Lighttpd - Web server more optimized for speed-critical environments.
• Nginx - Reverse proxy, load balancer, HTTP cache, and web server.
• uWSGI - The uWSGI project aims at developing a full stack for building hosting services.

Web Performance

• HAProxy - Software based load Balancing, SSL offloading and performance optimization, compression, and general web routing.
• Varnish - HTTP based web application accelerator focusing on optimizing caching and compression.

Wikis

Wiki software.

• DokuWiki - Simple to use and highly versatile wiki that doesn't require a database.
• Gollum - A simple, Git-powered wiki with a sweet API and local frontend.
• ikiwiki - A wiki compiler.
• Mediawiki - Used to power Wikipedia.
• MoinMoin - An advanced, easy to use and extensible WikiEngine with a large community of users.
• Ōlelo Wiki - A a wiki that stores pages in a Git repository.
• TiddlyWiki - Complete interactive wiki in JavaScript.

Resources

Various resources, such as books, websites and articles, for improving your skills and knowledge.

Blogs

• Code as Craft - Etsy's Ops blog, lots of technical posts.
• DevOpsGuys - Devops consultants who blog about operations.
• Rackspace Developers - Slightly biased blog with lots of Devops Topics.

Books

Sysadmin related books.

• The Linux Command Line - A book about the Linux command line by William Shotts.
• The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win - How DevOps techniques can fix the problems that happen in IT organizations.
• The Practice of System and Network Administration - The first and second editions describes the best practices of system and network administration, independent of specific platforms or technologies.
• The Visible Ops Handbook: Implementing ITIL in 4 Practical and Auditable Steps - Is a methodology designed to jumpstart implementation of controls and process improvement.
• UNIX and Linux System Administration Handbook - Approaches system administration from a practical perspective.

Editors

Open source code editors.

• Atom - A hackable text editor from Github.
• Brackets - Open source code editor for web designers and front-end developers.
• Eclipse - IDE written in Java with an extensible plug-in system.
• Geany - GTK2 text editor.
• GNU Emacs - An extensible, customizable text editor-and more.
• Haroopad - Markdown editor with live preview.
• ICEcoder - Code editor awesomeness, built with common web languages.
• jotgit - Git-backed real-time collaborative code editing.
• Light Table - The next generation code editor.
• Lime - Aims to provide an open source solution to Sublime Text
• TextMate - A graphical text editor for OS X.
• Vim - A highly configurable text editor built to enable efficient editing.

Newsletters

• Servers for Hackers - Newsletter for programmers who find themselves needing to know their way around a server.

Repositories

Software package repositories.

• Dotdeb - Repository with LAMP updated packages for Debian.
• Remi - Repository with LAMP updated packages for RHEL/Centos/Fedora.

Websites

Useful sysadmin related websites.

• Ops School - Comprehensive program that will help you learn to be an operations engineer.
• Digital Ocean Tutorials - A surprisingly vast resource for getting the basics of certain applications, tools, or even systems administration topics.

Contributing

Please see CONTRIBUTING for details.