5 Tools to Scan a Linux Server for Malware and Rootkits

As a Linux server administrator, ensuring your system remains secure from malware and rootkits is critical for protecting sensitive data and maintaining system integrity. Malicious software can compromise your server's security, steal information, or create backdoors for attackers. This article explores five essential tools that help detect and eliminate security threats on Linux servers.

ClamAV

ClamAV is an open-source antivirus engine designed for detecting viruses, trojans, and other malicious software. It's lightweight, regularly updated, and supports multiple file formats including compressed archives and email attachments.

Installation and Basic Usage

# Install ClamAV (Ubuntu/Debian)
sudo apt update && sudo apt install clamav clamav-daemon

# Update virus definitions
sudo freshclam

# Scan a directory recursively
clamscan -r /home/user

# Scan with verbose output and remove infected files
clamscan -r --remove --verbose /var/www

ClamAV can be configured to quarantine or automatically remove infected files. For real-time protection, the clamd daemon runs continuously and scans files as they are accessed.

Rkhunter (Rootkit Hunter)

Rkhunter specializes in detecting rootkits, backdoors, and local exploits by comparing system file checksums, scanning for hidden files, and checking system binaries for modifications.

Installation and Scanning

# Install Rkhunter (Ubuntu/Debian)
sudo apt install rkhunter

# Update definitions
sudo rkhunter --update

# Perform comprehensive system scan
sudo rkhunter --checkall --sk

# Check for rootkits only
sudo rkhunter --check --rootkits-only

Rkhunter generates detailed reports in /var/log/rkhunter.log and highlights suspicious findings that require administrator review.

Chkrootkit

Chkrootkit is a lightweight rootkit scanner that checks for over 70 types of rootkits, worms, and trojans. It examines system binaries, searches for rootkit signatures, and identifies suspicious network connections.

Installation and Usage

# Install Chkrootkit
sudo apt install chkrootkit

# Run quiet scan (only show findings)
sudo chkrootkit -q

# Perform expert mode scan
sudo chkrootkit -x

# Check specific rootkit
sudo chkrootkit -s knark

The tool outputs findings directly to the terminal and logs results for further analysis. Regular scans help detect newly installed rootkits.

Lynis

Lynis is a comprehensive security auditing tool that performs system hardening scans, vulnerability assessments, and malware detection. It provides actionable security recommendations and compliance reporting.

Installation and Security Audit

# Download and install Lynis
wget https://cisofy.com/files/lynis-3.0.8.tar.gz
tar -xzf lynis-3.0.8.tar.gz
cd lynis

# Run comprehensive audit
sudo ./lynis audit system

# Quick system scan
sudo ./lynis audit system --quick

# Generate compliance report
sudo ./lynis audit system --compliance

Lynis creates detailed reports in /var/log/lynis.log and provides hardening suggestions with priority levels to improve overall system security.

OSSEC

OSSEC is a powerful host-based intrusion detection system (HIDS) that monitors file integrity, analyzes log files, performs rootkit detection, and provides real-time alerting capabilities.

Installation and Configuration

# Download OSSEC
wget https://github.com/ossec/ossec-hids/archive/3.7.0.tar.gz
tar -xzf 3.7.0.tar.gz
cd ossec-hids-3.7.0

# Install OSSEC
sudo ./install.sh

# Start OSSEC
sudo /var/ossec/bin/ossec-control start

# Check OSSEC status
sudo /var/ossec/bin/ossec-control status

OSSEC continuously monitors system files, processes, and logs. It sends real-time alerts when detecting suspicious activities, unauthorized file modifications, or potential security breaches.

Comparison of Security Tools

Best Practices

• Schedule regular scans Set up cron jobs to run security scans automatically during off-peak hours.

• Keep definitions updated Regularly update virus signatures and rootkit databases for accurate detection.

• Layer your security Use multiple tools together for comprehensive coverage of different threat types.

• Monitor logs Review scan reports and system logs regularly to identify potential security issues.

• Establish baselines Create system baselines to detect unauthorized changes more effectively.

Conclusion

Protecting Linux servers requires a multi-layered approach using specialized security tools. ClamAV provides antivirus protection, while Rkhunter and Chkrootkit excel at rootkit detection. Lynis offers comprehensive security auditing, and OSSEC delivers real-time monitoring capabilities. Combining these tools with regular updates and monitoring creates a robust defense against malware and rootkits.