shazzam_design_overview.md

Shazzam: Port Scanning and OS Detection System

Shazzam is a sophisticated port scanning and host discovery system implemented in Java as part of the ServiceNow MID (Management, Instrumentation, and Discovery) server. It is designed to probe IP addresses for open ports and gather information about the target hosts, including attempting to determine their operating systems.

Core Architecture

Shazzam is built around several key components that work together to provide a scalable and extensible port scanning solution:

1. Base Classes and Inheritance Hierarchy

• ShazzamBase: Abstract base class that implements core functionality shared by various Shazzam probes
• Shazzam: Extends ShazzamBase to provide a full-featured port scanning implementation
• LightShazzam: A lighter weight version optimized for quicker scans
• DNSLegacy/DNSNameResolverLegacy: Specialized Shazzam implementations for DNS resolution

2. Port Scanning Engine

The heart of Shazzam is the PortScannerEngine class, which implements a highly parallel, non-blocking I/O-based scanner that can handle multiple scans simultaneously. Key features include:

• Scalable architecture that processes multiple scanners concurrently
• Support for both blocking and non-blocking operations
• Regulated packet transmission to avoid network congestion
• Thread management to control resource utilization

The engine works in four main steps:

1. Create an instance of the engine
2. Add all individual port scanners for a given run
3. Run the port scanner engine
4. Retrieve and process results

For high-throughput scenarios, Shazzam also offers ConcurrentPortScannerEngine that runs in multiple threads.

3. Scanner Types

Shazzam includes numerous port scanner implementations that inherit from abstract classes like APortScanner and AUDPPortScanner. These scanners include:

• GenericTCP: Basic TCP connection testing
• HTTP/HTTPS: Captures HTTP headers and server information
• SNMP: Queries devices using SNMP protocol and retrieves SysDescr fields
• NBT: NetBIOS scanner that retrieves host name and Windows domain name
• DNS: Performs reverse lookups of IP addresses to get hostnames
• ResolveDNS: Forward lookups of hostnames
• SLP: Service Location Protocol scanner
• CertificateCapture: Captures X.509 certificates from TLS ports
• BannerTCP: Likely captures service banners that help identify services

4. Scanner Queue Management

Shazzam uses two types of scanner queues:

• UnconditionalScannerQueue: Scans performed on all target IPs
• ConditionalScannerQueue: Scans only performed on IPs that are detected as "alive" first

This approach optimizes scanning by not wasting resources on hosts that aren't responding to basic probes.

5. Regulation and Throttling

To prevent overwhelming networks or triggering security systems, Shazzam includes sophisticated throttling through:

• BasicRegulator: Controls packet rates
• Configurable interval and packet-per-interval settings
• Chunking of IP ranges to limit simultaneous scans

OS Detection Capabilities

While not explicitly labeled as "OS detection" in the codebase, Shazzam performs operating system detection through several mechanisms:

1. SNMP queries: The SNMP scanner retrieves the "sysDescr" field which typically contains OS information.

2. Service fingerprinting: By identifying which ports are open and how services respond, Shazzam can infer OS information.

3. Banner grabbing: Multiple scanners capture service banners which often include OS details.

4. Protocol behavior: The way hosts respond to various protocol probes can indicate OS type.

5. TCP/IP stack fingerprinting: Though not explicitly shown in the code samples, the system likely analyzes TCP/IP response characteristics.

6. NBT queries: The NBT scanner can extract Windows domain information, indicating Windows systems.

7. DNS information: Hostname conventions from DNS often reveal OS types.

Shazzam collects all this information and aggregates it into structured results that can help determine the target's operating system.

Configuration and Customization

Shazzam is highly configurable with numerous parameters:

• Chunk size (number of IPs scanned in parallel)
• Regulator settings (packet rate control)
• Thread count and scanners per thread
• Protocol-specific timeouts and retry counts
• Reporting options (whether to report inactive/dead hosts)

Workflow

1. Initialization: Shazzam loads configuration and regulator settings
2. IP Collection: It processes IP ranges into manageable chunks
3. Port Scanner Setup: Initializes appropriate port scanners for each service to be probed
4. Execution: The PortScannerEngine executes scans in phases
5. Result Collection: Results are gathered, organized by IP address, and processed
6. Result Building: XML output is created with detailed information about each host

Integration with ServiceNow

Shazzam serves as a core discovery component for the ServiceNow CMDB (Configuration Management Database). It provides IP address, port, and service information to populate the CMDB with detailed information about networked devices. The OS detection capabilities specifically support automatic classification of discovered assets.

Security Features

Shazzam includes several security-minded features:

• Rate limiting to prevent network flooding
• Support for SNMP credentials and authentication
• TLS/SSL communication capabilities
• Configurable scan intensity

In summary, Shazzam is a comprehensive port scanning and host discovery system that uses multiple techniques to gather information about network hosts, including their operating systems. It uses a parallel, non-blocking architecture to efficiently scan large IP ranges while maintaining control over network resource utilization.