IoT_PreTester_V46_Demo/ARCHITECTURE.md

IoT PreTester 4.6 Demo — Architecture

Purpose and boundary

IoT PreTester 4.6 Demo analyzes existing .pcap and .pcapng files and turns packet-level observations into prioritized security findings. It is intentionally limited to network-capture analysis. Live capture, streaming, and firmware binary analysis are outside the demo scope.

Runtime components

Component	Responsibility
Next.js frontend	Authentication flow, device administration, capture upload, findings display, report download, and optional deep-analysis controls
Django REST backend	API, validation, persistence, user/device administration, and integration with the analysis core
analyzer package	Django-independent capture extraction, detection, scoring, provenance, and report rendering
SQLite	Demo persistence for devices, files, findings, extracted artifacts, and analysis status
tshark	Protocol dissection; driven directly through two streaming tshark -T ek passes
nmap/hashcat	Optional, explicitly initiated endpoint probing and offline hash recovery

Analysis flow

1. The frontend sends the capture and metadata to the authenticated Django API.
2. Django validates the upload, stores it in a dedicated volume, and creates the database records.
3. extract_tshark.py reads the capture into a normalized CaptureContext: one metadata pass over every packet, then one payload pass restricted to unencrypted flows. The device IP entered at upload time is stored as device_ip.
4. Ten independent detectors evaluate that shared context.
5. The engine deduplicates findings, attributes them to the device under test (affected_device, device_related), prioritizes them, attaches capture, packet, stream, and artifact provenance, and builds the observation inventory.
6. Django persists the result and extracted URL/hash artifacts.
7. The frontend displays findings and can request Markdown, text, or JSON reports.
8. Optional URL/hash enrichment runs only for artifacts selected by the user.

Analysis package

django_project/analyzer/
├── context.py              Normalized capture data model
├── extract_tshark.py       PCAP/PCAPNG extraction (tshark -T ek)
├── tls_tables.py           Cipher-suite and version names generated from tshark
├── allowlist.py            Expected provider domains (lowers severity, never hides)
├── inventory.py            Complete observation inventory for the report
├── engine.py               Detector orchestration and result assembly
├── findings.py             Shared finding taxonomy
├── provenance.py           Packet- and stream-level evidence references
├── scoring.py              Deduplication and prioritization
├── artifacts.py            URL and hash extraction
├── detectors/              Independent detection rules
├── enrich/                 Optional active enrichment
├── reporting/              Markdown, text, and JSON output
└── tests/                  Synthetic unit and regression tests

The package does not import Django. tshark and external tools are invoked only at the integration points that need them, so detector tests run on synthetic contexts without tshark; tests/test_regressions.py additionally exercises the real extractor against demo_files/ when tshark is available.

Certificates from TLS 1.2 handshakes are parsed from DER with cryptography (subject, issuer, validity, signature algorithm, key size, SAN). TLS 1.3 encrypts the certificate; the inventory states this explicitly.

Finding model

Every detector produces the same core fields: ID, title, category, severity, confidence, affected device, source and destination, protocol, port, time range, evidence, risk description, recommendation, packet references, and stream references.

Severity measures the potential impact if a finding is true. Confidence measures how strongly the capture supports it. Keeping these dimensions separate prevents low-confidence indicators from being presented as confirmed critical events.

When a device address is configured, every finding carries evidence.device_related and, if it touches the device, affected_device. Findings involving the device sort before equally severe findings about other hosts; the summary counts both groups.

Observation inventory

Besides findings, every result contains an inventory: capture facts, the device under test with all its peers, internal hosts, external endpoints (with resolved names), observed services, TLS posture per handshake, DNS names, HTTP hosts and file transfers, clear-text flows and service events, and a coverage section (encrypted vs. undecoded flows, truncation, TLS 1.3 certificate visibility). It is rendered in all report formats and in the findings page so that an analyst can verify that nothing was overlooked.

Detector set

1. Plaintext credentials
2. Password hashes and challenge-response material
3. Session tokens and secrets
4. Malware or command-and-control beaconing
5. Possible data exfiltration
6. Man-in-the-middle and local network manipulation
7. Insecure or unexpected protocols
8. Weak or obsolete TLS
9. DNS anomalies and tunneling indicators
10. Brute force, scanning, and suspicious file transfer

Design decisions

• Parse each capture with tshark's native field export instead of per-packet PDML; a 270 MB capture takes about 15 s instead of 10 minutes, and captures are read completely by default.
• Retain payload text only from unencrypted flows so secret scanning is never exhausted by ciphertext.
• Judge TLS by the negotiated ServerHello parameters; what a client merely offers is reported at lower severity.
• Base clear-text protocol findings on what tshark dissected, not on port numbers alone.
• Use the provider allowlist to lower severity of expected background traffic, never to suppress it.
• Parse each capture once and share normalized records across all detectors.
• Keep detector logic isolated from persistence, presentation, and external tools.
• Preserve evidence provenance so every finding can be traced back to the capture.
• Keep active probing opt-in because authorization can differ from authorization to inspect a capture.
• Use a small demo wordlist by default to keep distribution compact and behavior predictable.
• Store complete analysis results as JSON while keeping extracted artifacts and job status queryable.

Logging

Django logs to stdout (container log) and to a rotating backend.log; gunicorn's error output is routed to stdout as well. A request middleware logs every API call with duration, status, user and an X-Request-ID. Each capture analysis writes its own <capture>.analysis.log next to the upload (thread-filtered, so concurrent analyses do not mix), records the failure reason in File.error_message, and exposes both through the API and the findings page. The frontend keeps a ring buffer of client-side log entries with API timings and uncaught errors.

Known demo limitations

• SQLite and in-process background threads are suitable for a controlled demo, not horizontal production scaling.
• Capture quality and protocol encryption directly limit what can be detected; TLS 1.3 hides server certificates, and encrypted payloads are never inspected.
• Provider allowlisting is suffix-based; a device vendor's own cloud domains should be added via ANALYZER_BENIGN_DOMAINS to avoid low-severity noise.
• URL probes observe the endpoint at probe time, which can differ from capture time.
• The demo does not provide live packet capture, multi-tenant isolation, or production identity-provider integration.