Event Processing and Parsers ​

This document explains the event processing system and protocol parsers in eCapture. This system bridges the gap between raw eBPF events and formatted output, handling event deserialization, aggregation, protocol parsing, and output formatting.

For information about output formats (text, PCAP, keylog), see Output Formats. For information about how modules generate events, see Capture Modules.

Overview ​

The event processing pipeline consists of three major components:

1. Event Structures - Strongly-typed events implementing the IEventStruct interface that deserialize eBPF data
2. Event Processor - Manages worker lifecycle and routes events based on UUIDs
3. Protocol Parsers - Detect and parse application protocols (HTTP/1.x, HTTP/2) from aggregated payloads

Event Structure System ​

IEventStruct Interface ​

All events in eCapture implement the IEventStruct interface, which defines a standard contract for event handling:

Sources: user/event/ievent.go:41-52

The interface methods serve specific purposes:

Event Type Classification ​

Events are classified by Type enum to control their processing path:

Sources: user/event/ievent.go:26-37

• TypeOutput - Events written directly to output without aggregation (e.g., Bash commands, database queries)
• TypeModuleData - Events stored in module state for correlation (e.g., connection metadata, master keys)
• TypeEventProcessor - Events routed through EventProcessor for payload aggregation and parsing (e.g., SSL data events)

Concrete Event Implementations ​

The following diagram maps event structures to their source modules:

Sources: user/event/event_openssl.go:77-92, user/event/event_openssl.go:289-294, user/event/event_masterkey.go:37-55, user/event/event_masterkey.go:156-174, user/event/event_gnutls.go:25-35, user/event/event_nspr.go:26-36, user/event/event_mysqld.go:68-78, user/event/event_postgres.go:38-44, user/event/event_bash.go:37-47, user/event/event_openssl_tc.go:30-40

SSLDataEvent Structure ​

The SSLDataEvent is the most commonly used event, capturing SSL/TLS plaintext from OpenSSL/BoringSSL:

Sources: user/event/event_openssl.go:77-92, user/event/event_openssl.go:138-141

The UUID format includes the sock: prefix for socket-lifecycle-managed workers (see Worker Lifecycle Management section).

ConnDataEvent Structure ​

ConnDataEvent provides connection metadata from TC eBPF hooks:

Sources: user/event/event_openssl.go:272-308

Event Decoding Process ​

Event decoding follows a standard pattern using binary.Read():

Sources: user/event/event_openssl.go:94-132, user/event/event_bash.go:49-69, user/event/event_mysqld.go:80-109

All events use binary.LittleEndian to match eBPF data layout. Events containing timestamps call DecodeKtime() to convert kernel time to Unix nanoseconds.

EventProcessor Architecture ​

Core Components ​

The EventProcessor manages the event processing pipeline:

Sources: pkg/event_processor/processor.go:30-50

The processor maintains:

• incoming channel - Receives events from modules (buffer size 1024)
• outComing channel - Formatted output to logger (buffer size 1024)
• destroyConn channel - Socket destruction notifications
• workerQueue map - UUID to worker mapping for event routing
• logger io.Writer - Output destination (console, file, or CollectorWriter)

Event Dispatch Flow ​

Sources: pkg/event_processor/processor.go:66-109, pkg/event_processor/processor.go:130-148

Key aspects:

1. UUID-based routing - GetUUID() groups related events into the same worker
2. Lazy worker creation - Workers created on first event for a UUID
3. Reference counting - Get()/Put() prevent race conditions during deletion
4. Non-blocking writes - Events dropped if incoming channel is full

Worker System ​

IWorker Interface ​

Workers aggregate events and invoke parsers:

Sources: pkg/event_processor/iworker.go:35-49

eventWorker Implementation ​

Sources: pkg/event_processor/iworker.go:70-89

Worker Lifecycle Management ​

Workers support two lifecycle modes based on UUID prefix:

Sources: pkg/event_processor/iworker.go:57-63, pkg/event_processor/iworker.go:100-123

LifeCycleStateDefault:

• Used when UUID does not start with sock:
• Worker destroyed after idle timeout (1 second = 10 ticks × 100ms)
• Example: Bash events, MySQL queries with UUID PID_TID_Comm

LifeCycleStateSock:

• Used when UUID starts with sock: prefix
• Worker persists across idle periods
• Destroyed only when socket closes (external CloseEventWorker() call)
• Example: SSL data events with UUID sock:PID_TID_Comm_Fd_DataType_Tuple_Sock
• DestroyUUID contains socket pointer for cleanup matching

Worker Event Loop ​

Sources: pkg/event_processor/iworker.go:262-306

Key behaviors:

• Ticker resets on each event arrival (tickerCount = 0)
• Payload aggregation - Events accumulated in payload buffer
• Idle timeout - 10 ticks (1 second) without events triggers lifecycle action
• Socket lifecycle - closeChan signal from external socket destruction

Event Display and Parsing ​

When a worker is ready to output (timeout or socket close), it invokes the display pipeline:

Sources: pkg/event_processor/iworker.go:175-228, pkg/event_processor/iworker.go:248-260

The display process:

1. Aggregated payload passed to parser
2. Parser detects protocol and formats output
3. Output formatted based on logger type (text vs protobuf)
4. Worker state reset for next batch

Parser System ​

IParser Interface ​

Parsers detect and format application protocols:

Sources: pkg/event_processor/iparser.go:49-60

Parser Types ​

Sources: pkg/event_processor/iparser.go:40-47

Parser Selection ​

The NewParser() function auto-detects protocol:

Sources: pkg/event_processor/iparser.go:85-115

The detection sequence:

1. Try each registered parser's detect() method
2. First successful match creates typed parser instance
3. Fallback to DefaultParser if no match

Parser Registration ​

Parsers self-register during package initialization:

Sources: pkg/event_processor/http_request.go:159-163, pkg/event_processor/http_response.go:177-181, pkg/event_processor/iparser.go:64-73

HTTP Request Parser ​

The HTTPRequest parser handles HTTP/1.x requests:

Sources: pkg/event_processor/http_request.go:28-35, pkg/event_processor/http_request.go:54-81, pkg/event_processor/http_request.go:105-157

Key features:

• Incremental parsing - http.ReadRequest() called once when headers complete
• Body accumulation - Additional writes append to body buffer
• Gzip decompression - Automatic for Content-Encoding: gzip
• HTTP/2 detection - Returns raw bytes if Proto == "HTTP/2.0"

HTTP Response Parser ​

Similar to request parser but uses http.ReadResponse():

Sources: pkg/event_processor/http_response.go:28-37, pkg/event_processor/http_response.go:58-92, pkg/event_processor/http_response.go:115-175

Response parser handles:

• Chunked encoding - Detected via ContentLength < 0
• Truncated responses - Gracefully handles ErrUnexpectedEOF
• Content-Length mismatches - Logs warnings for debugging

DefaultParser ​

Fallback parser for non-HTTP protocols:

Sources: pkg/event_processor/iparser.go:117-166

The DefaultParser:

• Accumulates all data immediately (isdone = true)
• Auto-detects binary vs text (checks first byte)
• Uses hex dump for non-printable data
• Strips null terminators from C strings

Complete Event Flow Example ​

TLS Capture Pipeline ​

Sources: pkg/event_processor/processor.go:66-109, pkg/event_processor/iworker.go:262-306, pkg/event_processor/iworker.go:175-228, pkg/event_processor/iparser.go:85-115

Output Formatting ​

Text Mode ​

For CollectorWriter logger:

Sources: pkg/event_processor/iworker.go:175-228

Protobuf Mode ​

For protobuf logger (eCaptureQ GUI):

Sources: pkg/event_processor/iworker.go:214-227

Event Truncation ​

The EventProcessor supports payload truncation:

Sources: pkg/event_processor/iworker.go:230-245

Truncation prevents excessive memory usage for large payloads (configurable via --truncate flag).

Error Handling ​

Event Worker Errors ​

Workers use error channels for non-fatal errors:

Sources: pkg/event_processor/iworker.go:66-68, pkg/event_processor/processor.go:72-79

Key error behaviors:

• Non-blocking channels - Drops events rather than blocking if buffers full
• Parser errors - Logged but don't stop processing
• Worker panics - Get()/Put() panics indicate incorrect usage

Socket Lifecycle Cleanup ​

Socket-based workers cleaned up via destroyConn channel:

Sources: pkg/event_processor/processor.go:177-185, pkg/event_processor/processor.go:115-128, pkg/event_processor/iworker.go:142-148

This mechanism ensures workers for closed sockets are properly cleaned up, preventing memory leaks.

Summary ​

The event processing system provides:

1. Strong typing via IEventStruct interface for type-safe event handling
2. UUID-based routing to aggregate related events into workers
3. Dual lifecycle modes for efficient resource management
4. Automatic protocol detection and parsing for HTTP/1.x and HTTP/2
5. Multiple output formats supporting text, hex, and protobuf
6. Graceful error handling with non-blocking channels and event dropping

This architecture enables eCapture to handle high-volume event streams from multiple eBPF programs while maintaining structured output suitable for both human and machine consumption.