# hashline64

A lightweight, secure, and Unix-compliant filter for line-based generation of hash values from data streams.

## The FNV-1a Hash Algorithm

The tool utilizes the **FNV-1a algorithm**, a non-cryptographic hash function characterized by its simple implementation, high processing speed, and efficient bit distribution.

*   **Mathematical Principle:** The algorithm uses two constants tailored to the 64-bit width: the *offset basis* as the initial value (preventing identical hashes for empty or zero-filled strings) and the *FNV prime*. Multiplying by the prime causes a complete mixing of the bits with each step (avalanche effect).
*   **Difference from FNV-1:** Unlike FNV-1, FNV-1a performs the XOR operation before the multiplication. This improves the distribution and mixing of the final byte of a string.
*   **Comparison with djb2:** FNV-1a uses a 64-bit prime instead of djb2's factor of 33. This provides a better bit distribution and a lower collision rate for similar strings while maintaining nearly identical processing speeds.
*   **Use Cases:** Optimized for hash tables, in-memory checksums, and data deduplication.
*   **Limitation:** Because the function is not cryptographically secure, it is vulnerable to targeted collision attacks (hash flooding / DoS attacks).

## Buffer Behavior (Fail-Fast)

The program processes lines up to a hard limit of **64 KB (BUFSIZE / 65535 bytes)**. If a line exceeds this threshold, the tool follows a *fail-fast* approach:

*   It terminates the data stream gracefully with **exit code 0**.
*   No memory allocation occurs beyond the limit, no silent data manipulation takes place, and the program does not crash.
*   A separate error code is not returned because this is a defined, documented program behavior. The truncation can be detected directly within the data stream, as the final processed line will match the exact maximum buffer size (critical for traceability in log files).

## Architecture and Unix Philosophy

This restrictive behavior adheres strictly to the Unix philosophy (*"Do one thing and do it well"*). By avoiding features like dynamic memory management (`realloc`), the tool offers distinct advantages:

*   **Performance:** No runtime overhead from dynamic memory allocation.
*   **Security:** The static buffer structurally eliminates the risk of heap exploits.
*   **Compactness:** The small code size makes it ideal for static linking with `musl-gcc`.
*   **Platform Hardening:** Software-side hardening using `-fstack-protector-strong` and `-static-pie` mitigates remaining risks.

## Pipeline Integration

Adapting to data streams outside the norm must be handled by upstream tools within the pipeline.

### Preprocessing Options:

1. **Truncating Overlong Lines:** 
   An upstream `cut` command prevents the program from terminating and protects system resources:
   ```bash
   ... | cut -c-65532 | hashline64
   ```

2. **Enforcing Line Breaks:** 
   An upstream `fold` command actively breaks overlong data streams after a specified number of characters. This prevents program termination but alters the structure of the input data for hash calculation:
   ```bash
   ... | fold -w 65532 | hashline64
   ```

3. **Fragmentation with a Separator:** 
   An upstream `awk` command cleanly breaks overlong lines while injecting a separator marker (`\036`), allowing processing to continue. The fragments must be manually reassembled later:
   ```bash
   ... | awk '{while(length($0)>65535){printf "%s\036\n",substr($0,1,65532);$0=substr($0,65533)}print}' | hashline64
   ```

## Modifying the Buffer Size

The 64 KB limit is a deliberate choice to maintain a minimal memory footprint. Because the source code is released into the **Public Domain**, users can manually modify the `BUFSIZE` constant in the source code and compile a custom binary if needed.