# Logtool Plan

Python tool for viewing SC-F001 flash logs. Supports CLI table output, matplotlib GUI, and both file and HTTP sources.

---

## On-disk Entry Format

Each entry in flash:
```
[len u8] [payload (len-1 bytes)] [type u8]
```
Total bytes per entry = `len + 1`.

`log_write_blocking` does `len++` before writing, so `len` stored on disk = `payload_size + 1`, and `len - 1 = payload_size`. The write is correct and writes all payload bytes.

Padding bytes (0x00 = sector padding, 0xFF = erased flash) cause the reader to skip to the next sector boundary.

### Entry Types

| Type  | Name     | Payload (bytes) | Stored len | Total on disk |
|-------|----------|-----------------|------------|---------------|
| 0–12  | FSM log  | 39              | 40         | 41            |
| 100   | BAT log  | 12              | 13         | 14            |
| 101   | CRASH    | 9               | 10         | 11            |

All values **little-endian** (named `be_*` in source but no byte-swap — the names are misleading legacy).

### FSM payload layout (39 bytes)
```
[0:8]   ts_ms      u64  — ms since epoch (local-as-UTC, no timezone)
[8:12]  bat_V      f32  — battery voltage
[12:16] drive_A    f32  — drive bridge raw current
[16:20] jack_A     f32  — jack bridge raw current
[20:24] aux_A      f32  — aux bridge raw current
[24:26] counter    i16  — drive encoder counter (SENSOR_DRIVE)
[26]    sensors    u8   — bits[3:0]=stable state, bits[7:4]=raw state
                          bit position: SAFETY=0, JACK=1, DRIVE=2, AUX2=3
[27:31] drive_heat f32  — e-fuse thermal accumulator, drive bridge
[31:35] jack_heat  f32  — e-fuse thermal accumulator, jack bridge
[35:39] aux_heat   f32  — e-fuse thermal accumulator, aux bridge
```

### BAT payload layout (12 bytes)
```
[0:8]  ts_ms  u64
[8:12] bat_V  f32
```

### CRASH payload layout (9 bytes)
```
[0:8] ts_ms        u64
[8]   reset_reason u8  — esp_reset_reason_t value
```

### FSM state names (from control_fsm.h — single source of truth)
```
0  = STATE_IDLE
1  = STATE_MOVE_START_DELAY
2  = STATE_JACK_UP_START
3  = STATE_JACK_UP
4  = STATE_DRIVE_START_DELAY
5  = STATE_DRIVE
6  = STATE_DRIVE_END_DELAY
7  = STATE_JACK_DOWN
8  = STATE_UNDO_JACK_START
9  = STATE_CALIBRATE_JACK_DELAY
10 = STATE_CALIBRATE_JACK_MOVE
11 = STATE_CALIBRATE_DRIVE_DELAY
12 = STATE_CALIBRATE_DRIVE_MOVE
```

---

## HTTP /log Response Format

```
[4B json_len BE] [json_len bytes JSON] [4B tail_offset BE] [4B head_offset BE] [raw binary]
```
The raw binary is already linearized (tail→head wraparound handled server-side). Parse it sequentially as a stream of entries.

**Streaming**: GET once to get the full log + current head, then POST with the current head as the tail to get only new entries. Repeat on a timer.

---

## File Format Auto-detection

When opening a `.bin` file:
1. Read first 4 bytes as BE uint32 → `candidate_json_len`
2. If `candidate_json_len < 8192` AND byte at offset 4 is `{` → HTTP response format (parse header then binary)
3. Otherwise → raw flash binary (skip header, parse binary directly from offset 0)

---

## Timestamps

All timestamps are `ms since epoch` in **local-as-UTC** encoding: the device stores the user's local time directly as a Unix timestamp (no timezone offset). Display as-is using `datetime.utcfromtimestamp(ts_ms / 1000)` — no conversion needed.

---

## Implementation Plan

### File layout
```
logtool/
├── PLAN.md         ← this file
├── logtool.py      ← main entry point (CLI + dispatch)
├── parser.py       ← binary framing + struct unpack; reads state names from control_fsm.h
├── source.py       ← file reader and HTTP fetcher (GET full, POST incremental)
├── cli_view.py     ← tabular terminal output (tabulate)
└── gui_view.py     ← matplotlib time-series charts
```

### `parser.py`
- `load_fsm_states(fsm_h_path) -> dict[int, str]` — parses `control_fsm.h` to extract the `fsm_state_t` enum. Falls back to hardcoded dict if file not found.
- `parse_entries(data: bytes) -> list[dict]` — iterates raw binary, skips padding (0x00/0xFF), unpacks each entry by type
- `parse_response(blob: bytes) -> (dict, int, int, list[dict])` — parses full HTTP/file blob: json metadata, tail, head, entries

### `source.py`
- `read_file(path: str) -> bytes`
- `fetch_full(url: str) -> bytes` — HTTP GET /log
- `fetch_incremental(url: str, tail: int) -> (bytes, int)` — POST /log with tail as body, returns raw binary + new head
- `stream(url: str, callback, interval_s=2.0)` — poll loop: GET once for full log, then POST repeatedly; calls `callback(new_entries, new_head)` each iteration

### `cli_view.py`
- `print_table(entries: list[dict], fsm_states: dict)` — tabulate output
  - Columns: `time`, `type`, `state`, `bat_V`, `drive_A`, `jack_A`, `aux_A`, `counter`, `sensors_stable`, `sensors_raw`
  - BAT/CRASH rows show only relevant columns; FSM-only columns shown as `—`
  - CRASH rows prefixed with `***`
- `print_summary(entries)` — time range, count by type, min/max battery

### `gui_view.py`
- `show_plots(entries: list[dict])` — matplotlib figure with subplots:
  1. Battery voltage over time
  2. Currents (drive, jack, aux) over time
  3. FSM state over time (step/stairs plot)
  4. Thermal accumulators (drive_heat, jack_heat, aux_heat)
  - CRASH events: vertical red dashed lines across all subplots
- `live_plot(url: str, interval_s=2.0)` — `FuncAnimation` + streaming source, updates all axes

### `logtool.py` (CLI)
```
Usage:
  logtool.py <file.bin>                     # file → CLI table
  logtool.py <file.bin> --gui               # file → GUI
  logtool.py http://<ip>/log                # fetch once → CLI table
  logtool.py http://<ip>/log --gui          # fetch once → GUI
  logtool.py http://<ip>/log --stream       # live CLI (append rows)
  logtool.py http://<ip>/log --stream --gui # live GUI
  logtool.py <source> --type fsm|bat|crash  # filter by entry type
  logtool.py <source> --tail <offset>       # start from flash offset
  logtool.py <source> --fw ../main          # path to firmware source (for state names)
```

### Dependencies (`requirements.txt`)
```
requests
matplotlib
tabulate
```
Python 3.8+.

---

## Resolved Questions

1. **Entry size**: No bug. `len++` in `log_write_blocking` makes `len-1 = LOGSIZE = 39`, writing all payload bytes correctly.
2. **Streaming**: GET for initial full fetch, then POST with current head as tail for incremental polling.
3. **File format**: Auto-detect via 4-byte json_len + `{` check.
4. **Timestamps**: local-as-UTC, display with `datetime.utcfromtimestamp()`, no conversion.
5. **FSM state names**: Parse from `control_fsm.h` at startup (`--fw` flag or default `../main`). Fallback to hardcoded dict.
6. **tabulate**: Acceptable dependency.