# TODO - Developer Task Tracking

This file is used temporarily by the assistant to track progress on implementation tasks. It will be removed when all work is complete.

Milestone 0 – Bootstrap
- [x] M0-01: Create project skeleton directories and files
- [x] M0-02: Initialize Go module and pin go version
- [x] M0-03: Add .gitignore, .env.example, README
- [x] M0-04: Create Makefile (pending)

Milestone 1 – Config and Logging
- [x] M1-01: Implement logging setup using log/slog
- [x] M1-02: Implement configuration loader: env + flags + .env
- [x] M1-03: Define config schema and defaults
- [x] M1-04: Add config validation
- [ ] M1-05: Unit tests for config parsing precedence

Milestone 2 – Types and Parser
- [x] M2-01: Define normalized data types
- [x] M2-02: Define minimal Reddit API response structs
- [x] M2-03: Implement parser
- [x] M2-04: Implement deleted/removed filters
- [x] M2-05: Parser unit tests

Milestone 3 – Fetcher and Networking
- [x] M3-01: Build HTTP client
- [x] M3-02: Implement rate limiter
- [x] M3-03: Implement backoff with jitter
- [x] M3-04: URL builder for search.json
- [x] M3-05: Implement fetchPage
- [ ] M3-06: Fetcher tests
- [ ] M3-07: Implement metrics capture

Milestone 4 – Storage and Dedup
- [x] M4-01: Implement JSONL writer
- [x] M4-02: File naming and rotation
- [x] M4-03: Ensure output dir creation
- [x] M4-04: Implement dedup index
- [x] M4-05: Dedup persistence
- [ ] M4-06: Storage unit tests

Milestone 5 – Controller and Orchestration
- [x] M5-01: Implement controller orchestrator
- [x] M5-02: Pagination loop
- [x] M5-03: Integrate fetcher→parser→storage
- [x] M5-04: Progress reporting
- [x] M5-05: Graceful shutdown
- [x] M5-06: Summary report
- [x] M5-07: Wire CLI entrypoint
- [ ] M5-08: Error code taxonomy
- [ ] M5-09: Controller integration test

Milestone 6 – Nice-to-haves
- [ ] M6-01: Date-based subdir option
- [ ] M6-02: Optional compression on rollover

Milestone 7 – Performance
- [ ] M7-01: Performance runbook
- [ ] M7-02: Benchmark tuning

Milestone 8 – Docs and Release
- [ ] M8-01: README expansion
- [ ] M8-02: Cron examples
- [ ] M8-03: Sample data
- [ ] M8-04: CI steps
- [ ] M8-05: Tag and build release

Milestone 9 – Verification
- [ ] M9-01: Post-implementation checklist


Progress notes:
- Implemented project bootstrap, config loader, and logging.
- Implemented types and parser with unit tests for parser.
- Implemented fetcher client and page fetcher (rate limiting & Retry-After handling).
- Implemented storage (JSONL writer with rotation) and dedup index (persistent append-only file).
- Implemented controller orchestrator that coordinates fetching, parsing, deduping, and writing; supports graceful shutdown and basic retry/backoff.
- Remaining: add unit tests for fetcher & storage, wire CLI to controller, and add integration tests and metrics.