Commit Graph

20 Commits

Author SHA1 Message Date
6d5d0e3148 fix: all adaptor fetch() is async based 2025-05-14 18:18:44 +07:00
f99e49974a feat: add pipeline result data endpoint 2025-05-12 23:21:25 +07:00
e1cd04a07e feat: sse log on specific pipeline 2025-05-12 23:01:23 +07:00
b7aa490316 fix: fix import ingestors error, add color to log 2025-05-12 22:42:49 +07:00
701c2b0ae7 feat(config, sse): add default settings and implement SSE log streaming 2025-05-12 21:58:52 +07:00
fdef21443d feat: add new endpoint 2025-05-12 21:36:41 +07:00
31e35ff1b0 remove normalization 2025-05-12 21:36:06 +07:00
5cd5fcfeb3 fix: add pipeline test, add FAILED status and fix pydantic scheme error 2025-05-12 21:12:50 +07:00
f24886afdc feat: add scheduler to help manage pipeline 2025-05-12 19:16:47 +07:00
946f0b2a24 feat: add pipeline service with in-memory data stores 2025-05-12 18:34:21 +07:00
ff9de9f80a refactor: correct ingestor with new adapter - define export module list 2025-05-12 17:05:13 +07:00
eae46ac820 refactor: rename adaptor model to ingestion 2025-05-12 16:31:18 +07:00
9e8eabec04 refactor: fix typehint and add test 2025-05-12 16:12:28 +07:00
186c85bfde refactor: fix typehint 2025-05-12 15:25:46 +07:00
ec62a75b15 refactor: use generic typehint style 2025-05-12 15:12:42 +07:00
6a5fdf2c09 refactor: file adaptor now recive only UploadFile from user 2025-05-12 15:11:15 +07:00
6b445be016 move pydantic model file structure 2025-05-12 15:10:48 +07:00
93ccc23e2a ruff format and pyright lint 2025-05-12 14:26:27 +07:00
67f691a46a refactor: remove logging stream - rewrite later 2025-05-12 02:58:38 +07:00
c91526623c rename crawler ai folder 2025-05-11 17:35:04 +07:00