saiop-skills/skills/saiop-infrastructure/saiop-rag-ingestion/SKILL.md

name, description, version, author, license, platforms, prerequisites, metadata

name	description	version	author	license	platforms	prerequisites	metadata
saiop-rag-ingestion	Run the SAIOP document ingestion pipeline (scripts/ingestion/ingest.py) to load docs into Qdrant — network gotchas and how to verify it actually worked.	1.0.0	Claude Code (SAIOP ops session)	MIT	linux	env_vars commands QDRANT_API_KEY docker	hermes tags SAIOP RAG Qdrant ingestion Ollama

RAG ingestion into Qdrant (SAIOP)

When to use

Loading new SOPs, runbooks, or infrastructure docs into Qdrant so they're retrievable via RAG (Deployment Guide §6.2).

Key facts

• The reference bundle's scripts/ingestion/ingest.py has a real bug: it calls sys.exit(1) but never import sys. Add import sys before running it, or it crashes if invoked with neither --file nor --directory.
• It embeds via Ollama directly (OLLAMA_URL/api/embeddings, nomic-embed-text model) — NOT through the AI Router. Point OLLAMA_URL at http://host.docker.internal:11434 if running the script in a container (Ollama is a host systemd service, not a container).
• UFW only allows port 11434 from specific docker subnets (172.17.0.0/16, 172.19.0.0/16 — the ai-internal network). A container on a different network (e.g. knowledge, where Qdrant lives) will silently time out reaching Ollama via host.docker.internal, even though the hostname resolves fine. Fix: attach the ingestion container to both ai-internal (for Ollama) and knowledge (for Qdrant) — docker run -d --network ai-internal ... then docker network connect knowledge <container>.
• The five collections are atu_infrastructure, atu_operations, atu_incidents, atu_governance, atu_ai — pick by content type (the script's doc_type_map already does chunk-size tuning per collection).
• Don't trust "uploaded N chunks" alone as proof it worked. Verify with a real semantic search: embed a realistic question via the same Ollama endpoint, POST it to /collections/<name>/points/search, and check the returned chunk_text is actually relevant (score > ~0.7 and on-topic) — not just that points exist.