# aws_enterprise_knowledge_base **Repository Path**: novartis_knowledge_base/aws_enterprise_knowledge_base ## Basic Information - **Project Name**: aws_enterprise_knowledge_base - **Description**: No description available - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: cap-dev - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2026-06-22 - **Last Updated**: 2026-07-24 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # AWS Enterprise Knowledge Base [![Python](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/) [![AWS](https://img.shields.io/badge/AWS-Bedrock%20%7C%20Neptune%20%7C%20OpenSearch-orange.svg)](https://aws.amazon.com/) [![FastAPI](https://img.shields.io/badge/FastAPI-0.110+-green.svg)](https://fastapi.tiangolo.com/) [![Next.js](https://img.shields.io/badge/Next.js-14-black.svg)](https://nextjs.org/) Enterprise AI knowledge management platform - Transform unstructured business knowledge into queryable graph + vector hybrid systems with AI agent orchestration. ## Overview **MCS (Marketing & Commercial Set)** - AWS-powered enterprise knowledge platform combining GraphRAG (Graph + Vector hybrid retrieval) with AI agents for intelligent Q&A and training. **Core Capabilities:** - 📊 **Knowledge Construction**: Document parsing → Entity extraction → Graph + Vector indexing - 🔍 **Hybrid Retrieval**: Semantic search (OpenSearch) + Graph traversal (Neptune) + Keyword matching - 🤖 **AI Agent Orchestration**: Archive, Advisor, Coach agents with tool registration - 🏢 **Multi-tenant**: Independent knowledge bases per product/domain - 🌐 **Full Stack**: Next.js 14 frontend + FastAPI backend + AWS Bedrock LLMs --- ## Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ Application Services │ │ ┌──────────────┬──────────────┬──────────────┬──────────────┐ │ │ │ Ingestion │ Validation │ Tagging │ Analytics │ │ │ │ Service │ Service │ Service │ Service │ │ │ └──────────────┴──────────────┴──────────────┴──────────────┘ │ │ │ │ │ ▼ │ │ ┌────────────────────────────────────────────────────────────┐│ │ │ GraphRAG Core Platform (aws-solutions) ││ │ │ Query Service │ Ingestion │ Management │ LightRAG Core ││ │ └────────────────────────────────────────────────────────────┘│ │ │ │ │ ▼ │ │ ┌──────────────┬──────────────┬──────────────┬──────────────┐ │ │ │ Neptune │ OpenSearch │ DynamoDB │ S3 │ │ │ │ (Knowledge │ (Vector │ (Metadata) │ (Documents) │ │ │ │ Graph) │ Embeddings)│ │ │ │ │ └──────────────┴──────────────┴──────────────┴──────────────┘ │ └─────────────────────────────────────────────────────────────────┘ ``` **Design Philosophy:** - **GraphRAG Core** (`/aws-solutions`) = Knowledge infrastructure (graph construction, hybrid retrieval) - **Application Services** (`/services`) = Business logic consuming GraphRAG APIs - **Clear Boundaries** = Services call GraphRAG; GraphRAG doesn't know about services --- ## Project Structure ``` aws_enterprise_knowledge_base/ ├── aws-solutions/ # GraphRAG core platform │ └── graphrag-on-aws/ # Neptune + OpenSearch + LightRAG engine │ ├── services/ # Application services │ ├── ingestion/ # Document ingestion pipeline │ ├── processing/ # Document processing & embeddings │ ├── validation/ # Docling extraction validation │ ├── tagging/ # Auto-tagging service │ ├── analytics/ # Analytics & Snowflake integration │ ├── storage/ # Storage adapters (S3, Neptune, OpenSearch) │ ├── api-services/ # REST APIs (query, admin, agent) │ └── shared/ # Shared libraries │ └── unified_kb/ # Parser-agnostic document representation │ ├── tests/ # Unified test structure │ ├── unit/ # Fast isolated tests │ ├── integration/ # Cross-service workflows │ └── e2e/ # End-to-end scenarios │ ├── frontend/ # Next.js 14 React UI │ ├── src/ # TypeScript source │ └── components/ # shadcn/ui components │ ├── infrastructure/ # Deployment & operations │ ├── docker/ # Docker Compose, Dockerfiles │ └── scripts/ # Setup & verification scripts │ └── docs/ # Documentation ├── architecture/ # System design & analysis ├── platform-design/ # Integration architectures ├── guides/ # User & operator guides └── development/ # Developer setup & conventions ``` --- ## Quick Start ### Prerequisites - Python 3.11+ - Node.js 18+ - Docker & Docker Compose - AWS Account (for deployment) ### Local Development ```bash # 1. Clone repository git clone cd aws_enterprise_knowledge_base # 2. Setup development environment bash infrastructure/scripts/setup-dev.sh # 3. Start local services (PostgreSQL + MinIO) cd infrastructure/docker docker-compose up -d # 4. Verify setup cd ../.. bash infrastructure/scripts/verify-setup.sh # 5. Start development servers # Backend (FastAPI) - Terminal 1 cd services uvicorn api.main:app --reload --port 8000 # Frontend (Next.js) - Terminal 2 cd frontend npm run dev ``` **Access:** - Frontend: http://localhost:3000 - Backend API: http://localhost:8000 - API Docs: http://localhost:8000/docs --- ## Technology Stack | Layer | Technology | Purpose | |-------|-----------|---------| | **Frontend** | Next.js 14 + TypeScript | App Router, SSR | | **UI** | shadcn/ui + Tailwind CSS | Accessible component library | | **Backend** | FastAPI + Python 3.12 | Async REST API | | **GraphRAG Core** | LightRAG + AWS Services | Hybrid retrieval engine | | **Graph DB** | Amazon Neptune | Property graph (Gremlin/SPARQL) | | **Vector Search** | OpenSearch Serverless | Semantic search | | **LLM** | AWS Bedrock | Claude, Titan, Nova models | | **Agent Framework** | Strands Agents SDK | Production-grade orchestration | | **Authentication** | OIDC / Cognito | Enterprise SSO + JWT | | **Infrastructure** | AWS CDK | Type-safe IaC | --- ## Key Features ### 🎯 Knowledge Management - **Multi-KB Architecture**: Independent knowledge bases per product/domain - **Hybrid Storage**: Graph (relationships) + Vector (semantics) + Keyword (exact match) - **Document Validation**: Docling-based extraction with quality validation - **STAR Case Methodology**: Structured experience capture (Situation-Task-Action-Result) ### 🤖 AI Agent Orchestration - **Archive Agent**: Records capture, STAR extraction, GraphRAG storage - **Advisor Agent**: Pre-action consultation with semantic search + graph queries - **Coach Agent**: Training material generation from cases + evidence - **Tool Registry**: Dynamic tool loading (star_search, graph_query, skill_loader) ### 🔧 Enterprise-Ready - ✅ OIDC/Cognito authentication with JWT - ✅ Multi-tenant user isolation - ✅ Database migrations (Alembic) - ✅ API documentation (FastAPI auto-gen) - ✅ CloudWatch logging - ✅ CI/CD ready (CodeBuild) - ✅ Infrastructure as Code (AWS CDK) - ✅ Secrets management (AWS Secrets Manager) --- ## Development ### Running Tests ```bash # All tests pytest # Unit tests only pytest tests/unit/ -m unit # Integration tests pytest tests/integration/ -m integration # Specific test file pytest tests/integration/test_pipeline_logic.py -v ``` ### Database Migrations ```bash # Generate new migration alembic revision --autogenerate -m "description" # Apply migrations alembic upgrade head # Rollback alembic downgrade -1 ``` ### Code Quality ```bash # Type checking mypy services/ # Linting ruff check services/ # Format ruff format services/ ``` --- ## Deployment ### One-Click AWS Deployment ```bash # From root directory ./install.sh ``` **Configuration:** - Authentication: Cognito or OIDC - Custom domain: Optional - Serverless options: Neptune/OpenSearch/RDS serverless tiers **Deployment Architecture:** ``` CloudFront → ALB → ECS Fargate (Next.js + FastAPI) ↓ RDS PostgreSQL + Neptune + OpenSearch + S3 ``` For detailed deployment instructions, see [docs/guides/deployment.md](docs/guides/deployment.md) --- ## Documentation | Document | Description | |----------|-------------| | [Architecture Overview](docs/architecture/current_project_analysis.md) | Enterprise platform analysis | | [Setup Guide](docs/guides/setup.md) | Development environment setup | | [Testing Guide](docs/guides/testing.md) | Testing strategy & conventions | | [Pipeline Artifacts](docs/guides/pipeline-artifacts.md) | Pipeline testing documentation | | [UV Setup](docs/development/uv-setup.md) | UV package manager guide | | [API Documentation](http://localhost:8000/docs) | FastAPI auto-generated docs (local) | --- ## Key Modules ### Reusable Components 1. **Unified KB** (`services/shared/unified_kb/`) - Parser-agnostic document representation - Adapters: Docling, VLM, Application schemas 2. **GraphRAG Platform** (`aws-solutions/`) - L3 CDK construct for Neptune + OpenSearch + DynamoDB - Hybrid retrieval engine (graph + vector + keyword) 3. **Embedding Service** (`services/processing/embeddings/`) - Unified interface for Titan/Nova/custom embeddings - Model abstraction layer 4. **Agent Tools** (`services/api-services/agent-api/tools/`) - star_search: Semantic case retrieval - graph_query: Graph traversal - skill_loader: Dynamic prompt templates --- ## Contributing ### Code Conventions - **Python**: snake_case files/modules, PascalCase classes - **TypeScript**: PascalCase components, camelCase functions - **API Routes**: kebab-case URLs (`/knowledge-bases`) - **Database**: snake_case tables/columns ### Import Pattern ```python # After reorganization from services.shared.unified_kb import UnifiedDocument from services.tagging.autotagging_pipeline import runner from services.validation.docling_validation.src.at_poc import extractor ``` --- ## Common Commands ```bash # Check database status docker ps --filter "name=postgres" # Start development servers cd services && uvicorn api.main:app --reload # Backend cd frontend && npm run dev # Frontend # Run migrations alembic upgrade head # Type check mypy services/ ``` --- ## Support For issues and questions: - **GitHub Issues**: [Link to issues page] - **Documentation**: See `/docs` directory - **Architecture Decisions**: See [CLAUDE.md](CLAUDE.md) --- ## License [License Type] - See LICENSE file for details --- **Last Updated**: 2026-06-10 **Platform Version**: See [buildspec.yml](buildspec.yml) for current deployment version