feat: initial microservices project structure
Services: - api: FastAPI gateway with JWT auth, async endpoints, WebSocket - worker-gpu: CUDA sentence-transformers, FAISS IVFFlat, Ollama LLM - worker-indexer: Winnowing+MinHash plagiarism detection, PDF/DOCX extraction - worker-notifier: SMTP email notifications - worker-gost: GOST 7.1-2003 and GOST R 7.0.5-2008 formatting Infrastructure: - docker-compose.yml (production) + docker-compose.dev.yml (hot reload) - Nginx reverse proxy + WebSocket support - PostgreSQL 16 with Alembic migrations - Elasticsearch 8 with Russian/English analyzers - MinIO, RabbitMQ, Redis, Ollama Frontend: - React 18 + Vite + TypeScript + TailwindCSS + Zustand + React Query v5 - 9 pages: Home, Search, Cabinet, Task, Check, Bibliography, Pricing, Login, Register Scripts: - Parser stubs: OpenAlex, КиберЛенинка, arXiv (Phase 0 - to be filled) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
38
services/worker-gpu/app/celery_app.py
Normal file
38
services/worker-gpu/app/celery_app.py
Normal file
@@ -0,0 +1,38 @@
|
||||
"""Celery приложение GPU воркера."""
|
||||
|
||||
from celery import Celery
|
||||
from celery.utils.log import get_task_logger
|
||||
|
||||
from app.config import settings
|
||||
|
||||
celery_app = Celery(
|
||||
"worker_gpu",
|
||||
broker=settings.RABBITMQ_URL,
|
||||
backend=settings.REDIS_URL,
|
||||
include=["app.tasks.search", "app.tasks.plagiarism"],
|
||||
)
|
||||
|
||||
celery_app.conf.update(
|
||||
task_serializer="json",
|
||||
result_serializer="json",
|
||||
accept_content=["json"],
|
||||
timezone="Europe/Moscow",
|
||||
enable_utc=True,
|
||||
task_track_started=True,
|
||||
# Маршрутизация
|
||||
task_routes={
|
||||
"gpu.*": {"queue": "queue.gpu"},
|
||||
"index.*": {"queue": "queue.index"},
|
||||
"notify.*": {"queue": "queue.notify"},
|
||||
"gost.*": {"queue": "queue.gost"},
|
||||
},
|
||||
# Надёжность
|
||||
task_acks_late=True,
|
||||
task_reject_on_worker_lost=True,
|
||||
# GPU воркер — только 1 процесс
|
||||
worker_concurrency=1,
|
||||
# Результаты хранить 24 часа
|
||||
result_expires=86400,
|
||||
)
|
||||
|
||||
logger = get_task_logger(__name__)
|
||||
Reference in New Issue
Block a user