{
  "markdown": "# BestBrain\nunder development nothing runs currently . leave a star and keep updated\nBestBrain is a case-driven orchestration engine for simulation workflows across WebSocket + HTTP, manager buses, and QBRAIN-backed persistence.\n\n## Expansive Workflow (Visible)\n\n```mermaid\nflowchart TD\n    subgraph entryLayer [EntryLayer]\n        asgiInit[ASGI Init]\n        httpEntry[HTTP Entry]\n        wsEntry[WebSocket Entry]\n        asgiInit --> httpEntry\n        asgiInit --> wsEntry\n    end\n\n    subgraph httpLayer [HttpActions]\n        adminRoute[/admin/]\n        healthRoute[/health/]\n        worldRoute[/world/demo and /world/webhook/]\n        authRoute[/auth/access/]\n        graphRoute[/graph/view and /graph/brain/test/]\n        bqRoute[/bq/upsert and /bq/get non-mounted/]\n        httpEntry --> adminRoute\n        httpEntry --> healthRoute\n        httpEntry --> worldRoute\n        httpEntry --> authRoute\n        httpEntry --> graphRoute\n        httpEntry --> bqRoute\n    end\n\n    subgraph wsLifecycle [WebSocketLifecycle]\n        relayConnect[Relay Connect]\n        resolveUser[Resolve Or Create User]\n        initOrchestrator[Init Orchestrator]\n        initManagers[Init Managers]\n        resolveSession[Resolve Active Session]\n        sendBootstrap[Send SET_SID and LIST_USERS_SESSIONS]\n        receivePayload[Receive Payload]\n        wsEntry --> relayConnect --> resolveUser --> initOrchestrator --> initManagers --> resolveSession --> sendBootstrap --> receivePayload\n    end\n\n    subgraph orchestrationLayer [OrchestratorDecisionGraph]\n        normalizePayload[Normalize Payload]\n        detectFiles[Detect Files]\n        typedGate{Type Provided}\n        startSimBranch[START_SIM Branch]\n        chatBranch[CHAT Branch]\n        typedCaseBranch[Typed Case Branch]\n        classifyBranch[Classifier Branch]\n        resolveCase[Resolve Case]\n        goalStruct[Build Goal Struct]\n        autoFill[Auto Fill From Text History]\n        missingGate{Missing Required Values}\n        followUpQuestion[Return Follow Up Question]\n        dispatchHandler[Dispatch Handler]\n\n        receivePayload --> normalizePayload --> detectFiles\n        normalizePayload --> typedGate\n        typedGate -->|Yes| startSimBranch\n        typedGate -->|Yes| chatBranch\n        typedGate -->|Yes| typedCaseBranch\n        typedGate -->|No| classifyBranch --> resolveCase --> goalStruct --> autoFill --> missingGate\n        missingGate -->|Yes| followUpQuestion\n        missingGate -->|No| dispatchHandler\n        typedCaseBranch --> dispatchHandler\n        chatBranch --> dispatchHandler\n    end\n\n    subgraph filePipeline [FilePipeline]\n        processFiles[FileManager Process Upload Config]\n        ragUpsert[Vertex RAG Upsert]\n        extractComponents[Extract Params Fields Methods]\n        upsertComponents[Upsert Components and File Metadata]\n        detectFiles --> processFiles --> ragUpsert --> extractComponents --> upsertComponents\n    end\n\n    subgraph actionBus [DomainActionBus]\n        envCases[ENV Cases]\n        fieldCases[FIELD Cases]\n        injectionCases[INJECTION Cases]\n        sessionCases[SESSION Cases]\n        moduleCases[MODULE Cases]\n        paramCases[PARAM Cases]\n        methodCases[METHOD Cases]\n        fileCases[FILE Cases]\n        modelCases[MODEL Case non-wired]\n        smCases[SM Case non-wired]\n        gmailCases[GMAIL Cases non-wired]\n        dispatchHandler --> envCases\n        dispatchHandler --> fieldCases\n        dispatchHandler --> injectionCases\n        dispatchHandler --> sessionCases\n        dispatchHandler --> moduleCases\n        dispatchHandler --> paramCases\n        dispatchHandler --> methodCases\n        dispatchHandler --> fileCases\n        dispatchHandler --> modelCases\n        dispatchHandler --> smCases\n        dispatchHandler --> gmailCases\n    end\n\n    subgraph simulationLayer [SimulationLayer]\n        guardMain[Guard Main]\n        buildGraph[Build Graph Components]\n        streamGate{Grid Stream Enabled}\n        persistArtifacts[Persist Simulation Artifacts]\n        rotateSession[Deactivate and Create Session]\n        startSimBranch --> guardMain --> buildGraph --> streamGate --> persistArtifacts --> rotateSession\n    end\n\n    subgraph persistenceLayer [PersistenceLayer]\n        qbrainMgr[QBrainTableManager]\n        dbMgr[DBManager]\n        duckDb[(DuckDB)]\n        bigQuery[(BigQuery BQCore)]\n        vectorStore[(VectorStore)]\n        envCases --> qbrainMgr\n        fieldCases --> qbrainMgr\n        injectionCases --> qbrainMgr\n        sessionCases --> qbrainMgr\n        moduleCases --> qbrainMgr\n        paramCases --> qbrainMgr\n        methodCases --> qbrainMgr\n        fileCases --> qbrainMgr\n        upsertComponents --> qbrainMgr\n        persistArtifacts --> qbrainMgr\n        qbrainMgr --> dbMgr --> duckDb\n        dbMgr --> bigQuery\n        qbrainMgr --> vectorStore\n    end\n\n    subgraph outputLayer [OutputLayer]\n        wsSuccess[WS Typed Success]\n        wsError[WS Error]\n        wsFollowUp[WS Follow Up CHAT]\n        httpResponses[HTTP Responses]\n        dispatchHandler --> wsSuccess\n        followUpQuestion --> wsFollowUp\n        guardMain --> wsError\n        httpEntry --> httpResponses\n    end\n```\n\n## Project Component Tree (Full Map)\n\n```\nBestBrain/\n├── bm/                          # Django app (ASGI, settings, static)\n│   ├── urls.py                  # Root URL routing → world/, auth/, graph/, health/, admin/\n│   └── views.py                 # health(), spa_index (QDash catch-all)\n├── qbrain/\n│   ├── relay_station.py         # Relay (WebSocket consumer): connect, receive, send\n│   ├── predefined_case.py      # RELAY_CASES_CONFIG (ENV, FIELD, INJECTION, SESSION, MODULE, PARAM, METHOD, FILE)\n│   ├── urls.py                  # world/demo/, world/webhook/\n│   ├── auth/urls.py             # auth/access/\n│   ├── graph/\n│   │   ├── local_graph_utils.py # GUtils (NetworkX graph, add_node, add_edge, schemas)\n│   │   ├── brain.py             # Brain(GUtils): hydrate, ingest, classify_goal, execute_or_ask\n│   │   ├── brain_schema.py      # Node/edge types, GoalDecision, DataCollectionResult\n│   │   ├── brain_hydrator.py    # User-scoped DuckDB → LONG_TERM_STORAGE nodes\n│   │   ├── brain_classifier.py  # Hybrid goal classification (rule / vector / fallback)\n│   │   ├── brain_executor.py    # execute_or_request_more, debug metadata, payload guard\n│   │   ├── brain_workers.py     # Thread pool for embedding/hydration offload\n│   │   ├── models.py            # KnowledgeNode (CONTENT chunk schema)\n│   │   ├── processor/           # FileProcessorFacade, BaseProcessor, graph_builder\n│   │   ├── test.py              # Terminal Brain test (suite / interactive), Rich UI, JSON reports\n│   │   └── dj/\n│   │       ├── urls.py          # graph/view/, graph/brain/test/\n│   │       ├── visual.py        # GraphLookup (POST graph JSON → streaming HTML)\n│   │       └── brain_test.py    # HTTP Brain test chat (GET HTML, POST JSON chat/suite)\n│   ├── core/\n│   │   ├── orchestrator_manager/orchestrator.py  # Thalamus: handle_relay_payload, START_SIM, typed dispatch\n│   │   ├── guard.py             # Guard: main(env_id, env_data), build graph, pop_cmd(grid), persist model/anim\n│   │   ├── qbrain_manager/      # QBrainTableManager: MANAGERS_INFO, run_query, set_item, _generate_embedding\n│   │   ├── session_manager/     # SessionManager, session_manager (get_or_create_active_session)\n│   │   ├── env_manager/         # EnvManager (env CRUD, retrieve_env_from_id)\n│   │   ├── file_manager/        # FileManager: process_and_upload_file_config, RAG upsert, param/field/method extract\n│   │   ├── param_manager/       # ParamsManager\n│   │   ├── fields_manager/      # FieldsManager\n│   │   ├── method_manager/      # MethodManager\n│   │   ├── injection_manager/   # InjectionManager\n│   │   ├── module_manager/      # ModuleWsManager, ModuleLoader, Modulator\n│   │   ├── model_manager/       # ModelManager\n│   │   ├── user_manager/        # UserManager (get_or_create_user, initialize_qbrain_workflow)\n│   │   └── managers_context.py  # set_orchestrator, reset_orchestrator\n│   ├── _db/\n│   │   ├── manager.py           # DBManager (DuckDB), get_db_manager(), db_check, db_status\n│   │   └── vector_store.py      # VectorStore (create_store, upsert_vectors, similarity_search, classify)\n│   ├── chat_manger/             # AIChatClassifier (case classification from message)\n│   ├── qf_utils/                # QFUtils, FieldUtils, runtime_utils_creator\n│   ├── utils/                   # Utils, Manipulator, QueueHandler, serialize_complex, run_subprocess (pop_cmd)\n│   └── code_manipulation/       # StructInspector (AST → graph), handler registration\n├── jax_test/                    # External grid/simulation (optional)\n│   └── grid/                    # Grid run, streamer, animation_recorder\n├── docs/                        # PROMPT_*, GRID_STREAM_PROTOCOL, QDASH_GRID_CHECKLIST, etc.\n└── startup.py                   # Migrations, collectstatic, QDash build, nginx, daphne\n```\n\n## Component Interactions (Who Calls Whom)\n\n| From | To | Action |\n|------|-----|--------|\n| ASGI / Daphne | Relay | WebSocket at `/run/`; connect → receive → send |\n| Relay | UserManager | get_or_create_user |\n| Relay | SessionManager | get_or_create_active_session (via session_manager) |\n| Relay | Thalamus | Constructor(cases, user_id, relay=self) |\n| Relay | Orchestrator | handle_relay_payload(payload) for every message |\n| Orchestrator | AIChatClassifier | main(user_id, msg) when type missing or CHAT |\n| Orchestrator | FileManager | process_and_upload_file_config when files in payload |\n| Orchestrator | Guard | guard.main(env_id, env_data, ...) when data_type == START_SIM |\n| Orchestrator | Relay | send(text_data=...) to push SET_SID, LIST_*, typed success/error |\n| Guard | GUtils | add_node, add_edge (ENV, MODULE, FIELD, INJECTION, METHOD, PARAM) |\n| Guard | QBrainTableManager | row_from_id, set_item, upsert_copy (params, fields, methods, envs) |\n| Guard | pop_cmd (utils) | Run grid subprocess: `python -m jax_test.grid --cfg <cfg_path>` |\n| Guard | GridStreamer (optional) | put_frame(step, data) when GRID_STREAM_ENABLED |\n| FileManager | Param/Field/Method managers | Extract and upsert components; RAG upsert when corpus_id set |\n| QBrainTableManager | DBManager | run_query, execute, insert (DuckDB or BigQuery) |\n| Brain | GUtils | add_node (USER, GOAL, SUB_GOAL, SHORT_TERM_STORAGE, LONG_TERM_STORAGE, CONTENT), add_edge |\n| Brain | BrainHydrator | hydrate_user_long_term(user_id) → LONG_TERM_STORAGE from MANAGERS_INFO tables |\n| Brain | BrainClassifier | classify(query, long_term_nodes) → GoalDecision |\n| Brain | BrainExecutor | execute_or_request_more(case_item, resolved_fields, missing_fields) |\n| BrainClassifier | VectorStore | similarity_search for relay-case vectors; embed_fn from QBrain or deterministic fallback |\n| graph/test.py | Brain | execute_or_ask(query, user_payload); save report to graph/test_runs/ |\n| graph/dj/brain_test.py | Brain | POST JSON chat/suite → execute_or_ask; GET → HTML chat UI |\n\n## Action Catalog (All Main Actions)\n\n- **Relay**: `connect`, `receive`, `send`, `send_session`, `_send_all_user_sessions`, `_resolve_session`, `_save_session_locally`, `scan_dir_to_code_graph`\n- **Orchestrator**: `handle_relay_payload`, `_ensure_data_type_from_classifier`, `_handle_start_sim_process`, `_dispatch_relay_handler`, `_resolve_case`, file detection and FileManager invocation\n- **Guard**: `main(env_id, env_data)`, `create_nodes`, `data_handler`, `build_graph`, write config, `pop_cmd(grid)`, persist model/animation to env row\n- **QBrainTableManager**: `run_query`, `run_db`, `execute`, `insert`, `set_item`, `row_from_id`, `upsert_copy`, `get_managers_info`, `_generate_embedding`, `initialize_all_tables`\n- **DBManager**: `run_query`, `execute`, `close` (DuckDB or BigQuery)\n- **VectorStore**: `create_store`, `add_vectors`, `upsert_vectors`, `delete`, `similarity_search`, `batch_similarity_search`, `classify`, `count`, `reset`, `optimize`, `close`\n- **FileManager**: `process_and_upload_file_config`, `_step1_vertex_rag_upsert`, `_step2_extract_components_pipeline`, `_step3_upsert_components`, `_step4_upsert_files_table`, `_step5_upsert_module`\n- **Brain**: `hydrate_user_context`, `ingest_input`, `classify_goal`, `collect_required_data`, `execute_or_ask`, `_cleanup_goal_and_subgoals`, `close`\n- **BrainClassifier**: `classify(query, long_term_nodes)` → GoalDecision (rule / vector / fallback)\n- **BrainExecutor**: `execute_or_request_more` (need_data | executed | error; execution_debug; payload serialization guard)\n- **GUtils**: `add_node`, `add_edge`, `get_node`, `get_edge`, `local_batch_loader`, `save_graph`, `load_graph` (history/h_entry only when enable_data_store)\n- **graph/processor**: `FileProcessorFacade.process_file`, `process_to_graph(path, g)`, `build_graph(rows, g)` → CONTENT nodes + parent_of / follows\n- **HTTP**: `GET /health/`, `GET /world/demo/`, `GET /graph/view/`, `GET /graph/brain/test/`, `POST /graph/brain/test/` (chat/suite), SPA catch-all for QDash\n\n## Workflow Mastermap\n\n- Full visual workflow + exhaustive action catalog: `docs/PROJECT_WORKFLOW_MASTERMAP.md`\n\n## Running all apps locally\n\nThe `_admin` CLI can run every discovered project (Dockerfile or package.json/manage.py/requirements.txt) **without Docker**: it infers start commands from project type and context (startup.py, manage.py, main.py, Dockerfile CMD, package.json scripts) and runs them natively.\n\n- **Scan only** (print inferred command and cwd for each project):\n  ```bash\n  python -m _admin.main --run-local-scan-only\n  ```\n- **Run all** runnable projects (backend(s) and frontend(s) with default ports 8000, 3000):\n  ```bash\n  python -m _admin.main --run-local\n  ```\n- **Run one project** (path relative to repo root):\n  ```bash\n  python -m _admin.main --run-local --run-local-project qdash\n  python -m _admin.main --run-local --run-local-project grid\n  ```\n- **Custom ports**: `--run-local-port 8000` or `--run-local-port backend_drf:8000,frontend:3000`\n\nProjects are classified as `backend_drf`, `backend_fastapi`, `backend_py`, `frontend_react`, or `mobile_react_native`. The root Django app is run via `startup.py --backend-only` when present; the qbrain package and root backend are deduplicated (only one process). See `_admin/README.md` for the full inferred-command table and edge cases.\n\n## Current Core TODOs (from existing intent)\n\n- [ ] `get_data`: integrate BigQuery -> Sheets live data view (`table=ntype`, `col=px`, `row=ts state`).\n- [ ] Improve method extraction (bracket parsing and dedupe) and reliably inject method defs into `Guard.method_layer`.\n\n## Near-Term Engine Priorities\n\n- [ ] Implement relay case consumption hardening and payload contract validation.\n- [ ] Add Guard answer caching and cross-module parameter/field consistency checks.\n- [ ] Add observability (latency, error rates, case-level tracing) for Relay/Orchestrator/Guard.\n\n## Global TODO Rollup (docs + code)\n\n- **Core engine / orchestration**\n  - Unify case registry so wired actions and implemented handlers match one-to-one; make `CHAT` and `START_SIM` first-class registry entries (or equivalent contract wrappers) and close the `CONVERT_MODULE` placeholder with a concrete callable and tests.\n  - Add payload contract validation at relay ingress (required keys, type checks, unknown-field policy).\n  - Implement `get_data` live bridge objective (BigQuery → tabular display pipeline) and remove hardcoded test identity flows.\n  - Improve method extraction (bracket parsing and dedupe) and reliably inject method defs into `Guard.method_layer`.\n  - Add Guard consistency guarantees (answer caching, method‑param‑field reconciliation, deterministic ordering for repeated runs).\n  - Add per-case latency and error instrumentation and broader observability for Relay/Orchestrator/Guard; introduce async-safe batching and batched `get`/`link` operations in managers.\n  - Add session lifecycle controls (idle timeout, reconnect semantics, explicit closure audit) and stabilize graph serialization and the runtime graph/persistence boundary.\n\n- **Simulation / JAX GTM engine (`grid`)**\n  - Track energetic time distribution over time.\n  - Implement blur-based prefill of results from in-feature lines so not every value requires full computation; extend model payload with controller section, total time-step feature, and a configurable test switch.\n  - Evolve time-engine ideas: alternative-reality branches, iterator time-travel, time-step consistency checks, zero-shot horizon, rollback/replay, interpolation between steps, confidence/uncertainty signals, canonical vs alternative realities, cross-step invariants, and minimal state for offline validation.\n\n- **Frontend / QDash**\n  - Full terminal agent capabilities (intent handling, tool use) and a pure relay-based commit flow.\n  - Geometry-only drag-and-drop from the right-side view into the central grid background component (file conversion handled in the frontend).\n  - User self-management of the Gemini API key inside the app (settings or prompt-based).\n  - Deep integration with the `qbrain` backend/services.\n  - In-screen visualization mapping module parameters to visualization techniques (including retro n‑D views).\n  - “Add model” env switch wired into `conversation.models` with a clear rule for when the list is reset vs kept across terminal submits.\n\n- **Docs / workflow mastermap**\n  - Build workflow replay timelines per session and a policy-based optimizer that proposes minimal remediation for failed simulations.\n  - Add a schema drift detector and migration assistant for QBRAIN tables.\n  - Expand `MODEL` and `GMAIL` action buses into normalized case contracts.\n\n## Implemented Results Snapshot\n\n- **Session management (`core/session_manager`)**: Sessions table in the QBRAIN dataset, random numeric session IDs, Relay integration, and an 8-test suite; demo and integration flows verified, all tests passing.\n- **Injection management (`core/injection_manager`)**: BigQuery-backed `injections` table with full CRUD API and WebSocket handlers wired into `relay_station.py`; 9-test suite, all tests passing, ready to receive energy designer data.\n- **GTM JAX simulation engine (`grid`)**: End-to-end Guard → GNN workflow defined (DB build, simulation loop, export); iterator + time-controller architecture stabilized; scan-in/out feature scoring and indexing implemented.\n- **Nginx + deployment tooling (`qbrain/nginx`, `_admin`)**: Env-driven Nginx config rendering with `startup.sh` integration; `_admin` CLI can discover and run the monolith backend, QDash, and `grid` locally, including an optional QDash demo recording workflow.\n- **BigQuery Toolbox (`qbrain/_bigquery_toolbox`)**: Streamlit-based analytics and ingestion toolbox with RAG search, text-to-SQL, and vector search, including Local Core mode for direct engine integration.\n",
  "bytes": 18858,
  "sha": "e9198f99dbafdce6f63051695dc09003d7bdcda3ab72edbe2707e995c8293a94",
  "repo_slug": "wired87/core",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_wired87_core_e09eac6c/readme"
}