Compare commits

...

2 Commits

Author SHA1 Message Date
wzy-warehouse d402668a5c QGIS的docker管理 2026-06-22 11:01:15 +08:00
wzy-warehouse 154f0a968e QGIS的docker管理 2026-06-21 22:30:04 +08:00
12 changed files with 1033 additions and 383 deletions
+1
View File
@@ -59,3 +59,4 @@ htmlcov/
# QGIS 临时模板文件
app/data/template/*/tmp*.qgz
tmp*.qgz
/tmp/
+346
View File
@@ -0,0 +1,346 @@
# QGIS Docker 部署指南
QGIS 专题图产出通过 Docker 容器执行,主进程(Python 3.10)通过 `docker exec` 调用容器内的 QGIS 环境。
## 1. 环境要求
- DockerWindows: Docker DesktopLinux: docker-ce
- 项目代码已部署到服务器
- 输出文件目录已创建
## 2. 拉取 QGIS 镜像
```bash
# 官方 QGIS 镜像(含 QGIS 3.x + Python + Qt5
docker pull qgis/qgis:3.44.11
# 验证
docker run --rm qgis/qgis:3.44.11 qgis --version
```
### 无外网环境(离线导入)
```bash
# 在有网的机器上导出
docker save qgis/qgis:3.44.11 -o qgis-34411.tar
# 传输到目标服务器后导入
docker load -i qgis-34411.tar
```
## 3. 启动 QGIS 容器
```bash
# ---- Linux / macOS ----
PROJECT_ROOT=/home/xian/xian_algorithm_new
FILE_STORE=/data
docker run -d \
--name qgis-server \
--restart unless-stopped \
-v "${PROJECT_ROOT}:/app:ro" \
-v "${FILE_STORE}:${FILE_STORE}" \
qgis/qgis:3.44.11 \
sleep infinity
# ---- Windows (cmd.exe) ----
# 注意:Windows 路径必须用正斜杠 /,不能用反斜杠 \
# 挂载目标必须是 Linux 路径(容器是 Linux
docker run -d ^
--name qgis-server ^
--restart unless-stopped ^
-v "F:/project/xian/xian_algorithm_new:/app:ro" ^
-v "G:/files:/files" ^
qgis/qgis:3.44.11 ^
sleep infinity
```
### 参数说明
| 参数 | 说明 |
|------|------|
| `--name qgis-server` | 容器名称,需与 `settings.toml``QGIS_DOCKER_CONTAINER` 一致 |
| `-v 项目代码:/app:ro` | 项目代码只读挂载,容器内路径由 `QGIS_DOCKER_PROJECT_DIR` 配置 |
| `-v 输出目录:输出目录` | 文件输出目录读写挂载,保持主机与容器路径一致 |
| `sleep infinity` | 保持容器运行,等待 `docker exec` 调用 |
## 4. 预拷贝静态数据到容器本地 FS(必须,性能关键)
WSL2 9P 文件系统随机读取极慢(GPKG 62MB 耗时 6-10s/模板,模板 ZIP 也慢),拷贝到容器内 `/data/` 后读取仅需 ~0.5s。
### 方式一:Python 脚本(推荐)
```bash
python app/script/copy_data_to_container.py
```
输出示例:
```
=== 预拷贝静态数据到容器 qgis-server ===
[GPKG] 主机目录: F:\project\xian\xian_algorithm_new\app\data\gpkg
文件数: 23, 总大小: 62.3 MB
容器目标: qgis-server:/data/gpkg
拷贝完成: 3.2s, 容器内 23 个文件
[模板] 主机目录: F:\project\xian\xian_algorithm_new\app\data\template
文件数: 34, 总大小: 45.1 MB
容器目标: qgis-server:/data/template
拷贝完成: 2.1s, 容器内 34 个文件
=== 总耗时: 5.4s ===
```
其他用法:
```bash
python app/script/copy_data_to_container.py --dry-run # 仅查看信息
python app/script/copy_data_to_container.py --only gpkg # 只拷贝 GPKG
python app/script/copy_data_to_container.py --only template # 只拷贝模板
```
### 方式二:手动 docker cp
```bash
# 清理旧文件
docker exec qgis-server rm -rf /data/gpkg /data/template
# 拷贝 GPKG
docker cp F:\project\xian\xian_algorithm_new\app\data\gpkg qgis-server:/data/gpkg
# 拷贝模板
docker cp F:\project\xian\xian_algorithm_new\app\data\template qgis-server:/data/template
# 验证
docker exec qgis-server ls -la /data/gpkg
docker exec qgis-server ls -la /data/template/rainfall
docker exec qgis-server ls -la /data/template/earthquake
```
### 何时需要重新执行
- GPKG 或模板文件有更新时
- 容器重建后(`/data` 目录丢失)
- 首次部署时
## 5. 安装中文字体(手动)
QGIS 模板使用了 SimHei(黑体)、SimSun(宋体)、Microsoft YaHei(微软雅黑)等 Windows 中文字体,
Docker 镜像默认不包含这些字体,会导致中文全部乱码。**字体需手动安装,代码不会自动安装。**
### 步骤
```bash
# 1. 创建字体目录
docker exec qgis-server mkdir -p /usr/share/fonts/truetype/winfonts
# 2. 从主机复制 Windows 中文字体
# Windows 路径(根据实际字体文件位置调整):
docker cp "C:\Windows\Fonts\simhei.ttf" qgis-server:/usr/share/fonts/truetype/winfonts/
docker cp "C:\Windows\Fonts\simsun.ttc" qgis-server:/usr/share/fonts/truetype/winfonts/
docker cp "C:\Windows\Fonts\msyh.ttc" qgis-server:/usr/share/fonts/truetype/winfonts/
docker cp "C:\Windows\Fonts\msyhbd.ttc" qgis-server:/usr/share/fonts/truetype/winfonts/
# Linux 字体路径示例:
# docker cp /usr/share/fonts/truetype/droid/DroidSansFallbackFull.ttf qgis-server:/usr/share/fonts/truetype/winfonts/
# 3. 刷新字体缓存
docker exec qgis-server fc-cache -fv
# 4. 验证字体已识别
docker exec qgis-server python3 -c "
from PyQt5.QtGui import QFontDatabase
db = QFontDatabase()
zh = [f for f in db.families() if 'SimHei' in f or 'YaHei' in f or 'SimSun' in f]
print('中文字体:', zh)
"
```
### 持久化方案
容器重建后字体丢失。可选方案:
**方案 A:挂载单个字体文件**
```bash
docker run -d ... \
-v "C:\Windows\Fonts\simhei.ttf:/usr/share/fonts/truetype/winfonts/simhei.ttf" \
-v "C:\Windows\Fonts\simsun.ttc:/usr/share/fonts/truetype/winfonts/simsun.ttc" \
...
```
**方案 B:挂载整个字体目录(推荐)**
```bash
# 先在主机创建字体目录,放入所需字体文件
mkdir -p /opt/qgis-fonts
cp /usr/share/fonts/truetype/droid/DroidSansFallbackFull.ttf /opt/qgis-fonts/
docker run -d ... \
-v "/opt/qgis-fonts:/usr/share/fonts/truetype/winfonts:ro" \
...
```
## 6. 验证容器
```bash
# 检查容器状态
docker ps
# 测试 QGIS 可用性
docker exec qgis-server python3 -c "from qgis.core import QgsApplication; print('OK')"
# 查看容器内项目代码
docker exec qgis-server ls /app/app/services/qgis/
```
## 7. 配置文件
所有 Docker 相关配置集中在 `settings.toml``[default]` 段:
```toml
[default]
# ---- Docker QGIS 配置 ----
QGIS_DOCKER_CONTAINER = "qgis-server" # 容器名称(需与 docker run --name 一致)
QGIS_DOCKER_PROJECT_DIR = "/app" # 容器内项目代码挂载路径
QGIS_DOCKER_PYTHON = "/usr/bin/python3" # 容器内 Python 解释器路径
QGIS_DOCKER_IMAGE = "qgis/qgis:3.44.11" # Docker 镜像名称
QGIS_DOCKER_PREFIX_PATH = "/usr" # QGIS prefixPath(安装根目录)
QGIS_DOCKER_PYTHONPATH = [ # QGIS Python 包路径列表
"/usr/lib/python3/dist-packages",
"/usr/lib/python3.10/dist-packages",
"/usr/lib/python3.11/dist-packages",
"/usr/lib/python3.12/dist-packages",
]
QGIS_DOCKER_QT_PLATFORM = "offscreen" # Qt 无头模式
QGIS_DOCKER_KEEP_ALIVE = "sleep infinity" # 容器保活命令
# ---- 专题图参数 ----
QGIS_GPKG_DIR = "app/data/gpkg" # GPKG 目录(相对于项目根)
QGIS_DOCKER_GPKG_DIR = "/data/gpkg" # 容器内 GPKG 本地路径(预拷贝后读取)
QGIS_DOCKER_TEMPLATE_DIR = "/data/template" # 容器内模板本地路径(预拷贝后读取)
QGIS_EXPORT_DPI = 200 # 导出 DPI
QGIS_PARALLEL_WORKERS = 4 # 并行 docker exec 子进程数
QGIS_MAX_CONCURRENT = 2 # 最大并发请求数
# ---- 文件路径 ----
FILE_STORE_DIR = "G:/files" # 主机端文件输出目录
```
### 环境变量覆盖
```bash
export QGIS_DOCKER_CONTAINER="qgis-server"
export QGIS_DOCKER_PROJECT_DIR="/app"
export QGIS_DOCKER_PYTHON="/usr/bin/python3"
```
### 跨机器适配
不同机器只需修改 `settings.toml` 中的路径配置:
| 配置项 | 开发机 (Windows) | 生产机 (Linux) |
|--------|-----------------|---------------|
| `FILE_STORE_DIR` | `"G:/files"` | `"/data"` |
| `DB_HOST` | `"47.92.216.173"` | `"10.22.245.138"` |
| `DB_PORT` | `7654` | `54321` |
## 8. 目录结构
```
项目根目录/
├── app/
│ ├── data/
│ │ ├── gpkg/ # 静态底图 GeoPackage 文件
│ │ └── template/ # 专题图模板 (.qgz)
│ ├── services/qgis/
│ │ ├── qgis_env.py # Docker 环境配置(路径映射、命令构建)
│ │ ├── qgis_runner.py # 容器内子进程入口
│ │ ├── map_service.py # 地图生成主流程
│ │ ├── template_modifier.py # 模板 XML 修改
│ │ └── map_exporter.py # 图片导出
│ └── api/
│ └── qgis_map_export.py # FastAPI 专题图导出接口
├── script/
│ └── copy_data_to_container.py # GPKG + 模板预拷贝脚本
├── config.py # Dynaconf 配置入口
├── settings.toml # 全部配置
├── requirements.txt # Python 依赖
├── QGIS_DOCKER_README.md # 本文档
└── tmp/ # 临时文件目录(容器内映射为 /app/tmp/)
```
## 9. 临时文件
- 主机端临时 JSON(批量产图配置):写入 `{项目根}/tmp/`,容器内可通过 `/app/tmp/` 访问
- 容器端临时 .qgz(修改后的模板):写入容器内 `/tmp/`,由 runner 自动清理
## 10. 故障排查
```bash
# 容器未运行
docker ps -a | grep qgis-server
# 查看容器日志
docker logs qgis-server
# 手动测试 runner
docker exec qgis-server python3 /app/app/services/qgis/qgis_runner.py --help
# 检查 QGIS 依赖
docker exec qgis-server python3 -c "
from qgis.core import QgsApplication
QgsApplication.setPrefixPath('/usr', True)
app = QgsApplication([], False)
app.initQgis()
print('QGIS', QgsApplication.version())
app.exitQgis()
"
# 检查中文字体
docker exec qgis-server python3 -c "
from PyQt5.QtGui import QFontDatabase
db = QFontDatabase()
zh = [f for f in db.families() if any(k in f for k in ['SimHei','YaHei','SimSun','WenQuanYi'])]
print('中文字体:', zh if zh else '未安装!')
"
# 检查 GPKG 文件(容器本地路径,性能关键)
docker exec qgis-server ls /app/app/data/gpkg/ # 挂载路径(慢)
docker exec qgis-server ls /data/gpkg/ # 本地路径(快,需预拷贝)
# 检查模板文件(容器本地路径)
docker exec qgis-server ls /data/template/rainfall/ # 本地路径(快)
docker exec qgis-server ls /data/template/earthquake/
# 检查临时文件目录
docker exec qgis-server ls /app/tmp/
```
### 静态数据相关问题
| 问题 | 原因 | 解决 |
|------|------|------|
| 产图慢(>60s) | GPKG/模板从挂载目录读取(9P 慢) | 执行 `python app/script/copy_data_to_container.py` |
| 日志显示 `gpkg_dir=/app/...` | 未预拷贝或 `QGIS_DOCKER_GPKG_DIR` 为空 | 检查 settings.toml 配置 |
| 容器重建后变慢 | `/data` 目录丢失 | 重新执行拷贝脚本 |
| `docker cp` 权限错误 | 容器未运行 | `docker start qgis-server` |
## 11. 工作流程
```
用户请求 POST /qgis/export/map
→ docker inspect 检查容器是否运行
→ 查询推理结果
→ 扫描模板文件夹
→ 构建配置(路径映射到容器内路径)
→ 并行启动 docker exec 子进程
→ 容器内 qgis_runner.py
→ 加载 QGIS Python 包
→ 初始化 QgsApplication
→ 逐模板处理:
→ TemplateModifier 修改模板 XML(替换连接参数、静态层→GPKG)
→ project.read() 加载模板
→ 图层过滤、缩放、文本更新
→ 导出 JPG
→ 实时写入进度表
```
+112 -86
View File
@@ -10,7 +10,9 @@ import asyncio
import concurrent.futures
import os
import re
import subprocess
from datetime import datetime
from pathlib import Path
from typing import Optional
from fastapi import APIRouter, HTTPException
@@ -156,8 +158,43 @@ def _extract_center_from_condition(event_type: str, condition: dict) -> tuple:
def _build_qgis_config(batch_folder: str) -> dict:
"""构建 QGIS 服务配置(含批次输出目录)"""
from app.services.qgis.qgis_env import (
get_docker_container, get_host_file_store, get_container_file_store,
get_docker_project_dir,
)
gpkg_dir = get_gpkg_dir()
# Docker 模式:config 中的路径必须是容器内路径(模板修改器在容器内运行)
try:
result = subprocess.run(
["docker", "inspect", "--format={{.State.Running}}", get_docker_container()],
capture_output=True, text=True, timeout=5,
)
is_docker = result.stdout.strip() == "true"
except Exception:
is_docker = False
if is_docker:
# GPKG 目录:优先使用容器内本地路径(预拷贝后绕过 WSL2 9P)
# 需先执行 python script/copy_gpkg_to_container.py
gpkg_dir = getattr(settings, "QGIS_DOCKER_GPKG_DIR", "") or ""
if not gpkg_dir:
# fallback: 使用挂载路径(性能差)
project_dir = get_docker_project_dir()
gpkg_subdir = getattr(settings, "QGIS_GPKG_DIR", "app/data/gpkg")
gpkg_dir = f"{project_dir}/{gpkg_subdir}"
logger.warning(f"GPKG 将从挂载目录读取(慢),建议执行 copy_gpkg_to_container.py")
# batch_folder:主机路径 → 容器路径
host_fs = get_host_file_store().rstrip("/")
container_fs = get_container_file_store().rstrip("/")
batch_folder_container = batch_folder.replace("\\", "/")
if batch_folder_container.lower().startswith(host_fs.lower()):
batch_folder_container = container_fs + batch_folder_container[len(host_fs):]
batch_folder = batch_folder_container
logger.info(f"[Docker模式] gpkg_dir={gpkg_dir}, batch_folder={batch_folder}")
else:
logger.info(f"[本地模式] gpkg_dir={gpkg_dir}")
return {
"db": {
"host": settings.DB_HOST,
@@ -189,9 +226,18 @@ async def export_map(req: QgisMapExportRequest):
"""
根据模拟ID批量导出专题图。同一 inferenceId 共享文件夹,增量产出缺失图片。
"""
from app.services.qgis.qgis_env import is_qgis_available
if not is_qgis_available():
raise HTTPException(status_code=503, detail="QGIS 环境不可用")
from app.services.qgis.qgis_env import get_docker_container
try:
result = subprocess.run(
["docker", "inspect", "--format={{.State.Running}}", get_docker_container()],
capture_output=True, text=True, timeout=5,
)
if result.stdout.strip() != "true":
raise HTTPException(status_code=503, detail="QGIS Docker 容器未运行")
except HTTPException:
raise
except Exception:
raise HTTPException(status_code=503, detail="QGIS Docker 容器不可用")
async with _qgis_semaphore:
inference_id = req.inferenceId
@@ -320,10 +366,11 @@ async def export_map(req: QgisMapExportRequest):
def _generate_batch_maps(models: list, config: dict, batch_key: str,
inference_id: int = None, file_store: str = None) -> None:
"""并行启动多个 QGIS 子进程,实时读取每张图进度并写 DB"""
"""并行启动多个 docker exec 子进程,实时读取每张图进度并写 DB"""
import json, math, concurrent.futures, subprocess, tempfile, threading
from app.services.qgis.qgis_env import (
get_qgis_python_path, get_runner_script, build_qgis_subprocess_env,
get_docker_project_dir, get_container_python_path, build_docker_exec_cmd,
map_host_to_container, map_container_to_host, map_template_to_container,
)
max_workers = getattr(settings, "QGIS_PARALLEL_WORKERS", 4)
@@ -334,8 +381,12 @@ def _generate_batch_maps(models: list, config: dict, batch_key: str,
for i in range(0, len(models), chunk_size):
chunks.append(models[i:i + chunk_size])
project_dir = get_docker_project_dir()
python_in_container = get_container_python_path()
runner_in_container = f"{project_dir}/app/services/qgis/qgis_runner.py"
logger.info(
f"[批量产图] {len(models)} 张图 → {len(chunks)} 个并行子进程 "
f"[批量产图] {len(models)} 张图 → {len(chunks)} 个并行 docker exec "
f"(每进程 {chunk_size} 张)"
)
@@ -344,44 +395,87 @@ def _generate_batch_maps(models: list, config: dict, batch_key: str,
db_lock = threading.Lock() # 保护 DB 写入
def _run_chunk(chunk_models: list, chunk_idx: int):
"""单个子进程,逐张读取进度并实时写 DB"""
request = json.dumps({"config": config, "models": chunk_models}, ensure_ascii=False)
tmp = tempfile.NamedTemporaryFile(suffix=".json", delete=False, mode="w", encoding="utf-8")
"""单个 docker exec 子进程,逐张读取进度并实时写 DB"""
# 将主机路径映射为容器内路径(G:/files → /files
container_models = []
for m in chunk_models:
cm = dict(m)
if "outFile" in cm:
cm["outFile"] = map_host_to_container(cm["outFile"])
# 模板 path 映射到容器本地路径(预拷贝后绕过 9P)
if "path" in cm:
cm["path"] = map_template_to_container(cm["path"])
container_models.append(cm)
request = json.dumps({"config": config, "models": container_models}, ensure_ascii=False)
# 临时文件写到项目目录内的 tmp/ 子目录(挂载到容器 /app/tmp/),确保容器内可访问
project_dir_host = str(Path(__file__).parent.parent.parent)
tmp_dir = os.path.join(project_dir_host, "tmp")
os.makedirs(tmp_dir, exist_ok=True)
tmp = tempfile.NamedTemporaryFile(
suffix=".json", delete=False, mode="w", encoding="utf-8",
dir=tmp_dir,
)
tmp.write(request)
tmp.close()
qgis_root = getattr(settings, "QGIS_ROOT", "D:/QGIS")
python_path = get_qgis_python_path(qgis_root)
runner = get_runner_script()
cmd = [python_path, runner, tmp.name]
# 主机临时文件路径 → 容器内路径
tmp_in_container = tmp.name.replace("\\", "/")
project_root = str(Path(__file__).parent.parent.parent).replace("\\", "/")
if tmp_in_container.startswith(project_root):
tmp_in_container = tmp_in_container.replace(project_root, project_dir, 1)
cmd = build_docker_exec_cmd(python_in_container, runner_in_container, tmp_in_container)
logger.info(f"[Docker] chunk{chunk_idx} 命令: {' '.join(cmd)}")
proc = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL,
env=build_qgis_subprocess_env(qgis_root),
cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
text=True, encoding="utf-8", errors="replace",
)
# 并发读取 stderr 防止管道缓冲区满导致死锁
stderr_lines = []
def _drain_stderr():
for ln in proc.stderr:
stderr_lines.append(ln)
stderr_thread = threading.Thread(target=_drain_stderr, daemon=True)
stderr_thread.start()
chunk_results = []
for line in proc.stdout:
line = line.strip()
logger.debug(f"[chunk{chunk_idx}] stdout: {line[:200]}")
if line.startswith("PROGRESS:"):
try:
r = json.loads(line[len("PROGRESS:"):])
# 容器内路径映射回主机路径(/files → G:/files
if "output" in r:
r["output"] = map_container_to_host(r["output"])
chunk_results.append(r)
# ★ 每张图产出后立即写 DB
status = "FAIL" if "error" in r else "OK"
logger.info(f"[chunk{chunk_idx}] {status} {r.get('name', '?')}: {r.get('error', r.get('output', ''))[:100]}")
if inference_id and file_store and "error" not in r:
_write_single_path(inference_id, r.get("output", ""), file_store, db_lock)
except json.JSONDecodeError:
pass
logger.warning(f"[chunk{chunk_idx}] JSON解析失败: {line[:200]}")
proc.wait()
stderr_thread.join(timeout=5)
stderr_text = "".join(stderr_lines)
if stderr_text.strip():
logger.warning(f"[docker exec chunk{chunk_idx}] stderr:\n{stderr_text[:800]}")
try:
os.remove(tmp.name)
except OSError:
pass
if proc.returncode != 0:
raise RuntimeError(f"[子进程{chunk_idx}] 失败 (exit={proc.returncode})")
raise RuntimeError(
f"[docker exec chunk{chunk_idx}] 失败 (exit={proc.returncode}): "
f"{stderr_text[:300]}"
)
return chunk_results
@@ -412,74 +506,6 @@ def _generate_batch_maps(models: list, config: dict, batch_key: str,
raise RuntimeError(f"所有模型均失败 ({fail_count}张): {first_err}")
def _generate_maps_subprocess(chunk_models: list, config: dict, chunk_idx: int) -> list:
"""单个 QGIS 子进程,处理一批模板,返回结果列表"""
import json
import subprocess
import tempfile
from app.services.qgis.qgis_env import (
get_qgis_python_path, get_runner_script, build_qgis_subprocess_env,
)
request_data = json.dumps(
{"config": config, "models": chunk_models},
ensure_ascii=False,
)
tmp_json = tempfile.NamedTemporaryFile(
suffix=".json", delete=False, mode="w", encoding="utf-8"
)
tmp_json.write(request_data)
tmp_json.close()
try:
from config import settings
qgis_root = getattr(settings, "QGIS_ROOT", "D:/QGIS")
python_path = get_qgis_python_path(qgis_root)
if not python_path:
raise RuntimeError("未找到 QGIS Python 3.12 解释器")
runner_script = get_runner_script()
cmd = [python_path, runner_script, tmp_json.name]
logger.info(f"[子进程{chunk_idx}] 启动: {len(chunk_models)} 张图")
result = subprocess.run(
cmd,
capture_output=True,
timeout=600,
env=build_qgis_subprocess_env(qgis_root),
)
finally:
try:
os.remove(tmp_json.name)
except OSError:
pass
stdout_text = result.stdout.decode("utf-8", errors="replace").strip()
parsed_output = None
if stdout_text:
for line in reversed(stdout_text.split("\n")):
line = line.strip()
if line.startswith("{"):
try:
parsed_output = json.loads(line)
break
except json.JSONDecodeError:
continue
if parsed_output is not None:
results = parsed_output.get("results", [])
ok = sum(1 for r in results if "error" not in r)
logger.info(f"[子进程{chunk_idx}] 完成: {ok}/{len(results)}")
return results
if result.returncode != 0:
stderr_text = result.stderr.decode("utf-8", errors="replace").strip()
raise RuntimeError(f"[子进程{chunk_idx}] 失败: {stderr_text[:200]}")
logger.warning(f"[子进程{chunk_idx}] 无输出")
return []
# ============================================================
# file_path 数据库记录
+8 -2
View File
@@ -82,12 +82,18 @@ GPKG_SUBDIR = "app/data/gpkg"
def get_gpkg_dir(project_root: str = None) -> str:
"""获取 GPKG 目录绝对路径"""
"""获取 GPKG 目录绝对路径(从配置读取,避免硬编码)"""
try:
from config import settings
gpkg_subdir = getattr(settings, "QGIS_GPKG_DIR", None) or GPKG_SUBDIR
except Exception:
gpkg_subdir = GPKG_SUBDIR
if project_root is None:
project_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
)
gpkg_dir = os.path.join(project_root, GPKG_SUBDIR)
gpkg_dir = os.path.join(project_root, gpkg_subdir)
return os.path.normpath(gpkg_dir).replace("\\", "/")
+12 -8
View File
@@ -22,17 +22,21 @@ async def lifespan(app: FastAPI):
get_earthquake_model()
logger.info("DBN模型预加载完成")
# 检测 QGIS 子进程环境(产图时按需启动子进程)
qgis_root = getattr(settings, "QGIS_ROOT", None)
if qgis_root:
# 检测 Docker QGIS 容器是否可用
try:
from app.services.qgis.qgis_env import is_qgis_available
if is_qgis_available(qgis_root):
logger.info("QGIS 环境检测通过")
import subprocess
from app.services.qgis.qgis_env import get_docker_container
container = get_docker_container()
result = subprocess.run(
["docker", "inspect", "--format={{.State.Running}}", container],
capture_output=True, text=True, timeout=5,
)
if result.stdout.strip() == "true":
logger.info(f"Docker QGIS 容器 '{container}' 运行中")
else:
logger.warning("QGIS 环境不可用,专题图功能降级")
logger.warning(f"Docker QGIS 容器 '{container}' 未运行,专题图功能降级")
except Exception as e:
logger.error(f"QGIS 环境检测失败: {e}")
logger.error(f"Docker QGIS 检测失败: {e}")
yield
+159
View File
@@ -0,0 +1,159 @@
"""
将主机端 GPKG 和模板文件预拷贝到 Docker 容器本地文件系统。
WSL2 9P 文件系统随机读取极慢(GPKG 62MB 耗时 6-10s,模板 ZIP 也慢),
拷贝到容器内 /data/ 后读取仅需 ~0.5s。
用法:
python app/script/copy_data_to_container.py [--container qgis-server] [--dry-run] [--only gpkg|template]
前置条件:
Docker 容器已启动(docker start qgis-server
"""
import argparse
import os
import subprocess
import sys
import time
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
def _load_config():
"""从 settings.toml 读取配置"""
try:
from config import settings
return {
"gpkg_subdir": getattr(settings, "QGIS_GPKG_DIR", "app/data/gpkg"),
"template_subdir": "app/data/template",
"container": getattr(settings, "QGIS_DOCKER_CONTAINER", "qgis-server"),
"container_gpkg": getattr(settings, "QGIS_DOCKER_GPKG_DIR", "/data/gpkg"),
"container_template": getattr(settings, "QGIS_DOCKER_TEMPLATE_DIR", "/data/template"),
}
except ImportError:
pass
# fallback: 解析 TOML
cfg = {
"gpkg_subdir": "app/data/gpkg",
"template_subdir": "app/data/template",
"container": "qgis-server",
"container_gpkg": "/data/gpkg",
"container_template": "/data/template",
}
toml_path = PROJECT_ROOT / "settings.toml"
if toml_path.exists():
for line in toml_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
for key, prefix in [
("gpkg_subdir", "QGIS_GPKG_DIR"),
("container", "QGIS_DOCKER_CONTAINER"),
("container_gpkg", "QGIS_DOCKER_GPKG_DIR"),
("container_template", "QGIS_DOCKER_TEMPLATE_DIR"),
]:
if line.startswith(prefix) and "=" in line:
cfg[key] = line.split("=", 1)[1].strip().strip('"').strip("'")
return cfg
def _run(cmd, timeout=10, **kwargs):
"""subprocess.run 封装,统一 UTF-8 编码,避免 Windows GBK 报错"""
result = subprocess.run(
cmd, capture_output=True, timeout=timeout,
encoding="utf-8", errors="replace", **kwargs,
)
return result
def _check_container(container: str):
"""检查容器是否运行"""
result = _run(["docker", "inspect", "--format={{.State.Running}}", container], timeout=5)
if (result.stdout or "").strip() != "true":
raise RuntimeError(f"容器 {container} 未运行,请先 docker start {container}")
def _copy_dir_to_container(host_dir: Path, container: str, container_dest: str, label: str, dry_run: bool):
"""拷贝单个目录到容器"""
if not host_dir.is_dir():
print(f" [{label}] 目录不存在,跳过: {host_dir}")
return 0
files = list(host_dir.rglob("*"))
files = [f for f in files if f.is_file()]
if not files:
print(f" [{label}] 目录为空,跳过: {host_dir}")
return 0
total_size = sum(f.stat().st_size for f in files)
print(f" [{label}] 主机目录: {host_dir}")
print(f" 文件数: {len(files)}, 总大小: {total_size / 1024 / 1024:.1f} MB")
print(f" 容器目标: {container}:{container_dest}")
if dry_run:
print(f" [dry-run] 跳过")
return 0
# 确保目标目录存在
parent_dir = container_dest.rsplit("/", 1)[0]
_run(["docker", "exec", container, "mkdir", "-p", parent_dir])
# 清理旧目录
_run(["docker", "exec", container, "rm", "-rf", container_dest])
# docker cp
t0 = time.time()
result = _run(["docker", "cp", str(host_dir), f"{container}:{container_dest}"], timeout=120)
elapsed = time.time() - t0
if result.returncode != 0:
raise RuntimeError(f"[{label}] docker cp 失败: {result.stderr}")
# 验证
verify = _run(["docker", "exec", container, "find", container_dest, "-type", "f"])
stdout = (verify.stdout or "").strip()
count = len([l for l in stdout.splitlines() if l.strip()])
print(f" 拷贝完成: {elapsed:.1f}s, 容器内 {count} 个文件")
return elapsed
def copy_to_container(container: str = None, dry_run: bool = False, only: str = None):
"""将 GPKG 和模板拷贝到容器本地文件系统"""
cfg = _load_config()
if container is None:
container = cfg["container"]
_check_container(container)
t_total = time.time()
print(f"=== 预拷贝静态数据到容器 {container} ===\n")
if only != "template":
host_gpkg = PROJECT_ROOT / cfg["gpkg_subdir"]
_copy_dir_to_container(host_gpkg, container, cfg["container_gpkg"], "GPKG", dry_run)
print()
if only != "gpkg":
host_template = PROJECT_ROOT / cfg["template_subdir"]
_copy_dir_to_container(host_template, container, cfg["container_template"], "模板", dry_run)
elapsed = time.time() - t_total
print(f"\n=== 总耗时: {elapsed:.1f}s ===")
def main():
parser = argparse.ArgumentParser(description="将 GPKG 和模板预拷贝到 Docker 容器本地 FS")
parser.add_argument("--container", default=None, help="Docker 容器名称")
parser.add_argument("--dry-run", action="store_true", help="仅显示信息,不实际拷贝")
parser.add_argument("--only", choices=["gpkg", "template"], help="只拷贝指定类型")
args = parser.parse_args()
try:
copy_to_container(container=args.container, dry_run=args.dry_run, only=args.only)
except Exception as e:
print(f"错误: {e}", file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()
+2 -2
View File
@@ -27,5 +27,5 @@ __all__ = [
# 不在顶层导入 QGIS 依赖模块,避免主进程崩溃
# 使用方式:
# from app.services.qgis.map_service import MapService (仅在子进程中)
# from app.services.qgis.qgis_env import is_qgis_available (主进程安全)
# from app.services.qgis.map_service import MapService (仅在容器子进程中)
# from app.services.qgis.qgis_env import get_docker_container (主进程检查容器状态)
+73 -3
View File
@@ -26,24 +26,81 @@ import time
# 环境初始化(必须在 QGIS import 之前)
# ============================================================
QGIS_ROOT = os.environ.get("QGIS_ROOT", "D:/QGIS")
IS_WINDOWS = sys.platform == "win32"
def _detect_qgis_root():
"""自动检测 QGIS 安装根目录(跨平台)"""
root = os.environ.get("QGIS_ROOT")
if root and os.path.isdir(root):
return root
if IS_WINDOWS:
for candidate in ["D:/QGIS", "C:/OSGeo4W", "C:/QGIS"]:
if os.path.isdir(candidate):
return candidate
else:
for candidate in ["/usr", "/opt/QGIS", "/home/QGIS", "/usr/local"]:
if _is_valid_qgis_root_linux(candidate):
return candidate
return None
def _is_valid_qgis_root_linux(path: str) -> bool:
"""验证 Linux 路径是否为有效的 QGIS 安装根目录"""
if not os.path.isdir(path):
return False
return (
os.path.isdir(os.path.join(path, "share", "qgis"))
or any(os.path.isdir(os.path.join(path, "apps", n)) for n in ("qgis-ltr", "qgis"))
or os.path.isfile(os.path.join(path, "bin", "qgis"))
)
QGIS_ROOT = _detect_qgis_root()
def _detect_qgis_app_dir():
"""自动检测 QGIS 应用目录(跨平台)"""
if QGIS_ROOT is None:
raise RuntimeError("未检测到 QGIS 安装目录,请设置 QGIS_ROOT 环境变量")
if IS_WINDOWS:
for name in ("qgis-ltr", "qgis"):
d = os.path.join(QGIS_ROOT, "apps", name)
if os.path.isdir(d):
return d
raise RuntimeError(f"未找到 QGIS 应用目录: {QGIS_ROOT}\\apps\\qgis-ltr 或 qgis")
raise RuntimeError(
f"未找到 QGIS 应用目录: {QGIS_ROOT}\\apps\\qgis-ltr 或 qgis"
)
# Linux: 先检查独立安装器的 app 目录
for name in ("qgis-ltr", "qgis"):
d = os.path.join(QGIS_ROOT, "apps", name)
if os.path.isdir(d):
return d
# Linux apt 安装:返回 prefix 目录
for prefix in ("/usr", "/opt/QGIS"):
if os.path.isdir(os.path.join(prefix, "share", "qgis")):
return prefix
raise RuntimeError(
f"未找到 QGIS 应用目录: {QGIS_ROOT} 下无 apps/qgis-ltr 或 share/qgis"
)
def _setup_environment():
"""设置 QGIS 运行所需的环境变量和 DLL 搜索路径(跨平台)"""
os.environ["PYTHONUTF8"] = "1"
os.environ["GDAL_FILENAME_IS_UTF8"] = "YES"
os.environ["VSI_CACHE"] = "TRUE"
os.environ["VSI_CACHE_SIZE"] = "1000000"
if sys.platform == "win32":
if not IS_WINDOWS:
return
# ── Windows: DLL 预加载 ──
import ctypes
qgis_app_dir = _detect_qgis_app_dir()
for dll_dir in [
@@ -75,13 +132,26 @@ def _setup_environment():
def _setup_python_path():
"""将项目根目录和 QGIS Python 路径加入 sys.path(跨平台)"""
project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
if project_root not in sys.path:
sys.path.insert(0, project_root)
qgis_app_dir = _detect_qgis_app_dir()
if IS_WINDOWS:
qgis_python = os.path.join(qgis_app_dir, "python")
if os.path.isdir(qgis_python) and qgis_python not in sys.path:
sys.path.insert(0, qgis_python)
else:
for p in [
"/usr/lib/python3/dist-packages",
"/usr/lib/python3.10/dist-packages",
"/usr/lib/python3.11/dist-packages",
"/usr/lib/python3.12/dist-packages",
]:
if os.path.isdir(p) and p not in sys.path:
sys.path.insert(0, p)
def _scan_templates() -> list[str]:
+228 -167
View File
@@ -1,202 +1,263 @@
"""
QGIS 环境检测与子进程配置模块。
QGIS 环境配置模块 — Docker 模式
主进程运行在 Python 3.10,无法直接加载 QGIS 的 Python 3.12 C 扩展。
所有 QGIS 操作通过 subprocess 调用 QGIS Python 3.12 执行
所有 QGIS 操作通过 docker exec 在容器内执行,
主进程(Python 3.10)无需安装 QGIS 或处理 DLL 加载
容器内使用 qgis/qgis 官方镜像(QGIS 3.x + Python 3)。
本模块提供:
- get_qgis_python_path(): 检测 QGIS Python 3.12 解释器路径
- build_qgis_subprocess_env(): 构建子进程完整环境变量(替代 bat 包装器)
- is_qgis_available(): 检查 QGIS 是否可用
- get_runner_script(): 获取 qgis_runner.py 的路径
- get_docker_container(): 获取 Docker 容器名称
- get_docker_project_dir(): 获取容器内项目根目录
- build_docker_exec_cmd(): 构建 docker exec 命令
- build_docker_volume_mounts(): 构建 docker run 卷挂载参数
- get_runner_script(): 获取 qgis_runner.py 的绝对路径
- get_container_python_path(): 获取容器内 Python 解释器路径
"""
import os
import platform
from pathlib import Path
from app.config.paths import get_logger
logger = get_logger("qgis.env")
IS_WINDOWS = platform.system() == "Windows"
def get_qgis_python_path(qgis_root: str = None) -> str | None:
# ============================================================
# Docker 容器配置
# ============================================================
def get_docker_container() -> str:
"""获取 Docker 容器名称/ID"""
try:
from config import settings
container = getattr(settings, "QGIS_DOCKER_CONTAINER", "") or ""
if container.strip():
return container.strip()
except Exception:
pass
# 环境变量覆盖
env_container = os.environ.get("QGIS_DOCKER_CONTAINER", "")
if env_container.strip():
return env_container.strip()
return "qgis-server"
def get_docker_project_dir() -> str:
"""获取容器内项目根目录(代码挂载目标路径)"""
try:
from config import settings
project_dir = getattr(settings, "QGIS_DOCKER_PROJECT_DIR", "") or ""
if project_dir.strip():
return project_dir.strip()
except Exception:
pass
env_dir = os.environ.get("QGIS_DOCKER_PROJECT_DIR", "")
if env_dir.strip():
return env_dir.strip()
return "/app"
def get_container_python_path() -> str:
"""获取容器内 Python 解释器路径"""
try:
from config import settings
python_path = getattr(settings, "QGIS_DOCKER_PYTHON", "") or ""
if python_path.strip():
return python_path.strip()
except Exception:
pass
env_path = os.environ.get("QGIS_DOCKER_PYTHON", "")
if env_path.strip():
return env_path.strip()
# qgis/qgis 官方镜像默认路径
return "/usr/bin/python3"
# ============================================================
# docker exec 命令构建
# ============================================================
def build_docker_exec_cmd(python_path: str, runner_script: str, json_file: str) -> list:
"""
检测 QGIS 自带的 Python 3.12 解释器路径
构建 docker exec 命令
Windows: {QGIS_ROOT}/apps/Python312/python3.exe
Args:
python_path: 容器内 Python 路径(如 /usr/bin/python3
runner_script: 容器内 runner 脚本路径
json_file: 容器内 JSON 请求文件路径
Returns:
解释器绝对路径,不存在则返回 None
docker exec 命令列表
"""
if qgis_root is None:
qgis_root = _detect_qgis_root()
if qgis_root is None:
return None
container = get_docker_container()
project_dir = get_docker_project_dir()
import platform
if platform.system() == "Windows":
for name in ("python3.exe", "python.exe"):
candidate = os.path.join(qgis_root, "apps", "Python312", name)
if os.path.isfile(candidate):
logger.info(f"检测到 QGIS Python: {candidate}")
return candidate
logger.warning(f"未找到 QGIS Python 3.12")
return None
else:
import shutil
# Qt 平台插件(从配置读取,避免硬编码)
try:
from config import settings
qt_platform = getattr(settings, "QGIS_DOCKER_QT_PLATFORM", "offscreen")
except Exception:
qt_platform = "offscreen"
# 优先检查环境变量
env_python = os.environ.get("QGIS_PYTHON_PATH")
if env_python and os.path.isfile(env_python):
logger.info(f"QGIS_PYTHON_PATH 指定: {env_python}")
return env_python
# 搜索标准 Linux QGIS 安装路径
linux_paths = [
"/usr/bin/qgis", # Ubuntu/Debian apt 安装
"/usr/bin/qgis.bin",
"/opt/QGIS/apps/Python312/bin/python3", # 独立安装器
"/opt/QGIS/apps/Python3/bin/python3",
"/usr/libexec/qgis/python3", # Fedora/RHEL
cmd = [
"docker", "exec",
"-w", project_dir,
"-e", f"QT_QPA_PLATFORM={qt_platform}",
container,
python_path, runner_script, json_file,
]
for candidate in linux_paths:
if os.path.isfile(candidate):
logger.info(f"检测到 QGIS 可执行文件: {candidate}")
return candidate
# 回退到系统 Python(如果 qgis.core 可导入)
sys_python = shutil.which("python3") or shutil.which("python")
if sys_python:
logger.info(f"Linux 环境,使用系统 Python: {sys_python}")
return sys_python
return None
return cmd
def build_qgis_subprocess_env(qgis_root: str = None) -> dict:
# ============================================================
# docker run 卷挂载(首次启动容器用)
# ============================================================
def get_host_file_store() -> str:
"""获取主机端文件输出目录"""
try:
from config import settings
fs = getattr(settings, "FILE_STORE_DIR", "") or ""
if fs.strip():
return fs.strip().replace("\\", "/")
except Exception:
pass
return "G:/files"
def get_container_file_store() -> str:
"""获取容器内文件输出目录(Linux 路径)"""
# 挂载时主机 G:/files → 容器内 /files
host = get_host_file_store()
# 取最后一段目录名作为容器内路径
tail = host.rstrip("/").rsplit("/", 1)[-1]
return f"/{tail}"
def map_host_to_container(path: str) -> str:
"""将主机路径映射为容器内路径。
优先匹配 file_storeG:/files → /files),其次匹配项目根目录(F:/project/... → /app)。
"""
构建 QGIS Python 3.12 子进程的完整环境变量(替代 bat 包装器)。
normalized = path.replace("\\", "/")
策略:
1. 从 os.environ 继承,移除 venv 污染(PYTHONPATH/VIRTUAL_ENV 等)
2. 将 QGIS 的 bin 目录前置到 PATHQt5→qgis→gdal 顺序,与原 bat 一致)
3. 设置 PYTHONPATH / QGIS_PREFIX_PATH / QT_PLUGIN_PATH / GDAL_DATA / PROJ_DATA
4. 子进程 _setup_environment() 负责 os.add_dll_directory + ctypes 预加载
# 1. file_store 路径映射
host_fs = get_host_file_store()
container_fs = get_container_file_store()
if normalized.lower().startswith(host_fs.lower()):
return container_fs + normalized[len(host_fs):]
# 2. 项目根目录映射
project_root = str(Path(__file__).parent.parent.parent.parent).replace("\\", "/")
project_dir = get_docker_project_dir()
if normalized.lower().startswith(project_root.lower()):
return project_dir + normalized[len(project_root):]
return normalized
def map_container_to_host(path: str) -> str:
"""将容器内路径映射回主机路径"""
host = get_host_file_store()
container = get_container_file_store()
if path.startswith(container):
return host + path[len(container):]
return path
def map_template_to_container(host_path: str) -> str:
"""将主机端模板路径映射到容器内本地路径(预拷贝后绕过 9P)。
主机: F:/project/xian/xian_algorithm_new/app/data/template/rainfall/xxx.qgz
容器: /data/template/rainfall/xxx.qgz
"""
import platform
try:
from config import settings
container_tpl = getattr(settings, "QGIS_DOCKER_TEMPLATE_DIR", "") or ""
except Exception:
container_tpl = ""
# 继承当前环境,清理 venv 污染
env = dict(os.environ)
venv_root = env.get("VIRTUAL_ENV", "").lower()
if not container_tpl:
# fallback: 用通用映射(走 9P,慢)
return map_host_to_container(host_path)
for key in (
"PYTHONPATH",
"PYTHONHOME",
"VIRTUAL_ENV",
"PYTHONDONTWRITEBYTECODE",
):
env.pop(key, None)
normalized = host_path.replace("\\", "/").lower()
# 找 "app/data/template/" 后面的部分
marker = "app/data/template/"
idx = normalized.find(marker)
if idx >= 0:
relative = host_path.replace("\\", "/")[idx + len(marker):]
return f"{container_tpl.rstrip('/')}/{relative}"
if venv_root:
path_parts = env.get("PATH", "").split(";")
clean = []
for p in path_parts:
if not p or venv_root in p.lower():
continue
clean.append(p)
env["PATH"] = ";".join(clean)
# 检测 QGIS 安装路径
if qgis_root is None:
qgis_root = _detect_qgis_root() or "D:/QGIS"
env["QGIS_ROOT"] = qgis_root
if platform.system() != "Windows":
return env
qgis_app_dir = os.path.join(qgis_root, "apps", "qgis-ltr")
if not os.path.isdir(qgis_app_dir):
qgis_app_dir = os.path.join(qgis_root, "apps", "qgis")
if not os.path.isdir(qgis_app_dir):
raise RuntimeError(
f"未找到 QGIS 应用目录: {qgis_root}\\apps\\qgis-ltr 或 qgis"
)
# 前置 QGIS bin 目录到 PATHQt5 必须在最前面,QGIS 依赖它)
qgis_bin = os.path.join(qgis_app_dir, "bin")
qt5_bin = os.path.join(qgis_root, "apps", "Qt5", "bin")
gdal_lib = os.path.join(qgis_root, "apps", "gdal", "lib")
qt5_plugins = os.path.join(qgis_root, "apps", "Qt5", "plugins")
qtplugins = os.path.join(qgis_app_dir, "qtplugins")
qgis_python_dir = os.path.join(qgis_app_dir, "python")
env["PATH"] = f"{qt5_bin};{qgis_bin};{gdal_lib};{env.get('PATH', '')}"
# QGIS 核心环境变量(与 bat 包装器保持一致)
env["PYTHONPATH"] = qgis_python_dir
env["QGIS_PREFIX_PATH"] = qgis_app_dir
env["QT_PLUGIN_PATH"] = f"{qtplugins};{qt5_plugins}"
# GDAL / PROJ 数据目录(避免系统旧版 proj.db 干扰)
gdal_data = os.path.join(qgis_root, "apps", "gdal", "share", "gdal")
if os.path.isdir(gdal_data):
env["GDAL_DATA"] = gdal_data
for pd in (
os.path.join(qgis_app_dir, "share", "proj"),
os.path.join(qgis_root, "share", "proj"),
os.path.join(qgis_root, "apps", "gdal", "share", "proj"),
):
if os.path.isfile(os.path.join(pd, "proj.db")):
env["PROJ_DATA"] = pd
break
# UTF-8 / GDAL 编码辅助变量
env["PYTHONUTF8"] = "1"
env["GDAL_FILENAME_IS_UTF8"] = "YES"
env["VSI_CACHE"] = "TRUE"
env["VSI_CACHE_SIZE"] = "1000000"
logger.debug(
f"QGIS subprocess env built: root={qgis_root}, app={qgis_app_dir}, "
f"PATH prefixed with qgis_bin/qt5_bin/gdal_lib"
)
return env
return map_host_to_container(host_path)
def is_qgis_available(qgis_root: str = None) -> bool:
"""检查 QGIS 环境是否可用"""
return get_qgis_python_path(qgis_root) is not None
def build_docker_volume_mounts() -> list:
"""
构建 Docker 卷挂载参数列表(用于 docker run)。
挂载内容:
1. 项目代码目录 → 容器内 /app(只读)
2. 输出文件目录 → 容器内 /files(读写)
"""
project_root = str(Path(__file__).parent.parent.parent.parent)
host_file_store = get_host_file_store()
container_file_store = get_container_file_store()
project_dir = get_docker_project_dir()
mounts = [
f"{project_root}:{project_dir}:ro",
f"{host_file_store}:{container_file_store}",
]
return mounts
def build_docker_run_cmd(image: str = None) -> list:
"""
构建 docker run 启动命令(首次部署用)。
Returns:
docker run 命令列表
"""
if image is None:
try:
from config import settings
image = getattr(settings, "QGIS_DOCKER_IMAGE", "") or ""
if not image.strip():
image = "qgis/qgis:latest"
except Exception:
image = "qgis/qgis:latest"
# 容器保活命令(从配置读取)
try:
from config import settings
keep_alive = getattr(settings, "QGIS_DOCKER_KEEP_ALIVE", "sleep infinity")
except Exception:
keep_alive = "sleep infinity"
container = get_docker_container()
mounts = build_docker_volume_mounts()
cmd = [
"docker", "run", "-d",
"--name", container,
"--restart", "unless-stopped",
]
for m in mounts:
cmd.extend(["-v", m])
cmd.append(image)
# 保活命令
cmd.extend(keep_alive.split())
return cmd
# ============================================================
# Runner 脚本路径
# ============================================================
def get_runner_script() -> str:
"""获取 qgis_runner.py 的绝对路径"""
"""获取 qgis_runner.py 的绝对路径(主机端)"""
return str(Path(__file__).parent / "qgis_runner.py")
def _detect_qgis_root() -> str | None:
"""
自动检测 QGIS 安装根目录。
优先级:
1. 环境变量 QGIS_ROOT
2. Windows 默认路径 D:/QGIS
3. Linux 常见路径
"""
env_root = os.environ.get("QGIS_ROOT")
if env_root and os.path.isdir(env_root):
return env_root
import platform
if platform.system() == "Windows":
for candidate in ["D:/QGIS", "C:/OSGeo4W", "C:/QGIS"]:
if os.path.isdir(candidate):
logger.info(f"检测到 QGIS 根目录: {candidate}")
return candidate
else:
for candidate in ["/usr", "/opt/QGIS", "/home/QGIS"]:
if os.path.isdir(candidate):
logger.info(f"检测到 QGIS 根目录: {candidate}")
return candidate
logger.warning("未检测到 QGIS 安装目录")
return None
+32 -63
View File
@@ -1,9 +1,8 @@
#!/usr/bin/env python3
"""
QGIS 专题图生成子进程入口。
QGIS 专题图生成子进程入口 — Docker 容器内运行
由主进程 (Python 3.10) 通过 subprocess 调用,
运行在 QGIS 自带的 Python 3.12 环境中。
由主进程通过 docker exec 调用,运行在 QGIS Docker 容器内。
支持两种模式:
- 批量模式(推荐):单次启动 QgsApplication,顺序处理多个模板
@@ -23,102 +22,70 @@ import sys
import time
# ============================================================
# 环境初始化(必须在任何 QGIS/Qt import 之前
# 环境初始化(Docker 容器内,QGIS 通过 apt 安装在 /usr
# ============================================================
QGIS_ROOT = os.environ.get("QGIS_ROOT", "D:/QGIS")
def _detect_qgis_app_dir():
"""自动检测 QGIS 应用目录(qgis-ltr 或 qgis"""
for name in ("qgis-ltr", "qgis"):
d = os.path.join(QGIS_ROOT, "apps", name)
if os.path.isdir(d):
return d
raise RuntimeError(
f"未找到 QGIS 应用目录: {QGIS_ROOT}\\apps\\qgis-ltr 或 qgis"
)
def _setup_python_path():
"""将项目根目录和 QGIS Python 路径加入 sys.path"""
"""将项目根目录和 QGIS dist-packages 加入 sys.path"""
project_root = os.path.dirname(
os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
)
if project_root not in sys.path:
sys.path.insert(0, project_root)
qgis_app_dir = _detect_qgis_app_dir()
qgis_python = os.path.join(qgis_app_dir, "python")
if os.path.isdir(qgis_python) and qgis_python not in sys.path:
sys.path.insert(0, qgis_python)
# QGIS Python 包路径(从配置读取,避免硬编码)
try:
from config import settings
pythonpath = getattr(settings, "QGIS_DOCKER_PYTHONPATH", None) or []
except Exception:
pythonpath = []
for p in pythonpath:
if os.path.isdir(p) and p not in sys.path:
sys.path.insert(0, p)
def _setup_environment():
"""设置 QGIS 运行所需的环境变量和 DLL 搜索路径。
QGIS_PREFIX_PATH / QT_PLUGIN_PATH / GDAL_DATA / PROJ_DATA
已由主进程通过 build_qgis_subprocess_env() 设置,子进程不重复设。
这里注册 DLL 搜索目录并预加载核心 DLL 以确保正确加载顺序。
"""
"""设置 QGIS 运行所需的环境变量"""
os.environ["PYTHONUTF8"] = "1"
os.environ["GDAL_FILENAME_IS_UTF8"] = "YES"
os.environ["VSI_CACHE"] = "TRUE"
os.environ["VSI_CACHE_SIZE"] = "1000000"
if sys.platform == "win32":
import ctypes
qgis_app_dir = _detect_qgis_app_dir()
dll_dirs = [
os.path.join(qgis_app_dir, "bin"),
os.path.join(QGIS_ROOT, "apps", "Qt5", "bin"),
os.path.join(QGIS_ROOT, "apps", "gdal", "lib"),
]
for dll_dir in dll_dirs:
if os.path.isdir(dll_dir):
os.add_dll_directory(dll_dir)
# 预加载核心 DLL —— 强制从 QGIS 目录加载,防止系统 PATH 中同名 DLL 干扰
_preload_dlls = [
"qgis_core.dll", "qgispython.dll",
"Qt5Core.dll", "Qt5Gui.dll", "Qt5Widgets.dll",
"Qt5Network.dll", "Qt5Svg.dll", "Qt5Xml.dll",
"Qt5Concurrent.dll", "Qt5PrintSupport.dll",
]
for dll_dir in dll_dirs:
for dll_name in _preload_dlls:
dll_path = os.path.join(dll_dir, dll_name)
if os.path.isfile(dll_path):
try:
ctypes.WinDLL(dll_path)
except OSError:
pass
# ============================================================
# 主逻辑
# ============================================================
def _init_qgis():
"""初始化 QgsApplication只做一次"""
"""初始化 QgsApplication从配置读取 prefixPath,避免硬编码"""
from qgis.core import QgsApplication
qgis_app_dir = _detect_qgis_app_dir()
QgsApplication.setPrefixPath(qgis_app_dir, True)
try:
from config import settings
prefix_path = getattr(settings, "QGIS_DOCKER_PREFIX_PATH", "/usr")
except Exception:
prefix_path = "/usr"
QgsApplication.setPrefixPath(prefix_path, True)
qgs_app = QgsApplication([], False)
qgs_app.initQgis()
return qgs_app
def _process_single(service, model):
"""处理单个模板,返回结果 dict"""
"""处理单个模板,返回结果 dict。成功/失败均输出 PROGRESS 行。"""
name = service.generate(model)
result = {"name": name, "output": model["outFile"]}
# ★ 实时进度:每完成一张图就输出到 stdout
print(f"PROGRESS:{json.dumps(result, ensure_ascii=False)}", flush=True)
return result
def _emit_progress(result: dict):
"""输出 PROGRESS 行(成功或失败均调用)"""
print(f"PROGRESS:{json.dumps(result, ensure_ascii=False)}", flush=True)
def main():
t_start = time.time()
@@ -166,7 +133,9 @@ def main():
f"耗时 {elapsed:.1f}s — {error_msg}",
file=sys.stderr,
)
results.append({"name": model.get("name", ""), "output": "", "error": error_msg})
fail_result = {"name": model.get("name", ""), "output": "", "error": error_msg}
_emit_progress(fail_result)
results.append(fail_result)
# 输出结果
if len(models) == 1 and not request.get("models"):
+6 -2
View File
@@ -39,12 +39,13 @@ class TemplateModifier:
"""
将 GPKG 文件路径对应的 <provider ...>postgres</provider> 改为 ogr。
策略:按 <maplayer> 块处理,若块内 datasource 是文件路径(盘符开头),
策略:按 <maplayer> 块处理,若块内 datasource 是文件路径(盘符开头或 Linux 绝对路径),
则该块内的 provider 改为 ogr。避免跨层误改。
"""
maplayer_re = re.compile(r'(<maplayer[^>]*>.*?</maplayer>)', re.DOTALL)
provider_re = re.compile(r'(<provider[^>]*>)postgres(</provider>)')
file_ds_re = re.compile(r'<datasource>([A-Za-z]:/[^<]+)</datasource>')
# 匹配 Windows 盘符路径 (G:/...) 或 Linux 绝对路径 (/app/...)
file_ds_re = re.compile(r'<datasource>(?:[A-Za-z]:/|/)[^<]*</datasource>')
def _fix_layer(m):
layer_xml = m.group(1)
@@ -60,8 +61,11 @@ class TemplateModifier:
has_static = bool(self._static_map)
if not override and not has_static:
logger.info(f"模板无需修改(无 override 且 static_map 为空),直接返回: {template_path}")
return template_path
logger.info(f"模板修改: static_map有{len(self._static_map)}条, override={'' if override else ''}")
orig = override.get("original") if override else None
actual = override.get("actual") if override else None
+19 -15
View File
@@ -8,9 +8,13 @@ RAIN_STATION_GRID_DIR = "/xian/rainfall/grid/images/:id"
REDIS_RAIN_STATION_GRID_KEY = "xian:rainfall:rain_station_grid"
REDIS_RAIN_STATION_IDENTIFIER_KEY = "xian:rainfall:rain_station_identifier"
PREDICT_PROBABILITY_THRESHOLD = 50
# 静态底图 GeoPackage 目录(相对于项目根目录)
# 静态底图 GeoPackage 目录
QGIS_GPKG_DIR = "app/data/gpkg"
# 专题图输出子目录(相对于 FILE_STORE_DIR
# 容器内 GPKG 本地路径
QGIS_DOCKER_GPKG_DIR = "/data/gpkg"
# 容器内模板本地路径
QGIS_DOCKER_TEMPLATE_DIR = "/data/template"
# 专题图输出子目录
QGIS_OUTPUT_DIR = "xian/qgis/map/:eventType/:inferenceId"
# 专题图默认参数
QGIS_DEFAULTS_MAP_LAYOUT = "A4"
@@ -19,10 +23,21 @@ QGIS_DEFAULTS_ZOOM_VALUE = "5"
QGIS_DEFAULTS_MAP_UNIT = "制图单位:西安市应急管理局"
# 专题图DPI
QGIS_EXPORT_DPI = 200
# 并行子进程数(每进程独立 QGIS 实例)
# 并行 docker exec 子进程数
QGIS_PARALLEL_WORKERS = 4
# 最大并发请求数(防止多人同时触发资源耗尽)
# 最大并发请求数
QGIS_MAX_CONCURRENT = 2
# ============================================================
# Docker QGIS 配置
# ============================================================
# 容器名称/ID
QGIS_DOCKER_CONTAINER = "qgis-server"
# 容器内项目代码挂载目标路径
QGIS_DOCKER_PROJECT_DIR = "/app"
# 容器内 Python 解释器路径
QGIS_DOCKER_PYTHON = "/usr/bin/python3"
# Docker 镜像名称
QGIS_DOCKER_IMAGE = "qgis/qgis:3.44.11"
# 优先产出模板
QGIS_PRIORITY_TEMPLATES = ["暴雨地质灾害风险区分布图", "暴雨滑坡潜在隐患点及人口分布图", "暴雨山洪潜在隐患点及人口分布图", "暴雨泥石流潜在隐患点及人口分布图", "暴雨内涝潜在隐患点及人口分布图", "暴雨避难场所分布图"]
@@ -80,13 +95,6 @@ REDIS_DB = 0
# 文件路径配置
# ============================================================
FILE_STORE_DIR = "G:/files"
# ============================================================
# QGIS 配置
# ============================================================
QGIS_ROOT = "D:/QGIS"
# 专题图输出子目录
QGIS_OUTPUT_DIR = "xian/qgis/map/:eventType/:inferenceId"
QGIS_DEFAULTS_MAP_UNIT = "制图单位:西安市应急管理局"
# ============================================================
# 生产环境
@@ -121,7 +129,3 @@ REDIS_DB = 0
# 文件路径配置
# ============================================================
FILE_STORE_DIR = "/data"
# ============================================================
# QGIS 配置
# ============================================================
QGIS_ROOT = "/home/QGIS"