添加 DevOps 自动化部署系统
- 创建 Git 仓库监听器 (monitor.py) - 创建部署执行器 (deployer.py) - 添加 Java 和 UI 部署脚本 - 添加配置文件和文档 - 支持自动构建、部署和提交子模块更新
This commit is contained in:
孙小云
parent
f054526a09
commit
790e182322
|
|
@ -0,0 +1,209 @@
|
|||
# RuoYi-Cloud DevOps 自动化部署系统
|
||||
|
||||
## 简介
|
||||
|
||||
这是一个基于 Python 的 Git 仓库监听和自动化部署系统，用于监听多个 Git 子仓库的提交，并自动触发构建和部署流程。
|
||||
|
||||
## 功能特性
|
||||
|
||||
- ✅ 监听多个 Git 仓库的指定分支
|
||||
- ✅ 检测到新提交自动触发部署
|
||||
- ✅ 支持 Java 和 Node.js 项目构建
|
||||
- ✅ 自动复制构建产物到 Docker 目录
|
||||
- ✅ 自动执行 Docker Compose 部署
|
||||
- ✅ 自动提交子模块更新到主仓库
|
||||
- ✅ 完整的日志记录
|
||||
|
||||
## 目录结构
|
||||
|
||||
```
|
||||
.devops/
|
||||
├── config.yaml # 配置文件
|
||||
├── monitor.py # Git 监听器
|
||||
├── deployer.py # 部署执行器
|
||||
├── start.sh # 启动脚本
|
||||
├── scripts/ # 部署脚本目录
|
||||
│ ├── deploy-java.sh # Java 服务部署脚本
|
||||
│ ├── deploy-ui.sh # UI 部署脚本
|
||||
│ └── deploy-common.sh # 通用函数库
|
||||
└── logs/ # 日志目录
|
||||
```
|
||||
|
||||
## 环境要求
|
||||
|
||||
- Python 3.6+
|
||||
- Git
|
||||
- Docker & Docker Compose
|
||||
- Maven (Java 项目)
|
||||
- Node.js & npm (前端项目)
|
||||
|
||||
## 安装依赖
|
||||
|
||||
```bash
|
||||
# 安装 Python 依赖
|
||||
pip3 install PyYAML
|
||||
```
|
||||
|
||||
## 配置说明
|
||||
|
||||
编辑 `.devops/config.yaml` 文件：
|
||||
|
||||
### 1. 仓库配置
|
||||
|
||||
每个仓库需要配置以下信息：
|
||||
|
||||
```yaml
|
||||
repositories:
|
||||
- name: ruoyi-auth # 仓库名称
|
||||
url: http://... # Git 仓库 URL
|
||||
branch: main # 监听的分支
|
||||
path: ruoyi-auth # 在主仓库中的路径
|
||||
type: java # 项目类型 (java/nodejs)
|
||||
deploy_script: deploy-java.sh # 部署脚本
|
||||
build_commands: # 构建命令列表
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar # 构建产物路径
|
||||
docker_path: docker/ruoyi/auth/jar # Docker 目录
|
||||
docker_service: ruoyi-auth # Docker Compose 服务名
|
||||
```
|
||||
|
||||
### 2. 监听配置
|
||||
|
||||
```yaml
|
||||
monitor:
|
||||
poll_interval: 60 # 轮询间隔（秒）
|
||||
enabled_repos: [] # 监听的仓库列表（空=全部）
|
||||
```
|
||||
|
||||
### 3. 部署配置
|
||||
|
||||
```yaml
|
||||
deploy:
|
||||
docker_compose_path: ./docker/docker-compose.yml
|
||||
auto_commit: true # 是否自动提交子模块更新
|
||||
commit_message: "自动更新子模块: {repo_name} 到最新版本"
|
||||
```
|
||||
|
||||
## 使用方法
|
||||
|
||||
### 1. 启动持续监听
|
||||
|
||||
```bash
|
||||
# 使用启动脚本
|
||||
bash .devops/start.sh
|
||||
|
||||
# 或直接运行 Python
|
||||
python3 .devops/monitor.py
|
||||
```
|
||||
|
||||
### 2. 执行一次检查
|
||||
|
||||
```bash
|
||||
python3 .devops/monitor.py --once
|
||||
```
|
||||
|
||||
### 3. 指定配置文件
|
||||
|
||||
```bash
|
||||
python3 .devops/monitor.py --config /path/to/config.yaml
|
||||
```
|
||||
|
||||
## 工作流程
|
||||
|
||||
1. **监听器检测到子仓库有新提交**
|
||||
2. **进入 runtime 目录**
|
||||
3. **克隆/更新主仓库**（如果不存在）
|
||||
4. **初始化所有子模块**
|
||||
5. **进入特定子模块目录**
|
||||
6. **拉取最新代码**
|
||||
7. **执行构建命令**（mvn/npm）
|
||||
8. **复制构建产物到 docker 目录**
|
||||
9. **执行 docker-compose 部署**
|
||||
10. **回到主仓库，提交子模块更新**
|
||||
11. **推送到远程主仓库**
|
||||
|
||||
## 日志
|
||||
|
||||
日志文件位置：`.devops/logs/devops.log`
|
||||
|
||||
日志级别可在配置文件中设置：
|
||||
- DEBUG：详细调试信息
|
||||
- INFO：一般信息（默认）
|
||||
- WARNING：警告信息
|
||||
- ERROR：错误信息
|
||||
|
||||
## 部署脚本说明
|
||||
|
||||
### deploy-java.sh
|
||||
|
||||
Java 服务部署脚本，执行以下操作：
|
||||
1. 重新构建 Docker 镜像
|
||||
2. 启动 Docker 服务
|
||||
3. 检查服务状态
|
||||
|
||||
### deploy-ui.sh
|
||||
|
||||
前端 UI 部署脚本，执行以下操作：
|
||||
1. 重启 Nginx 服务
|
||||
2. 检查服务状态
|
||||
|
||||
### deploy-common.sh
|
||||
|
||||
通用函数库，提供：
|
||||
- 日志输出函数
|
||||
- 命令检查函数
|
||||
- Docker 检查函数
|
||||
- 健康检查函数
|
||||
|
||||
## 故障排查
|
||||
|
||||
### 1. 监听器无法启动
|
||||
|
||||
检查 Python 依赖是否安装：
|
||||
```bash
|
||||
pip3 list | grep PyYAML
|
||||
```
|
||||
|
||||
### 2. 构建失败
|
||||
|
||||
查看日志文件：
|
||||
```bash
|
||||
tail -f .devops/logs/devops.log
|
||||
```
|
||||
|
||||
### 3. Docker 部署失败
|
||||
|
||||
检查 Docker 服务是否运行：
|
||||
```bash
|
||||
docker info
|
||||
docker-compose ps
|
||||
```
|
||||
|
||||
### 4. Git 操作失败
|
||||
|
||||
检查 Git 配置和权限：
|
||||
```bash
|
||||
git config --list
|
||||
ssh -T git@your-git-server
|
||||
```
|
||||
|
||||
## 注意事项
|
||||
|
||||
1. 确保有足够的磁盘空间用于 runtime 目录
|
||||
2. 首次运行会克隆主仓库，可能需要较长时间
|
||||
3. 构建过程可能需要较长时间，请耐心等待
|
||||
4. 建议在测试环境先验证配置正确性
|
||||
5. 定期清理 runtime 目录和日志文件
|
||||
|
||||
## 扩展功能
|
||||
|
||||
可以根据需要添加：
|
||||
- 钉钉/企业微信通知
|
||||
- Web 管理界面
|
||||
- 部署回滚功能
|
||||
- 健康检查和监控
|
||||
- 多环境支持
|
||||
|
||||
## 许可证
|
||||
|
||||
本项目遵循 MIT 许可证
|
||||
|
|
@ -0,0 +1,210 @@
|
|||
# DevOps 自动化部署配置文件
|
||||
|
||||
# Git 仓库配置
|
||||
repositories:
|
||||
# 认证服务
|
||||
- name: ruoyi-auth
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-auth.git
|
||||
branch: main
|
||||
path: ruoyi-auth
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/auth/jar
|
||||
docker_service: ruoyi-auth
|
||||
|
||||
# 网关服务
|
||||
- name: ruoyi-gateway
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-gateway.git
|
||||
branch: main
|
||||
path: ruoyi-gateway
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/gateway/jar
|
||||
docker_service: ruoyi-gateway
|
||||
|
||||
# 前端UI
|
||||
- name: ruoyi-ui
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-ui.git
|
||||
branch: main
|
||||
path: ruoyi-ui
|
||||
type: nodejs
|
||||
deploy_script: deploy-ui.sh
|
||||
build_commands:
|
||||
- npm install
|
||||
- npm run build:prod
|
||||
artifact_path: dist
|
||||
docker_path: docker/nginx/html/dist
|
||||
docker_service: ruoyi-nginx
|
||||
|
||||
# 系统服务
|
||||
- name: ruoyi-system
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-system.git
|
||||
branch: main
|
||||
path: ruoyi-modules/ruoyi-system
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/system/jar
|
||||
docker_service: ruoyi-modules-system
|
||||
|
||||
# 文件服务
|
||||
- name: ruoyi-file
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-file.git
|
||||
branch: main
|
||||
path: ruoyi-modules/ruoyi-file
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/file/jar
|
||||
docker_service: ruoyi-modules-file
|
||||
|
||||
# 代码生成
|
||||
- name: ruoyi-gen
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-gen.git
|
||||
branch: main
|
||||
path: ruoyi-modules/ruoyi-gen
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/gen/jar
|
||||
docker_service: ruoyi-modules-gen
|
||||
|
||||
# 定时任务
|
||||
- name: ruoyi-job
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-job.git
|
||||
branch: main
|
||||
path: ruoyi-modules/ruoyi-job
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/job/jar
|
||||
docker_service: ruoyi-modules-job
|
||||
|
||||
# 监控服务
|
||||
- name: ruoyi-monitor
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-ruoyi-visual.git
|
||||
branch: main
|
||||
path: ruoyi-visual/ruoyi-monitor
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/visual/monitor/jar
|
||||
docker_service: ruoyi-visual-monitor
|
||||
|
||||
# 设备服务
|
||||
- name: tuoheng-device
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-tuoheng-device.git
|
||||
branch: main
|
||||
path: ruoyi-modules/tuoheng-device
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/device/jar
|
||||
docker_service: tuoheng-modules-device
|
||||
|
||||
# 审批服务
|
||||
- name: tuoheng-approval
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-tuoheng-approval.git
|
||||
branch: main
|
||||
path: ruoyi-modules/tuoheng-approval
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/approval/jar
|
||||
docker_service: tuoheng-modules-approval
|
||||
|
||||
# 航线服务
|
||||
- name: tuoheng-airline
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-tuoheng-airline.git
|
||||
branch: main
|
||||
path: ruoyi-modules/tuoheng-airline
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/airline/jar
|
||||
docker_service: tuoheng-modules-airline
|
||||
|
||||
# 任务服务
|
||||
- name: tuoheng-task
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-tuoheng-task.git
|
||||
branch: main
|
||||
path: ruoyi-modules/tuoheng-task
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/task/jar
|
||||
docker_service: tuoheng-modules-task
|
||||
|
||||
# FMS服务
|
||||
- name: tuoheng-fms
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-tuoheng-fms.git
|
||||
branch: main
|
||||
path: ruoyi-modules/tuoheng-fms
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/fms/jar
|
||||
docker_service: tuoheng-modules-fms
|
||||
|
||||
# 媒体服务
|
||||
- name: tuoheng-media
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-tuoheng-media.git
|
||||
branch: main
|
||||
path: ruoyi-modules/tuoheng-media
|
||||
type: java
|
||||
deploy_script: deploy-java.sh
|
||||
build_commands:
|
||||
- mvn clean package -DskipTests
|
||||
artifact_path: target/*.jar
|
||||
docker_path: docker/ruoyi/modules/media/jar
|
||||
docker_service: tuoheng-modules-media
|
||||
|
||||
# 主仓库配置
|
||||
main_repository:
|
||||
url: http://th.local.t-aaron.com:13000/THENG/a-cloud-all.git
|
||||
branch: main
|
||||
runtime_path: ./runtime
|
||||
|
||||
# 监听配置
|
||||
monitor:
|
||||
poll_interval: 60 # 轮询间隔（秒）
|
||||
enabled_repos: [] # 空数组表示监听所有仓库，或指定具体仓库名称列表
|
||||
|
||||
# 部署配置
|
||||
deploy:
|
||||
docker_compose_path: ./docker/docker-compose.yml
|
||||
auto_commit: true # 是否自动提交子模块更新到主仓库
|
||||
commit_message: "自动更新子模块: {repo_name} 到最新版本"
|
||||
|
||||
# 日志配置
|
||||
logging:
|
||||
level: INFO # DEBUG, INFO, WARNING, ERROR
|
||||
file: .devops/logs/devops.log
|
||||
max_size: 10485760 # 10MB
|
||||
backup_count: 5
|
||||
|
|
@ -0,0 +1,275 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
部署执行器
|
||||
负责执行具体的部署任务
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import logging
|
||||
import subprocess
|
||||
import shutil
|
||||
import glob
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class Deployer:
|
||||
"""部署执行器"""
|
||||
|
||||
def __init__(self, config):
|
||||
"""初始化部署器"""
|
||||
self.config = config
|
||||
self.logger = logging.getLogger('Deployer')
|
||||
self.runtime_path = Path(config['main_repository']['runtime_path'])
|
||||
self.main_repo_url = config['main_repository']['url']
|
||||
self.main_repo_branch = config['main_repository']['branch']
|
||||
|
||||
def run_command(self, cmd, cwd=None, timeout=600):
|
||||
"""执行命令"""
|
||||
self.logger.info(f"执行命令: {cmd}")
|
||||
try:
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
shell=True,
|
||||
cwd=cwd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=timeout
|
||||
)
|
||||
|
||||
if result.stdout:
|
||||
self.logger.debug(f"输出: {result.stdout}")
|
||||
|
||||
if result.returncode != 0:
|
||||
self.logger.error(f"命令执行失败: {result.stderr}")
|
||||
return False
|
||||
|
||||
return True
|
||||
except subprocess.TimeoutExpired:
|
||||
self.logger.error(f"命令执行超时: {cmd}")
|
||||
return False
|
||||
except Exception as e:
|
||||
self.logger.error(f"命令执行异常: {e}")
|
||||
return False
|
||||
|
||||
def ensure_main_repo(self):
|
||||
"""确保主仓库存在并是最新的"""
|
||||
repo_path = self.runtime_path / 'a-cloud-all'
|
||||
|
||||
if not repo_path.exists():
|
||||
self.logger.info("主仓库不存在，开始克隆...")
|
||||
self.runtime_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
cmd = f"git clone --recurse-submodules {self.main_repo_url} {repo_path}"
|
||||
if not self.run_command(cmd, cwd=self.runtime_path):
|
||||
self.logger.error("克隆主仓库失败")
|
||||
return False
|
||||
|
||||
self.logger.info("主仓库克隆成功")
|
||||
else:
|
||||
self.logger.info("主仓库已存在，更新代码...")
|
||||
|
||||
# 切换到指定分支
|
||||
if not self.run_command(f"git checkout {self.main_repo_branch}", cwd=repo_path):
|
||||
return False
|
||||
|
||||
# 拉取最新代码
|
||||
if not self.run_command("git pull", cwd=repo_path):
|
||||
return False
|
||||
|
||||
# 更新所有子模块
|
||||
if not self.run_command("git submodule update --init --recursive", cwd=repo_path):
|
||||
return False
|
||||
|
||||
self.logger.info("主仓库更新成功")
|
||||
|
||||
return True
|
||||
|
||||
def update_submodule(self, repo_config):
|
||||
"""更新指定的子模块"""
|
||||
repo_path = self.runtime_path / 'a-cloud-all'
|
||||
submodule_path = repo_path / repo_config['path']
|
||||
|
||||
self.logger.info(f"更新子模块: {repo_config['name']}")
|
||||
|
||||
# 进入子模块目录
|
||||
if not submodule_path.exists():
|
||||
self.logger.error(f"子模块目录不存在: {submodule_path}")
|
||||
return False
|
||||
|
||||
# 切换到指定分支
|
||||
branch = repo_config['branch']
|
||||
if not self.run_command(f"git checkout {branch}", cwd=submodule_path):
|
||||
return False
|
||||
|
||||
# 拉取最新代码
|
||||
if not self.run_command("git pull origin " + branch, cwd=submodule_path):
|
||||
return False
|
||||
|
||||
self.logger.info(f"子模块更新成功: {repo_config['name']}")
|
||||
return True
|
||||
|
||||
def build_project(self, repo_config):
|
||||
"""构建项目"""
|
||||
repo_path = self.runtime_path / 'a-cloud-all'
|
||||
submodule_path = repo_path / repo_config['path']
|
||||
|
||||
self.logger.info(f"开始构建: {repo_config['name']}")
|
||||
|
||||
# 执行构建命令
|
||||
for cmd in repo_config['build_commands']:
|
||||
self.logger.info(f"执行构建命令: {cmd}")
|
||||
if not self.run_command(cmd, cwd=submodule_path, timeout=1800):
|
||||
self.logger.error(f"构建失败: {cmd}")
|
||||
return False
|
||||
|
||||
self.logger.info(f"构建成功: {repo_config['name']}")
|
||||
return True
|
||||
|
||||
def copy_artifacts(self, repo_config):
|
||||
"""复制构建产物到 docker 目录"""
|
||||
repo_path = self.runtime_path / 'a-cloud-all'
|
||||
submodule_path = repo_path / repo_config['path']
|
||||
|
||||
self.logger.info(f"复制构建产物: {repo_config['name']}")
|
||||
|
||||
# 获取构建产物路径
|
||||
artifact_pattern = submodule_path / repo_config['artifact_path']
|
||||
artifacts = glob.glob(str(artifact_pattern))
|
||||
|
||||
if not artifacts:
|
||||
self.logger.error(f"未找到构建产物: {artifact_pattern}")
|
||||
return False
|
||||
|
||||
# 目标目录
|
||||
docker_path = repo_path / repo_config['docker_path']
|
||||
docker_path.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# 复制文件
|
||||
for artifact in artifacts:
|
||||
artifact_path = Path(artifact)
|
||||
if artifact_path.is_file():
|
||||
dest = docker_path / artifact_path.name
|
||||
shutil.copy2(artifact, dest)
|
||||
self.logger.info(f"复制文件: {artifact_path.name}")
|
||||
elif artifact_path.is_dir():
|
||||
# 如果是目录（如 dist），清空目标目录后复制
|
||||
if docker_path.exists():
|
||||
for item in docker_path.iterdir():
|
||||
if item.name != '.gitkeep':
|
||||
if item.is_dir():
|
||||
shutil.rmtree(item)
|
||||
else:
|
||||
item.unlink()
|
||||
shutil.copytree(artifact, docker_path, dirs_exist_ok=True)
|
||||
self.logger.info(f"复制目录: {artifact_path.name}")
|
||||
|
||||
self.logger.info("构建产物复制完成")
|
||||
return True
|
||||
|
||||
def run_deploy_script(self, repo_config):
|
||||
"""执行部署脚本"""
|
||||
repo_path = self.runtime_path / 'a-cloud-all'
|
||||
script_name = repo_config['deploy_script']
|
||||
script_path = repo_path / '.devops' / 'scripts' / script_name
|
||||
|
||||
if not script_path.exists():
|
||||
self.logger.error(f"部署脚本不存在: {script_path}")
|
||||
return False
|
||||
|
||||
self.logger.info(f"执行部署脚本: {script_name}")
|
||||
|
||||
# 准备脚本参数
|
||||
docker_service = repo_config.get('docker_service', '')
|
||||
docker_compose_path = self.config['deploy']['docker_compose_path']
|
||||
|
||||
# 执行脚本
|
||||
cmd = f"bash {script_path} {repo_config['name']} {docker_service} {docker_compose_path}"
|
||||
|
||||
if not self.run_command(cmd, cwd=repo_path, timeout=600):
|
||||
self.logger.error("部署脚本执行失败")
|
||||
return False
|
||||
|
||||
self.logger.info("部署脚本执行成功")
|
||||
return True
|
||||
|
||||
def commit_submodule_update(self, repo_config):
|
||||
"""提交子模块更新到主仓库"""
|
||||
if not self.config['deploy'].get('auto_commit', False):
|
||||
self.logger.info("自动提交已禁用，跳过")
|
||||
return True
|
||||
|
||||
repo_path = self.runtime_path / 'a-cloud-all'
|
||||
|
||||
self.logger.info("提交子模块更新到主仓库")
|
||||
|
||||
# 添加子模块更改
|
||||
submodule_path = repo_config['path']
|
||||
if not self.run_command(f"git add {submodule_path}", cwd=repo_path):
|
||||
return False
|
||||
|
||||
# 检查是否有更改
|
||||
result = subprocess.run(
|
||||
"git diff --cached --quiet",
|
||||
shell=True,
|
||||
cwd=repo_path
|
||||
)
|
||||
|
||||
if result.returncode == 0:
|
||||
self.logger.info("没有需要提交的更改")
|
||||
return True
|
||||
|
||||
# 提交更改
|
||||
commit_msg = self.config['deploy']['commit_message'].format(
|
||||
repo_name=repo_config['name']
|
||||
)
|
||||
|
||||
if not self.run_command(f'git commit -m "{commit_msg}"', cwd=repo_path):
|
||||
return False
|
||||
|
||||
# 推送到远程
|
||||
if not self.run_command(f"git push origin {self.main_repo_branch}", cwd=repo_path):
|
||||
self.logger.warning("推送失败，但部署已完成")
|
||||
return True
|
||||
|
||||
self.logger.info("子模块更新已提交并推送")
|
||||
return True
|
||||
|
||||
def deploy(self, repo_config):
|
||||
"""执行完整的部署流程"""
|
||||
self.logger.info(f"=" * 60)
|
||||
self.logger.info(f"开始部署: {repo_config['name']}")
|
||||
self.logger.info(f"=" * 60)
|
||||
|
||||
try:
|
||||
# 1. 确保主仓库存在
|
||||
if not self.ensure_main_repo():
|
||||
return False
|
||||
|
||||
# 2. 更新子模块
|
||||
if not self.update_submodule(repo_config):
|
||||
return False
|
||||
|
||||
# 3. 构建项目
|
||||
if not self.build_project(repo_config):
|
||||
return False
|
||||
|
||||
# 4. 复制构建产物
|
||||
if not self.copy_artifacts(repo_config):
|
||||
return False
|
||||
|
||||
# 5. 执行部署脚本
|
||||
if not self.run_deploy_script(repo_config):
|
||||
return False
|
||||
|
||||
# 6. 提交子模块更新
|
||||
if not self.commit_submodule_update(repo_config):
|
||||
return False
|
||||
|
||||
self.logger.info(f"部署完成: {repo_config['name']}")
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"部署过程中发生异常: {e}", exc_info=True)
|
||||
return False
|
||||
|
|
@ -0,0 +1,189 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Git 仓库监听器
|
||||
监听多个 Git 仓库的指定分支，检测到新提交时触发部署
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import yaml
|
||||
import logging
|
||||
import subprocess
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
# 添加当前目录到 Python 路径
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from deployer import Deployer
|
||||
|
||||
|
||||
class GitMonitor:
|
||||
"""Git 仓库监听器"""
|
||||
|
||||
def __init__(self, config_path='.devops/config.yaml'):
|
||||
"""初始化监听器"""
|
||||
self.config_path = config_path
|
||||
self.config = self._load_config()
|
||||
self._setup_logging()
|
||||
self.deployer = Deployer(self.config)
|
||||
self.last_commits = {} # 存储每个仓库的最后一次提交 hash
|
||||
|
||||
self.logger.info("Git 监听器初始化完成")
|
||||
|
||||
def _load_config(self):
|
||||
"""加载配置文件"""
|
||||
with open(self.config_path, 'r', encoding='utf-8') as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
def _setup_logging(self):
|
||||
"""设置日志"""
|
||||
log_config = self.config.get('logging', {})
|
||||
log_level = getattr(logging, log_config.get('level', 'INFO'))
|
||||
log_file = log_config.get('file', '.devops/logs/devops.log')
|
||||
|
||||
# 确保日志目录存在
|
||||
os.makedirs(os.path.dirname(log_file), exist_ok=True)
|
||||
|
||||
# 配置日志格式
|
||||
formatter = logging.Formatter(
|
||||
'%(asctime)s - %(name)s - %(levelname)s - %(message)s'
|
||||
)
|
||||
|
||||
# 文件处理器
|
||||
file_handler = logging.FileHandler(log_file, encoding='utf-8')
|
||||
file_handler.setFormatter(formatter)
|
||||
|
||||
# 控制台处理器
|
||||
console_handler = logging.StreamHandler()
|
||||
console_handler.setFormatter(formatter)
|
||||
|
||||
# 配置 logger
|
||||
self.logger = logging.getLogger('GitMonitor')
|
||||
self.logger.setLevel(log_level)
|
||||
self.logger.addHandler(file_handler)
|
||||
self.logger.addHandler(console_handler)
|
||||
|
||||
def get_remote_commit(self, repo_url, branch):
|
||||
"""获取远程仓库的最新提交 hash"""
|
||||
try:
|
||||
cmd = f"git ls-remote {repo_url} refs/heads/{branch}"
|
||||
result = subprocess.run(
|
||||
cmd, shell=True, capture_output=True, text=True, timeout=30
|
||||
)
|
||||
if result.returncode == 0 and result.stdout:
|
||||
commit_hash = result.stdout.split()[0]
|
||||
return commit_hash
|
||||
return None
|
||||
except Exception as e:
|
||||
self.logger.error(f"获取远程提交失败 {repo_url}: {e}")
|
||||
return None
|
||||
|
||||
def check_repository(self, repo_config):
|
||||
"""检查单个仓库是否有新提交"""
|
||||
repo_name = repo_config['name']
|
||||
repo_url = repo_config['url']
|
||||
branch = repo_config['branch']
|
||||
|
||||
self.logger.debug(f"检查仓库: {repo_name}")
|
||||
|
||||
# 获取最新提交
|
||||
current_commit = self.get_remote_commit(repo_url, branch)
|
||||
if not current_commit:
|
||||
self.logger.warning(f"无法获取 {repo_name} 的最新提交")
|
||||
return False
|
||||
|
||||
# 检查是否有新提交
|
||||
last_commit = self.last_commits.get(repo_name)
|
||||
if last_commit is None:
|
||||
# 首次检查，记录当前提交
|
||||
self.last_commits[repo_name] = current_commit
|
||||
self.logger.info(f"初始化 {repo_name} 提交记录: {current_commit[:8]}")
|
||||
return False
|
||||
|
||||
if current_commit != last_commit:
|
||||
self.logger.info(
|
||||
f"检测到 {repo_name} 有新提交: {last_commit[:8]} -> {current_commit[:8]}"
|
||||
)
|
||||
self.last_commits[repo_name] = current_commit
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
def get_enabled_repos(self):
|
||||
"""获取需要监听的仓库列表"""
|
||||
enabled = self.config['monitor'].get('enabled_repos', [])
|
||||
all_repos = self.config['repositories']
|
||||
|
||||
if not enabled:
|
||||
# 空列表表示监听所有仓库
|
||||
return all_repos
|
||||
|
||||
# 只返回启用的仓库
|
||||
return [repo for repo in all_repos if repo['name'] in enabled]
|
||||
|
||||
def run_once(self):
|
||||
"""执行一次检查"""
|
||||
repos = self.get_enabled_repos()
|
||||
self.logger.info(f"开始检查 {len(repos)} 个仓库...")
|
||||
|
||||
for repo_config in repos:
|
||||
try:
|
||||
if self.check_repository(repo_config):
|
||||
# 检测到新提交，触发部署
|
||||
self.logger.info(f"触发部署: {repo_config['name']}")
|
||||
success = self.deployer.deploy(repo_config)
|
||||
|
||||
if success:
|
||||
self.logger.info(f"部署成功: {repo_config['name']}")
|
||||
else:
|
||||
self.logger.error(f"部署失败: {repo_config['name']}")
|
||||
except Exception as e:
|
||||
self.logger.error(f"处理仓库 {repo_config['name']} 时出错: {e}", exc_info=True)
|
||||
|
||||
def run(self):
|
||||
"""持续监听运行"""
|
||||
poll_interval = self.config['monitor']['poll_interval']
|
||||
self.logger.info(f"开始监听 Git 仓库，轮询间隔: {poll_interval} 秒")
|
||||
self.logger.info("按 Ctrl+C 停止监听")
|
||||
|
||||
try:
|
||||
while True:
|
||||
self.run_once()
|
||||
time.sleep(poll_interval)
|
||||
except KeyboardInterrupt:
|
||||
self.logger.info("收到停止信号，退出监听")
|
||||
except Exception as e:
|
||||
self.logger.error(f"监听过程中发生错误: {e}", exc_info=True)
|
||||
|
||||
|
||||
def main():
|
||||
"""主函数"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description='Git 仓库监听器')
|
||||
parser.add_argument(
|
||||
'--config',
|
||||
default='.devops/config.yaml',
|
||||
help='配置文件路径'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--once',
|
||||
action='store_true',
|
||||
help='只执行一次检查，不持续监听'
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
monitor = GitMonitor(args.config)
|
||||
|
||||
if args.once:
|
||||
monitor.run_once()
|
||||
else:
|
||||
monitor.run()
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
#!/bin/bash
|
||||
# 通用函数库
|
||||
# 提供部署脚本使用的通用函数
|
||||
|
||||
# 颜色输出
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 日志函数
|
||||
log_info() {
|
||||
echo -e "${GREEN}[INFO]${NC} $1"
|
||||
}
|
||||
|
||||
log_warn() {
|
||||
echo -e "${YELLOW}[WARN]${NC} $1"
|
||||
}
|
||||
|
||||
log_error() {
|
||||
echo -e "${RED}[ERROR]${NC} $1"
|
||||
}
|
||||
|
||||
# 检查命令是否存在
|
||||
check_command() {
|
||||
if ! command -v $1 &> /dev/null; then
|
||||
log_error "命令不存在: $1"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# 检查 Docker 服务是否运行
|
||||
check_docker() {
|
||||
if ! docker info &> /dev/null; then
|
||||
log_error "Docker 未运行"
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# 等待服务健康检查
|
||||
wait_for_healthy() {
|
||||
local service=$1
|
||||
local max_wait=${2:-60}
|
||||
local count=0
|
||||
|
||||
log_info "等待服务健康检查: $service"
|
||||
|
||||
while [ $count -lt $max_wait ]; do
|
||||
if docker-compose ps $service | grep -q "Up (healthy)"; then
|
||||
log_info "服务已就绪: $service"
|
||||
return 0
|
||||
fi
|
||||
sleep 2
|
||||
count=$((count + 2))
|
||||
done
|
||||
|
||||
log_warn "服务健康检查超时: $service"
|
||||
return 1
|
||||
}
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
#!/bin/bash
|
||||
# Java 服务部署脚本
|
||||
# 参数: $1=服务名称, $2=docker服务名, $3=docker-compose路径
|
||||
|
||||
set -e # 遇到错误立即退出
|
||||
|
||||
SERVICE_NAME=$1
|
||||
DOCKER_SERVICE=$2
|
||||
DOCKER_COMPOSE_PATH=$3
|
||||
|
||||
echo "=========================================="
|
||||
echo "部署 Java 服务: $SERVICE_NAME"
|
||||
echo "Docker 服务: $DOCKER_SERVICE"
|
||||
echo "=========================================="
|
||||
|
||||
# 检查参数
|
||||
if [ -z "$SERVICE_NAME" ] || [ -z "$DOCKER_SERVICE" ]; then
|
||||
echo "错误: 缺少必要参数"
|
||||
echo "用法: $0 <服务名称> <docker服务名> <docker-compose路径>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 获取脚本所在目录
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
|
||||
echo "项目根目录: $PROJECT_ROOT"
|
||||
|
||||
# 进入 docker 目录
|
||||
cd "$PROJECT_ROOT/docker"
|
||||
|
||||
echo "重新构建 Docker 镜像..."
|
||||
docker-compose build --no-cache "$DOCKER_SERVICE"
|
||||
|
||||
echo "启动服务..."
|
||||
docker-compose up -d "$DOCKER_SERVICE"
|
||||
|
||||
echo "等待服务启动..."
|
||||
sleep 5
|
||||
|
||||
echo "检查服务状态..."
|
||||
docker-compose ps "$DOCKER_SERVICE"
|
||||
|
||||
echo "=========================================="
|
||||
echo "部署完成: $SERVICE_NAME"
|
||||
echo "=========================================="
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
#!/bin/bash
|
||||
# UI 前端部署脚本
|
||||
# 参数: $1=服务名称, $2=docker服务名, $3=docker-compose路径
|
||||
|
||||
set -e # 遇到错误立即退出
|
||||
|
||||
SERVICE_NAME=$1
|
||||
DOCKER_SERVICE=$2
|
||||
DOCKER_COMPOSE_PATH=$3
|
||||
|
||||
echo "=========================================="
|
||||
echo "部署前端服务: $SERVICE_NAME"
|
||||
echo "Docker 服务: $DOCKER_SERVICE"
|
||||
echo "=========================================="
|
||||
|
||||
# 检查参数
|
||||
if [ -z "$SERVICE_NAME" ] || [ -z "$DOCKER_SERVICE" ]; then
|
||||
echo "错误: 缺少必要参数"
|
||||
echo "用法: $0 <服务名称> <docker服务名> <docker-compose路径>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 获取脚本所在目录
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
|
||||
echo "项目根目录: $PROJECT_ROOT"
|
||||
|
||||
# 进入 docker 目录
|
||||
cd "$PROJECT_ROOT/docker"
|
||||
|
||||
echo "重启 Nginx 服务..."
|
||||
docker-compose restart "$DOCKER_SERVICE"
|
||||
|
||||
echo "等待服务启动..."
|
||||
sleep 3
|
||||
|
||||
echo "检查服务状态..."
|
||||
docker-compose ps "$DOCKER_SERVICE"
|
||||
|
||||
echo "=========================================="
|
||||
echo "部署完成: $SERVICE_NAME"
|
||||
echo "=========================================="
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
#!/bin/bash
|
||||
# DevOps 监听器启动脚本
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
|
||||
echo "=========================================="
|
||||
echo "RuoYi-Cloud DevOps 自动化部署系统"
|
||||
echo "=========================================="
|
||||
|
||||
# 检查 Python 环境
|
||||
if ! command -v python3 &> /dev/null; then
|
||||
echo "错误: 未找到 python3"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 检查依赖
|
||||
echo "检查 Python 依赖..."
|
||||
pip3 list | grep -q PyYAML || pip3 install PyYAML
|
||||
|
||||
# 进入项目根目录
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# 启动监听器
|
||||
echo "启动 Git 监听器..."
|
||||
python3 .devops/monitor.py "$@"
|
||||
Loading…
Reference in New Issue