Add CI/CD pipeline, logging enhancements, and release management

- Create a GitHub Actions workflow for testing with Python 3.12 and 3.13.
- Update Makefile to include release management commands and pipeline checks.
- Document the CI/CD pipeline structure and usage in PIPELINE.md.
- Add structlog for structured logging and enhance logging utilities.
- Implement release management script for automated versioning and tagging.
- Modify logging configuration to support structured logging and improved formatting.
- Update dependencies in pyproject.toml and poetry.lock to include structlog.
- Enhance access logging in server and middleware to include structured data.
This commit is contained in:
Илья Глазунов
2025-09-03 00:13:21 +03:00
parent ff093b020f
commit 537b783726
16 changed files with 1054 additions and 241 deletions
+82
View File
@@ -0,0 +1,82 @@
# Automated Release Configuration
## How to use the pipeline
### 1. Linting (executed on every push)
```bash
# Triggers:
- push to any branch
- pull request to any branch
# Checks:
- Black (code formatting)
- isort (import sorting)
- flake8 (linting)
- mypy (type checking)
```
### 2. Tests (executed for dev, master, main)
```bash
# Triggers:
- push to branches: dev, master, main
- pull request to branches: dev, master, main
# Checks:
- pytest on Python 3.12 and 3.13
- coverage reports
```
### 3. Build and release (executed for tags)
```bash
# Triggers:
- push tag matching v*.*.*
- manual dispatch through Gitea interface
# Actions:
- Package build via Poetry
- Draft release creation
- Artifact upload (.whl and .tar.gz)
```
## Release workflow
1. **Release preparation:**
```bash
# Update version in pyproject.toml
poetry version patch # or minor/major
# Commit changes
git add pyproject.toml
git commit -m "bump version to $(poetry version -s)"
```
2. **Tag creation:**
```bash
# Create tag
git tag v$(poetry version -s)
git push origin v$(poetry version -s)
```
3. **Automatic process:**
- Pipeline starts automatically
- Linting and tests execute
- Package builds
- Draft release created
4. **Release finalization:**
- Go to Gitea interface
- Find created draft release
- Edit description according to template
- Publish release
## Environment variables
For correct pipeline operation, ensure:
- `GITHUB_TOKEN` - for release creation
- Repository permissions for release creation
## Customization
- Change Python versions in `test.yaml` if needed
- Add additional checks in `lint.yaml`
- Configure notifications in `pipeline.yaml`
+52
View File
@@ -0,0 +1,52 @@
name: Lint Code
run-name: ${{ gitea.actor }} started code linting
on:
push:
branches: ["*"]
pull_request:
branches: ["*"]
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.12'
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --with dev
- name: Run Black (Code formatting check)
run: poetry run black --check pyserve/
- name: Run isort (Import sorting check)
run: poetry run isort --check-only pyserve/
- name: Run flake8 (Linting)
run: poetry run flake8 pyserve/
- name: Run mypy (Type checking)
run: poetry run mypy pyserve/
- name: Lint completed
run: echo "Code passed all linting checks!"
+49
View File
@@ -0,0 +1,49 @@
name: CI/CD Pipeline
run-name: ${{ gitea.actor }} started full pipeline
on:
push:
branches: ["*"]
tags: ["v*"]
pull_request:
branches: ["dev", "master", "main"]
workflow_dispatch:
inputs:
version:
description: 'Release version (e.g., v0.6.1)'
required: false
default: ''
jobs:
lint:
uses: ./.gitea/workflows/lint.yaml
test:
if: github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main' || github.event_name == 'pull_request'
needs: lint
uses: ./.gitea/workflows/test.yaml
build-and-release:
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
needs: [lint, test]
uses: ./.gitea/workflows/release.yaml
with:
version: ${{ github.event.inputs.version }}
notify:
runs-on: ubuntu-latest
needs: [lint, test, build-and-release]
if: always()
steps:
- name: Pipeline Summary
run: |
echo "## Pipeline Execution Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "| Stage | Status |" >> $GITHUB_STEP_SUMMARY
echo "|-------|--------|" >> $GITHUB_STEP_SUMMARY
echo "| Linting | ${{ needs.lint.result == 'success' && 'Success' || 'Failed' }} |" >> $GITHUB_STEP_SUMMARY
echo "| Tests | ${{ needs.test.result == 'success' && 'Success' || needs.test.result == 'skipped' && 'Skipped' || 'Failed' }} |" >> $GITHUB_STEP_SUMMARY
echo "| Build and Release | ${{ needs.build-and-release.result == 'success' && 'Success' || needs.build-and-release.result == 'skipped' && 'Skipped' || 'Failed' }} |" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [[ "${{ needs.build-and-release.result }}" == "success" ]]; then
echo "**Draft release created!** Check and publish in Gitea interface." >> $GITHUB_STEP_SUMMARY
fi
-19
View File
@@ -1,19 +0,0 @@
name: Gitea Actions Demo
run-name: ${{ gitea.actor }} is testing out Gitea Actions 🚀
on: [push]
jobs:
Explore-Gitea-Actions:
runs-on: ubuntu-latest
steps:
- run: echo "🎉 The job was automatically triggered by a ${{ gitea.event_name }} event."
- run: echo "🐧 This job is now running on a ${{ runner.os }} server hosted by Gitea!"
- run: echo "🔎 The name of your branch is ${{ gitea.ref }} and your repository is ${{ gitea.repository }}."
- name: Check out repository code
uses: actions/checkout@v4
- run: echo "💡 The ${{ gitea.repository }} repository has been cloned to the runner."
- run: echo "🖥️ The workflow is now ready to test your code on the runner."
- name: List files in the repository
run: |
ls ${{ gitea.workspace }}
- run: echo "🍏 This job's status is ${{ job.status }}."
+155
View File
@@ -0,0 +1,155 @@
name: Build and Release
run-name: ${{ gitea.actor }} preparing release
on:
push:
tags:
- 'v*'
workflow_dispatch:
inputs:
version:
description: 'Release version (e.g., v0.6.1)'
required: true
default: 'v0.6.1'
jobs:
build:
runs-on: ubuntu-latest
needs: []
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.12'
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --with dev
- name: Build package
run: |
poetry build
echo "Package built successfully!"
ls -la dist/
- name: Generate changelog
id: changelog
run: |
echo "## What's new in this version" > CHANGELOG.md
echo "" >> CHANGELOG.md
echo "### New Features" >> CHANGELOG.md
echo "- Add description of new features" >> CHANGELOG.md
echo "" >> CHANGELOG.md
echo "### Bug Fixes" >> CHANGELOG.md
echo "- Add description of bug fixes" >> CHANGELOG.md
echo "" >> CHANGELOG.md
echo "### Technical Changes" >> CHANGELOG.md
echo "- Add description of technical improvements" >> CHANGELOG.md
echo "" >> CHANGELOG.md
echo "### Dependencies" >> CHANGELOG.md
echo "- Updated dependencies to latest versions" >> CHANGELOG.md
echo "" >> CHANGELOG.md
echo "---" >> CHANGELOG.md
echo "**Full Changelog:** https://gitea.example.com/${{ gitea.repository }}/compare/v0.5.0...${{ github.ref_name }}" >> CHANGELOG.md
- name: Upload build artifacts
uses: actions/upload-artifact@v3
with:
name: dist-${{ github.ref_name }}
path: |
dist/
CHANGELOG.md
- name: Build completed
run: echo "Build completed! Artifacts ready for release."
release:
runs-on: ubuntu-latest
needs: build
if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Download build artifacts
uses: actions/download-artifact@v3
with:
name: dist-${{ github.ref_name || github.event.inputs.version }}
- name: Read changelog
id: changelog
run: |
if [ -f CHANGELOG.md ]; then
echo "CHANGELOG<<EOF" >> $GITHUB_OUTPUT
cat CHANGELOG.md >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
else
echo "CHANGELOG=Automatically generated release" >> $GITHUB_OUTPUT
fi
- name: Create Release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.ref_name || github.event.inputs.version }}
release_name: PyServeX ${{ github.ref_name || github.event.inputs.version }}
body: |
${{ steps.changelog.outputs.CHANGELOG }}
## Installation
```bash
pip install pyserve==${{ github.ref_name || github.event.inputs.version }}
```
## Usage
```bash
pyserve --help
```
draft: true
prerelease: false
- name: Upload Release Asset (wheel)
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./dist/pyserve-*.whl
asset_name: pyserve-${{ github.ref_name || github.event.inputs.version }}.whl
asset_content_type: application/octet-stream
- name: Upload Release Asset (tarball)
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./dist/pyserve-*.tar.gz
asset_name: pyserve-${{ github.ref_name || github.event.inputs.version }}.tar.gz
asset_content_type: application/gzip
- name: Release created
run: echo "Draft release created! Check and publish in Gitea interface."
+56
View File
@@ -0,0 +1,56 @@
name: Run Tests
run-name: ${{ gitea.actor }} started tests
on:
push:
branches: ["dev", "master", "main"]
pull_request:
branches: ["dev", "master", "main"]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.12', '3.13']
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install Poetry
uses: snok/install-poetry@v1
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v3
with:
path: .venv
key: venv-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('**/poetry.lock') }}
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --with dev
- name: Run tests
run: poetry run pytest tests/ -v
- name: Run tests with coverage
run: poetry run pytest tests/ -v --cov=pyserve --cov-report=xml --cov-report=term
- name: Upload coverage to artifacts
uses: actions/upload-artifact@v3
with:
name: coverage-report-${{ matrix.python-version }}
path: coverage.xml
- name: Tests completed
run: echo "All tests passed successfully on Python ${{ matrix.python-version }}!"