Add CI/CD pipeline, logging enhancements, and release management
Lint Code / lint (push) Failing after 2m2s
CI/CD Pipeline / lint (push) Successful in 0s
Run Tests / test (3.12) (push) Successful in 54s
CI/CD Pipeline / build-and-release (push) Has been cancelled
CI/CD Pipeline / test (push) Has been cancelled
CI/CD Pipeline / notify (push) Has been cancelled
Run Tests / test (3.13) (push) Has been cancelled
Lint Code / lint (push) Failing after 2m2s
CI/CD Pipeline / lint (push) Successful in 0s
Run Tests / test (3.12) (push) Successful in 54s
CI/CD Pipeline / build-and-release (push) Has been cancelled
CI/CD Pipeline / test (push) Has been cancelled
CI/CD Pipeline / notify (push) Has been cancelled
Run Tests / test (3.13) (push) Has been cancelled
- Create a GitHub Actions workflow for testing with Python 3.12 and 3.13. - Update Makefile to include release management commands and pipeline checks. - Document the CI/CD pipeline structure and usage in PIPELINE.md. - Add structlog for structured logging and enhance logging utilities. - Implement release management script for automated versioning and tagging. - Modify logging configuration to support structured logging and improved formatting. - Update dependencies in pyproject.toml and poetry.lock to include structlog. - Enhance access logging in server and middleware to include structured data.
This commit is contained in:
+178
@@ -0,0 +1,178 @@
|
||||
# CI/CD Pipeline for PyServeX
|
||||
|
||||
This document describes the complete CI/CD pipeline for the PyServeX project, including linting, testing, building, and automated release creation.
|
||||
|
||||
## Pipeline Structure
|
||||
|
||||
### 1. Linting Stage (`lint.yaml`)
|
||||
**Triggers:**
|
||||
- Push to any branch
|
||||
- Pull request to any branch
|
||||
|
||||
**Checks:**
|
||||
- Black (code formatting)
|
||||
- isort (import sorting)
|
||||
- flake8 (code analysis)
|
||||
- mypy (type checking)
|
||||
|
||||
### 2. Testing Stage (`test.yaml`)
|
||||
**Triggers:**
|
||||
- Push to branches: `dev`, `master`, `main`
|
||||
- Pull request to branches: `dev`, `master`, `main`
|
||||
|
||||
**Checks:**
|
||||
- pytest on Python 3.12 and 3.13
|
||||
- Coverage report generation
|
||||
- Artifact storage with reports
|
||||
|
||||
### 3. Build and Release Stage (`release.yaml`)
|
||||
**Triggers:**
|
||||
- Push tag matching `v*.*.*`
|
||||
- Manual trigger through Gitea interface
|
||||
|
||||
**Actions:**
|
||||
- Package build via Poetry
|
||||
- Changelog generation
|
||||
- Draft release creation
|
||||
- Artifact upload (.whl and .tar.gz)
|
||||
|
||||
### 4. Main Pipeline (`pipeline.yaml`)
|
||||
Coordinates execution of all stages and provides results summary.
|
||||
|
||||
## How to Use
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Environment initialization
|
||||
make init
|
||||
|
||||
# Check all stages locally
|
||||
make pipeline-check
|
||||
|
||||
# Code formatting
|
||||
make format
|
||||
|
||||
# Run tests
|
||||
make test-cov
|
||||
```
|
||||
|
||||
### Creating a Release
|
||||
|
||||
#### Automatic method (recommended):
|
||||
```bash
|
||||
# Patch release (x.x.X)
|
||||
make release-patch
|
||||
|
||||
# Minor release (x.X.0)
|
||||
make release-minor
|
||||
|
||||
# Major release (X.0.0)
|
||||
make release-major
|
||||
```
|
||||
|
||||
#### Manual method:
|
||||
```bash
|
||||
# 1. Update version
|
||||
poetry version patch # or minor/major
|
||||
|
||||
# 2. Commit changes
|
||||
git add pyproject.toml
|
||||
git commit -m "bump version to $(poetry version -s)"
|
||||
|
||||
# 3. Create tag
|
||||
git tag v$(poetry version -s)
|
||||
|
||||
# 4. Push to server
|
||||
git push origin main
|
||||
git push origin v$(poetry version -s)
|
||||
```
|
||||
|
||||
## Working with Releases
|
||||
|
||||
### What happens automatically:
|
||||
1. **When tag** `v*.*.*` is created, pipeline starts
|
||||
2. **Linting executes** - code quality check
|
||||
3. **Tests run** - functionality verification
|
||||
4. **Package builds** - wheel and tarball creation
|
||||
5. **Draft release created** - automatic creation in Gitea
|
||||
|
||||
### What needs manual action:
|
||||
1. **Go to Gitea interface** in Releases section
|
||||
2. **Find created draft** release
|
||||
3. **Edit description** according to template in `RELEASE_TEMPLATE.md`
|
||||
4. **Publish release** (remove "Draft" status)
|
||||
|
||||
## Configuration
|
||||
|
||||
### Pipeline files:
|
||||
- `.gitea/workflows/lint.yaml` - Linting
|
||||
- `.gitea/workflows/test.yaml` - Testing
|
||||
- `.gitea/workflows/release.yaml` - Build and release
|
||||
- `.gitea/workflows/pipeline.yaml` - Main pipeline
|
||||
- `.gitea/RELEASE_TEMPLATE.md` - Release template
|
||||
|
||||
### Scripts:
|
||||
- `scripts/release.sh` - Automated release creation
|
||||
- `Makefile` - Development and release commands
|
||||
|
||||
## Environment Setup
|
||||
|
||||
### Gitea Actions variables:
|
||||
- `GITHUB_TOKEN` - for release creation (usually configured automatically)
|
||||
|
||||
### Access permissions:
|
||||
- Repository release creation permissions
|
||||
- Tag push permissions
|
||||
|
||||
## Monitoring
|
||||
|
||||
### Stage statuses:
|
||||
- **Success** - stage completed successfully
|
||||
- **Failure** - stage failed with error
|
||||
- **Skipped** - stage skipped (e.g., tests for non-listed branches)
|
||||
|
||||
### Artifacts:
|
||||
- **Coverage reports** - test coverage reports
|
||||
- **Build artifacts** - built packages (.whl, .tar.gz)
|
||||
- **Changelog** - automatically generated changelog
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common issues:
|
||||
|
||||
1. **Linting fails:**
|
||||
```bash
|
||||
make format # Auto-formatting
|
||||
make lint # Check issues
|
||||
```
|
||||
|
||||
2. **Tests fail:**
|
||||
```bash
|
||||
make test # Local test run
|
||||
```
|
||||
|
||||
3. **Build error:**
|
||||
```bash
|
||||
make clean # Clean temporary files
|
||||
make build # Rebuild
|
||||
```
|
||||
|
||||
4. **Tag already exists:**
|
||||
```bash
|
||||
git tag -d v1.0.0 # Delete locally
|
||||
git push origin :refs/tags/v1.0.0 # Delete on server
|
||||
```
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- [Poetry documentation](https://python-poetry.org/docs/)
|
||||
- [Gitea Actions documentation](https://docs.gitea.io/en-us/actions/)
|
||||
- [Pytest documentation](https://docs.pytest.org/)
|
||||
- [Black documentation](https://black.readthedocs.io/)
|
||||
|
||||
---
|
||||
|
||||
**Author:** Ilya Glazunov
|
||||
**Project:** PyServeX
|
||||
**Documentation version:** 1.0
|
||||
Reference in New Issue
Block a user