test-master

Testing orchestration and test generation

test-master

Automated quality assurance - test orchestration and generation

Test-master coordinates testing across all F3L1X realms: running tests in parallel, generating tests from code, detecting untested branches, and reporting coverage metrics. It ensures code quality stays high and bugs stay rare.

What It Does

Test-master handles all testing concerns:

  1. Test Execution - Run all tests across all realms in parallel
  2. Test Generation - Auto-generate test stubs from code structure
  3. Coverage Analysis - Detect untested code paths
  4. Performance Testing - Track test execution speed
  5. Reporting - Generate HTML coverage reports and metrics

Think of test-master as your quality control team—running thousands of tests instantly to catch problems before users do.

Key Capabilities

Test Orchestration

  • Parallel Execution: Run tests across all realms simultaneously
  • Isolated Test Suites: Each realm runs in isolated database transaction
  • Dependency Management: Automatically set up test fixtures and mocks
  • Flake Detection: Re-run failing tests to detect intermittent failures
  • Performance Tracking: Monitor test execution speed over time

Test Generation

  • Stub Creation: Generate test file structure from models
  • Edge Case Suggestions: Identify untested scenarios
  • Mock Generation: Auto-create mocks for external services
  • Assertion Suggestions: Recommend what to assert
  • Refactoring Support: Update tests when code changes

Coverage Analysis

  • Branch Coverage: Track tested code branches
  • Function Coverage: Monitor which functions have tests
  • Line Coverage: Show untested lines in code
  • Coverage Trends: Track if coverage improving/declining
  • Hotspot Detection: Find most critical untested code

Reporting

  • HTML Reports: Browse coverage with highlighted untested code
  • Coverage Badges: Embed coverage badges in README
  • PDF Reports: Generate reports for stakeholders
  • Email Summaries: Automated coverage reports to team
  • Trend Charts: Show coverage trends over time

Accessing test-master

Web Dashboard

URL: http://127.0.0.1:8009

The test-master dashboard shows test status, coverage metrics, and reports.

Command Line Interface

python manage.py test-all
python manage.py test-realm herald
python manage.py generate-tests --realm herald
python manage.py coverage-report --html
python manage.py performance-test

Testing API

Endpoint Purpose Method
/api/tests/run/ Start test execution POST
/api/tests/status/ Get test status GET
/api/coverage/ Get coverage metrics GET
/api/coverage/<realm>/ Coverage for specific realm GET

Common Use Cases

Use Case 1: Run Full Test Suite

python manage.py test-all

Runs 3,847 tests across 42 realms in parallel.

Use Case 2: Find Untested Code

Dashboard shows which functions and branches lack tests.

Use Case 3: Track Coverage Regression

Charts show if coverage is improving or declining over time.

Use Case 4: Generate Tests for New Feature

Automatically generates test stubs for newly added code.

Important Notes

Test Structure Convention

All realms follow Django test structure in apps/{app}/tests/ with test_models.py, test_views.py, test_services.py, etc.

Coverage Thresholds

Pipeline-go enforces: Overall 80% required, New code 85% required, Critical code 95% required.

Test Isolation

Each test runs in isolated database transaction (auto-rollback), gets clean filesystem, mocks external services.

Troubleshooting

Tests fail locally but pass in CI

Check for environment-specific issues, database state, file paths.

Coverage report shows incorrect percentage

Regenerate coverage: python manage.py coverage-report --force

Test generation creates wrong stubs

Generated tests are stubs—edit them to match actual functionality.

Tests timeout or hang

Some tests may be slow, profile with --profile flag to find slow tests.

Parallel test execution fails

Tests may have undeclared dependencies or shared state, isolate the issue.

  • pipeline-go - Test-master integrates with CI/CD pipeline
  • realm-spawn - New realms get test framework automatically
  • worker-bee - May trigger test runs on code changes
  • All active realms - Have test suites orchestrated by test-master

Further Reading