GlyphRunner System
131 lines
3.4 KiB
Markdown
Executable File
131 lines
3.4 KiB
Markdown
Executable File
# GlyphRunner Integration Test Suite
|
|
|
|
Comprehensive tests for the GlyphRunner pipeline: compilation, execution, inspection, and summarization.
|
|
|
|
## Tests
|
|
|
|
### test_compile.py
|
|
**Coverage**: Source → .gx compilation pipeline
|
|
- Basic compile with explicit output path
|
|
- Auto output naming (derive .gx from .py)
|
|
- Verify magic bytes (XIC header)
|
|
- File creation and size validation
|
|
|
|
**Run**: `python3 test_compile.py`
|
|
|
|
### test_run.py
|
|
**Coverage**: Execution and code verification
|
|
- Basic .gx execution
|
|
- Code execution verification (variables, output capture)
|
|
- Segment counting and timing
|
|
|
|
**Run**: `python3 test_run.py`
|
|
|
|
### test_inspect.py
|
|
**Coverage**: .gx file inspection
|
|
- Manifest section display
|
|
- Segments section display
|
|
- Payload information
|
|
- All required manifest fields (version, origin, source_file, etc.)
|
|
|
|
**Run**: `python3 test_inspect.py`
|
|
|
|
### test_summary.py
|
|
**Coverage**: Human-readable summary generation
|
|
- Summary basic output
|
|
- All expected fields (GX File, Source, Type, Segments, Compressed, Version)
|
|
- Human-readable formatting
|
|
|
|
**Run**: `python3 test_summary.py`
|
|
|
|
### test_errors.py
|
|
**Coverage**: Error handling and graceful failure
|
|
- Compile with missing source file
|
|
- Run with missing .gx file
|
|
- Inspect with missing .gx file
|
|
- Summary with missing .gx file
|
|
- No command (help text)
|
|
|
|
All error cases return exit code 1 and print helpful error messages.
|
|
|
|
**Run**: `python3 test_errors.py`
|
|
|
|
### test_determinism.py
|
|
**Coverage**: Deterministic output and reproducibility
|
|
- Manifest structure consistency across runs
|
|
- File size determinism
|
|
- Magic bytes (XIC header) consistency
|
|
- Compressed payload determinism
|
|
|
|
**Note**: Full byte-for-byte determinism is not guaranteed due to timestamp fields in manifests. This test verifies structural and payload determinism instead.
|
|
|
|
**Run**: `python3 test_determinism.py`
|
|
|
|
## Running All Tests
|
|
|
|
**Complete test suite**:
|
|
```bash
|
|
python3 run_all_tests.py
|
|
```
|
|
|
|
This runs all test suites in sequence and prints a summary.
|
|
|
|
## Test Design
|
|
|
|
- **Plain Python**: No pytest/unittest frameworks, just subprocess and assertions
|
|
- **Exit codes**: Tests exit 0 on all-pass, non-zero on any failure
|
|
- **Clear output**: Each test prints PASS/FAIL per assertion
|
|
- **Isolated**: Tests use /tmp for output, don't interfere with each other
|
|
- **Independent**: Each test can be run individually
|
|
|
|
## Sample Coverage
|
|
|
|
All tests use `/home/dave/sample_code.py` as the test source:
|
|
- Real Python code (functions, classes, control flow)
|
|
- ~600 bytes source
|
|
- Compiles to ~960 byte .gx file
|
|
- 6 segments identified
|
|
- ~280 bytes compressed
|
|
|
|
## Expected Results
|
|
|
|
All 6 test suites should pass with ~25 individual test cases across:
|
|
- 2 compile tests
|
|
- 2 run tests
|
|
- 3 inspect tests
|
|
- 3 summary tests
|
|
- 5 error handling tests
|
|
- 3 determinism tests
|
|
|
|
## Pipeline Verification
|
|
|
|
The test suite exercises the full pipeline:
|
|
|
|
```
|
|
sample_code.py (source)
|
|
↓
|
|
gx_cli compile (via commands.cmd_compile)
|
|
↓
|
|
gx_compiler.compiler.GXCompiler
|
|
↓
|
|
sample_code.gx (compiled)
|
|
↓
|
|
gx_cli run/inspect/summary
|
|
↓
|
|
runtime_executor (gx_loader, execution, integration)
|
|
↓
|
|
Output + Execution Results
|
|
```
|
|
|
|
## Constraints Satisfied
|
|
|
|
✓ Tests in superdave/integration_tests/
|
|
✓ Plain Python scripts (no frameworks)
|
|
✓ Runnable as `python3 test_xxx.py`
|
|
✓ Exit non-zero on failure
|
|
✓ Clear PASS/FAIL messages
|
|
✓ Only touch /home/dave/superdave and /tmp
|
|
✓ Happy path with real sample_code.py
|
|
✓ Error handling (missing files, bad paths)
|
|
✓ Determinism verification
|