C# to Aspose Migration Plan¶
Executive Summary¶
This document outlines a systematic approach to migrating Excel manipulation logic from C# tools (SCRIBE and AFR Templater) to TypeScript using Aspose. The migration is structured into discrete tasks, each completable in 10 hours or less, designed to be shipped incrementally with feature flags.
Migration Priorities¶
- SCRIBE Foundation (Core Excel operations)
- Budget Reports (Simpler business logic, validates approach)
- AFR Reports (Complex domain-specific logic)
Key Principles¶
- Each task is a complete, shippable unit
- Feature flags enable gradual rollout (C# ↔ Aspose switching)
- Comprehensive testing for each component
- Start with FY2025/FY2026, maintain C# for historical years
- Simplify architecture where possible
Timeline Estimate¶
- SCRIBE Foundation: 8-10 tasks
- Budget Migration: 4-6 tasks
- AFR Migration: 12-16 tasks
- Total: 24-32 tasks
Phase 1: SCRIBE Foundation (8-10 Tasks)¶
The SCRIBE tool is the foundation for all Excel operations. This phase establishes core patterns that Budget and AFR will build upon.
Task 1.1: Aspose Abstraction Layer & Basic File Operations¶
Priority: Critical Path Dependencies: None
Description Create a TypeScript abstraction layer over Aspose.Cells that
mirrors the pattern from the C# SpireHelper. This establishes the foundation
for all Excel operations.
Deliverables
- Create
@app/worker/src/utils/aspose-helper.ts - Implement interface:
typescript interface IExcelHelper { loadWorkbook(path: string): Workbook saveWorkbook( workbook: Workbook, path: string, format: ExcelFormat ): void getWorksheet(workbook: Workbook, name: string): Worksheet getCellValue(worksheet: Worksheet, address: string): CellValue setCellValue( worksheet: Worksheet, address: string, value: CellValue ): void getCellType(worksheet: Worksheet, address: string): CellType } - Implement basic file operations:
- Load workbook from file path
- Save workbook (support .xls, .xlsx, .xlsm formats)
- Get worksheet by name
- Create comprehensive unit tests with sample Excel files
Testing Requirements
- Load various Excel formats (.xls, .xlsx, .xlsm)
- Save workbooks without modification (binary comparison)
- Error handling (missing files, corrupted files)
- Memory leak testing (load/save cycle)
Acceptance Criteria
- [ ] All tests pass
- [ ] Successfully loads and saves Excel 97-2003 format (.xls)
- [ ] Successfully loads and saves Excel 2007+ format (.xlsx, .xlsm)
- [ ] Feature flag allows switching between implementations
Task 1.2: Cell Reading & Type Detection¶
Priority: High Dependencies: Task 1.1
Description Implement comprehensive cell reading and type detection logic
that matches the C# SpireHelper.ExtractValue() functionality.
Deliverables
- Extend
AsposeHelperwith cell reading methods:typescript interface CellValue { type: CellValueType // FORMULA | NUMBER | TEXT | DATE | BOOLEAN | BLANK value: string | number | boolean | Date | null formula?: string displayText?: string } - Implement type detection algorithm:
- Check for formula (has
=prefix) - Check for blank cell
- Check for date/datetime
- Check for boolean (TRUE/FALSE)
- Check for number
- Default to text
- Check for formula (has
- Extract numeric values from cells (handle formula results)
- Create comprehensive test suite covering all cell types
Testing Requirements
- Test all cell value types (formula, number, text, date, boolean, blank)
- Test formula evaluation (numeric formulas, text formulas)
- Test edge cases (empty strings, zero values, large numbers)
- Test date serial number handling (Excel 1900 bug compatibility)
- Snapshot testing against C# tool outputs
Acceptance Criteria
- [ ] All cell types correctly identified
- [ ] Formula results match C# implementation
- [ ] Date values match Excel serial number conversion
- [ ] Outputs match C#
ExtractValue()for test cases
Task 1.3: Cell Writing & Type Handling¶
Priority: High Dependencies: Task 1.2
Description Implement cell writing functionality supporting all SCRIBE data types (TEXT, NUMBER, DOLLAR, DATE, FORMULA, BOOLEAN, DROPDOWN, BLANK).
Deliverables
- Implement
setCellValue()for all data types:- TEXT: Plain string value
- NUMBER: Numeric value
- DOLLAR: Currency value with formatting
- DATE: MM/dd/yyyy string → Excel serial number (with 1900 bug handling)
- FORMULA: Excel formula (starts with =)
- BOOLEAN: TRUE/FALSE
- DROPDOWN: Data validation dropdown
- BLANK: Clear cell contents
- Implement number formatting:
typescript setNumberFormat(worksheet: Worksheet, address: string, format: string): void - Excel date conversion logic (matching C# implementation):
- Handle Excel 1900 leap year bug
- Convert MM/dd/yyyy strings to serial numbers
- Create comprehensive test suite
Testing Requirements
- Test writing all 8 data types
- Test number format application (currency, percentage, date)
- Test date conversion accuracy (including 1900 bug)
- Test formula preservation
- Test data validation dropdowns
- Compare outputs with C# SCRIBE tool (cell-by-cell comparison)
Acceptance Criteria
- [ ] All 8 data types write correctly
- [ ] Date conversion matches C# implementation exactly
- [ ] Number formats apply correctly
- [ ] Formulas preserve and evaluate correctly
- [ ] Data validation dropdowns work
- [ ] Outputs match C# tool for test cases
Task 1.4: Formula Calculation & Cell Copy Operations¶
Priority: High Dependencies: Task 1.3
Description Implement formula calculation and cell copy/paste operations that preserve cell types and formatting.
Deliverables
- Implement formula calculation methods:
typescript calculateWorkbook(workbook: Workbook): void calculateWorksheet(worksheet: Worksheet): void calculateCell(worksheet: Worksheet, address: string): void calculateRange(worksheet: Worksheet, startAddress: string, endAddress: string): void - Implement cell copy operations:
typescript copyCellValue(source: CellValue): CellValue copyCellFormat(sourceWs: Worksheet, sourceAddr: string, destWs: Worksheet, destAddr: string): void - Copy operations preserve:
- Cell value and type
- Number format
- Formula (not result)
- Create test suite with complex formulas
Testing Requirements
- Test formula calculation at workbook/worksheet/cell/range levels
- Test cell copy preserves all attributes
- Test formula copying (relative vs absolute references)
- Test cross-sheet formulas
- Performance testing (large formula chains)
Acceptance Criteria
- [ ] Formula calculation matches Excel behavior
- [ ] Cell copy preserves type, value, format
- [ ] Formula references update correctly when copied
- [ ] Cross-sheet formulas work correctly
- [ ] Performance acceptable for large workbooks
Task 1.5: SCRIBE Instruction Parser (Pure Logic)¶
Priority: Medium Dependencies: None (pure logic, no Excel operations)
Description Port the C# InstructionParser to TypeScript. This is pure
string parsing logic with no Excel dependencies.
Deliverables
- Create
@app/worker/src/utils/scribe/instruction-parser.ts -
Implement parser:
```typescript interface ParsedInstruction { operation: string // LOAD, SAVE, COPY, etc. args: string[] lineNumber: number }
class InstructionParser { parse(scribeFilePath: string): ParsedInstruction[] } ```
-
Support all 12 SCRIBE operations
- Error handling with line numbers
- Comprehensive test suite with sample .scribe files
Testing Requirements
- Test parsing all 12 instruction types
- Test error handling (invalid operations, wrong arg counts)
- Test malformed instruction syntax
- Test empty files, comment lines (if supported)
- Unit tests with mock file system
Acceptance Criteria
- [ ] Parses all 12 SCRIBE operations correctly
- [ ] Error messages include line numbers
- [ ] Handles edge cases gracefully
- [ ] Matches C# parser behavior
Task 1.6: SCRIBE Store & Instruction Base Classes¶
Priority: Medium Dependencies: Task 1.2 (for CellValue type)
Description Port C# ScribeStore and BaseInstruction infrastructure to
TypeScript.
Deliverables
-
Create
@app/worker/src/utils/scribe/scribe-store.ts:```typescript class ScribeStore { private workbooks: Map
private cellRanges: Map private cellValues: Map addWorkbook(name: string, workbook: Workbook): void getWorkbook(name: string): Workbook addCellRange(name: string, range: CellRange): void getCellRange(name: string): CellRange addCellValue(name: string, value: CellValue): void getCellValue(name: string): CellValue} ```
-
Create base instruction interface:
```typescript interface InstructionContext { store: ScribeStore helper: IExcelHelper }
interface IInstruction { execute(context: InstructionContext): void | Promise
} ``` -
Create test suite
Testing Requirements
- Test store add/get operations
- Test error handling (missing keys)
- Test memory management (large workbooks)
- Unit tests with mocks
Acceptance Criteria
- [ ] Store correctly manages workbooks, ranges, values
- [ ] Error handling for missing keys
- [ ] No memory leaks
Task 1.7: SCRIBE Core Instructions (LOAD, SAVE, COPY, PASTE)¶
Priority: High Dependencies: Tasks 1.4, 1.5, 1.6
Description Implement core SCRIBE instructions: LOAD, SAVE, COPY, PASTE, COPY_RANGE, PASTE_RANGE.
Deliverables
- Create instruction implementations:
LoadInstruction- Load Excel file to storeSaveInstruction- Save workbook from store to fileCopyInstruction- Copy single cell to storePasteInstruction- Paste single cell from storeCopyRangeInstruction- Copy cell range to storePasteRangeInstruction- Paste cell range from store
- Integration tests with real Excel files
Testing Requirements
- Test LOAD: various Excel formats, missing files
- Test SAVE: all Excel formats, overwriting files
- Test COPY/PASTE: all cell types
- Test COPY_RANGE/PASTE_RANGE: various range sizes
- Integration tests with full .scribe files
- Compare outputs with C# SCRIBE
Acceptance Criteria
- [ ] All 6 instructions work correctly
- [ ] Cell types and formats preserved in copy/paste
- [ ] Range operations handle edge cases
- [ ] Outputs match C# SCRIBE tool
Task 1.8: SCRIBE Write & Calculation Instructions¶
Priority: High Dependencies: Task 1.7
Description Implement WRITE and CALCULATE instructions.
Deliverables
- Create instruction implementations:
WriteInstruction- Write typed values to cells (all 8 types)CalculateInstruction- Trigger formula recalculation (workbook/worksheet/cell/range)
- Integration tests
Testing Requirements
- Test WRITE with all 8 data types
- Test CALCULATE at all levels
- Test formula dependencies
- Integration tests with complex formulas
- Compare with C# outputs
Acceptance Criteria
- [ ] WRITE handles all data types correctly
- [ ] CALCULATE triggers at correct scope
- [ ] Formula results match Excel
- [ ] Outputs match C# tool
Task 1.9: SCRIBE Arithmetic Instructions (ADD, MULTIPLY)¶
Priority: Medium Dependencies: Task 1.8
Description Implement arithmetic instructions: ADD_CELL, ADD_RAW, MULTIPLY_RANGE.
Deliverables
- Create instruction implementations:
AddCellInstruction- Add one cell’s value to anotherAddRawInstruction- Add raw number to cellMultiplyRangeInstruction- Multiply range by scalar with rounding
- Integration tests
Testing Requirements
- Test ADD_CELL with various numeric types
- Test ADD_RAW with large numbers, decimals
- Test MULTIPLY_RANGE with precision/rounding
- Integration tests
- Compare with C# outputs
Acceptance Criteria
- [ ] Arithmetic operations accurate
- [ ] Decimal precision handled correctly
- [ ] Range multiplication with rounding works
- [ ] Outputs match C# tool
Task 1.10: SCRIBE Instruction Invoker & Integration¶
Priority: High Dependencies: Tasks 1.7, 1.8, 1.9
Description Create the instruction invoker and integrate all SCRIBE components. Replace C# SCRIBE with Aspose implementation.
Deliverables
- Create
@app/worker/src/utils/scribe/instruction-invoker.ts:typescript class InstructionInvoker { execute( instructions: ParsedInstruction[], context: InstructionContext ): Promise<void> } - Update
ScribeHelper:- Add method
executeAspose(instructionsPath: string) - Use feature flag to switch between C# and Aspose
- Add method
- Create instruction factory (map operation name → instruction class)
- Comprehensive integration tests
- Update worker tasks to use feature flag
- Documentation
Testing Requirements
- End-to-end tests with real .scribe files
- Test all 12 instructions in combination
- Test error propagation with line numbers
- Performance testing vs C# implementation
- Side-by-side comparison (C# vs Aspose outputs)
Acceptance Criteria
- [ ] All 12 SCRIBE operations work via invoker
- [ ] Error messages include line numbers and context
- [ ] Feature flag allows switching implementations in the Scribe helper
- [ ] 100% output parity with C# for test cases
- [ ] Documentation updated
Migration Notes
- This task completes SCRIBE foundation
- Budget and AFR tasks can now proceed
- Keep C# SCRIBE available via feature flag for rollback
Phase 2: Budget Migration (4-6 Tasks)¶
Budget reports are simpler than AFR and validate the SCRIBE foundation with real-world usage.
Task 2.1: Budget Template Data Extraction & Preparation¶
Priority: High Dependencies: Task 1.10
Description Create utilities to extract and prepare budget data for SCRIBE template compilation. This replaces portions of the current budget task logic.
Deliverables
-
Create
@app/worker/src/utils/budget/budget-data-extractor.ts:```typescript interface BudgetData { proposalYear: number previousYear: number entityName: string ctdsCode: string districtContacts: DistrictContacts categoryStates: CategoryStates // … other budget-specific data }
class BudgetDataExtractor { async extractBudgetData( accountId: string, fiscalYearId: string ): Promise
} ``` -
Port budget-specific data gathering logic from
generate_budget.ts - Unit tests with mock database data
Testing Requirements
- Test data extraction for various district configurations
- Test category states handling
- Test district contacts parsing
- Unit tests with mocked DB
Acceptance Criteria
- [ ] Extracts all required budget data
- [ ] Handles missing/optional data gracefully
- [ ] Unit tests pass
Task 2.2: Budget SCRIBE Template Compilation (Aspose)¶
Priority: High Dependencies: Task 2.1
Description Migrate budget SCRIBE template compilation to use Aspose-backed SCRIBE execution.
Deliverables
- Update
ScribeHelper.compile()to support Aspose execution:typescript async compileAndExecute( templatePath: string, data: Record<string, unknown>, outputPath: string, useAspose: boolean = false ): Promise<void> - Create budget-specific SCRIBE helper:
typescript class BudgetScribeProcessor { async processBudget( budgetData: BudgetData, templateDir: string, outputDir: string ): Promise<string> // returns output file path } - Integration with existing budget template flow
- Feature flag:
ENABLE_ASPOSE_BUDGET_SCRIBE - Comprehensive testing with budget templates
Testing Requirements
- Test Handlebars template compilation
- Test SCRIBE execution with Aspose
- Test budget file output correctness
- Side-by-side comparison (C# vs Aspose)
- Integration tests with full budget workflow
Acceptance Criteria
- [ ] Budget templates compile correctly
- [ ] SCRIBE executes via Aspose
- [ ] Output files match C# implementation
- [ ] Feature flag allows switching implementations
- [ ] Integration tests pass
Task 2.3: Budget Task Integration & Testing¶
Priority: High Dependencies: Task 2.2
Description Integrate Aspose-backed budget generation into generate_budget
task with feature flag.
Deliverables
- Update
@app/worker/src/tasks/generate_budget.ts:- Add feature flag check
- Route to Aspose or C# based on flag
- Preserve all existing functionality
- Create comprehensive integration tests
- Update monitoring/logging for both paths
- Documentation updates
Testing Requirements
- End-to-end test with real budget data
- Test feature flag switching
- Test error handling and rollback
- Performance comparison (C# vs Aspose)
- Test with various district configurations
Acceptance Criteria
- [ ] Feature flag correctly routes to Aspose or C#
- [ ] Budget generation works end-to-end with Aspose
- [ ] Error handling and logging work correctly
- [ ] Performance acceptable (within 2x of C#)
- [ ] Output files match C# implementation
- [ ] Can safely roll back to C# via feature flag
Task 2.4: Budget Revision SCRIBE Migration¶
Priority: Medium Dependencies: Task 2.3
Description Migrate budget revision task to use Aspose-backed SCRIBE.
Deliverables
- Update
@app/worker/src/tasks/generate_budget_revision.ts:- Extract BUDG75 data (may already use Aspose)
- Use Aspose SCRIBE for budget revision operations
- Feature flag for C# vs Aspose routing
- Create budget revision-specific SCRIBE processor
- Integration tests with budget revision workflow
- Feature flag:
ENABLE_ASPOSE_BUDGET_REVISION
Testing Requirements
- Test BUDG75 data extraction
- Test budget revision SCRIBE execution
- Test file downloads and processing
- End-to-end integration tests
- Compare outputs with C# implementation
Acceptance Criteria
- [ ] Budget revision works end-to-end with Aspose
- [ ] BUDG75 data correctly extracted and applied
- [ ] Outputs match C# implementation
- [ ] Feature flag allows switching
- [ ] Integration tests pass
Phase 3: AFR Migration (12-16 Tasks)¶
AFR reports contain complex, domain-specific business logic for Arizona education financial reporting.
Task 3.1: Normalized Record Aggregator (Pure Logic)¶
Priority: Critical Path Dependencies: None (pure logic, no Excel operations)
Description Port the C# NormalizedRecordAggregator to TypeScript. This is
the most critical piece of AFR business logic.
Deliverables
-
Create
@app/worker/src/utils/afr/normalized-record-aggregator.ts:```typescript interface NormalizedRecord { fund_code: string program_code: string function_code: string object_code: string unit_code: string amount: string // in cents description: string source_line: number original_account_code: string }
interface MaskPattern { fund: string program: string function: string object: string unit: string }
class NormalizedRecordAggregator { aggregateRecordAmounts( records: NormalizedRecord[], inclusionMasks: MaskPattern[][], exclusionMasks?: MaskPattern[][], options?: { flipSign?: boolean; forcePositive?: boolean } ): number // returns amount in cents
bulkAggregate( records: NormalizedRecord[], aggregations: Record< string, { inclusionMasks: MaskPattern[][] exclusionMasks?: MaskPattern[][] } > ): Record<string, number> containsRecordMatchingMasks( records: NormalizedRecord[], inclusionMasks: MaskPattern[][], exclusionMasks?: MaskPattern[][] ): boolean} ```
-
Implement mask matching algorithm:
- Wildcard support (
Xmatches any digit) - Range support (
{1000-1999}) - Multiple values (
{001;010;020}) - Compound patterns (
{61XX;6200-6299})
- Wildcard support (
- Comprehensive test suite with edge cases
- Documentation with examples
Testing Requirements
- Test all mask pattern types (wildcards, ranges, multiples, compounds)
- Test inclusion/exclusion logic
- Test flipSign and forcePositive options
- Test bulk aggregation performance
- Test with real normalized records from AFR
Acceptance Criteria
- [ ] All mask patterns work correctly
- [ ] Aggregation logic matches C# implementation exactly
- [ ] Bulk aggregation optimized for performance
- [ ] Documentation includes usage examples
- [ ] Outputs match C# for all test cases
Task 3.2: Code Converter System (Pure Logic)¶
Priority: Critical Path Dependencies: None (pure logic)
Description Port the C# CodeConverter base class and FY2025/FY2026
converters to TypeScript.
Deliverables
-
Create
@app/worker/src/utils/afr/code-converter.ts:```typescript abstract class CodeConverter { protected rollupFundCodes: Map
> protected primaryUnitCodeMap: Map protected balanceObjectCodeRanges: [number, number] protected operatingObjectCodeRanges: [number, number] calcRollUpFund(fundCode: string): string calcFinalFund(fundCode: string, objectCode: string): string calcPrimaryUnitCode(unitCode: string): string isBalanceRecord(objectCode: string): boolean calcBalanceFundTotals(records: NormalizedRecord[]): Map<string, number>} ```
-
Implement FY2025 and FY2026 converters:
FiscalYear2025CodeConverter extends CodeConverterFiscalYear2026CodeConverter extends CodeConverter
- Create code converter factory:
typescript class CodeConverterFactory { static create(fiscalYear: number): CodeConverter } - Comprehensive test suite
- Documentation
Testing Requirements
- Test fund rollup calculations
- Test primary unit code mapping
- Test balance vs operating record detection
- Test balance fund totals calculation
- Test with real AFR data
Acceptance Criteria
- [ ] All converter methods work correctly
- [ ] FY2025 and FY2026 converters implemented
- [ ] Factory creates correct converter by year
- [ ] Outputs match C# implementation
- [ ] Documentation complete
Task 3.3: AFR Config Validation & Preparation¶
Priority: High Dependencies: None
Description Create validation and preparation logic for AFR configuration
data. Since we’re no longer calling C# tools via command line, we can pass the
TemplaterConfig object directly instead of writing/reading JSON files.
However, we still need validation to catch issues early and fail gracefully.
Deliverables
-
Create
@app/worker/src/utils/afr/afr-config-validator.ts:```typescript interface ValidationResult { valid: boolean errors: ValidationError[] warnings: ValidationWarning[] }
interface ValidationError { field: string message: string value?: any }
class AFRConfigValidator { validate(config: TemplaterConfig): ValidationResult
private validateSchoolCount(schoolCount: number): ValidationError[] // School count must be 1-45 // Warn if 26-45 (requires macro-enabled workbook) private validateCTDS(ctds: string): ValidationError[] // CTDS must be 9 digits, valid format private validateSchools(schools: School[]): ValidationError[] // Each school needs valid CTDS, unit codes, primary unit code // Private schools properly flagged private validateDistrictContacts( contacts: DistrictContacts ): ValidationError[] // Required contacts present isType3CTDS(ctds: string): boolean // JTED detection (7th digit == '3') getSingleSchoolDistrict(config: TemplaterConfig): boolean // Helper to determine if single school} ```
-
Create config preparation utilities:
```typescript class AFRConfigPreparation { // Convert TemplaterConfig to format expected by processor prepareForAFR(config: TemplaterConfig): AFRProcessorConfig
// Categorize schools (public vs private) getPublicSchools(config: TemplaterConfig): School[] // Determine report types needed getRequiredReportTypes(config: TemplaterConfig): ReportType[] // Based on deseg, foodService, schoolCount, etc.} ```
-
Integration with TemplaterHelper:
- Validate config before starting AFR generation
- Fail fast with clear error messages
- Return validation errors to user
-
Unit tests with various config scenarios
- Documentation
Testing Requirements
- Test validation logic (valid/invalid configs)
- Test school count boundaries (1, 25, 26, 45, 46)
- Test CTDS validation and Type 3 detection
- Test missing required fields
- Test edge cases (no schools, all private schools, etc.)
- Test validation error messages are clear and actionable
Acceptance Criteria
- [ ] Validation catches common configuration errors
- [ ] School count boundaries enforced with clear errors
- [ ] CTDS validation works correctly
- [ ] Type 3 CTDS detection accurate
- [ ] Validation errors are user-friendly
- [ ] Config preparation utilities handle edge cases
- [ ] No JSON file I/O needed (pass objects directly)
Simplification Note
This eliminates the need for JSON serialization/deserialization that was
required for C# tool communication. The TemplaterConfig object from
TemplaterHelper can be validated and passed directly to the AFR processor.
Task 3.4: AFR Template Base Processor Structure¶
Priority: High Dependencies: Tasks 3.1, 3.2, 3.3
Description Create the base AFR processor structure that year-specific processors will extend.
Deliverables
-
Create
@app/worker/src/utils/afr/afr-processor.ts:```typescript abstract class AFRProcessor { protected config: AFRConfig protected records: NormalizedRecord[] protected aggregator: NormalizedRecordAggregator protected converter: CodeConverter protected helper: IExcelHelper
abstract fillAccountingData( workbook: Workbook, records: NormalizedRecord[] ): Promise<void> abstract fillCustomPages( workbook: Workbook, records: NormalizedRecord[] ): Promise<void> abstract getPageMapping(): PageMapping abstract getBalanceAccounts(): BalanceAccount[] // Template method pattern async generateAFR(): Promise<string> { const workbook = await this.loadTemplate() await this.populateAFR(workbook) return await this.saveReport(workbook) } protected async populateAFR(workbook: Workbook): Promise<void> { await this.fillCoverPage(workbook) await this.fillAccountingData(workbook, this.records) await this.fillSchoolData(workbook) await this.fillBalanceSheet(workbook) await this.fillCustomPages(workbook, this.records) await this.performFinalCalculations(workbook) } // Common methods protected async fillCoverPage(workbook: Workbook): Promise<void> protected async fillSchoolData(workbook: Workbook): Promise<void> protected async fillBalanceSheet(workbook: Workbook): Promise<void> protected async performFinalCalculations( workbook: Workbook ): Promise<void>} ```
-
Create processor factory:
typescript class AFRProcessorFactory { static create( fiscalYear: number, config: AFRConfig, records: NormalizedRecord[] ): AFRProcessor } - Basic test structure
- Documentation
Testing Requirements
- Test base class methods with mocks
- Test template method pattern flow
- Unit tests for common methods
Acceptance Criteria
- [ ] Base processor structure matches C# design
- [ ] Template method pattern implemented
- [ ] Common methods functional
- [ ] Factory creates processors by year
- [ ] Tests pass
Task 3.5: FY2025 AFR - Cover Page & School Listing¶
Priority: High Dependencies: Task 3.4
Description Implement FY2025 AFR processor with cover page and school listing population.
Deliverables
- Create
@app/worker/src/utils/afr/fy2025-processor.ts:typescript class FiscalYear2025Processor extends AFRProcessor { // Override abstract methods async fillAccountingData( workbook: Workbook, records: NormalizedRecord[] ): Promise<void> async fillCustomPages( workbook: Workbook, records: NormalizedRecord[] ): Promise<void> getPageMapping(): PageMapping getBalanceAccounts(): BalanceAccount[] } - Implement cover page population:
- Entity name, CTDS, county
- District contacts (prepared by, superintendent, business manager)
- Fiscal year
- Implement school listing population:
- School roster with names and CTDS codes
- Handle single school vs multi-school districts
- Cell mapping system for cover page
- Integration tests with FY2025 template
- Feature flag:
ENABLE_ASPOSE_AFR
Testing Requirements
- Test cover page population with various district configs
- Test school listing for 1, 2-25, 26+ schools
- Test private school handling
- Compare outputs with C# implementation
- Integration tests with real FY2025 template
Acceptance Criteria
- [ ] Cover page populates correctly
- [ ] School listing handles all school count scenarios
- [ ] Private schools excluded where appropriate
- [ ] Outputs match C# implementation
- [ ] Integration tests pass
Task 3.6: FY2025 AFR - Balance Sheet Population¶
Priority: High Dependencies: Task 3.5
Description Implement balance sheet population (Page 1-2) for FY2025 AFR.
Deliverables
- Implement
fillBalanceSheet()for FY2025:- Page 1: Assets
- Page 2: Liabilities & Fund Balance
- Use code converter for balance fund calculations
- Use aggregator for balance record aggregation
- Implement balance sheet structure for FY2025
- Integration tests
Testing Requirements
- Test balance sheet with various fund structures
- Test with positive/negative balances
- Test fund totals and subtotals
- Compare with C# outputs
- Integration tests with real data
Acceptance Criteria
- [ ] Balance sheet populates correctly
- [ ] Fund calculations accurate
- [ ] Subtotals and totals correct
- [ ] Outputs match C# implementation
- [ ] Integration tests pass
Task 3.7: FY2025 AFR - Accounting Data Population (Part 1)¶
Priority: Critical Path Dependencies: Task 3.6
Description Implement first portion of fillAccountingData() - the most
complex method in AFR processing. Break into two parts due to size (~500+
lines).
Deliverables
- Implement Accounting Data population for rows 1-100:
- Revenue accounts
- Expenditure accounts (functions 1000-2999)
- Use bulk aggregation for performance
- Cell mapping for Accounting Data sheet
- Convert cents to dollars (divide by 100)
- Apply number formatting (#,##0.00)
- Integration tests
Testing Requirements
- Test revenue aggregations
- Test expenditure aggregations by function
- Test with various fund/function/object combinations
- Verify decimal precision
- Compare with C# outputs (cell-by-cell)
- Performance testing (bulk aggregation)
Acceptance Criteria
- [ ] Rows 1-100 populate correctly
- [ ] Amounts accurate to the cent
- [ ] Number formatting correct
- [ ] Performance acceptable (use bulk aggregation)
- [ ] Outputs match C# implementation
- [ ] Integration tests pass
Task 3.8: FY2025 AFR - Accounting Data Population (Part 2)¶
Priority: Critical Path Dependencies: Task 3.7
Description Complete fillAccountingData() implementation for FY2025.
Deliverables
- Implement Accounting Data population for rows 101-end:
- Additional expenditure accounts (functions 3000+)
- Special aggregations (deseg, food service)
- Balance forward amounts
- Subtotals and totals
- Complete integration tests
Testing Requirements
- Test remaining expenditure accounts
- Test desegregation fund handling
- Test food service integration (FY2025+)
- Test balance forward calculations
- Complete cell-by-cell comparison with C# outputs
- Integration tests with full AFR workflow
Acceptance Criteria
- [ ] All Accounting Data rows populate correctly
- [ ] Desegregation funds handled correctly
- [ ] Food service integrated (FY2025+)
- [ ] Balance forwards accurate
- [ ] Complete output parity with C# implementation
- [ ] Integration tests pass
Task 3.9: FY2025 AFR - Custom Pages (COVID, Food Service)¶
Priority: Medium Dependencies: Task 3.8
Description Implement FY2025-specific custom pages (Page 10: COVID-19 reporting, Page 7: Food Service).
Deliverables
- Implement
fillCustomPages()for FY2025:- Page 10: COVID-19 federal relief funds
- Page 7: Food service operations (integrated in FY2025)
- COVID fund aggregations (ESSER, GEER)
- Food service aggregations
- Integration tests
Testing Requirements
- Test COVID page with/without COVID funds
- Test food service integration
- Test with districts that have/don’t have food service
- Compare with C# outputs
- Integration tests
Acceptance Criteria
- [ ] COVID page populates correctly
- [ ] Food service integration works
- [ ] Conditional logic works (districts with/without these features)
- [ ] Outputs match C# implementation
- [ ] Integration tests pass
Task 3.10: FY2025 AFR - Final Calculations & Validation¶
Priority: High Dependencies: Task 3.9
Description Implement final calculations, formula recalculation, and AFR validation logic.
Deliverables
- Implement
performFinalCalculations():- Trigger workbook formula recalculation
- Verify formula results
- AFR validation logic:
- Balance sheet balances
- Cross-page totals match
- Required fields populated
- Error reporting
- Integration tests
Testing Requirements
- Test formula recalculation
- Test validation catches errors
- Test with valid and invalid AFRs
- Integration tests with full AFR generation
- Compare final outputs with C# implementation
Acceptance Criteria
- [ ] Formula recalculation works correctly
- [ ] Validation catches issues
- [ ] Error messages are clear
- [ ] Complete AFR generation works end-to-end
- [ ] Outputs match C# implementation
- [ ] Integration tests pass
Task 3.11: Desegregation & School-Level Reports (FY2025)¶
Priority: Medium Dependencies: Task 3.10
Description Implement desegregation AFR and school-level report generation for FY2025.
Deliverables
- Implement
generateDesegAFR():- Filter records for desegregation fund
- Generate separate deseg AFR
- Implement
generateSchoolReport():- Handle 1 school (use AFR Page 9)
- Handle 2-25 schools (.xls workbook)
- Handle 26-45 schools (.xlsm with macros)
- School-level aggregations by unit code
- Macro preservation for 26+ schools
- Integration tests
Testing Requirements
- Test deseg AFR with/without deseg funds
- Test school reports for various school counts (1, 10, 26, 45)
- Test macro preservation for .xlsm files
- Test private school exclusion
- Compare with C# outputs
- Integration tests
Acceptance Criteria
- [ ] Deseg AFR generates correctly
- [ ] School reports handle all school count scenarios
- [ ] Macros preserved for 26+ schools
- [ ] Private schools handled correctly
- [ ] Outputs match C# implementation
- [ ] Integration tests pass
Task 3.12: IDC Report Generation (FY2025)¶
Priority: Medium Dependencies: Task 3.10
Description Implement Indirect Cost (IDC) report generation using Aspose.
Deliverables
- Implement
generateIDCReport():- Aggregate indirect costs (function 2300-2600)
- Aggregate direct costs (function 1000-2200)
- Calculate IDC rate percentage
- Populate IDC template
- IDC-specific record transformations (max caps, exclusions)
- Integration tests
- Feature flag:
ENABLE_ASPOSE_IDC
Testing Requirements
- Test IDC calculations with various cost structures
- Test max cap logic (food service, sub-awards)
- Test exclusion logic
- Compare with C# outputs
- Integration tests
Acceptance Criteria
- [ ] IDC rate calculates correctly
- [ ] Max caps applied correctly
- [ ] Exclusions handled correctly
- [ ] Outputs match C# implementation
- [ ] Integration tests pass
Task 3.13: FY2025 AFR Task Integration¶
Priority: High Dependencies: Tasks 3.10, 3.11, 3.12
Description Integrate FY2025 AFR generation into generate_report task with
feature flag.
Deliverables
- Update
@app/worker/src/tasks/generate_report.ts:- Add feature flag routing (C# vs Aspose)
- Integrate AFR processor
- Preserve all existing functionality
- Handle all report types (AFR, Deseg, School, IDC, CSR)
- Update
TemplaterHelperto support Aspose execution - Comprehensive integration tests
- Error handling and rollback logic
- Monitoring and logging
Testing Requirements
- End-to-end tests for all FY2025 report types
- Test feature flag switching
- Test error handling and rollback
- Performance comparison (C# vs Aspose)
- Test with various district configurations
- Load testing
Acceptance Criteria
- [ ] Feature flag correctly routes to Aspose or C#
- [ ] All FY2025 report types generate correctly
- [ ] Error handling and logging work
- [ ] Performance acceptable (within 2x of C#)
- [ ] All outputs match C# implementation
- [ ] Can safely roll back to C# via feature flag
- [ ] Integration tests pass
Task 3.15: Historical Year Processors (Optional)¶
Priority: Low Dependencies: Task 3.13
Description Implement historical year processors (FY2022-FY2024) if needed. Each year is a separate task.
Deliverables (per year)
- Create processor for specific fiscal year
- Implement year-specific logic
- Integration tests
Notes
- Only implement if historical years are actively used
- C# tools remain available for historical years
- Can be deferred indefinitely if not needed
- Each year is a separate 8-10 hour task
Task 3.16: AFR Performance Optimization & C# Deprecation¶
Priority: Low Dependencies: Tasks 3.13, 3.14
Description Optimize AFR performance and plan C# tool deprecation.
Deliverables
- Performance profiling and optimization:
- Optimize bulk aggregation
- Minimize formula recalculations
- Optimize file I/O
- Memory management
- Create C# deprecation plan:
- Migration timeline
- Rollback procedures
- Communication plan
- Clean up feature flags (consolidate if appropriate)
- Final documentation
- Performance benchmarks
Testing Requirements
- Performance benchmarking vs C# implementation
- Load testing with large AFRs
- Memory profiling
- Stress testing
Acceptance Criteria
- [ ] Performance meets or exceeds C# implementation
- [ ] No memory leaks
- [ ] Deprecation plan documented
- [ ] All tests pass
- [ ] Documentation complete
Migration Notes
- This task completes AFR migration
- C# tools can be removed from deployment (with rollback plan)
- Feature flags can be cleaned up after confidence period
Migration Strategy & Best Practices¶
Feature Flag Strategy¶
Flag Hierarchy
ENABLE_ASPOSE (master flag)
├── ENABLE_ASPOSE_SCRIBE (SCRIBE foundation)
│ ├── ENABLE_ASPOSE_FILE_OPS
│ ├── ENABLE_ASPOSE_CELL_READING
│ ├── ENABLE_ASPOSE_CELL_WRITING
│ └── ... (other SCRIBE flags)
├── ENABLE_ASPOSE_BUDGET (Budget reports)
│ ├── ENABLE_ASPOSE_BUDGET_EXTRACTION
│ ├── ENABLE_ASPOSE_BUDGET_SCRIBE
│ └── ENABLE_ASPOSE_BUDGET_REVISION
└── ENABLE_ASPOSE_AFR (AFR reports)
├── ENABLE_ASPOSE_AFR_FY2025
├── ENABLE_ASPOSE_AFR_FY2026
└── ... (other AFR flags)
Flag Implementation
// In config or environment
const featureFlags = {
ENABLE_ASPOSE: process.env.ENABLE_ASPOSE === 'true',
ENABLE_ASPOSE_SCRIBE: process.env.ENABLE_ASPOSE_SCRIBE === 'true'
// ... other flags
}
// Usage in code
if (featureFlags.ENABLE_ASPOSE_SCRIBE) {
await scribeHelper.executeAspose(instructionsPath)
} else {
await scribeHelper.execute(instructionsPath) // C# fallback
}
Testing Strategy¶
Test Pyramid
E2E Tests (10%)
├── Full AFR generation workflow
├── Full Budget generation workflow
└── Feature flag switching
Integration Tests (30%)
├── SCRIBE instruction execution
├── AFR processor workflows
├── Budget processor workflows
└── Database integration
Unit Tests (60%)
├── Instruction parsing
├── Mask matching
├── Aggregation logic
├── Code converter logic
└── Cell operations
Snapshot Testing
For Excel outputs, use snapshot testing:
test('AFR output matches snapshot', async () => {
const output = await generateAFR(testConfig, testRecords)
const serialized = await serializeExcelFile(output)
expect(serialized).toMatchSnapshot()
})
Simplification Opportunities¶
1. Fiscal Year Architecture: Configuration vs Inheritance¶
Current C# Approach:
// Base class + inheritance + factory
abstract class CodeConverter { ... }
class FiscalYear2025CodeConverter : CodeConverter { ... }
class FiscalYear2026CodeConverter : CodeConverter { ... }
// Factory creates correct subclass
Analysis: The year-to-year differences are primarily:
- Data mappings (rollupFundCodes, primaryUnitCodeMap)
- Minor logic variations in calcFinalFund()
- Balance object code ranges
Proposed Refactor: Configuration-Based Approach
// 1. Define year configuration schema
interface FiscalYearConfig {
year: number
rollupFundCodes: Record<string, string[]> // rollup -> detail funds
primaryUnitCodeMap: Record<string, string> // unit -> primary unit
balanceObjectCodeRanges: [number, number]
operatingObjectCodeRanges: [number, number]
// Optional custom logic
customFundLogic?: (fundCode: string, objectCode: string) => string
customBalanceCalculation?: (
records: NormalizedRecord[]
) => Map<string, number>
}
// 2. Single converter class that reads config
class CodeConverter {
constructor(private config: FiscalYearConfig) {}
calcRollUpFund(fundCode: string): string {
// Look up in config.rollupFundCodes
for (const [rollup, details] of Object.entries(
this.config.rollupFundCodes
)) {
if (details.includes(fundCode)) return rollup
}
return fundCode // Default to itself
}
calcFinalFund(fundCode: string, objectCode: string): string {
// Try custom logic first
if (this.config.customFundLogic) {
const custom = this.config.customFundLogic(fundCode, objectCode)
if (custom) return custom
}
// Default logic
return this.calcRollUpFund(fundCode)
}
// ... other methods use config
}
// 3. Load configs from files or constants
// @app/worker/src/utils/afr/fiscal-year-configs/fy2025.ts
export const FY2025_CONFIG: FiscalYearConfig = {
year: 2025,
rollupFundCodes: {
'100': ['001', '010', '011', '012', '013', '014'],
'200': ['020', '021', '022', '023', '024']
// ... all mappings
},
primaryUnitCodeMap: {
'001': '001',
'002': '001' // Branch maps to main
// ... all mappings
},
balanceObjectCodeRanges: [9100, 9999],
operatingObjectCodeRanges: [1000, 8999],
customFundLogic: (fund, object) => {
// FY2025-specific fund logic if needed
if (object.startsWith('65') && fund === '610') {
return '600' // Food service special case
}
return null // Use default logic
}
}
// 4. Simple factory
class CodeConverterFactory {
private static configs = new Map<number, FiscalYearConfig>([
[2025, FY2025_CONFIG],
[2026, FY2026_CONFIG]
// Easy to add new years
])
static create(fiscalYear: number): CodeConverter {
const config = this.configs.get(fiscalYear)
if (!config) throw new Error(`No config for FY${fiscalYear}`)
return new CodeConverter(config)
}
}
Pros:
- ✅ Easier to add new fiscal years (just add config file)
- ✅ Year differences are explicit data, not hidden in code
- ✅ Easier to test (can test config validity separately)
- ✅ Config files could even be JSON/YAML for non-developers to update
- ✅ Less code duplication
- ✅ Can compare year configs easily (git diff on config files)
Cons:
- ❌ Complex year-specific logic harder to express
- ❌ Less type-safe than class methods
- ❌ If years diverge significantly, might need inheritance anyway
Recommendation: Use configuration-based approach with escape hatches for custom logic. This fits well because:
- Most year changes are data mapping updates
- Config files in
fiscal-year-configs/directory are easier to maintain - Can still use functions for complex logic (
customFundLogic) - If a year becomes too different, can still create a subclass
Same approach for AFRProcessor:
// Instead of separate FiscalYear2025Processor, FiscalYear2026Processor
interface FiscalYearProcessorConfig {
year: number
templateFileName: string
pages: {
cover: string
accountingData: string
balanceSheet: [string, string] // Page 1, Page 2
covidReporting?: string // Optional for FY2023+
foodService?: string // Different per year
}
accountingDataMappings: AccountingDataMapping[]
balanceSheetMappings: BalanceSheetMapping[]
// Hooks for custom logic
customPagePopulation?: (
workbook: Workbook,
config: AFRConfig
) => Promise<void>
}
class AFRProcessor {
constructor(
private yearConfig: FiscalYearProcessorConfig,
private config: AFRConfig,
private records: NormalizedRecord[]
) {
this.aggregator = new NormalizedRecordAggregator()
this.converter = CodeConverterFactory.create(yearConfig.year)
}
async generateAFR(): Promise<string> {
const workbook = await this.loadTemplate(
this.yearConfig.templateFileName
)
await this.fillCoverPage(workbook) // Generic, uses yearConfig.pages
await this.fillAccountingData(workbook) // Uses yearConfig.accountingDataMappings
await this.fillBalanceSheet(workbook) // Uses yearConfig.balanceSheetMappings
// Custom pages
if (this.yearConfig.customPagePopulation) {
await this.yearConfig.customPagePopulation(workbook, this.config)
}
return await this.saveReport(workbook)
}
private async fillAccountingData(workbook: Workbook): Promise<void> {
const ws = workbook.worksheets[this.yearConfig.pages.accountingData]
// Bulk aggregate all mappings at once
const aggregations: Record<string, BulkAggregateParams> = {}
for (const mapping of this.yearConfig.accountingDataMappings) {
aggregations[mapping.cellAddress] = {
inclusionMasks: mapping.inclusionMasks,
exclusionMasks: mapping.exclusionMasks,
flipSign: mapping.flipSign
}
}
const results = this.aggregator.bulkAggregate(
this.records,
aggregations
)
// Write results to cells
for (const [cellAddress, amount] of Object.entries(results)) {
const amountInDollars = amount / 100
this.helper.setCellValue(ws, cellAddress, amountInDollars)
this.helper.setNumberFormat(ws, cellAddress, '#,##0.00')
}
}
}
Accounting Data Mappings Config Example:
// fiscal-year-configs/fy2025-accounting-data.ts
export const FY2025_ACCOUNTING_DATA: AccountingDataMapping[] = [
{
cellAddress: 'F10',
description:
'Fund 001 - Instruction Salaries (Function 1000, Object 6100-6199)',
inclusionMasks: [
[
{
fund: '001',
program: 'XXX',
function: '1000',
object: '{6100-6199}',
unit: 'XXX'
}
]
]
},
{
cellAddress: 'F11',
description:
'Fund 001 - Instruction Benefits (Function 1000, Object 6200-6299)',
inclusionMasks: [
[
{
fund: '001',
program: 'XXX',
function: '1000',
object: '{6200-6299}',
unit: 'XXX'
}
]
]
}
// ... 200+ more mappings
]
Pros:
- ✅ Extremely clear what each cell represents
- ✅ Easy to update for new years (copy & modify config)
- ✅ Can version control configs as data
- ✅ Could generate configs from spreadsheet documentation
- ✅ Testing is easier (validate configs, test generic processor)
Cons:
- ❌ 200+ mappings per year = large config files
- ❌ Complex conditional logic harder to express
- ❌ Debugging might be harder (less code to step through)
Recommendation: Use hybrid approach:
- Config files for straightforward mappings (80% of cases)
- Custom functions for complex logic (COVID reporting, conditional pages)
- Keep processor logic generic and data-driven
2. Unified Excel Helper¶
Instead of separate SCRIBE and AFR Excel helpers, create one unified helper:
// Before: Multiple helpers with different interfaces
class SpireHelper { ... }
class AsposeHelper { ... }
// Different tools use different helpers inconsistently
// After: One unified helper with clear interface
interface IExcelHelper {
// File operations
loadWorkbook(path: string): Workbook
saveWorkbook(workbook: Workbook, path: string, format: ExcelFormat): void
// Cell operations
getCellValue(worksheet: Worksheet, address: string): CellValue
setCellValue(worksheet: Worksheet, address: string, value: CellValue): void
getCellType(worksheet: Worksheet, address: string): CellType
// Range operations
getCellRange(worksheet: Worksheet, start: string, end: string): CellValue[]
setCellRange(worksheet: Worksheet, start: string, values: CellValue[]): void
// Formula operations
calculateWorkbook(workbook: Workbook): void
calculateWorksheet(worksheet: Worksheet): void
// Formatting
setNumberFormat(worksheet: Worksheet, address: string, format: string): void
copyCellFormat(sourceWs: Worksheet, sourceAddr: string, destWs: Worksheet, destAddr: string): void
}
class ExcelHelper implements IExcelHelper {
// Single implementation backed by Aspose
// All tools use the same helper
}
Pros:
- ✅ Consistent interface across all tools
- ✅ Single source of truth for Excel operations
- ✅ Easier to test (mock IExcelHelper once)
- ✅ Easier to swap libraries if needed
- ✅ Better TypeScript type safety
Cons:
- ❌ Might be slightly more generic than specialized helpers
- ❌ Need to ensure interface covers all use cases
Recommendation: Yes, do this. It’s a clear win with minimal downsides.
3. Simplified Instruction System (Consider Carefully)¶
Current SCRIBE Approach:
// 12 separate instruction classes
class LoadInstruction implements IInstruction { ... }
class SaveInstruction implements IInstruction { ... }
class CopyInstruction implements IInstruction { ... }
// ... 9 more
Potential Simplification:
enum InstructionType {
LOAD,
SAVE,
COPY,
PASTE,
COPY_RANGE,
PASTE_RANGE,
WRITE,
CALCULATE,
ADD_CELL,
ADD_RAW,
MULTIPLY_RANGE,
DUMP
}
interface InstructionArgs {
// Union type of all possible arg combinations
workbookName?: string
worksheetName?: string
cellAddress?: string
value?: any
dataType?: DataType
// ... all args
}
class Instruction {
constructor(
public type: InstructionType,
public args: InstructionArgs,
public lineNumber: number
) {}
async execute(context: InstructionContext): Promise<void> {
// Big switch statement
switch (this.type) {
case InstructionType.LOAD:
return this.executeLoad(context)
case InstructionType.SAVE:
return this.executeSave(context)
// ...
}
}
private async executeLoad(context: InstructionContext): Promise<void> {
// Load logic here
}
}
Pros:
- ✅ Less file structure complexity
- ✅ Easier to see all instructions in one place
- ✅ Single class to test
Cons:
- ❌ Loss of separation of concerns
- ❌ Giant switch statement (code smell)
- ❌ Less type-safe args (union type is messy)
- ❌ Harder to extend/modify individual instructions
- ❌ Loses Open/Closed Principle (open for extension, closed for modification)
Recommendation: Keep separate instruction classes. The current pattern is better:
- Each instruction is self-contained and testable
- Type-safe arguments per instruction
- Easy to add new instructions without modifying existing code
- Clear separation of concerns
- The 12 files are not a burden, they’re organized
4. Shared Report Processor Base Class¶
Current Approach:
// Budget and AFR have separate logic
class BudgetProcessor { ... }
class AFRProcessor { ... }
// Some duplication in aggregation, validation, etc.
Simplification:
abstract class ReportProcessor {
protected aggregator: NormalizedRecordAggregator
protected converter: CodeConverter
protected helper: IExcelHelper
protected config: ReportConfig // Generic config type
// Shared methods
protected async loadTemplate(filename: string): Promise<Workbook> { ... }
protected async saveReport(workbook: Workbook, filename: string): Promise<string> { ... }
protected convertCentsToDollars(cents: number): number { return cents / 100 }
protected async bulkWriteAggregations(
worksheet: Worksheet,
mappings: CellMapping[],
records: NormalizedRecord[]
): Promise<void> {
// Common logic for writing aggregated data to cells
}
// Abstract methods subclasses must implement
abstract generate(): Promise<string>
abstract validate(): ValidationResult
}
class BudgetProcessor extends ReportProcessor {
async generate(): Promise<string> {
// Budget-specific generation
}
}
class AFRProcessor extends ReportProcessor {
async generate(): Promise<string> {
// AFR-specific generation
}
}
Pros:
- ✅ Eliminates duplication
- ✅ Consistent patterns across report types
- ✅ Shared utilities reduce bugs
- ✅ Easier to add new report types
Cons:
- ❌ Introduces coupling between report types
- ❌ Need to ensure base class doesn’t get too bloated
- ❌ Changes to base class affect all reports
Recommendation: Yes, create shared base class. Budget and AFR have enough commonality:
- Both use NormalizedRecordAggregator
- Both use CodeConverter
- Both populate Excel templates with aggregated data
- Both need validation
Keep the base class focused on truly shared operations, not domain-specific logic.
5. Declarative Cell Mappings (For AFR Accounting Data)¶
Current C# Approach:
// 500+ lines of imperative code in FillAccountingData()
void FillAccountingData(Workbook workbook, List<NormalizedRecord> records) {
var ws = workbook.Worksheets["Accounting Data"];
// Row 10: Fund 001, Function 1000, Objects 6100-6199
var masks = new List<List<string>> {
new List<string> { "001.XXX.1000.{6100-6199}.XXX" }
};
long amountInCents = aggregator.AggregateRecordAmounts(records, masks);
double amountInDollars = amountInCents / 100.0;
ws.Range["F10"].NumberValue = amountInDollars;
ws.Range["F10"].NumberFormat = "#,##0.00";
// Row 11: Fund 001, Function 1000, Objects 6200-6299
// ... 200+ more similar blocks
}
Problem: Lots of repetitive code that’s hard to:
- Maintain (easy to make copy-paste errors)
- Review (hard to verify correctness)
- Update for new years (have to rewrite whole method)
Simplification:
// Define mappings declaratively in config file
interface AccountingDataMapping {
cellAddress: string
description: string // For documentation
inclusionMasks: MaskPattern[][]
exclusionMasks?: MaskPattern[][]
flipSign?: boolean
forcePositive?: boolean
numberFormat?: string // Defaults to '#,##0.00'
}
// fiscal-year-configs/fy2025-accounting-data.ts
export const FY2025_ACCOUNTING_DATA_MAPPINGS: AccountingDataMapping[] = [
{
cellAddress: 'F10',
description: 'Fund 001 - Instruction Salaries (1000, 6100-6199)',
inclusionMasks: [[
{ fund: '001', program: 'XXX', function: '1000', object: '{6100-6199}', unit: 'XXX' }
]]
},
{
cellAddress: 'F11',
description: 'Fund 001 - Instruction Benefits (1000, 6200-6299)',
inclusionMasks: [[
{ fund: '001', program: 'XXX', function: '1000', object: '{6200-6299}', unit: 'XXX' }
]]
},
{
cellAddress: 'G15',
description: 'Fund 001 - Net Revenue (excluding transfers)',
inclusionMasks: [[
{ fund: '001', program: 'XXX', function: 'XXXX', object: '5XXX', unit: 'XXX' }
]],
exclusionMasks: [[
{ fund: '001', program: 'XXX', function: 'XXXX', object: '5200', unit: 'XXX' }
]]
},
// ... 200+ more mappings, clearly documented
]
// Generic processor method
async fillAccountingData(workbook: Workbook): Promise<void> {
const ws = workbook.worksheets[this.yearConfig.pages.accountingData]
// Build bulk aggregation request (single pass through records)
const aggregations: Record<string, BulkAggregateParams> = {}
for (const mapping of this.yearConfig.accountingDataMappings) {
aggregations[mapping.cellAddress] = {
inclusionMasks: mapping.inclusionMasks,
exclusionMasks: mapping.exclusionMasks,
flipSign: mapping.flipSign,
forcePositive: mapping.forcePositive
}
}
// Single pass through all records
const results = this.aggregator.bulkAggregate(this.records, aggregations)
// Write all results to cells
for (const mapping of this.yearConfig.accountingDataMappings) {
const amountInCents = results[mapping.cellAddress]
const amountInDollars = amountInCents / 100
this.helper.setCellValue(ws, mapping.cellAddress, amountInDollars)
this.helper.setNumberFormat(
ws,
mapping.cellAddress,
mapping.numberFormat || '#,##0.00'
)
}
}
Pros:
- ✅ Much easier to review: “Is F10 mapped correctly?” Just look at config
- ✅ No code duplication: One generic method handles all mappings
- ✅ Self-documenting: Description field explains each mapping
- ✅ Easy to update: New year = copy config file, modify mappings
- ✅ Easy to generate: Could generate from spreadsheet documentation
- ✅ Easy to test: Can unit test config validity separately
- ✅ Performance: Bulk aggregation is more efficient than 200+ individual calls
- ✅ Version control friendly: Config changes show as data diffs, not code diffs
Cons:
- ❌ Large config files (200+ mappings per year)
- ❌ Less flexible for complex conditional logic
- ❌ Need to ensure config format is validated
- ❌ Initial setup takes time to extract all mappings
Recommendation: Absolutely do this. This is the biggest win in the entire migration. The benefits far outweigh the costs:
- Correctness: Much easier to verify mappings match state requirements
- Maintainability: Year updates become config updates, not code rewrites
- Performance: Bulk aggregation is faster than individual aggregations
- Documentation: The config IS the documentation
Implementation approach:
- Start with a few mappings to establish pattern
- Use TypeScript to validate config structure
- Add config linting (detect duplicate cell addresses, invalid masks, etc.)
- Could even generate config from Excel documentation files
Example config validation:
function validateAccountingDataMappings(
mappings: AccountingDataMapping[]
): ValidationError[] {
const errors: ValidationError[] = []
const cellAddresses = new Set<string>()
for (const mapping of mappings) {
// Check for duplicate cell addresses
if (cellAddresses.has(mapping.cellAddress)) {
errors.push({
field: 'cellAddress',
message: `Duplicate cell address: ${mapping.cellAddress}`,
value: mapping.cellAddress
})
}
cellAddresses.add(mapping.cellAddress)
// Validate mask patterns
for (const maskGroup of mapping.inclusionMasks) {
for (const mask of maskGroup) {
if (!isValidMaskPattern(mask)) {
errors.push({
field: 'inclusionMasks',
message: `Invalid mask pattern in ${mapping.cellAddress}`,
value: mask
})
}
}
}
// Validate Excel cell address format
if (!/^[A-Z]+\d+$/.test(mapping.cellAddress)) {
errors.push({
field: 'cellAddress',
message: `Invalid Excel cell address: ${mapping.cellAddress}`,
value: mapping.cellAddress
})
}
}
return errors
}
Summary of Simplification Recommendations¶
| Simplification | Recommendation | Effort | Impact | Priority |
|---|---|---|---|---|
| 1. Config-based fiscal years | ✅ DO IT | Medium | High | High |
| 2. Unified Excel helper | ✅ DO IT | Low | Medium | High |
| 3. Simplified instructions | ❌ Skip | N/A | N/A | N/A |
| 4. Shared report base class | ✅ DO IT | Low | Medium | Medium |
| 5. Declarative cell mappings | ✅ DO IT | High | Very High | Critical |
Implementation Order:
- Unified Excel Helper (Task 1.1) - Foundation for everything
- Shared Report Base Class (can add during Phase 2-3) - Opportunistic refactor
- Config-based Fiscal Years (Task 3.2) - When implementing code converter
- Declarative Cell Mappings (Task 3.7-3.8) - When implementing accounting data population
The declarative cell mappings approach is the most impactful simplification and should be a core part of the AFR migration strategy.
2. Simplified Instruction System
Consider simplifying SCRIBE instructions:
// Current: 12 separate instruction types
LoadInstruction, SaveInstruction, CopyInstruction, ...
// Potential: Unified instruction with operation enum
class Instruction {
constructor(
public operation: InstructionType,
public args: InstructionArgs
) {}
}
3. Consolidated AFR Processors
If year-to-year changes are minimal, consider:
// Instead of separate processors per year
class AFRProcessor {
constructor(
public fiscalYear: number,
public config: YearConfig // year-specific config
) {}
}
4. Shared Aggregation Logic
Budget and AFR share aggregation patterns:
// Shared base class
abstract class ReportProcessor {
protected aggregator: NormalizedRecordAggregator
protected converter: CodeConverter
// ... shared methods
}
class BudgetProcessor extends ReportProcessor { ... }
class AFRProcessor extends ReportProcessor { ... }
Error Handling & Monitoring¶
Error Categories
- Validation Errors: Invalid config, missing data
- Processing Errors: Formula errors, type mismatches
- I/O Errors: File not found, permission denied
- Business Logic Errors: Balance sheet doesn’t balance, invalid account codes
Error Handling Pattern
try {
await generateAFR(config, records)
} catch (error) {
if (error instanceof ValidationError) {
// User-fixable error
logger.warn('AFR validation failed', { error })
throw new UserFacingError('Please check your configuration')
} else if (error instanceof ProcessingError) {
// System error, might need rollback
logger.error('AFR processing failed', { error })
await rollbackToC#()
} else {
// Unknown error
logger.error('Unexpected error', { error })
throw error
}
}
Conclusion¶
This migration plan provides a systematic, incremental approach to replacing C# Excel tools with TypeScript/Aspose implementation. By breaking the work into discrete, shippable tasks and using feature flags for gradual rollout, we minimize risk while maintaining production stability.
The migration follows a logical progression:
- SCRIBE Foundation - Establishes core patterns
- Budget Reports - Validates approach with simpler logic
- AFR Reports - Tackles complex domain logic
Each task is designed to be completed in 10 hours or less and can be easily converted into a GitHub issue. The use of feature flags allows for safe, gradual rollout with the ability to roll back at any point.