Engineering

A conversational interface in a multi-tenant learning platform has to do more than produce a plausible answer. The same words can mean different things for a learner, teacher, or administrator. A useful response must respect the caller's role, the active school, the current application context, and the difference between describing an action and carrying it out.

I designed and built the harness across context construction, intent routing, structured outputs, validation, and application integration. The work connects model behavior to the real product: which capabilities exist, which users may invoke them, and how an action is represented back to the person making the request.

I built and led the Virtu Prep product organization; colleagues and contractors contributed to the surrounding product, interfaces, content, and operational workflows. This case study does not attribute that collective work to me alone.

Provider model APIs, application services, tenant and role data, and upstream libraries. Model output is treated as a proposal, not as authorization.

Each turn builds a context pack containing the request and the capabilities available within its scope. The model proposes an intent and parameters. Application code then validates that proposal, checks unresolved inputs, and applies role-aware corrections only within the permitted capability set.

The dispatcher separates actions that can proceed from requests that need clarification or confirmation. Consequential changes stop at a confirmation boundary before execution. The response reflects the action's actual disposition rather than treating generated prose as evidence that something happened.

The tests exercise interpretation, validation, and the confirmation boundary independently. A destructive-action test uses an executor that fails if it is called during the proposal turn. That makes the boundary observable: producing a reassuring confirmation message is not enough if the underlying action has already run.

A speech model needs to carry the intended pronunciation into an application-friendly voice. A student model can also learn a teacher's mistakes. The challenge is therefore not simply to generate more audio: it is to establish that the corpus is complete, correctly formatted, and suitable for the behavior the student should learn.

I built the pipeline around teacher synthesis, corpus preparation, student training, and candidate evaluation. This is work with existing model architectures and upstream weights, not a claim to have invented them. My contribution is in adapting those components to the speech task and making the training-to-evaluation process inspectable.

I built and led the Virtu Prep product organization; the wider team defined product needs and application context. This case study isolates my pipeline work from broader team outcomes.

Teacher-generated audio, Piper/VITS architecture, upstream weights, training and ONNX tooling, and speech-recognition diagnostics. I did not invent these architectures.

The teacher model produces audio paired with the training text. Corpus preparation deduplicates recordings, checks that manifest entries have corresponding audio, and normalizes the audio to the format expected by the student trainer. A sample-rate mismatch can otherwise produce a model that trains without an obvious error but sounds wrong.

The student is a warm-started Piper/VITS voice trained on that prepared corpus. Candidate checkpoints are exported to ONNX for synthesis and comparison. Tracking checkpoint identity through export and rendering matters: a cached file with the right size is not necessarily the model selected for evaluation.

Exported candidates are compared using held-out material, diagnostic checks, and direct listening. Speech-recognition transcripts can omit pronunciation distinctions, so transcription-based scores cannot settle every question. Listening judgments inform checkpoint and inference-setting choices, with their coverage limits kept explicit.

A better aggregate score does not establish that an agent is ready to use. Routing, refusals, confirmation, and workflow handoff can move in different directions. Evaluation also has to measure the system that actually executes, rather than an intermediate model label that application code may legitimately correct.

I worked on dataset construction, supervised fine-tuning, model comparisons, evaluation harnesses, and the criteria used to judge candidates. I also investigated cases where the evaluation itself produced a misleading account of system behavior, separating genuine model errors from incorrect labels, missing context, and scoring at the wrong stage.

I built and led the Virtu Prep product organization; application requirements and product decisions were developed with the Virtu Prep team. This record describes my work on data, training, comparisons, and evaluation instruments.

Base models, provider and training toolchains, task-specific examples, application corrections, and private evaluation records.

The training workflow assembles task-specific examples and separates training material from evaluation sets. Fine-tuned candidates are compared with their base models on defined behavioral dimensions, with acceptance criteria stated before interpreting the result. Separate work compares model adaptation with in-context grounding and deterministic application controls.

The broader harness includes adversarial cases and role-aware checks. Matched-pair comparisons rescore the same recorded outputs under a common rule, avoiding a fresh model call as a source of variation in that comparison. Repeated trials help distinguish persistent errors from inconsistent behavior.

Some fine-tunes improved action selection while missing other acceptance criteria. Those outcomes remain mixed results, not success stories reconstructed from the best metric. Instrument corrections also mattered: scoring a raw intent before a valid role-aware correction can mark a correct dispatched action as a failure. The work treats both the candidate and the measuring instrument as things that need testing.

Creating a specialized learning environment involves many related decisions: the audience, mission, program, content structure, and public presentation. A prose answer cannot be consumed reliably by an application. The system needs an explicit configuration contract that connects an administrator's requirements to a usable environment without assuming that every requested capability already exists.

I designed and built the configuration-generation workflow and its connection to the learning platform. The work translates product requirements into structured generation, defines the output contract, and connects the resulting configuration to application rendering. It began inside Virtu Prep before becoming a separate project.

Product, content, design, hosting, and school-launch work extended beyond the configuration workflow and involved other contributors.

The Virtu Prep application contract, JSON Schema validation, model APIs, and separate operational provisioning systems.

The workflow captures the school's requirements and uses structured generation to produce configuration for the platform. The result is validated against the expected JSON schema before it is treated as application data. Desktop and mobile experiences share that configuration rather than requiring a separately authored implementation for each school.

The generated configuration describes the environment; the application remains responsible for interpreting it. This boundary keeps the generation step distinct from the software capabilities it can configure and from the operational work required to make a school accessible on its own domain.

Schema validation establishes a structural contract, not a guarantee that the program is educationally complete or appropriate. The configuration still needs product review and checks against the administrator's requirements. The engineering decision is to make these responsibilities explicit rather than treating a successful model response as a completed launch.

Grades and credentials can obscure the evidence behind a claim of ability. A useful record needs to show what happened, how it was interpreted, and what can responsibly be concluded.

I authored and published the protocol to carry context and accountability information with capability claims; the format does not itself establish scientific validity.

I authored and published the protocol. Future contributors and implementers should be credited where documented.

Evidence provenance, validation, governance, and conformance concepts; a representation is only as sound as the evidence and methods behind it.

The published protocol distinguishes observations, derived measures, inferences, and validation records. It is designed to carry context and accountability information; using its format does not itself establish scientific validity.

The published protocol can be inspected as an evidence vocabulary and accountability structure. Publication is distinct from the validity of an automated inference, implementation conformance, or adoption.