Warisha Arshad

Biography

Warisha Arshad is the Evaluation Engineer on llm-eval-kit, part of the INFERENCE Lab Engineering Fellowship (Cohort 01). Her focus is on robust quality assurance, adversarial evaluation of AI systems, and creating reliable test fixtures for continuous delivery.

Areas of Expertise

NLPGenerative AITrustworthy AI

Engineering Journals & Technical Reports (1)

llm-eval-kit

Building a Deterministic Offline Evaluation Engine for LLM Systems

We redesigned llm-eval-kit into a deterministic, zero-network evaluation pipeline built for reliable continuous evaluation in CI/CD environments. The new architecture introduces a decoupled criteria registry, fail-fast orchestration, hybrid semantic and symbolic verification, and graceful handling of inapplicable evaluation criteria. The implementation reached 93% test coverage across 88 tests, with cross-version CI validation from Python 3.9 to 3.12. The work demonstrates how carefully defined evaluation contracts and offline heuristics can provide reproducible, privacy-preserving model assessment without relying on external LLM-as-a-judge APIs.

Read Journal