ERP Test Discovery Platform — Shows which parts of an ERP nobody is testing, read from the system itself

Overview

The project at a glance.

223automated tests passing across 18 test files

10vendor-neutral ERP concepts in the canonical model

8boundary test cases checked by CI on every push

9direct third-party dependencies, tracked in a generated manifest

ERP Test Discovery is a VNE product for businesses running Microsoft Dynamics 365 Finance and Operations. Instead of helping people automate tests they have already chosen, it reads the ERP itself to list what the system can do, shows which of those capabilities are not being tested, and says plainly where it cannot see. The core is built and tested on Microsoft's Contoso demo data, and connecting it to a live sandbox comes next.

Who it serves
Dynamics 365 programme and test managersA view of which capabilities in which legal entities have no test coverage, before a go-live or upgrade rather than after.
ERP partners and consultanciesA discovery deliverable produced from the customer's own system, with blind spots stated plainly rather than hidden.
QA and test automation engineersA machine-readable coverage map with provenance on every claim, ready to feed test planning and tooling.

The challenge

Nobody can say what is not being tested

An ERP such as Dynamics 365 Finance and Operations can do hundreds of things across finance, procurement, sales, warehousing and more, in every legal entity a business runs. Test packs cover the processes somebody thought to write down, and existing automation tools help a person automate a test they have already decided to write. The question that matters most goes unanswered: what can the system do that nobody is testing?

Those gaps tend to show up during go-lives and upgrades, and customisations are the part most likely to break on upgrade. A report that looks complete but cannot see the customisations is worse than no report, because it gives false confidence to the people signing off.

What we built

Let the ERP say what it can do, then compare that with what is tested

VNE designed and built a discovery platform that connects to Dynamics 365 Finance and Operations read-only. It reads the system's legal entities, its inventory of capabilities from the platform's own metadata and the shape of its configuration, then translates all of it into a neutral description that would fit any ERP.

A coverage engine turns that description into a coverage map: every capability in every legal entity is marked covered, uncovered or unknown, and every mark records where it came from. Once the customer supplies the list of capabilities it already tests, unknowns become covered or uncovered findings, and modules are listed least-tested first. Using transaction history as a second source of evidence is the next phase.

Underneath is a layered TypeScript monorepo in which only the adapters are allowed to know about Dynamics, and the build itself enforces that rule, so Business Central or SAP can later be added as one new adapter rather than a rewrite.

Key features

What it does.

Coverage map as the deliverable

Every capability in every legal entity becomes a claim marked covered, uncovered or unknown, saved as a file other tools can read, with a summary and a module breakdown sorted least-tested first.

Reads the ERP, not a checklist

Legal entities, capabilities and configuration are read from the live system rather than a hardcoded list, so the inventory follows whichever release the customer is running.

Blind spots stated up front

Each adapter declares what it can and cannot read, and the report puts those gaps above the numbers. When customisations cannot be read, customisation risk is reported as unknown, never zero.

The customer's own test list

Supply the capabilities already tested and unknowns become covered or uncovered findings. Without any test evidence, the tool refuses to call anything uncovered.

Probe before discovery

A probe command reports what a new environment exposes: the platform version it could confirm, the areas it cannot read and how many observations of each kind it collected.

Data protection by construction

Configuration settings are recorded by type only and their values are never kept, transaction data must be pseudonymised before it is accepted, and credentials cannot be printed to logs.

Read-only and well-behaved

It only ever reads from the ERP, signs in as a registered Microsoft Entra ID application, and waits when the platform asks it to slow down.

Runs with no environment at all

A demo mode replays bundled fixtures of Contoso demo data, so the full pipeline can be seen and tested end to end without credentials or a sandbox.

Vendor-neutral by design

Ten neutral ERP concepts, sketched against Business Central and SAP before any code was written, keep every analysis layer free of Dynamics-specific names.

Inside the product

See it in action.

ERP Test Discovery is a command-line tool, so the gallery shows its real output from runs against fictional Contoso demo fixtures, re-rendered in a readable viewer, alongside its lint and test results and a layer diagram drawn from the repository.

Two coverage run summaries: 72 claims all unknown with no test evidence, and 18 covered, 54 uncovered after supplying a list of three tested capabilities — The same discovery run twice against the bundled Contoso demo fixtures. With no test evidence connected, all 72 claims are unknown and the tool says why. Given the customer's list of three tested capabilities, it reports 18 covered and 54 uncovered, with the most untested modules first.

Top of the coverage summary showing a customisation risk unknown warning and a list of areas the adapter can only partly read or cannot read — Before any numbers, the summary warns that customisation risk is unknown and lists each area the adapter cannot fully read, with the reason. The project's research found no API for listing installed extensions in Finance and Operations, so the tool reports that risk as unknown instead of scoring it as zero.

The coverage map JSON file with summary counts, a module breakdown and individual uncovered and unknown claims — The deliverable is a JSON coverage map that other tools can consume. Every claim carries its status, a confidence score or a reason, and provenance recording which adapter, which ERP version and which method produced it.

Probe command output showing the discovered ERP version, the declared blind spots and counts of observations by kind — The probe command is the first thing to run against a new environment. Shown here against the demo fixtures, it reports the platform version it could confirm, the blind spots it declares and how many entities, modules, capabilities and settings it observed.

A deliberately illegal import from an analysis engine into the Dynamics adapter, the lint errors it produces, and the boundary verification script passing — Architecture that enforces itself. An analysis engine importing the Dynamics adapter fails the build with an explanation of what to do instead, and a CI job proves on every push that each deliberately illegal fixture is still rejected and each legal one still passes.

Vitest report showing 223 passing tests, 6 skipped and 0 failing across 18 test files, with the end-to-end discovery tests expanded — The test suite: 223 passing tests across 18 files, from schema and pseudonymisation rules to an end-to-end discovery run. The 6 skipped tests are the conformance kit's extraction checks, which need a connection the Business Central stub cannot make, so they are reported as skipped rather than passed.

Diagram of the six architecture layers with built, stub and reserved packages, the canonical model highlighted as the boundary — The layer model drawn from the repository: which packages are built today, which are stubs and which are reserved for later phases, and the four ways the vendor boundary is enforced.

Workflow

How a coverage map is built

01
Connect
Connection details come only from environment variables. The adapter signs in with Microsoft Entra ID and tries to read the platform version, saying so plainly if it cannot.
02
Read the system
It lists legal entities across all companies, builds the capability inventory from the platform's metadata and notes what kind of value each setting holds, such as on/off or a threshold, without keeping the value itself.
03
Declare what it cannot see
Before any number is calculated, the adapter names every area it can only partly read or cannot read at all, and why.
04
Build the coverage map
The coverage engine creates a claim for each capability in each legal entity, carries the provenance through, and ranks modules by how much is untested.
05
Add evidence
The customer's list of tested capabilities turns unknowns into covered and uncovered findings. Transaction history is the planned second source of evidence.

Under the hood

How it's built.

The platform has one part that knows how to talk to a particular ERP, and everything else works from a neutral description of what any ERP contains. The Dynamics 365 adapter reads the system and translates what it finds into that neutral model; the coverage engine and the command line never learn which ERP they are looking at.

Technically, it is a pnpm TypeScript monorepo of 22 workspace packages across six layers: adapters, the canonical model, analysis engines, generation, execution and presentation. Eight packages hold working code today; the other fourteen are placeholders reserved for later phases. The canonical model is a set of Zod schemas that validate data at runtime, supply the compile-time types and generate a versioned JSON Schema. A single composition-root package is the only place allowed to construct a concrete adapter, which the rest of the system resolves by id through an adapter registry. Workspace dependencies, two custom ESLint rules and a CI job enforce these boundaries.

01Command line

erp-discover probeWhat the environment exposes and what it does not
erp-discover discoverCoverage map JSON plus a readable summary
Demo and fixture replayRuns end to end with no credentials

02Wiring

Composition rootThe only package allowed to import an adapter
Adapter registryResolves the adapter by id from configuration

03Analysis and model

Coverage engineTri-state claims, module ranking, provenance
Canonical model10 Zod concepts, versioned JSON Schema

04Adapters

Adapter coreErpAdapter contract, pseudonymiser, conformance kit
Dynamics 365 F&O adapterOData and metadata reader, GET only
Business Central stubDeclares that it can read nothing yet

05External services

Dynamics 365 Finance & OperationsPersonal Contoso sandbox only
Microsoft Entra IDOAuth 2.0 client credentials

Technology

Backend

TypeScript 5.7 (strict)
Node.js 22
pnpm 10 workspaces

Data

Zod 4 canonical schemas
Versioned JSON Schema
JSON coverage map

Integrations & APIs

Dynamics 365 F&O OData
F&O metadata service
Retry-After-aware retry

Auth & Security

Entra ID client credentials
HMAC-SHA256 pseudonymisation
Redacting Secret type

Hosting & DevOps

GitHub Actions (3 jobs)
Generated dependency manifest
Environment-variable configuration

Testing & Quality

Vitest 2
ESLint 9 (two custom rules)
typescript-eslint 8
Prettier 3
Fixture record and replay

Engineering highlights

The hard parts, solved.

01
Keeping the product vendor-neutral, mechanically
A platform built against one ERP drifts towards that ERP's shape. VNE wrote two custom ESLint rules: one fails the build when a package imports across a forbidden layer, the other bans Dynamics-shaped names above the canonical model. A CI job lints six deliberately illegal and two legal fixtures and fails if any is judged wrongly, so the enforcement is itself tested. The vocabulary rule was tuned so ordinary English such as "legal entity" in prose is not flagged.
02
Making "we don't know" impossible to lose
Coverage status is a three-way type, covered, uncovered or unknown, and an unknown must carry one of seven reasons. Every adapter must declare its support for eight extraction areas, and one shared function decides what an absence means at each support level. The project's research found no API for listing installed extensions in Finance and Operations, so the adapter declares that area unreadable and the map reports customisation risk as unknown rather than zero.
03
Data protection built into the types
Configuration values are classified (enabled, a numeric threshold, a selection and so on) and discarded in the same step. A transaction observation that has not been pseudonymised fails schema validation, so the rule is a parse error rather than a review note. Pseudonymisation uses an allowlist with domain-separated HMAC-SHA256 tokens, credentials are wrapped in a type that prints as redacted, and the fixture recorder refuses to write anything that looks like a secret or token.
04
Reading a system without assuming its version
The tool must work across Dynamics releases, so nothing is hardcoded. The capability inventory comes from the platform's metadata service, field names are matched against lists of candidates, paged results are followed to the end with a safety limit, and throttling honours the Retry-After header. No documented endpoint for the platform version was found, so several candidates are tried in turn and, if none answers, the version is recorded as unknown along with the endpoints attempted.
05
Testing an ERP integration without an ERP
The adapter talks to the outside world through a single GET-only transport with live, record and replay implementations, so it is tested in CI against fixtures of Contoso demo data, hand-authored to the expected response shapes until a sandbox is connected. An adapter conformance kit was written before the first real adapter, and a Business Central stub that declares no capabilities exercises the degradation path end to end.

Outcome

Working end to end on demo data, with a live sandbox next

The foundation phase is complete, and the first stage of the next phase has delivered the Dynamics 365 Finance and Operations adapter, the coverage engine and a command line that produces the coverage map end to end. All of it is tested against hand-authored fixtures of Microsoft's Contoso demo data, which will be replaced by real recordings once a personal sandbox is connected.

Development follows rules written into the repository: the tool is never pointed at a client environment, and no client data is stored, without written permission. Risk ranking, transaction-based evidence, regulatory packs and the web interface are designed and scheduled but not yet built.

The project at a glance.

Nobody can say what is not being tested

Let the ERP say what it can do, then compare that with what is tested

What it does.

Coverage map as the deliverable

Reads the ERP, not a checklist

Blind spots stated up front

The customer's own test list

Probe before discovery

Data protection by construction

Read-only and well-behaved

Runs with no environment at all

Vendor-neutral by design

See it in action.

How a coverage map is built

Connect

Read the system

Declare what it cannot see

Build the coverage map

Add evidence

How it's built.

Technology

Backend

Data

Integrations & APIs

Auth & Security

Hosting & DevOps

Testing & Quality

The hard parts, solved.

Keeping the product vendor-neutral, mechanically

Making "we don't know" impossible to lose

Data protection built into the types

Reading a system without assuming its version

Testing an ERP integration without an ERP

Working end to end on demo data, with a live sandbox next

Let's buildyours next.

Let's build
yours next.