Evidencing compliance: release registers and security testing per release

The email nobody is ready for

Picture a firm of twelve people in Bristol that sells a booking product to dental practices. An email arrives from a large customer's procurement team, or from the insurer at renewal, or from an auditor a bank has sent to look at the accounting system. It contains a questionnaire. Question fourteen says something like: describe your secure development lifecycle and provide evidence of security testing for the current production version.

The owner genuinely takes security seriously. The hosting is good, and the developer knows what SQL injection is. And yet there is nothing to attach. Not because nothing was done, but because nothing was written down in a form anyone else could check.

This article is about closing that gap. It is not mainly about the law. I have written separately about the Cyber Resilience Act, NIS2 and the AI Act, and those pieces tell you what is required. This one is about the plain machinery that lets a small firm say, with a straight face and a PDF: here is what we released, here is who tested it, here is the report, and here is the version running right now.

The firm in Bristol and every other business in the examples is invented. The law and the standards are footnoted. Where I describe what I do for clients, I describe only what is published on my site today.

What evidence actually means

Most of the confusion lives in one word, so it is worth defining it.

"We take security seriously" is a claim. "We follow best practice" is a claim. A policy document that says releases are tested before deployment is also a claim, and a weak one, because anyone can write a policy.

Evidence is a record that a third party can check without trusting you. It has a date. It names a specific version of a specific system. It names who did the thing and points at what they produced. And it sits somewhere it could not have been quietly changed afterwards.

That sounds pedantic until you sit on the other side of the table. An auditor is not paid to believe you. An insurer's claims handler, once something has gone wrong and money is at stake, looks for the gap between what you said at renewal and what you can show. A procurement officer has a checklist and a legal department, and "trust us" ticks no box.

So everything below is built around one question. For any production system, at any moment, can you produce a record that says which version is running, what changed in it, who tested it and when, and where the report is? If yes, most questionnaires become copy and paste. If no, every questionnaire is a scramble.

What the law asks you to be able to show

The rules differ in scope, but they share one verb: demonstrate.

GDPR. Article 5(2) makes the controller responsible for complying with the data protection principles and able to demonstrate that it does. Article 32 requires security appropriate to the risk and lists, among the measures, "a process for regularly testing, assessing and evaluating the effectiveness of technical and organisational measures".¹ A test that left no record does not demonstrate anything.

UK GDPR. The same two articles sit in the UK GDPR.² The regulator you would show them to has changed its name: since 30 September 2026 the Information Commissioner's Office has been replaced by the Information Commission, under sections 118 and 119 of the Data (Use and Access) Act 2025.³

NIS2. Article 21 obliges essential and important entities to take risk management measures that include supply chain security, security in development and maintenance "including vulnerability handling and disclosure", and policies to assess whether the measures work. Article 23 sets the reporting clock for significant incidents: an early warning within 24 hours, a notification within 72 hours, a final report within a month.⁴ If one of your customers is in scope, the supply chain clause is why their letter lands on your desk.

NIS2 in the countries around you. Germany's implementing act was promulgated on 5 December 2025 and rewrote the BSI Act.⁵ Austria's NISG 2026 was published on 23 December 2025 and has applied since 1 October 2026.⁶ Ireland has not yet transposed the directive: the Commission referred it to the Court of Justice in July 2026, and the Government said in October that the National Cyber Security Bill was still being drafted. As I check this on 11 November 2026, no such Bill appears in the Oireachtas list of bills.⁷⁸ In the UK, the Cyber Security and Resilience (Network and Information Systems) Bill was introduced to Parliament on 12 November 2025 and updates the UK's own NIS rules.⁹

The Cyber Resilience Act. If you place software products on the EU market, the duty to report actively exploited vulnerabilities and severe incidents applies from 11 September 2026, and the rest of the Regulation from 11 December 2027.¹⁰ Its essential requirements expect security updates and a record of vulnerabilities handled over the life of the product, which is a release register by another name.

PCI DSS. If you take card payments, the Payment Card Industry Data Security Standard is currently at version 4.0.1, published in June 2024, and the requirements that were future dated became mandatory on 31 March 2025.¹¹ Among other things it asks for changes to production to follow documented procedures, critical security patches within a month of release, penetration testing at least every twelve months and after significant changes, and audit logs kept for at least twelve months with the latest three immediately available.¹²

ISO/IEC 27001:2022. The certifiable standard for information security management. Its Annex A controls include a secure development life cycle (8.25), security testing in development and acceptance (8.29), change management (8.32), logging (8.15) and management of technical vulnerabilities (8.8).¹³ Everything below maps onto those controls, which is useful if certification ever comes up.

The European Accessibility Act. Since 28 June 2025 it has applied to ecommerce services sold to consumers in the EU.¹⁴ That includes UK sellers trading into the EU. Accessibility is a property of the release too: a checkout that passed in March can fail in April. I have covered the details in the accessibility requirements piece.

Six different texts. None of them asks for perfection. All of them ask whether you can show what you did.

The one habit everything hangs off

Every production release gets a version number and release notes. That is the habit.

Every release, not only the big ones. The Tuesday afternoon deploy that fixed a typo gets a version, because next month, when someone asks what changed between the version the penetration tester looked at and the version running now, the answer has to be a list, not a shrug.

The scheme matters less than the discipline, but choose one and write it down. For products that ship on a schedule, semantic versioning, the familiar 2.4.1 pattern, works fine. For applications deployed several times a week, calendar versions are often clearer: year, month and a counter, so 2026.11.3 is the third release in November 2026. You can see at a glance how old the running version is, which is exactly what an insurer wants to know.

Release notes are not marketing. They are three short lists, what was added, what changed, what was removed, plus a timestamp and who approved the release. That is how I document releases for clients: production releases are versioned, timestamped and written up in those three lists, so the question "what exactly went live, and when" always has an answer.¹⁵

Everything else in this article attaches to that register. Security tests reference a version. Critical path approvals reference a version. Audit log queries are bounded by versions. Take the register away and the rest is a pile of documents with no spine.

The register is the spine. Each step after it points back to a version number.

Security testing is a property of a version, not a policy

This is the shift that makes the questionnaire easy.

Most small firms, if they test security at all, treat it as an event. "We had a pen test in 2024. There is a PDF somewhere." The trouble is that the system tested in 2024 no longer exists. It has been deployed forty times since. The report proves that a system which no longer runs was once reasonably secure.

The better model records security testing against a version. The record says: version 2026.11.3 of the customer portal, assessed on this date, by this person or tool, here is the report, pass or fail. Not every release needs the same depth, and that is fine, as long as the record says what was done for which version.

What the assessment contains depends on the change and the stack. For a Rails application a sensible baseline looks like this.

Static analysis on every build. Brakeman scans Rails code for security problems such as SQL built from strings, unescaped output, mass assignment and redirects to user supplied addresses.¹⁶ It is free, it takes a minute, and the build can fail on new warnings.

A dependency audit on every build. The bundler audit gem checks your libraries against a database of known vulnerabilities, with an equivalent for the JavaScript side.¹⁷ This is the least expensive answer to the PCI patching requirement and the NIS2 vulnerability handling clause, because it tells you the day a library you use becomes a problem.

Scans of the running application. Automated scanning against the common injection and cross site scripting classes, aimed at the deployed system rather than the code alone, because it tries the thousand variations a person would not.

Penetration testing. A person with a brief, trying to get in. For a major release or a new system, the full exercise. For a routine change, a focused look at what changed. On the systems I build, major production releases get web application security testing on top of the automated suites, grounded in formal offensive security training.¹⁵

Load and resilience drills where they matter. If downtime costs money, and NIS2 explicitly cares about availability, then someone has pushed the system until it fell over and written down where.

Keep the reports in the product's documentation repository, in git, named with the version they relate to. A git repository is a ledger: every change is timestamped and attributed, and rewriting history is both hard and visible. When an auditor asks how they know a report was not written last week, the commit history answers.

A traffic light per system

One device turns all of this into something an owner can read without opening a single report: a traffic light per company or per system.

Red means no security assessment has ever been recorded. Amber means the version running in production has not yet been assessed, usually because something was released and the assessment is still pending. Green means every production application's current version has a passing assessment on record.

That is the whole rule. It cannot be gamed with a nice policy, and it cannot be green because someone feels good. It turns amber the moment a release ships and green again when the assessment catches up. Amber for three weeks is a conversation. Red is an emergency. Green is the screenshot you attach to the questionnaire.

It also changes behaviour. If every release turns the light amber until it is assessed, people batch small changes and test properly, which is exactly the habit the Cyber Resilience Act's vulnerability handling rules try to create.

Who asks, and what they actually want

The questionnaires come from six directions, and they are variations of the same few questions.

The auditor. Usually a financial auditor, or an IT auditor working for one, who needs assurance that a system producing financial figures does what it claims and has not been tampered with. They ask which version is running, what controls exist over changes, who can make them, and whether every change is recorded. A release register with recorded test runs is close to the whole answer, and the logic itself is covered by verified critical paths, below.

The insurer. Cyber insurance questionnaires typically ask about multi factor authentication, patching within a defined window, backups tested for restore, logging of access and changes, a recent penetration test and an incident response plan. At renewal they want ticks. At claim time they want proof the ticks were true on the day. Dated assessment records and an exportable audit log make the ticks defensible.

The NIS2 supplier letter. A customer in scope must manage supply chain risk under Article 21, so they ask whether you handle vulnerabilities, how fast you patch, whether you test releases, and whether you can tell them about incidents quickly enough for their own 24 hour clock.⁴ The register and the per version assessments are the evidence. Incident notification is a paragraph in your incident plan.

Enterprise procurement. The longest form, often a spreadsheet of two hundred rows, most of which do not fit a company your size. The rest ask about secure development, access control, encryption, logging, business continuity and subprocessors. Here a "no, but" with evidence beats a "yes" without it. "There is no dedicated security team, but every release is recorded, here is the register and here are the last three reports" gets through.

The data protection regulator. Rarer, but after a complaint or a breach, the Information Commission asks what technical and organisational measures you had under Article 32 and whether you can demonstrate them.² Same register, same reports.

The bank or payment provider. Lenders and payment providers increasingly ask a short version of the insurer's questions before extending terms, and anyone taking cards meets the PCI DSS questions above.¹²

Six audiences, one register, one set of assessment records, one audit log, one place for signed documents. They are all asking the same thing in the end: can you show me, or do I have to take your word for it?

Verified critical paths, for when the logic itself is the risk

Some systems contain logic that, if wrong, is not a bug but a liability. The arithmetic that splits costs between members of a cooperative. The rule that decides which legal documents a customer accepts for which product. A tariff calculation. A commission split.

For these, testing that the code does what the developer intended is not enough. The question is whether what the developer intended is what the client, and the client's auditor, believe the rule to be.

A verified critical path answers that. The logic is written up in plain language, for the client rather than for developers, next to the executable tests that prove the system behaves that way. The client reads it and approves it, and at that moment it is frozen. Any later change to the logic reopens the approval, so the record cannot drift away from what the system actually does. That is how I handle critical business logic for clients: written up in plain language, approved by the client, frozen at that state, exportable as a PDF in English and German for the auditor, with changes reopening the approval.¹⁵

What the auditor receives is not "the developer says it works". It is the rule in words, the client's approval of that rule on a date, the tests that check the system follows it, and the evidence that nothing has changed since. ISO 27001 calls the underlying ideas security testing in acceptance and change management.¹³

Writing the plain language version has a side effect worth having. It forces a conversation about what the rule really is, and that conversation is where missing cases surface, before they are code.

Test gated releases, and why the tests are for the client

I will not repeat the case for test driven and behaviour driven development here. It is in my Cucumber piece and the Rails testing guide. The compliance angle is narrower.

No production release goes out until the full suite has run green. Staging is where you find out; production is not. And the run itself, when, on which commit, with what coverage, is recorded. On the projects I deliver, the full TDD and BDD suites must pass before anything reaches production, and the runs are recorded as evidence with coverage tracked.¹⁵

Recording it is the point. "We have tests" is a claim. "Version 2026.11.3 went live after a run of 1,412 scenarios at 84% coverage on this date, report attached" is evidence, and it is exactly what an auditor's question about change control is looking for. PCI DSS asks for the same thing in its change management requirement.¹²

Behaviour driven scenarios have a second job. They are written in plain language, so the client's tester can read them and confirm they describe what the business wants. Clients can read those tests in plain language in their client hub.¹⁸ The confirmation on staging is recorded, then the person with authority to accept the release accepts it. Request, tests, staging confirmation, acceptance, release, assessment: every step has a name and a date.

That sounds heavy. In practice it is a few clicks per release, and it is the difference between a change history that means something and one that does not.

Audit logging that an insurer can actually use

Most applications log something. Almost none of it helps when it matters.

When an insurer assesses a claim or an auditor chases a discrepancy, the questions are specific: who changed this record, when, from where, and what was it before. Logs full of request timings and stack traces do not answer that. An audit trail does: an append only record of who did what to which business object, kept apart from the data itself.

A useful audit log covers every action that touches an accounting relevant record, a user account, a permission, a document or a release, each with actor, target, change and timestamp, plus logins and failed logins. Its retention rule is written down. PCI DSS, for anyone in its scope, sets a floor of twelve months with the latest three immediately available.¹² A release register and an audit log are part of what I list as proof on the client hub.¹⁸

And it must be exportable. An insurer does not want to log into your admin. They want a file, filtered by date range and system, that their own analyst can read. Build that export once, calmly, rather than at two in the morning during an incident.

There is a GDPR wrinkle. Audit logs contain personal data, at minimum the actor's identity, so they need a lawful basis and a retention period like anything else. Recital 49 recognises network and information security as a legitimate interest, which covers it comfortably, but record it in your records of processing under Article 30.¹ An audit log you cannot explain under GDPR is a liability of its own.

The vault: signed documents in one place

The last piece is the least technical and the one most often missing.

Every engagement produces documents someone will eventually ask for: the signed contract, the data processing agreement, the insurance schedules, the offboarding clauses, the approval PDFs from the critical paths. In most small firms they live in email attachments, a shared folder with an inconsistent naming scheme, and a solicitor's filing system.

A vault puts them in one place, per client, with signed versions that cannot be overwritten. In my client portal, contracts, terms and insurance documents live in a vault, signed versions are locked for good, every view and download is logged, and onboarding includes a formal review and acceptance step that can go straight to the client's legal team.¹⁵

When procurement asks for your insurance certificate and your DPA, the answer is a link, not an afternoon. I wrote about the buyer's side of that question in how data processors can get you fined. The vault is the mirror image: being the supplier who can answer.

A worked example: one release on a normal Tuesday

Back to an invented company, this time a Leeds firm selling invoicing software at invoicing.example, to show how light the routine is.

The client's tester writes a request: the invoice export should include the customer's VAT number. The scenario is written in plain language and the tests follow. The tests run red, the code makes them green, and the full suite runs before the change goes to staging. The tester checks it on staging and confirms. The signatory accepts. The release goes out as 2026.11.4 with a one line note under "changed", and the traffic light turns amber.

Brakeman and the dependency audit already ran on the build and passed. That evening an injection scan runs against production, the assessment is recorded against 2026.11.4 with a link to the report, and the light turns green. Had the change touched the VAT calculation itself, the verified critical path would have flagged it, and the signatory would have approved again before release.

The overhead beyond the actual work is perhaps twenty minutes. In those twenty minutes, the questions of all six audiences have been answered for that release.

The Monday morning version

You do not need a portal to start. You need the habit and a place to write things down. Here is a version any small firm can begin next week.

Tag every production release in git. A tag with the version number and a message holding the three lists. That is your release register. If you deploy from a branch, tag the commit you deployed.

Keep a spreadsheet with one row per release. Version, date, one sentence on what changed, who approved it, test suite result, whether a security assessment was done, its date and a link to the report. Five minutes per release.

Add Brakeman and a dependency audit to your build, or the equivalents for your stack, and make the build fail on new findings. A morning's work once, and it runs for ever.

Get one penetration test on the current version and file the report with the version number in its name. For each later release, note in the spreadsheet whether the change was within that test's scope or needs another look.

Start an audit log if your application has none. Rails has mature libraries that do it in an afternoon. Make it append only and write down how long you keep it.

Put signed documents in one folder with a fixed naming scheme and no deletions: contract, DPA, insurance schedules, and a text file listing what is there and when it was signed.

Write a one page policy that describes the above. Not aspirations, what you actually do. It exists so a questionnaire can reference it, and it is only worth something if the spreadsheet proves you follow it.

That minimum will carry a firm of five through an insurer's renewal, a NIS2 supplier letter and most procurement forms. It will not carry you through a financial audit of critical logic, and it starts to hurt at about three systems and two people releasing, because a spreadsheet stops being trusted the moment two people edit it.

How it grows

A portal is what the spreadsheet becomes when the spreadsheet is no longer enough, and the transitions are predictable.

The spreadsheet becomes a register in a system when someone other than the person filling it in needs to trust it, usually the first client who asks to see it. The folder of reports becomes linked assessments when you run more than one production application and "is the running version tested" needs an answer per application. The traffic light appears when the owner wants to know the state without asking. Verified critical paths appear the first time an auditor asks not whether the system was tested but whether its rule is the right rule. The audit log export appears the first time an insurer asks for one, and you would rather it appeared before. The vault appears when the third procurement form in a year asks for the same four documents.

None of it needs building before you need it. All of it needs building before the email arrives, which is a different thing, and the minimum version above buys the time to build the rest.

What I would not bother with at this size

Certification first. ISO/IEC 27001 is a real standard and a genuine signal, and for a company under fifty people it is also a substantial project in time and money, for a certificate most of your customers will not require.¹³ Do the programme above first. If a customer makes certification a condition of a contract that pays for it, do it then, and you will find most of the groundwork done.

Tools before habits. There is a whole industry selling compliance automation platforms to small firms. They are fine. They are also useless if nobody tags releases and records tests, and if everybody does, a spreadsheet lasts longer than the vendors would like you to believe.

Policies nobody reads. A forty page security policy written for the questionnaire, describing a company that does not exist, is worse than a one page policy that is true. Auditors can tell, and when something goes wrong, a policy you did not follow is evidence against you.

The point of all of it

The programme is not mainly about questionnaires. It makes releases better. Knowing that every release will be versioned, tested, assessed and recorded makes people release deliberately. Writing critical logic in plain language finds gaps in conversation, before they are code. An audit log settles "who changed the price" in a minute instead of a week of email archaeology. A vault turns due diligence from a search into a link.

Compliance is a side effect of running software properly and writing it down. The questionnaire is simply the moment when writing it down pays for itself.

Footnotes

1. European Parliament and Council, Regulation (EU) 2016/679 (General Data Protection Regulation), Articles 5(2), 30 and 32 and recital 49, EUR-Lex, https://eur-lex.europa.eu/eli/reg/2016/679/oj/eng ↩ ↩²

2. The UK General Data Protection Regulation, Article 32 (security of processing) and Article 5 (principles), legislation.gov.uk, latest available version, https://www.legislation.gov.uk/eur/2016/679/article/32 and https://www.legislation.gov.uk/eur/2016/679/article/5 ↩ ↩²

3. The Data (Use and Access) Act 2025 (Commencement No. 9 and Transitional and Saving Provisions) Regulations 2026, SI 2026/1015, commencing sections 118 and 119 (the Information Commission) on 30 September 2026, legislation.gov.uk, https://www.legislation.gov.uk/uksi/2026/1015/made ↩

4. European Parliament and Council, Directive (EU) 2022/2555 (NIS2 Directive), Articles 21 and 23, Official Journal L 333, 27 December 2022, https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32022L2555 ↩ ↩²

5. Federal Republic of Germany, Gesetz zur Umsetzung der NIS 2 Richtlinie (NIS2UmsuCG), Bundesgesetzblatt 2025 I Nr. 301, 5 December 2025, https://www.recht.bund.de/bgbl/1/2025/301/VO.html ↩

6. Republic of Austria, Netz und Informationssystemsicherheitsgesetz 2026 (NISG 2026), BGBl. I Nr. 94/2025, 23 December 2025, in force 1 October 2026, https://www.ris.bka.gv.at/Dokumente/BgblAuth/BGBLA_2025_I_94/BGBLA_2025_I_94.html ↩

7. Law Society Gazette, Ireland referred to CJEU over cyber bill, 8 July 2026, https://www.lawsociety.ie/gazette/top-stories/2026/july/ireland-referred-to-cjeu-over-cyber-bill/ and Government pledge on delayed cyber bill, 7 October 2026, https://www.lawsociety.ie/gazette/top-stories/2026/october/government-pledge-on-delayed-cyber-bill/ ↩

8. Houses of the Oireachtas, Bills and Acts, checked 11 November 2026, https://www.oireachtas.ie/en/bills/ ↩

9. Department for Science, Innovation and Technology, Cyber Security and Resilience (Network and Information Systems) Bill factsheets: summary of the Bill, GOV.UK, updated 3 November 2026, https://www.gov.uk/government/publications/cyber-security-and-resilience-network-and-information-systems-bill-factsheets/summary-of-the-bill ↩

10. European Parliament and Council, Regulation (EU) 2024/2847 (Cyber Resilience Act), Articles 13, 14 and 71 and Annex I, EUR-Lex, https://eur-lex.europa.eu/eli/reg/2024/2847/oj/eng ↩

11. PCI Security Standards Council, Just published: PCI DSS v4.0.1, 11 June 2024, https://blog.pcisecuritystandards.org/just-published-pci-dss-v4-0-1 ↩

12. PCI Security Standards Council, PCI DSS v4.0.1, Requirements 6.3.3, 6.5.1, 10.5.1 and 11.4, document library, https://www.pcisecuritystandards.org/document_library/ ↩ ↩² ↩³ ↩⁴

13. International Organization for Standardization, ISO/IEC 27001:2022 Information security, cybersecurity and privacy protection, Information security management systems, Requirements, Annex A controls 8.8, 8.15, 8.25, 8.29 and 8.32, https://www.iso.org/standard/27001 ↩ ↩² ↩³

14. European Parliament and Council, Directive (EU) 2019/882 (European Accessibility Act), Articles 2 and 31, EUR-Lex, https://eur-lex.europa.eu/eli/dir/2019/882/oj/eng ↩

15. Keferboeck, How every engagement runs (professional standards), published on the SaaS development page, https://keferboeck.com/en-gb/product-and-services/saas-development ↩ ↩² ↩³ ↩⁴ ↩⁵

16. Brakeman, static analysis security scanner for Ruby on Rails, https://brakemanscanner.org/ ↩

17. RubyGems.org, bundler audit gem, https://rubygems.org/gems/bundler-audit ↩

18. Keferboeck, Why experience matters (proof points: tests in plain language in your client hub, critical paths verified and frozen behind approval, a release register and an audit log), https://keferboeck.com/en-gb/product-and-services/saas-development ↩ ↩²