WikiStart: nsec3-workshop2-report-draft.txt

1. Introduction

A second workshop to test and refine the NSEC3 extension for DNSSEC
was held September 18-20, 2006, in Dulles, Virginia, USA, and hosted
by VeriSign.  Attending were (in alphabetical order by first name):

  David Blacka (VeriSign)
  Edward Lewis (NeuStar)
  Geoff Sisson (Nominet)
  Jelte Jansen (NLNet Labs)
  John Dickinson (Nominet)
  Mark Andrews (ISC)
  Matt Larson (VeriSign)
  Peter Koch (DENIC)
  Roy Arends (Nominet)
  Sam Weiler (SPARTA)
  Scott Rose (VeriSign)
  Suresh Krishnaswamy (SPARTA)
  Suzanne Woolf (ISC)
  Wouter Wijngaards (NLNet Labs)
  Ólafur Guðmundsson (OGUD Consulting)

After the first NSEC3 workshop in Frankfurt in May 2006, the attendees
felt, along with others in the dnsext working group, that a second
workshop was needed to address additional issues that couldn't be
tested at the first workshop for various reasons.


2. Issues tested

In this second NSEC3 workshop we tested five main areas:

  (a) Signalling NSEC3 and traversing various combinations

We tested the mechanism proposed in the -07 NSEC3 draft for a zone to
signal its use of NSEC3.  This mechanism requires the zone to be
signed with new DNSKEY algorithm IDs that correspond to existing
algorithm IDs but specifically indicate NSEC3 is in use.  For example,
DNSKEY algorithm ID 5 indicates a zone signed with RSA/SHA-1.  For
this workshop, algorithm ID 133 (5+128) indicated RSA/SHA-1 and also
possibly NSEC3.

The goal of this signalling mechanism is to "blind" legacy,
NSEC3-oblivious DNSSEC validators to NSEC3 zones, causing these
validators to treat responses containing NSEC3 records as insecure
rather than bogus.  We tested different delegation paths alternating
between NSEC3 and NSEC and including opt-out.  Details of the
workshop's test zone structure environment are described below in
section 3.1.

  (b) Transitioning from NSEC to NSEC3

We tested how a zone could transition from signing with NSEC to
signing with NSEC3, again as described in the -07 NSEC3 draft, while
maintaining a secure state and not requiring the zone to become
unsigned at any point during the transition.

  (c) Zone signing

We tested the interoperability of NSEC3-capable DNSSEC zone signers.

  (d) Zone loading and transfer

We tested the ability of several authoritative servers to successfully
load NSEC3-signed zones and then the interoperability of these servers
to transfer the NSEC3-signed zones without any errors or loss of
information.

  (e) Handling brokenness

We tested the reaction of different NSEC3-capable validators to
different scenarios involving deliberately broken packets.  The
specific brokenness is discussed below, but broken responses included,
for example, missing or malformed NSEC3 records.


3. Test environment

3.1. Zone structure 

In contrast to some previous DNSSEC workshops, where the structure of
test zones was decided during the workshop and configured on
attendees' workstations, for this workshop the test zone structure was
created before the workshop by Roy Arends of Nominet and hosted on
publicly accessible name servers at Nominet.  This method saved
considerable time, since we could begin the workshop with minimal
on-site setup.  Participants needed only to set up
resolvers/validators on their workstations, which then queried the
test zone structure on the Nominet name servers.

The test zone structure was hosted by three authoritative servers.
The 'trust-anchored zone' (ws.nsec3.org) resided on one server, the
second level zones resided on another server, and the third level
zones resided on yet another server.  The ws.nsec3.org zone was
DNSSEC-NSEC signed and contained delegations to signed zones and one
unsigned zone.  The delegations immediately under ws.nsec3.org were:

        n0      unsigned zone
        n1u     signed zone with unsigned delegations, using NSEC
        n1s     signed zone with signed delegations, using NSEC
        n3o     signed zone with opted out, unsigned delegations, using NSEC3
        n3u     signed zone with unsigned delegations, using NSEC3
        n3s     signed zone with signed delegations, using NSEC3

The zones with unsigned delegations (n0,n1u,n3o,n3u) had a delegation
to n0.(n0,n1u,n3o,n3u).  The zones with signed delegations (n1s,n3s)
had delegations to n1.n3s and n3.(n1s,n3s).

3.2. Software

The workshop tests used different authoritative servers,
resolver/validators and zone signers.  Implementation names are
obfuscated to avoid even unintentionally casting any implementations
in a negative light.

Five name server implementations were used as authoritative servers
and are designated A1 through A5 in this report.  (Different major
versions of the same software are counted as a different
implementation.)  The zones in the test structure were hosted by two
different NSEC3-capable authoritative server implementations, A1 and
A4.  The zone loading and transfer tests used all five.

Participants configured multiple different resolver/validators on
their workstations to query the workshop zone structure hosted at
Nominet and validate the responses.  Eleven different implementations
were used, although similarly to the authoritative servers, different
major versions of the same software count as different
implementations.  In some cases, the software was used only as a
resolver and in other cases a trust anchor was configured and the
software was also used as a DNSSEC validator.  These implementations
are referred to as R1 through R11 when used only in a resolver context
(i.e., no DNSSEC validation).  The same eleven implementations are
referred to as V1 through V11 when configured with a trust anchor and
performed DNSSEC validation.

For the zone signing tests, there were four different NSEC3-capable
DNSSEC zone signers that were tested for interoperability and their
ability to generate equivalent signed zones from the same unsigned
input.  These signers are designated S1 through S4.

Finally, for the brokenness tests, we used Roy Arends's "packet
server", which is special-purpose name server designed to serve
specially constructed deliberately broken packets to test validator
behavior to known-bad responses.


4. Signalling NSEC3 and traversing various combinations

Setup

This series of test were developed to test the NSEC/NSEC3 signaling
mechanism.  Previous work showed that non-NSEC3 aware validating
resolvers could not follow a unsigned delegation from a NSEC3 zone.
The proposed solution was to signal NSEC3 signed zones by using an
alternative DNSKEY algorithm code, which non-NSEC3 aware validators
would interpret as insecure and not attempt to validate security
information from those zones.  In this experiment, the algorithm code
for "NSEC3 signed" zones was 133.

Test Format

The tree used for the signaling and traversing tests are summarized below.  
For resolvers that required DNSSEC, the KSK for ws.nsec3.org was used as the 
trust anchor.  

* ws.nsec3.org (signed with nsec)
  * n0.ws.nsec3.org
      * n0.n0.ws.nsec3.org
  * n1u.ws.nsec3.org
      * n0.n1u.ws.nsec3.org
  * n1s.ws.nsec3.org
     * n3.n1s.ws.nsec3.org
  * n3u.ws.nsec3.org
     * n0.n3u.nsec3.org
  * n3s.ws.nsec3.org
     * n1.n3s.ws.nsec3.org
     * n3.n3s.ws.nsec3.org
  * n3o.ws.nsec3.org
     * n0.n3o.ws.nsec3.org
  * full.ws.nsec3.org

The labels indicate the following:

n0 - unsigned zone data
n1 - DNSSEC signed data (NSEC chain)
n1u - DNSSEC signed data (NSEC chain) with unsigned delegations
n1s - DNSSEC signed data (NSEC chain) with signed delegations
n3 - DNSSEC with NSEC3 chain
n3u - DNSSEC with NSEC3 chain, unsigned delegations
n3s - DNSSEC with NSEC3 chain, signed delegations
n3o - DNSSEC with NSEC3 chain with Opt-Out
full - DNSSEC with NSEC3, includes wildcards, empty non-terminals, etc.

Each zone contained a single host (www) and DNSKEY RRs as appropriate.  There 
was two versions of this tree - one served by A1 servers and one served by
A4 servers.  Each type of resolver sent queries for the www host in each of 
the following zones:

www.n0.n0.ws.nsec3.org
www.n0.n1u.ws.nsec3.org
www.n3.n1s.ws.nsec3.org
www.n0.n3u.ws.nsec3.org
www.n0.n3o.ws.nsec3.org
www.n1.n3s.ws.nsec3.org
www.n3.n3s.ws.nsec3.org

Along with various queries to full.ws.nsec3.org as time permitted.
For each resolver implementation, the expected results depended on the
characteristics of the resolver:

Expected results:

Tree authoritative servers:  A1, A4
NSEC3-aware: R9, R1, R7, R10, R9
NSEC-aware: R8, R3, R10
DNSSEC unaware: R11, R5, R4, R6

                        NSEC3-aware             NSEC-aware      DNSSEC-unaware

www.n0.n0       insecure                insecure        traditional resp.       

www.n0.n1u      insecure                insecure        traditional resp.

www.n1.n1s      nxdomain                nxdomain        traditional resp.
                                        
www.n3.n1s      secure/AD               insecure        traditional resp.

www.n0.n3u      insecure                insecure        traditional resp.

www.n0.n3o      insecure                insecure        traditional resp.

www.n1.n3s      secure/AD               insecure        traditional resp.

www.n3.n3s  secure/AD           insecure        traditional resp.


Actual results from tests:


                           R9      R1      R11     R8      R5       R3      R6

        www.n0.n0               P               P               P               P               P                P      P

        www.n0.n1u              P               P               P               P               P                P      P

        www.n1.n1s              P               P               P               P               P                P      P
                
        www.n3.n1s              P               P               P               P               P                P      P

        www.n0.n3u              P               P               P               P               P                P      P

        www.n0.n3o              P           *           P               P               P                P      P

        www.n1.n3s              P           P           P               P               P                P      P

        www.n3.n3s              P           P           P               P               P                P      P




                                   R3       R4            R7             R10            R10
                                (dnssec)                             (with nsec3)       (no nsec3)      (nsec3)
                                
        www.n0.n0               P                P                    P                   P                  P

        www.n0.n1u              P                P                    P                   P                  P
                        
        www.n1.n1s              P                P                    P                   P                  P
                
        www.n3.n1s              P                P                    P                   P                  P

        www.n0.n3u              P                P                    P                   P                  P

        www.n0.n3o              P                P                    P                   P                  P

        www.n1.n3s              P                P                    P                   P                  P

        www.n3.n3s              P                P                    P                   P                  P


Discussion

From the results in the tables above, the "*" indicates that the tree
using A1 returning ServFail for that query.  It was later believed to
be an environment bug, not a protocol error.  One that may repeat if
the zones are not set up correctly.  All other queries resulted in
matching behavior with both domain trees and the expected results
(marked with the 'P' for "pass").

These tests do not cover all possible DNS environments.  More work
would need to be done as corner cases and unique zones are
encountered.  For the majority of expected scenarios, the expected
results (from the resolvers point of view) were recieved.  That is,
when a non-NSEC3 aware validating resolver encounters a zone along a
resolution path, it will see the unknown key algorithm and consider
that zone (and all children of that zone) to be insecure.


5. Transitioning from NSEC to NSEC3

Synopsis:

This tested the observed behaviour of several different
resolvers/validators before, during and after a roll from NSEC to
NSEC3.


Roll states:

State 1: Initial state (pre-roll):

        An NSEC3-capable authoritative server with NSEC-signed zone
        using key algorithm 5 (RSASHA1) which had three sub-zones.
        These were:

        - an unsigned sub-zone [n0.roll.ws.nsec3.org]
        - a signed sub-zone using NSEC [n1.roll.ws.nsec3.org]
        - a signed sub-zone using NSEC3 [n3.roll.ws.nsec3.org]

State 2: Intermediate state (during roll):

        We performed a key rollover to an NSEC-signed zone using
        key algorithm 133 (NSEC3RSASHA1, i.e. RSASHA1 algorithm with
        code specially assigned to signal possible use of NSEC3).

State 3: Final state (post-roll):

        We then signed the zones, replacing the NSEC chain with an
        NSEC3 chain -- again using key algorithm 133 -- and reloaded
        the server.


Testing:

A number of different resolvers/validators were tested:

- two different NSEC3-aware validators: V1 and V7.

- two different NSEC-only validators: V3 and V8.

- several different resolvers which were either DNSSEC-unaware or
  had DNSSEC disabled: R3, R4, R5, R10, R11.


Result:

All resolvers/validators returned expected results throughout all
three states of the test.

- The NSEC3-aware validators returned consistent authoritative results
  during all three states.

- The NSEC-only-aware validators returned authoritative answers in
  State 1 and non-authoritative answers in States 2 and 3.

- The non-DNSSEC-aware resolvers returned expected answers throughout
  the test.


6. Zone signing

Intro: 

The signing tests used two different zones, in two different modes,
without opt-in and with.

There where 4 different signers used in this test: 
jdns_signzone, ldns_signzone, pdns_signzone, Bind dnssec-signzone-9.5-pre

In this report the names of the actual signers will be obfuscated,
signers will be referred to as S1, S2, S3, S4 (not the same order as
above). 

Structure: 

In the signing test all signers used the same signing parameters
  Signature Inception time: 20060919000000
  Signature Expiration time: 20061231000000
  TTL on all RRsets:         3600
  NSEC3 iterations:          10
  NSEC3 salt:                AFFE  (hex value) 
  same DNSKEY's              2 keys of alg 5 
                             2 keys of alg 133 

By doing this the resulting zones where "directly" comparable, 

The first zone used [REF1] had 10 different names including 
apex, there were single label names, multiple label names and a 
wild-card at multi-label name. There were delegations in the zone,
with and without DS records. 

The second zone was an empty zone (only data at APEX). 

Expected Results: 
David Blacka had written a "zone normalizer" to convert the zones to a
standard format that then could be compared by diff to find any
discrepancies. 

Actual Results:
The normalizer turned out to have two issues in processing the zones,
from various signers. The first issue was mixing of cases in names,
the second issue was ordering of RRSIG's covering a RRset. 
Both issues have been addressed. 
The initial report from the normalizer was that all zones differed. 

At this point two humans where assigned to figure out the differences
in the zones, and what was correct. 

John's tests
Checked the empty zones first. Visual check that all the signed zones
had the same
    number of each type of RR
    NSEC3-PARAM
    hashes in the NSEC3 RR's
    signatures in the RRSIG's
    TTL's in all RR's

Several of the signatures had been done with incorrect
parameters. These were resigned. and the checks repeated.
Some signers signed zone with both algorithms and some only with alg
133. The room determined that this was an open issue that should not
be used to judge the signers validity. 


Olafur's tests: 
Started work on the non-empty zones, he wrote scripts to check the
consistency of the NSEC3 chains and if the NSEC3's matched the
NSEC3PARAM record at the apex.  
The scripts evolved from simply putting out the NSEC3 records to
actually checking the chain no matter what order provided. 

Errors seen: 
For the empty zones S2 and S1 were found to agree. S4 was found to use
the wrong TTL for DNSKEY's and S3 did not create a NSEC3-PARAM RR or
its RRSIG. 

for the non-empty zones: 
One implementation S4 did not include an NSEC3PARAM.
One implementation S1 had the right number of NSEC3 records but
multiple NSEC3's pointed to the same target. 


Second run: 
After reporting these results the developers sat down and updated
their signers to address the issues. 

After second round of zone signing 3 of the signers  (S1, S2, S3)
agree on what should be in the NSEC3PARAM record and in the NSEC3
chain. S4 agrees with the the others on the NSEC3 chain but does not 
include that NSEC3PARAM record in the zone. 

At this point the opt-out zones where tested and they all passed, all
opted out the same name and had the same NSEC3 chain. 

Conclusions: 
  (any difference between expected and actual results) 

Signers agree on what NSEC3 records should be for the same NSEC3
settings. All of the signers generate NSEC3 from input flags or signer
configuration file. 
Most of the issues seen where addressed and fixed at the workshop. S4
addressed their issue after testing completed. 

Open issues: 
Signing algorithms used. There is an open issue what to do if there are
both algorithm 133 and 5 DNSKEY records present. 
Some singers sign with both algorithms but do NOT include NSEC chain. 
One signer (S1) does not include the DNSKEY RR's with algorithm 5 when
instructed to generate NSEC3 chain. 

DNSKEY record with algorithm 5 requires RRSIG's on the zone and/or
NSEC chain. What does it mean to have NSEC and NSEC3 specific DNSKEY
algorithms listed in the DNSKEY set at the apex? 

Nit: 
Some singers put their name and version into the signed zone, others
did not. Testers found the presence implementation information useful. 


7. Zone loading and transfer

o Setup

We set up a chain of authoritative servers that had master/slave
relations to each other. Some of these servers understood NSEC3 (the
first one, to read the zone file) and some intermediate servers did
not understand NSEC3 (but were RFC 3597 ready). The zone was
transferred to the slaves down the chain by means of AXFR. This way
the behaviour of NSEC3 unaware servers when presented with an NSEC3
zone became subject to the test. A second objective was to test the
behaviour when the NSEC3 parameters were changed and the zone was
transferred to the secondaries. The secondaries that were NSEC3 aware
were expected to detect the parameter change. The chain of servers is
shown below.

Tested AXFR down this chain
1. A1
2. A2
3. A3
4. A4
5. A5
6. A1

The A1 server implementation was both used at the start and end, so it
got to be both a master and a slave in this sequence (as every other
server).

o Structure of the zones

Two zones were used for this test, both originating from the previous
signing test. The first zone was "empty", consisting only of the SOA
RR, the NS RRs and the necessary DNSSEC RRSets. The second zone was
more populated, called "full", and is available for reference at
<place URL here?>.
  
  This "full" zones contained wildcards, delegations and Empty Non Terminals.

  Both zones were signed with NSEC3.

o Queries we sent

The zones at the servers were checked:

* We sent AXFR queries to every server in the chain with an NSEC3
  aware querying tool, normalized and sorted the output to eventually
  hash it by MD5.

* Also, we sent queries to solicit NXDOMAIN responses to the NSEC3
  capable servers in the chain. These servers would only be capable of
  including the NSEC3 RRs in the response if they had detected the
  NSEC3 parameters.

After some time we updated the "full" zone to a new serial number, and
changed the NSEC3 parameter set inside this zone. After sending down
this modified zone down the chain of servers, we repeated the tests
mentioned above.

o Expected results

The NSEC3 unaware servers in between should treat the NSEC3 and
NSEC3PARAM records as unknown RR data (as of RFC 3597). Direct queries
should be answered, including AXFR queries from their slaves. No other
special handling is to be expected. This would still enable the NSEC3
aware servers to enable authenticated denial for the zone, even though
the zone is transferred to them via the NSEC3 unaware servers. The
zone must not change in transit (a.k.a "the telephone game").

The NSEC3 aware servers must detect the parameter change and adapt to
give correct NSEC3 RRs in NXDOMAIN responses.

o Actual results

All the AXFRed zones could be verified to be equal to each other and
also matched the original master zone file (after normalization to
sort the entries), except that the AXFR responses showed the SOA RR
twice, as usual.

After the zone update all servers successfully transferred the new
zone from their respective masters, and queries were answered
correctly with NXDOMAIN and (updated) NSEC3 records. The AXFRs from
the servers again showed the correct zone, i.e. same result as after
the first distribution down the server chain.

o Conclusion/discussion

The tested servers are either NSEC3 aware or RFC 3597 compliant and
treat NSEC3 and NSEC3PARAM as unknown RR types where necessary. This
enables the NSEC3 aware servers to operate correctly, and these NSEC3
aware servers detect the parameter update correctly. In conclusion,
the test is passed by all participating implementations.


8. Handling brokenness

Setup:

For the broken packets test we used a secure delegation to
'b.ws.nsec3.org' to Packet-server. Packet-server is not an
authoritative server in itself, but uses pre-compiled responses to
answer requests. The packet-server resides on ns4.infra.ws.nsec3.org.

The participants used validators v1, v7 and v9. These were capable of
understanding NSEC3, and configured with a trust anchor for
ws.nsec3.org. Other implementations were used as well, but since this
test focussed on nsec3 validation, we do not include it in this
report.

Structure of the packets:

The packets itself are mostly responses that miss some part of the
authenticated denial proof. Exceptions are the packets that are
ambiguous, for instance returning a NODATA for type A, while existence
of A is reflected in the NSEC3 type list.

The broken packets are in 10 different sets, what follows is a general
description of the sets

Set A. NXDOMAIN

An NXDOMAIN response was created containing three NSEC3 records. One
showing the Closest Encloser (CE), one proving the Closest Encloser
(NC), and one proving the absence of the wildcard at the Closest
Encloser (NW). The different responses in this set show none, one, two
or all three records.

Set B. standard NODATA

A standard NODATA response contains one NSEC3, showing the absence of
a type for that name. One response is a NODATA, but the NSEC3 shows
the presence of the type. One response shows a different NSEC3. One
shows the presence of the CNAME type.

Set C. Wildcard NODATA

A wildcard NODATA response contains thee NSEC3s. The CE, the NC and
the NSEC3 at the wildcard showing that the type does not exist for the
wildcard. The different responses in this set show none, one, two or
all three records

Set D. Wildcard expansion

A wildcard expansion has two NSEC3s. The CE and the NC. The different
responses in this set show none, one or two NSEC3s.

set E. Referrals to unsigned zones

A delegation to unsigned zones contain an NSEC3 that shows the absence
of the DS record. One response shows a missing NSEC3, one response
shows an NSEC3 with the DS bit set.

set F. Referrals to opted out unsigned zones.

An opted out delegation contains two NSEC3s. One shows the CE, the
other the NC, with the OO bit set. The responses contain zero or one
NSEC3, and one with the OO bit clear.

set G. NSEC3s from multiple chains.

This is an NXDOMAIN response where the three NSEC3s have different parameters.

set H. skewed NSEC3

These responses contain NSEC3s where the ownername sorts after the
next hashed ownername.

set I. same name

These responses contain NSEC3s where the ownername is equal to the
next hashed ownername

set J. multiple labels NXDOMAIN

These responses contain the responses similar to A, but the CE is not
the ownername. Furthermore these contain responses with the DNAME bit
in the CE, and a combination of other proofs.

What follows is a matrix that shows where the actual results differ
from the expected results


empty fields conform to expected result.

name    expected result                         R7
a1.b    regular NXDOMAIN                        
a2.b    missing CE                              
a3.b    missing NC                              
a4.b    missing NW                              
a5.b    missing CE, NC                          
a6.b    missing CE, NW                          
a7.b    missing NC, NW                          
a8.b    missing NC, NW, NC                      Can't prove denial,
                                                missing dnssec data

b1.b    regular NODATA                          
b2.b    NODATA, but type exists                 
b3.b    NODATA, but no matching NSEC3           No CE
b4.b    NODATA, but CNAME exists                

c1.b    regular NODATA                          
c2.b    missing CE                              
c3.b    missing NC                              
c4.b    missing NW                              no matching NSEC3
c5.b    missing CE, NC                          
c6.b    missing CE, NW                          
c7.b    missing NC, NW                          
c8.b    missing NC, NW, CE                      Can't prove denial,
                                                no dnssec data

d1.b    regular NODATA                          
d2.b    missing CE (result should be OK)        
d3.b    missing NC                              
d4.b    missing NW                              Wildcard exp, no denial
                                                of original data

ref.e1  DS missing, bit set in                  DS query gave SERVFAIL
ref.e2  missing                                 DS query gave SERVFAIL

oo.f1   missing CE                              Bogus response,
                                                probably bug.
oo.f2   missing NC                              SERVFAIL on DS query
oo.f3   missing NC, CE                          SERVFAIL on DS query
oo.f4   oo bit not set in NC                    SERVFAIL on DS query

g1.b                                            'Could not find single
                                                set of params'

h1      NC skewed                               
h2      CE skewed                               
h3      NW skewed                               

i1      nxdomain, NC same name                  AD bit set
i2      nxdomain, NW same name                  AD bit set
i3      nxdomain, CE same name                  AD bit set

1.a.j.b regular NXDOMAIN                        
2.a.j.b missing CE                              
3.a.j.b missing NC                              
4.a.j.b missing NW                              
5.a.j.b missing CE, NC                          
6.a.j.b missing CE, NW                          
7.a.j.b missing NC, NW                          
8.a.j.b missing NC, NW, CE                      Can't prove denial,
                                                no dnssec data
9.a.j.b DNAME bit in CE                         
10.a.j.b NODATA, but is delegation              
11.a.j.b delegation, but is APEX                SERVFAIL on query
12.a.j.b delegation, but is NODATA              SERVFAIL on query


name    R1              R10                     R9
a1.b    
a2.b    
a3.b    
a4.b    
a5.b    
a6.b    
a7.b    
a8.b                                            Missing DNSSEC data

b1.b    
b2.b                    Type does not exist
b3.b    
b4.b    No error        No error                No error

c1.b    
c2.b    
c3.b    
c4.b    
c5.b    
c6.b    
c7.b    
c8.b                                            Missing DNSSEC data

d1.b                    Proof not checked
d2.b                    Proof not checked       Error No CE
d3.b                    Proof not checked
d4.b                    Proof not checked

ref.e1  Untestable                              No DNSSEC data
ref.e2  Untestable                              No DNSSEC data

oo.f1                   Provably unsecured
oo.f2   Untestable      SERVFAIL on DS
oo.f3   Untestable      SERVFAIL on DS
oo.f4   Untestable      SERVFAIL on DS

g1.b                                            No CE, no NW

h1                      Validates OK            No CE
h2                                              No CE
h3                                              No NW

i1                      Validates OK            No CE
i2                      Validates OK            No CE
i3                      Validates OK            Validates OK

1.a.j                                           Unable to verify
2.a.j                                           Unable to verify
3.a.j                                           Unable to verify
4.a.j                                           Unable to verify
5.a.j                                           Unable to verify
6.a.j                                           Unable to verify
7.a.j                                           Unable to verify
8.a.j                                           Unable to verify
9.a.j   No error        No error                Unable to verify
10.a.j                  SERVFAIL                Unable to verify
11.a.j  Untestable      SERVFAIL                Unable to verify
12.a.j  Untestable      SERVFAIL                Unable to verify