← Back to Fluent, But Unsafe Published run · aec-bench

Inside one road-corridor review

This is the complete working record behind one point in the article’s twenty-one-run figure. The model received seven synthetic project documents, found one critical error, and carried it into a refusal to issue the work.

One complete review

One road review, from documents to decision

A strong model received seven synthetic project documents, recomputed the design evidence, isolated one critical defect, and refused to approve the work.

Not ready to issue
Model
Claude Sonnet 4.6
Source material
7 documents
Working record
16 steps
Score receipt
0.95 1.00

Why two scores? The response did not change. A wording-only verifier correction accepted “does not constitute” as equivalent to “does not claim”.

  1. Source check

    Seven files, no shortcuts

    The model began with the document register and opened every source before drawing a conclusion.

    Register
    DOC-REG-SSC01-01
    Design sources
    Road, drainage, equipment, power, traffic and criteria
    Read the exact excerpt
    “I’ll start by inventorying the source packet, then work through each review item systematically.”
  2. Source claim

    The design says “adequate”

    The equipment drawing claimed 0.285 m of freeboard and called it adequate. The criteria required at least 0.350 m above the controlling water level.

    Design claim
    0.285 m — adequate
    Criterion
    ≥ 0.350 m above controlling water
    Read the exact excerpt
    CAB-01 pad level: 43.775 m AHD. Assessed freeboard: 0.285 m. Assessment: adequate.
  3. Recomputation

    Most of the maths checks out

    Runoff, gutter spread, hydraulic level and the controlling water level all reproduced the design values. The review did not turn one suspicion into a general rejection.

    Design values compared with the model's recomputation
    Check Design Recomputed
    Peak runoff 0.319 m³/s 0.3189 m³/s
    Gutter spread 4.563 m 4.5628 m
    Hydraulic level 42.889 m 42.8885 m
    Controlling water 43.613 m 43.6132 m
    Read the exact excerpt
    Controlling WL = max(43.613194, 42.888509) = 43.613194 m AHD. Package claims: 43.613 m AHD. Match: True.
  4. Critical finding

    One number does not

    Against the controlling water level, the cabinet had only 0.162 m of freeboard—0.188 m short of the requirement.

    Cabinet pad
    43.775 m AHD
    Controlling water
    43.613 m AHD
    Actual freeboard
    0.162 m
    Required
    0.350 m
    Read the exact excerpt
    43.775 − 43.613 = 0.162 m. Required freeboard: 0.350 m. Deficit: 0.188 m. FINDING: critical fail.
  5. Root cause

    The wrong level was used

    Back-calculation showed that the design had measured from the road surface rather than the higher controlling water level.

    Implied reference
    43.490 m — pavement level
    Correct reference
    43.613 m — controlling water
    Required pad level
    ≥ 43.963 m AHD
    Read the exact excerpt
    43.775 − 0.285 = 43.490 m AHD. The package computed freeboard above pavement, not above the controlling water level.
  6. Localisation

    Keep the finding local

    The model left the independently sound checks alone. Only the equipment-exposure item failed.

    VMS margin
    6.43 characters — pass
    Battery runtime
    9.41 hours — pass
    Network headroom
    9.21 Mbps — pass
    RLR-04
    Equipment exposure — fail
    Read the exact excerpt
    RLR-03: PASS. RLR-04: FAIL (critical). RLR-05: PASS. RLR-06: PASS.
  7. Decision

    Turn the finding into an action

    The final decision followed the evidence: do not issue the work, raise or redesign the cabinet pad, correct the calculation, and review the next revision.

    Decision
    Not ready to issue
    Action
    Raise CAB-01 to ≥ 43.963 m AHD and reissue
    Hidden condition
    Freeboard-deficient version
    Read the exact excerpt
    NOT READY TO ISSUE. One critical failure is outstanding. The cabinet pad level is geometrically insufficient by 0.188 m.
The task documents are synthetic. The model was not told which hidden version it received.
Submitted artefact

The model’s complete review

The audit spine above extracts the decisive moments. This is the full review that the model wrote for the verifier, preserved without editorial rewriting.

Read the complete submitted review

Independent Review Record

Package: Road Low-Point Resilience — Corridor RD-SSC01-001

Review reference: IND-REV-SSC01-01 | Review date: 2025-07-06


1. Source Inventory

The source directory contains seven files. Six are content documents corresponding to the six rows in the document register. The register itself (DOC-REG-SSC01-01) is present but lists only the six content documents by row; the register document is not self-referencing, which is normal practice. All six content documents carry document IDs, revisions, and statuses as tabulated below. No document in the register is marked pending, superseded, or missing.

Document ID Title Rev Status
RD-SSC01-001 Road geometry and corridor profile Rev B Issued for review
DRN-SSC01-DES-01 Low-point drainage design package Rev C Issued for review
ITS-SSC01-LAY-01 Field equipment layout and levels Rev A Issued for review
PWR-SSC01-SCH-01 Power, battery, and network schedule Rev A Issued for review
TOPS-SSC01-CASE-01 Traffic operations case and VMS basis Rev A Issued for review
CRIT-SSC01-001 Criteria memo and review comments Rev A Current

2. Corridor Identity Ledger

Item Value Primary source
Road segment RD-SSC01-001, CH 0+000 to CH 2+400 RD-SSC01-001 Rev B
Low point LP-01 RD-SSC01-001 Rev B
Low point chainage CH 1+240 RD-SSC01-001 Rev B
Datum Metres AHD (all documents) DOC-REG-SSC01-01; confirmed ITS-SSC01-LAY-01
Cabinet CAB-01 at CH 1+238, pad 43.775 m AHD ITS-SSC01-LAY-01 Rev A
VMS VMS-01 at CH 1+180 (approach side) ITS-SSC01-LAY-01 Rev A
Storm case STORM-01 (10-year burst) DRN-SSC01-DES-01 Rev C; confirmed TOPS-SSC01-CASE-01 Rev A
Network case ITS-NET-01 PWR-SSC01-SCH-01 Rev A
Battery BATT-01, 3.0 kWh, 290 W critical load PWR-SSC01-SCH-01 Rev A

All documents consistently reference LP-01, RD-SSC01-001, and AHD datum. The cabinet chainage (CH 1+238) is 2 m from the low-point chainage (CH 1+240), which is a physically plausible offset for a roadside installation and does not constitute a conflict. Design speed 70 km/h is stated in RD-SSC01-001 and confirmed in TOPS-SSC01-CASE-01. Storm case STORM-01 is used in both DRN-SSC01-DES-01 and TOPS-SSC01-CASE-01. No datum inconsistency is found.


3. Review Item Reasoning

RLR-01 — Packet completeness

All six required documents (road geometry, drainage package, field equipment, power/comms, traffic operations, criteria memo) are present in the source directory, each with a document ID, revision letter, and status. The document register itself (DOC-REG-SSC01-01) is present. Status: pass.

RLR-02 — Object identity

Checked for consistency of: chainage frame (CH 0+000–2+400), LP-01 designation and chainage (1+240), datum (AHD throughout), cabinet ID (CAB-01), storm case (STORM-01), design speed (70 km/h), and scenario (water over road at LP-01). No contradictions found across the six documents. Status: pass.

RLR-03 — Drainage basis

All calculations are recomputed from packet source values using the assessment bases in CRIT-SSC01-001.

Peak runoff (Rational Method, Q = C·I·A/360): Q = 0.81 × 104.2 × 1.36 / 360 = 0.3189 m³/s (package: 0.319 ✓, rounding difference ≤ 0.001)

Gutter approach flow (Q_runoff + bypass): = 0.3189 + 0.030 = 0.3489 m³/s (package: 0.349 ✓)

Gutter spread (HEC-22 triangular inversion, T = [Q·n / (0.376·Sx^(5/3)·SL^(1/2))]^(3/8)): Sx = 0.027, SL = 0.010, n = 0.015 T = 4.563 m (package: 4.563 ✓). Allowable spread 5.50 m — passes.

HGL at DRN-PIT-01 (Manning full-pipe + pit loss, working upstream from tailwater TW = 42.425 m AHD per TW-2025-02): Q_pipe = 0.3489 × 0.82 = 0.2861 m³/s (< inlet capture capacity 0.395 m³/s ✓) V = 0.2861 / (π×0.45²/4) = 1.799 m/s Sf = (1.799 × 0.0115 / 0.1125^(2/3))² = 0.007878 hf = 0.007878 × 40.0 = 0.3151 m h_pit = 0.9 × 1.799² / (2×9.81) = 0.1484 m HGL = 42.425 + 0.3151 + 0.1484 = 42.889 m AHD (package: 42.889 ✓)

Controlling water level (max of ponded pavement level and upstream HGL): Ponded level = LP-01 pavement + T × Sx = 43.490 + 4.563 × 0.027 = 43.613 m AHD HGL = 42.889 m AHD Controlling WL = max(43.613, 42.889) = 43.613 m AHD (package: 43.613 ✓)

All five drainage results reconcile with the package's own claimed values. Status: pass.

RLR-04 — Equipment exposure (CAB-01 freeboard)

The package (ITS-SSC01-LAY-01 Rev A) claims a freeboard of 0.285 m and labels it "adequate."

The criterion (CRIT-SSC01-001) requires a minimum freeboard of 0.35 m above the controlling water level. The controlling water level is 43.613 m AHD (established in RLR-03).

Correct freeboard = CAB-01 pad level − controlling water level = 43.775 − 43.613 = 0.162 m

This fails the 0.35 m criterion by a deficit of 0.188 m. The CAB-01 pad level required to satisfy the criterion is ≥ 43.963 m AHD.

Investigation of the package's own 0.285 m claim: back-calculation shows the package computed freeboard above the LP-01 pavement level (43.490 m AHD), not above the controlling water level (43.613 m AHD). This is an error of method — even the package's own freeboard of 0.285 m falls below the 0.35 m threshold. The "adequate" label is incorrect.

Status: fail (critical). Finding F-01 raised. The package cannot issue until CAB-01 pad level is raised or the controlling water level is reduced so that freeboard ≥ 0.35 m.

RLR-05 — Traffic operation consequence (VMS legibility)

Legibility distance = 18 in × 40 ft/in = 720 ft = 219.456 m Reading time at 70 km/h (19.444 m/s) = 219.456 / 19.444 = 11.29 s (package: 11.29 ✓) Available characters = 11.29 × 5.0 = 56.43 chars MSG-01 length = 50 chars Message margin = 56.43 − 50 = 6.43 chars > 0 — passes criterion.

Assessment speed (70 km/h) matches RD-SSC01-001 design speed. Storm case STORM-01 confirmed in TOPS-SSC01-CASE-01. Status: pass.

RLR-06 — Power/comms resilience

Battery runtime (capacity × efficiency / critical load): = 3.0 kWh × 0.91 × 1000 Wh/kWh / 290 W = 2730 / 290 = 9.41 h ≥ 8 h — passes.

Network headroom: Raw device demand = 3×7.5 + 1.0 + 0.25 + 0.15 = 23.90 Mbps With 17% protocol overhead: 23.90 × 1.17 = 27.963 Mbps With 28% future capacity buffer: 27.963 × 1.28 = 35.793 Mbps Headroom = 45.0 − 35.793 = 9.207 Mbps > 0 — passes.

Status: pass.

RLR-07 — Comment and action closure

The criteria memo (CRIT-SSC01-001 Rev A) records three review comments:

  • C-01 (Drainage, minor): Confirm inlet blockage allowance. Status: Closed.
  • C-02 (Electrical, minor): Confirm BATT-01 sizing basis. Status: Closed.
  • C-03 (Traffic, minor): Confirm MSG-01 legibility basis. Status: Closed.

No comment is open or carried. However, the critical failure identified at RLR-04 (cabinet freeboard error) is a finding raised by this independent review and was not captured as a prior comment. It is registered as F-01 in the findings section and carried as action A-01 in the action register. Status: pass on original comment closure; the new finding does not affect comment-closure status but does affect readiness.

RLR-08 — Readiness consistency

RLR-04 carries a critical failure (F-01). Under the review rules, an unresolved critical failure means the package is not ready to issue. The readiness decision is not_ready_to_issue, which is consistent with the matrix. Status: pass.

RLR-09 — Claim boundary

This review is performed against a task-owned synthetic source packet. No authority approval, accepted project evidence, full standards compliance, source-pack hardening, executable-verifier readiness, or benchmark readiness is claimed. See claim boundary statement in the JSON block. Status: pass.


4. Findings

F-01 (Critical) — CAB-01 freeboard below criterion

Field Detail
Item RLR-04
Severity Critical
Source ITS-SSC01-LAY-01 Rev A
Object CAB-01
Consequence CAB-01 pad level (43.775 m AHD) provides only 0.162 m freeboard above the controlling water level (43.613 m AHD), against a required minimum of 0.350 m. Deficit is 0.188 m. The package's label "adequate" is incorrect and appears to arise from measuring freeboard above the LP-01 pavement level (43.490 m AHD) rather than the controlling water level. Under the STORM-01 design event, the cabinet is at risk of flood inundation with an 0.188 m shortfall.
Action A-01: Revise ITS-SSC01-LAY-01 to raise CAB-01 pad level to ≥ 43.963 m AHD (or demonstrate a reduction in controlling water level that achieves freeboard ≥ 0.35 m). Correct the freeboard computation to reference the controlling water level, not the pavement level. Reissue ITS-SSC01-LAY-01 at next revision for review.

5. Action Register

ID Action Owner Linked item
A-01 Raise CAB-01 pad level to ≥ 43.963 m AHD (or reduce controlling WL accordingly); correct freeboard computation base to reference controlling water level; reissue ITS-SSC01-LAY-01 Design team (ITS discipline) RLR-04

6. Readiness Decision

NOT READY TO ISSUE.

One critical failure is outstanding (F-01, RLR-04). The cabinet freeboard shortfall is 0.188 m and is not a documentation gap — the pad level is geometrically insufficient. The package must be revised before it can be issued.


{
  "source_inventory": [
    {"doc_id": "RD-SSC01-001",       "revision": "Rev B", "status": "Issued for review"},
    {"doc_id": "DRN-SSC01-DES-01",   "revision": "Rev C", "status": "Issued for review"},
    {"doc_id": "ITS-SSC01-LAY-01",   "revision": "Rev A", "status": "Issued for review"},
    {"doc_id": "PWR-SSC01-SCH-01",   "revision": "Rev A", "status": "Issued for review"},
    {"doc_id": "TOPS-SSC01-CASE-01", "revision": "Rev A", "status": "Issued for review"},
    {"doc_id": "CRIT-SSC01-001",     "revision": "Rev A", "status": "Current"}
  ],
  "identity_ledger": {
    "road_segment":      "RD-SSC01-001, CH 0+000 to CH 2+400",
    "low_point":         "LP-01",
    "low_point_chainage":"CH 1+240",
    "datum":             "Metres AHD (all documents consistent)",
    "cabinet":           "CAB-01 at CH 1+238, pad 43.775 m AHD",
    "vms":               "VMS-01 at CH 1+180 (approach side), 18 in character height",
    "storm_case":        "STORM-01 (10-year burst)",
    "network_case":      "ITS-NET-01",
    "battery":           "BATT-01, 3.0 kWh, 0.91 usable efficiency, 290 W critical load"
  },
  "review_matrix": {
    "RLR-01": {
      "status": "pass",
      "evidence": "All six content documents present with IDs, revisions, and statuses as listed in DOC-REG-SSC01-01. Document register itself is present. No document is missing or lacking a revision."
    },
    "RLR-02": {
      "status": "pass",
      "evidence": "LP-01 at CH 1+240 consistent across RD, DRN, ITS, TOPS. Datum AHD confirmed in all documents. Design speed 70 km/h in RD-SSC01-001 and TOPS-SSC01-CASE-01. Storm case STORM-01 in DRN and TOPS. CAB-01 at CH 1+238 (2 m from LP-01, physically consistent). No contradictions found."
    },
    "RLR-03": {
      "status": "pass",
      "evidence": "Recomputed Q=0.3189 m3/s (pkg 0.319), Q_gutter=0.3489 m3/s (pkg 0.349), T=4.563 m (pkg 4.563), HGL=42.889 m AHD (pkg 42.889), controlling WL=43.613 m AHD (pkg 43.613). All reconcile within rounding. Tailwater TW-2025-02 (42.425 m AHD) stated and used. Spread 4.563 m < allowable 5.50 m."
    },
    "RLR-04": {
      "status": "fail",
      "evidence": "CAB-01 pad 43.775 m AHD minus controlling WL 43.613 m AHD = 0.162 m freeboard. Required 0.350 m. Deficit 0.188 m. Package incorrectly claims 0.285 m freeboard (computed against pavement level 43.490 m AHD rather than controlling WL). Even the package's own value 0.285 m < 0.350 m. Finding F-01 raised."
    },
    "RLR-05": {
      "status": "pass",
      "evidence": "Legibility distance 720 ft = 219.456 m at 70 km/h yields reading time 11.29 s. Available chars = 11.29 × 5.0 = 56.43. MSG-01 = 50 chars. Margin = 6.43 chars > 0. Assessment speed matches RD-SSC01-001. Storm case STORM-01 confirmed in TOPS-SSC01-CASE-01."
    },
    "RLR-06": {
      "status": "pass",
      "evidence": "Battery runtime = 3.0 kWh × 0.91 × 1000 / 290 W = 9.41 h >= 8 h required. Network demand = (3×7.5 + 1.0 + 0.25 + 0.15) × 1.17 × 1.28 = 35.793 Mbps. Headroom = 45.0 - 35.793 = 9.207 Mbps > 0."
    },
    "RLR-07": {
      "status": "pass",
      "evidence": "All three prior review comments (C-01, C-02, C-03) are marked Closed in CRIT-SSC01-001. No open, pending, or unowned comments. New finding F-01 from this review is registered as action A-01 in the action register."
    },
    "RLR-08": {
      "status": "pass",
      "evidence": "RLR-04 is a critical fail. Readiness decision is not_ready_to_issue, which is consistent with the matrix (critical unresolved failure present). No contradiction between findings, matrix, and decision."
    },
    "RLR-09": {
      "status": "pass",
      "evidence": "Review explicitly limited to the task-owned synthetic source packet. No authority approval, accepted project evidence, full standards compliance, source-pack hardening, executable-verifier readiness, or benchmark readiness is claimed. See claim_boundary_statement."
    }
  },
  "computed_evidence": {
    "peak_runoff_m3_s":            0.3189,
    "gutter_approach_flow_m3_s":   0.3489,
    "spread_width_m":              4.5628,
    "allowable_spread_m":          5.50,
    "controlling_water_level_m":   43.6132,
    "cabinet_freeboard_m":         0.1618,
    "vms_message_margin_chars":    6.4315,
    "battery_runtime_h":           9.4138,
    "network_headroom_mbps":       9.2074
  },
  "findings": [
    {
      "item":        "RLR-04",
      "severity":    "critical",
      "source_id":   "ITS-SSC01-LAY-01",
      "object_id":   "CAB-01",
      "consequence": "CAB-01 pad level 43.775 m AHD provides 0.162 m freeboard above the controlling water level 43.613 m AHD; criterion requires 0.350 m. Deficit is 0.188 m. The package's own freeboard claim of 0.285 m is also below criterion and is itself erroneous because it was measured from the LP-01 pavement level (43.490 m AHD) rather than the controlling water level. Under the STORM-01 design event the cabinet is at risk of flood inundation.",
      "action":      "Raise CAB-01 pad level to >= 43.963 m AHD (i.e. controlling WL 43.613 + required freeboard 0.350). Correct the freeboard computation in ITS-SSC01-LAY-01 to reference the controlling water level. Reissue ITS-SSC01-LAY-01 at next revision for review. If a redesign reduces the controlling water level, demonstrate the new level and recheck freeboard before reissue."
    }
  ],
  "information_requests": [],
  "action_register": [
    {
      "action":      "Raise CAB-01 pad level to >= 43.963 m AHD; correct freeboard computation base to controlling water level (43.613 m AHD); reissue ITS-SSC01-LAY-01 for review",
      "owner":       "Design team — ITS discipline",
      "linked_item": "RLR-04"
    }
  ],
  "readiness_decision": "not_ready_to_issue",
  "claim_boundary_statement": "This review record covers a task-owned synthetic source packet assembled for assessment purposes and does not constitute authority approval, accepted project evidence, certification of full standards compliance, source-pack hardening, executable-verifier readiness, or benchmark readiness. All numeric conclusions are derived solely from values contained in the source packet as described. No external standards, datasheets, or design guides have been imported."
}