TailSpend Docs

New Relic Validation — 11 October 2026

Error and Source Resolution

A standalone minified asset was loaded only in a beta browser test tab. New Relic captured tailspend-nr-browser-source-probe-20261011 and resolved it to browser-source-probe.ts, line 4, including the original source code.

Mobile probes ran only on a separate apmtest EAS channel: iOS Simulator and a separate Android validation app, preserving the user's installed Android app and its data. Preview and production channels were unchanged.

The first mobile errors exposed two issues: Expo's OTA download filename is different from the export filename, and Android's extensionless hash is not recognized by New Relic's stack parser. OTA error reporting now copies and normalizes only Expo hash filenames to index.bundle, preserving Hermes line/bytecode offsets. Maps upload as index.bundle.map under the platform update ID and native app version. Embedded and development stacks are unchanged. The app's original Error object is unchanged.

The corrected OTA reported tailspend-nr-acceptance-js-normalized-20261011 on both platforms. New Relic resolved the application frame to apps/mobile/src/monitoring/acceptance-probe.ts, line 18:19, including the source snippet. Android update: 01a12aca-987c-7c09-a874-e4b60c5c6ddf; iOS update: 01a12aca-987c-709f-b61f-437dc585edc6.

The UI still warns about InternalBytecode.js.map and cannot parse the bare generator next frame. Those are Hermes internals; the application frame and its original throw site resolve correctly. The synthetic fixture is removed from committed source, and the test channel was restored to clean application code. The iOS simulator's preview channel was restored.

A native crashNow probe also ran on both test clients. iOS reported NewRelicDemoException with occurrence 32D653B2-06D2-4EBB-B975-39CD82E222DE. Its matching simulator dSYM UUID was verified and uploaded, followed by the official symbol-map converter with native app version 1.2.0. A second fresh occurrence (99AEAF08-A61D-473C-A2C7-F7334F7AE8E6) still exposes raw native addresses after conversion. The symbolicated=true flag is insufficient proof: native frame resolution has not passed. The agent skips simulator symbols automatically; physical beta build validation with its matching symbols is still required. The Android native probe executed, but its corresponding MobileCrash event has not been observed; do not count it as passed.

Matched Lambda Overhead

Two temporary Lambda functions used identical current beta scan-worker code, execution role, Node.js 24 ARM64 runtime, 1024 MB memory, and environment, except the New Relic wrapper/layer/configuration. The test input referenced an already completed job, which returns without OCR calls or expense writes. Each mode had 8 forced cold starts and 24 warm invocations. Both temporary functions were deleted after sampling.

Metric Monitoring Off Monitoring On Difference
Median cold initialization 687.48 ms 1862.30 ms +1174.82 ms
Median cold billed duration 824.50 ms 2160.00 ms +1335.50 ms
Median warm billed duration 8.00 ms 15.00 ms +7.00 ms
Mean warm billed duration 8.29 ms 41.58 ms +33.29 ms
Maximum warm billed duration 10 ms 184 ms +174 ms
Median cold client elapsed 1072.41 ms 2488.23 ms +1415.82 ms
Median warm client elapsed 121.22 ms 142.08 ms +20.86 ms

This is a small controlled sample, not a production traffic study or a full OCR workload benchmark. The cold-start increase is material; warm medians are small but occasional flushes increase billed duration. Retain CloudWatch monitoring and the backend/web rollback option. This result does not quantify browser CPU or mobile battery/startup overhead.

Beta Rollback Validation

The Deploy workflow adds a beta-only validate_new_relic option. It deploys with monitoring disabled, verifies every service Lambda's handler, layer and monitoring configuration plus the deployed web artifact, and runs the existing integration suite. It then restores monitoring and repeats runtime verification and integration tests. Restoration is scheduled even when the disable deployment or its tests fail.

The workflow uses explicit per-run overrides; it does not modify repository, EAS or SSM variables. Normal deploy behavior is unchanged. Production cannot use the validation override. Mobile monitoring is independent of this backend/web switch. Other beta deployment runs should be avoided during the validation cycle; each deployment is serialized, but the entire cycle does not reserve beta between jobs.

The live cycle failed during the API migration before monitoring runtime verification. The monitoring-off deployment could not remove old HTTP API route settings because their routes had already been removed. CloudFormation rollback also failed while removing the new GET /admob/ssv setting, leaving TailSpend-beta-Service in UPDATE_ROLLBACK_FAILED. The monitoring restoration job ran as intended but could not update that failed stack. Both integration jobs were skipped, so this cycle does not satisfy rollback acceptance.

Recovery restored the seven existing route throttle settings before resuming CloudFormation rollback without skipping resources. The repair unblocked a subsequent main-pipeline deployment, and the service stack reached UPDATE_COMPLETE. All 17 beta Lambda functions existing during cleanup had New Relic wrappers, layers and account configuration. The deployed web bundle contains the beta browser application ID, and /welcome returns HTTP 200. No stored monitoring switch was changed. The failed cycle remains failed; a successful repeat after the current main pipeline finishes is still required. The HTTP API migration must preserve route/settings ordering during deletion as well as creation; adding a dependency on the new routes alone was insufficient for the initial attempt.

The live cycle must finish successfully before marking backend/web rollback validation complete. Client Release remains disabled while the remaining acceptance gates are unresolved.

Local Verification

View Markdown source