Delivery rate answers the wrong question
A delivery receipt describes what the network reported about a message. The question the business is asking is different: did the user complete verification while they were still in the flow? Those two numbers can move in opposite directions, and only the second one is an outcome.
The model below is built around outcomes and the distributions that explain them. It deliberately contains no target values. Every threshold worth alerting on comes from the programme's own recent history, per route and per destination — an authentication flow in one market with one provider is not comparable to another.
The outcome metrics
Verification completion. Of the authentication sessions where a code was issued, the share where verification succeeded. This is the headline number. Segment it before drawing any conclusion.
First-attempt completion and resend behaviour. How often verification completes without a resend, and the distribution of resends per session. A rising resend rate on one route or destination usually appears before completion falls, because users compensate for late messages until they give up.
Duplicate issuance. Sessions where more than one code was issued and remained valid, and sessions where a user entered a code that had been superseded. Both are self-inflicted failures and both are fixable in code.
Stale and out-of-order codes. Verification attempts using an expired code, or an earlier code arriving after a later one. High counts point at latency, not at users.
User retry loops. Sessions where the user requested a code repeatedly and never completed, and where they abandoned. This is the population support tickets come from.
The latency distributions
Three intervals, reported as distributions rather than averages, because the tail is where failure lives:
- Issue to accept — your service decides to issue a code until the provider accepts the submission. This is your own stack plus the provider's ingestion.
- Submit to final status — acceptance to a final status from the route. This is the provider, the destination network and the operator.
- Issue to verification — issuance to a successful verification attempt. This includes the human, and it is the interval that determines whether your expiry is realistic.
Compare the third distribution against the expiry you have set. The expiry should be a decision informed by that distribution and by your security policy, not a copied default — and the same reasoning defines the useful deadline after which a further attempt no longer helps the user.
The breakdowns that locate a problem
Aggregates hide the failure; every metric above needs to be sliceable by:
- Provider and route
- Country and, where available, operator
- Sender identity and template
- Device platform and application version
- Traffic class and journey (login, registration, recovery, payment approval)
- Time of day
Two comparisons are worth watching continuously:
- Delivery versus verification gap per route. A route with strong reported delivery and weak verification is delivering late, delivering something users do not trust, or reporting status that does not reflect reality.
- Application errors on the verification endpoint, separated from user errors. A validation bug looks like a delivery problem in the aggregate.
Canonical failure reasons
Provider error codes are inconsistent, so map them once into your own set, grouped by who can act:
| Group | Examples | Owner |
|---|---|---|
| Application | invalid destination format, missing template, endpoint error | Your engineering |
| Policy | rate limit reached, suppressed destination, sender not registered for market | Your operations |
| Provider | route rejected, throttled, submission error, no final status returned | Provider management |
| Destination network | filtered, blocked sender, network reject | Provider plus operator escalation |
| Recipient | unreachable, absent subscriber, handset state | Not actionable per message |
| Unknown | status not returned within the deadline | Investigate as a data-quality issue |
Retain the raw provider status and code alongside the canonical one. Without that, changing provider changes your reporting rather than your reliability.
Cost and abuse
- Cost per successful verification, not cost per message. A cheaper route that produces more resends and fewer completions is more expensive.
- Fraud and abuse signals: request velocity per number, account and network; concentrations towards unusual or high-cost destinations; requests that never lead to verification. These affect both spend and the integrity of every other metric.
Correlating the records without logging the code
Five record types have to join: the authentication session, the OTP issuance, each message attempt, each status update, and the verification outcome. One correlation identifier — generated when your service decides to issue, carried through every subsequent record and stored on both the application and messaging sides — is what makes the join possible.
The code itself is never part of that trail. Store only what is needed to validate a submitted code, plus a non-reversible reference to the issuance if you need to tie an attempt to a specific code. Ordinary application logs, analytics and support screens should never contain code values.
Measurement schema
auth_session: session_id, user_ref, journey, started_at, outcome, outcome_at
otp_issuance: issuance_id, session_id, correlation_id, issued_at, expires_at,
attempt_index, superseded_by, channel_policy
message_attempt: attempt_id, correlation_id, issuance_id, provider, route,
sender_id, country, operator, submitted_at, accepted_at,
provider_message_ref, client_reference
message_status: attempt_id, canonical_status, canonical_reason, raw_status,
raw_code, status_at, is_final
verification: verification_id, session_id, issuance_id, attempted_at, result,
failure_kind, device_platform, app_version
Every metric in this article is derivable from those five tables. Nothing in them contains a code value.
Diagnosis table
| Observation | Likely cause | First check |
|---|---|---|
| Delivery reported strong, verification down on one route | Late delivery or untrusted content on that route | Submit-to-final-status distribution and issue-to-verification distribution for that route |
| Resends up, completion flat | Messages arriving late but inside the window | Tail of submit-to-final-status by operator |
| Resends up, completion down, one country | Filtering or registration problem in that market | Canonical reasons in the destination-network group; sender registration state |
| Expired-code attempts rising | Expiry shorter than real user behaviour, or latency growth | Issue-to-verification distribution against the configured expiry |
| Superseded-code attempts rising | Multiple valid codes per session | Issuance records per session; deduplication at session level |
| Verification failures with valid, timely codes | Application or validation defect | Failure kinds on the verification endpoint, split by app version |
| Missing final statuses on one route | Provider status reporting gap | Share of attempts with no final status by provider; raise with provider |
| Cost per verification rising, volume flat | Resend growth or abuse | Requests per completed verification; velocity by number and network |
Alerting
Alert on movement against the programme's own baseline for that route and destination, over a window long enough to exclude normal variance for that volume. Every alert names an owner and an action: escalate to the provider, shift the route, pause a campaign, or open an application defect. An alert with no action attached will be ignored within a fortnight.
The related delivery mechanics are in A Practitioner's Guide to A2P SMS Delivery.