Survey data validation in Python starts before a calculation. A percentage needs a question, a population and a declared base. Otherwise, a merge can produce a tidy table that tells the wrong story. Our small Nigerian example shows how to protect those relationships with an explicit data contract.
We ran a standard-library validator against 12 deliberately designed records. Four passed the contract, four lacked required metadata and four carried incompatible values. All 12 outcomes matched their written expectations. These records are teaching fixtures, not respondents or a national accuracy sample.
Survey data validation in Python: read the source first
NITDA’s Nigeria Tech Outlook Report, listed on 13 November 2026, describes 2,147 adult telephone interviews during 26 September–9 October. Its B4 connection-reliability question instead uses 920 smartphone owners and phone sharers. The reported weighted shares are 17.1% for very frequent failure or slowness and 61.0% for occasional problems.
Annex A discloses no telephone-coverage or sex adjustment and no national demographic post-stratification. The achieved sample skews male and older. State estimates are unsupported. These limitations travel with the figures; a successful Python check cannot remove them.

Why multiplying a percentage by a base is unsafe
A weighted percentage does not necessarily equal an integer numerator divided by the displayed unweighted base. Multiplication can generate a plausible decimal, but it does not recover the observed respondent count. Rounding that decimal does not solve the problem.
Keep an estimate and its metadata in the same record. In particular, separate the unweighted respondent base from a weighted total or population denominator. If the source supplies no raw numerator, leave that field unknown rather than inventing one from a published percentage.
The small contract we implemented
Our schema stores an indicator ID, question, source page, population, unweighted base, estimate, unit, weighting status and three dates. For this teaching exercise, the allowed B4 contract names the subgroup explicitly and requires a weighted percentage between zero and 100.
The code checks required fields before numeric conversion. Next, it checks the question-specific population, base, unit and weighting tuple. Finally, it parses ISO dates and rejects a fieldwork end before its start or a publication date before fieldwork ends. The download contains the exact implementation.

Keep authentic evidence separate from fictional fixtures
The published-evidence CSV contains only two attributed aggregate rows. A separate fixture CSV holds our invented test records. Each fixture includes an expected outcome written as part of the test design. None of those invented values should enter a chart about real respondents.
Four consistent fixtures exercise the allowed range, including zero and 100. Four missing-field fixtures remove the population, source page, weighting status or fieldwork start. The final four replace the subgroup base with the full sample, use a respondent-count unit, change the population or exceed the percentage range.
That design tests specific failure paths. It does not cover every possible error. For example, a plausible but false source-page reference could still pass this simple validator. A human must trace the citation back to the document; schema validation cannot read and authenticate evidence automatically.
Observed results from the 12-case run

All 12 actual outcomes matched their expected labels on our recorded Python environment. Report this as a fixture check, with a denominator of 12 designed cases. Do not describe it as perfect real-world accuracy, a representative benchmark or independent validation of the survey.
The ledger retains the explanatory message for every row. That matters because a bare red cell does not tell a learner what to repair. Missing provenance calls for finding the source. An incompatible base calls for checking the question, not merely changing a number until the script turns green.
A safer join and review routine
Start with a stable indicator identifier and a declared question scope. Check that the ID is unique in the metadata table before joining. Then require the population, unit, period and weighting fields to match the intended analysis. Two columns called “percentage” are not enough evidence that their rows are comparable.
After the join, inspect unmatched rows and duplicate keys. Do not silently drop them. Preserve the rejected records in a review queue with a reason and source reference. If a table has changed between editions, record the edition rather than assuming the old contract still applies.
Unlock Your Unique Tech Path
Get expert tech consulting tailored just for you. Receive personalized advice and solutions within 1-3 business days.
Get StartedFor a classroom extension, add one invalid date and one duplicate metadata key, write their expected outcomes, and rerun the lab. Keep those new cases distinct from our published 12-case ledger. The Python CSV documentation explains how the standard library reads these text fields; our validator supplies the explicit rules.
Download the evidence and runnable exercise
- Published aggregate evidence with source metadata (CSV)
- Twelve deliberately designed validation fixtures (CSV)
- Expected and observed fixture outcomes (CSV)
- Blank survey data-contract worksheet (CSV)
- Python validator, fixtures and instructions (ZIP)
For another evidence boundary, read our CI file-coverage study. Our relational masking lab also shows why connected data needs checks that preserve relationships.
Methodology and limits
We executed the downloadable Python script locally on 15 November 2026 using public aggregates and fictional fixtures. The environment record, input rows, expected labels and actual messages are retained. No microdata, personal information, paid service or account changes were involved.
This is a deliberately narrow question-specific validator. It cannot recover unpublished weights, correct selection bias, establish representativeness or support state rankings. It also cannot prove every citation is correct. Its practical value is making a few dangerous assumptions visible and testable before they reach a report.
