What is the estimated trade matrix?
ct_get_data() gives you exactly what countries have
reported to the UN Comtrade Database, and nothing else. The
main database contains no
estimates. That is usually what you want — but it means a country
that filed nothing simply is not there, and a world total built from
reported data is silently short by however much that country trades.
The Trade Matrix endpoint is a separate analytical
product that fills those gaps. It is maintained by the UN Statistics
Division, which complements reported figures with estimates so that the
reporter × partner matrix covers world trade completely.
ct_get_trade_matrix() queries it.
How much difference does that make? For food and live animals (SITC
section "0") in 2022:
trade_matrix_coverage
#> reporters_total reporters_not_reporting value_share_not_reporting
#> 1 209 43 0.0313791443 of those 209 reporters have no reported figure anywhere in this
matrix — every one of their flows is a UN estimate. They account for
only about 3% of world section-"0" exports by value, but
that is not the point. The point is which countries they are
(the ten largest are shown here):
trade_matrix_nonreporters
#> reporter_iso reporter_desc total_value
#> 1 RUS Russian Federation 30656396940
#> 2 BLR Belarus 4769748693
#> 3 HND Honduras 3206227436
#> 4 FRO Faroe Isds 2005628144
#> 5 SYR Syria 965707332
#> 6 GRL Greenland 902167565
#> 7 SDN Sudan 649557587
#> 8 BGD Bangladesh 611261071
#> 9 VEN Venezuela 545125342
#> 10 GNB Guinea-Bissau 500206359If your analysis involves Russia, Belarus or Honduras in 2022,
ct_get_data() will not tell you they are missing. It will
just return a smaller matrix. That is the case for reaching for this
endpoint.
One caution, which the next-but-one section explains: “no reported
figure in this matrix” is not quite the same as “filed nothing”. Iraq
also appears on the full list, yet it did file 2022 data with
Comtrade — the matrix estimated its food exports regardless. Read the
list as countries whose figures here are entirely estimated,
and check individual cases against ct_get_data() when the
distinction matters.
The rest of this vignette covers the things about it that are easy to get wrong.
Authentication
Just like ct_get_data(), this function needs a Comtrade
API token. If you have not set one up yet, head over to the main comtradr vignette, which walks
through obtaining and storing your key.
A basic call
Let’s build the world export matrix for food and
live animals in 2022. Leaving reporter and
partner at their default "everything" returns
the full matrix, estimates included.
matrix_0 <- ct_get_trade_matrix(
commodity_code = "0",
flow_direction = "export",
start_date = 2022,
end_date = 2022
)To keep this vignette fast and network-free, we do not call the API
when it is built. The objects shown throughout were computed from that
same query — data-raw/DATASET.R runs it once with
include_world = TRUE, so that the World rows are available
for the next section, and derives everything else from the bilateral
rows.
trade_matrix_sample is a small illustrative slice —
Montenegro’s exports in 2022, a single reporter whose partner flows mix
reported values with UN estimates. It keeps only a handful of display
columns:
str(trade_matrix_sample)
#> 'data.frame': 46 obs. of 11 variables:
#> $ reporter_iso : chr "MNE" "MNE" "MNE" "MNE" ...
#> $ reporter_desc : chr "Montenegro" "Montenegro" "Montenegro" "Montenegro" ...
#> $ partner_iso : chr "SRB" "BIH" "_X " "ITA" ...
#> $ partner_desc : chr "Serbia" "Bosnia Herzegovina" "Areas, nes" "Italy" ...
#> $ flow_desc : chr "Export" "Export" "Export" "Export" ...
#> $ cmd_code : chr "0" "0" "0" "0" ...
#> $ cmd_desc : chr "Food and live animals" "Food and live animals" "Food and live animals" "Food and live animals" ...
#> $ primary_value : num 24883516 4757266 3567222 3031961 2064945 ...
#> $ ref_year : int 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 ...
#> $ classification_code: chr "SS" "SS" "SS" "SS" ...
#> $ is_reported : logi FALSE FALSE TRUE TRUE FALSE FALSE ...
head(trade_matrix_sample)
#> reporter_iso reporter_desc partner_iso partner_desc flow_desc cmd_code
#> 1 MNE Montenegro SRB Serbia Export 0
#> 2 MNE Montenegro BIH Bosnia Herzegovina Export 0
#> 3 MNE Montenegro _X Areas, nes Export 0
#> 4 MNE Montenegro ITA Italy Export 0
#> 5 MNE Montenegro DEU Germany Export 0
#> 6 MNE Montenegro MKD North Macedonia Export 0
#> cmd_desc primary_value ref_year classification_code is_reported
#> 1 Food and live animals 24883516 2022 SS FALSE
#> 2 Food and live animals 4757266 2022 SS FALSE
#> 3 Food and live animals 3567222 2022 SS TRUE
#> 4 Food and live animals 3031961 2022 SS TRUE
#> 5 Food and live animals 2064945 2022 SS FALSE
#> 6 Food and live animals 1496952 2022 SS FALSEThe World rows, and why they will ruin your totals
This is the one that bites hardest, so it comes first.
The endpoint returns aggregate World rows — reporter
code 0, partner code 0, ISO W00 —
interleaved with the bilateral flows. They are the row margins, the
column margins and the grand total of the matrix. They are labeled
World in reporter_desc /
partner_desc, but no flag column marks them as
aggregates, so nothing stops you summing straight over them.
Here is what the full 2022 section-"0" export response
actually contains:
trade_matrix_blocks
#> block n_rows total_value
#> 1 World / World (grand total) 1 1.511925e+12
#> 2 bilateral flows 18237 1.511925e+12
#> 3 partner = World (row margin) 209 1.511925e+12
#> 4 reporter = World (column margin) 243 1.511925e+12Four blocks, each summing to the same world total. Add up
primary_value over the response as it arrives from the API
and you get four times the right answer.
ct_get_trade_matrix() therefore drops these rows by
default. If you want the margins or the grand total, ask for them
explicitly. (One exception: if you request
partner = "World", that margin is the thing you asked for
and is kept — the reporter margin and grand total are still dropped, so
the result is still safe to sum.)
# bilateral flows only -- safe to aggregate (the default)
bilateral <- ct_get_trade_matrix(
commodity_code = "0", flow_direction = "export",
start_date = 2022, end_date = 2022
)
# margins and grand total included -- do NOT sum across the whole frame
with_margins <- ct_get_trade_matrix(
commodity_code = "0", flow_direction = "export",
start_date = 2022, end_date = 2022,
include_world = TRUE
)Do not try to filter these rows using
is_aggregate. The column exists in the response,
but this endpoint never populates it: it is FALSE on every
row, World rows included. Reporter or partner code 0 is the
only reliable marker.
What is_reported actually means
The obvious reading of is_reported is “TRUE if the
country reported this, FALSE if the UN estimated it because the country
did not report”. That reading is wrong, and it will lead you astray.
is_reported is a per-cell provenance
flag:
-
TRUE— the value is the reporting country’s own figure. -
FALSE— the cell was produced or adjusted by the UNSD estimation pipeline: extrapolated from a nearby reported year, derived by mirror inversion from partner data, manually adjusted for under-reported or confidential trade, or produced by redistributing non-specified partners.
Three consequences worth internalizing:
A country that reports fully still has FALSE
cells. Germany reported 2022 in full, yet 77 of its 233 partner
cells are flagged FALSE — and 60 of those agree with the
reported figure to within rounding (a median difference of about $31,000
on values in the millions). Only 17 are revised materially, almost all
upward. The flag marks passage through the estimation pipeline, not
necessarily a changed number.
Aggregates inherit the flag. A TOTAL
row is FALSE if any underlying section was
estimated, even where every other section was reported as filed.
So the reported/estimated split is not “extra
coverage”. Over the bilateral rows of our 2022
section-"0" matrix:
trade_matrix_totals
#> is_reported total_value n_flows
#> 1 FALSE 1.022290e+11 2476
#> 2 TRUE 1.409696e+12 15761That FALSE bucket — about 7% of the total — is not
“trade the UN estimated for countries that had not reported”. It is
every cell that passed through the pipeline for any reason. Do not
present it as the coverage gap; the coverage gap is the 43 countries
shown at the top of this vignette.
To find those countries, group by reporter and ask whether any row is reported, rather than reading the flag row by row:
Commodities
Commodities are classified by SITC, and
classification_code is "SS" on every row — the
“combined” SITC list that spans revisions. (The TM you see
in the request URL is the endpoint’s dataset selector, not a
classification.) The estimation itself is done on SITC Rev.3.
commodity_code accepts:
-
"TOTAL"— all commodities combined. - One-, two- and three-digit SITC codes, e.g.
"0","01","011". - The five four-/five-digit codes the UN estimates in addition:
"7812","7841","7851","7852","78531". - The level selectors
"ag1"through"ag5", which return every code with that many digits.everythingis a synonym for"ag1". -
"all_levels", which returns the entire hierarchy at once.
The one-digit sections are:
| Code | SITC section |
|---|---|
| 0 | Food and live animals |
| 1 | Beverages and tobacco |
| 2 | Crude materials, inedible, except fuels |
| 3 | Mineral fuels, lubricants and related materials |
| 4 | Animal and vegetable oils, fats and waxes |
| 5 | Chemicals and related products |
| 6 | Manufactured goods classified chiefly by material |
| 7 | Machinery and transport equipment |
| 8 | Miscellaneous manufactured articles |
| 9 | Commodities not classified elsewhere |
A second double-counting trap lurks here. "all_levels"
returns TOTAL, the sections, and the two- and
three-digit codes nested inside them — so summing across that result
over-counts just as the World rows do. The function warns you when you
ask for it. If you want “all ten sections”, use everything
or "ag1", which return one level only.
Values, dates and flows
Only primary_value is populated, in
current US dollars, following the usual Comtrade convention: CIF-type
for imports, FOB-type for exports. The cifvalue,
fobvalue, qty and net_wgt columns
are always NA here. That is by design — the estimation is
applied to trade value only, and quantity information is stripped from
the source data.
Annual goods only. start_date and
end_date must be plain years (yyyy), as a
number or a string; 2022 and "2022" both work.
Passing "2022-01" is rejected rather than quietly
interpreted as a year. The API accepts at most 12 periods per query.
Imports and exports only. The trade matrix carries
no re-exports or other flows, so flow_direction accepts
"import", "export" or
"everything". Other values from
ct_get_ref_table('flow_direction') are rejected up front,
rather than returning a confusingly empty result.
Caveats worth carrying into your analysis
These come from the UN’s own Note on the Trade Estimation:
- Mirror-derived estimates “may not fully reflect the data collected and compiled by the countries”, because trade flows reported by partners do not match those reported by the compiling country.
- The manual adjustment of under-reported and confidential trade “is based on the best judgment of the staff” at UNSD’s Business and Trade Statistics Section.
- Aggregate totals from this matrix usually run slightly higher than the reported series.
- Because the main database holds no estimates,
ct_get_data()andct_get_trade_matrix()are not interchangeable. Figures from the trade matrix should not be presented as reported statistics.
Finally, note that this endpoint is not part of the UN Comtrade
public API documentation, which is why
ct_get_trade_matrix() is marked experimental. Its field
semantics may change without notice.
When publishing, cite the data source as “UN Comtrade”.
See also
-
ct_get_data()— the standard endpoint, which returns only reported values. Reach for it when you want exactly what countries filed, and forct_get_trade_matrix()when a missing country would distort your answer. - The main
comtradrvignette for token setup, lookup helpers and general usage.