Question 1

ABRAMS world trade wiki – about us

Accepted Answer

ABRAMS world trade wiki creates the foundation based on billions of data from a wide variety of sources e.g. World Trade Organization (http://wto.org), UN Comtrade (https://comtrade.un.org) as well as national government- and government related institutions, granting fully legal access to raw data based on the Freedom of Information Act. (https://de.wikipedia.org/wiki/Freedomofinformation) On ABRAMS world trade wiki all these international data are systematically processed, to create, with support of transparency, security and confidence in global trade. We share this principle with various world organizations that have been of a great help in the development of the world trade for decades through their international commitment and network:

World Trade Organization WTO:
https://www.wto.org/english/thewto_e/whatis_e/what_we_do_e.htm

United Nations Comtrade:
https://comtrade.un.org

World Customs Organization WCO:
http://www.wcoomd.org/en/about-us.aspx also http://www.wcoomd.org/en.aspx

Question 2

For whom has the knowledge portal been developed?

Accepted Answer

The far-reaching targetgroups can be roughly divided in:

Governments

Journalists

Logistics

Companies

Consultants

Researchers

Question 3

Is this data legal and why is it published?

Accepted Answer

Yes, the origin of these legally acquired raw data are publicly available sources. Any other interested party could also acquire these raw data from all relevant institutions and theoretically, consolidate it to the same processed knowledge as in the case of ABRAMS wiki, achieved by years of painstaking work, comprehensive programming and magnificent dedication of professional employees.

Countries all over the world, some since the sixties, grant fully legal access to various forms of world trade data, based on the Freedom of Information Act. (https://en.wikipedia.org/wiki/Freedomofinformation). The motivation to publish these data can be diverse, reaching from strengthening the own national economy (exporting companies can promote their products and performance as on a physical trade fair, being identified as a potential supplier, as for importing companies where global suppliers are better able to identify their product requirements, improving the position of the domestic company benefiting from competing suppliers.) till identifying illegal business practices and corruption e.g. counterfeit, import/export of prohibited or restricted goods and duty fraud.

Disclosure of corporate information is not practiced equally in all countries, in Germany for example, the annual balance sheets and profit and loss accounts of tens of thousands of companies are disclosed (www.bundesanzeiger.de) and open for the public. In other countries however, the disclosure of financial information is completely unknown and would be regarded as unthinkable, according to public opinion, financial information of often private companies should be confidential (listed companies are already subject to other publication requirements). In Germany however, yearly disclosure of financial information is a legal obligation and punished with high penalties if not made on time.

In other countries however, it is common sense to publish or grant access to trade data from companies, in order to improve world trade with maximum transparency.
Examples of benefits of transparency in world trade:

Compliance with laws and treaties, e.g. anti-dumping regulations which are registered in a structured manner by the World Trade Organization (WTO):
https://www.wto.org/english/tratop_e/adp_e/adp_e.htm

Eradicating cartels and encouraging competition by e.g. national and international antitrust authorities (see https://en.wikipedia.org/wiki/Competition_regulator):

Antitrust laws are the laws that apply to virtually all industries and to every level of business, including manufacturing, transportation, distribution and marketing. They prohibit a variety of practices that restrain trade. Examples of illegal practices are price-fixing conspiracies, corporate mergers likely to reduce the competitive vigor of particular markets, and predatory acts designed to achieve or maintain monopoly power. These kind of laws or authorities are common in almost every country with an open market economy.

Antitrust authorities in many countries share their experiences in organizations such as the European Competition Authority, the European Competition Network, the International Competition Network or the OECD.

In Germany for example, the Bundeskartellamt (federal antitrust agency) is responsible for a functioning and competitive regulatory principle. This is explicitly based on the antitrust law GWB (see https://www.gesetze-im-internet.de/gwb/), preventing unlawful or prohibited competition practices restraining free trade.

Protection of trademarks e.g. fighting organized crime of product counterfeiting by the World Customs Organization (WCO):
http://www.wcoomd.org/en.aspx

The positive arguments for transparency in world trade are obvious and are very well summarized by the basic intentions of the WTO, WCO and UN Comtrade: World Trade Organization WTO:
https://www.wto.org/english/thewto_e/whatis_e/10thi_e/10thi00_e.htm

World Customs Organization WCO:
http://www.wcoomd.org/en/about-us.aspx

UN Comtrade:
http://unstats.un.org/unsd/tradekb/Knowledgebase/What-is-UN-COMTRADE

Question 4

What is the date range of the data and are they up-to-date?

Accepted Answer

From many sources we have data since 2007 in our database. Due to reasons of clarity and performance, we have made a basic segmentation:

In "Market Intelligence" we have prepared the primary statistical import and export data from 2010 onwards. Updating a calendar year usually takes place in the middle of the following year by summarizing all reports of the reporting countries (for decades most countries of the world report categories of goods to UN Comtrade by HS codes. We are connected via an API with their databases where we process the data afterwards).

In order to present the data as complete and accurate as possible, we have interpolated missing country reports by means of algorithms and extrapolated still pending reports which are visualized as a trend. These values are marked accordingly.

In the sections for detailed analysis of commercial data e.g. "Company Transparency", we provide uniform standardized data of different databases as of 2013, because here actuality and above all, comparability of the data are most important. The preset time frame is visible in the upper left corner of the tools and can be adjusted according your needs.

In the "Free Search" section you have general access to all data back to 2007, depending on the source. Please note that some country databases start as of 2013.

Depending on the source, trade data are updated on a daily, monthly or quarterly basis. It cannot be precluded that in individual cases the data are delivered with delay. As a general rule, we will prepare and provide the data as quickly and as completely as possible. We must emphasize however, that we can not take any liability for the completeness of all trading data. All statistics, calculations and visualizations presented, are always based on data made available to us and do not represent all correlations in the total world trade.

Question 5

How to transfer data into knowledge?

Accepted Answer

The purpose of our platform is to convert a big volume of trade data (more than 6 billion records) into value for our users, enabling them to transform this information into knowledge by combining insights, generated by our various tools. In this way our customers are able to make better strategic decisions or predictions, maintaining their competitive edge.

Before we are able to publish or visualize this information, the data has to be processed and analyzed to guarantuee a certain quality and accessibility.

To get a better understanding of this tranformation process we have described each step in detail for you below:

1. Collect or Extract

Our data is coming from a variety of sources, most of the time semi- or unstructured and transaction based. This means that the data are not conform to a standard relational model, not always selfdescribing, in various formats and not typically hierarchical in nature. The data are mostly generated from operational systems using data entry applications which process day-to-day transactions (shipments) but also by scanned documents through OCR software (Optical Character Recognition), generating data (text) with a certain failure rate. Or statistical data reported by various countries, mostly structured but not always validated.

2. Transform

Before the data can be imported/loaded we have to transform the data into a standard format by analyzing, decoding, renaming, transposing and mapping the data, so it will fit into our data warehouse.

3. Verify

We verify our data by by quantitiv and qualititiv analysis using a statistical and data consistency framework. These frameworks monitor the total process, from collecting till publishing/visualizing the data on our platform. In this way we can evaluate the completeness and correctness of our data against the methods and procedures of our objectives.

4. Validate

To guarantee our system meets our requirements and specifications of accuracy, repeatability, reproducibility, stability and safety, we validate our data and systems by data type-, code and cross reference- and structured validation. A few samples of our validation methods are:

Plausibility checks: Checks fields to ensure data in these fields corresponds to the expected value

Cross-system plausibility checks: Compares data in different systems to ensure it is consistent

Checksums: This is a total done on one or more numeric fields which appears in every record

Cardinality check: Checks that record has a valid number of related records

5. Cleansing

Prior to the process of organizing information in a data warehouse, data cleansing or data scrubbing is crucial. Important tasks in data cleansing are:

deduplicate / merge records

detect incorrect or missing data values because of data entry errors

inconsistent value naming conventions because of different entry formats and use of abbreviations such as ONE vs "1" or Ltd. instead of Limited

Company names are written in various ways, misspelled or addresses are incorrect or polluted with additional information.

6. Organize and standardize

Before the data can be mined they have to be organized, structured and standardized, having a common definition, format, elements and structure, fitting into later defined "business layers" for analysis and interpretation. To do so we use more than 20 types of complex algorithms and various programming languages.

7. Data enrichment

Through data enrichment we are able to offer a significant improvement in the business value of our integrated data, a few examples of enrichment are:

Matching incoming records with existing data for identification of new shipments, products, customers or suppliers

Correcting invalid data based on other data in the record or database

Interpolating missing values based on other available data

8. Data analysis

One of the most important steps in the transformation to knowledge is the analysis of the data. Besides data exploration, datamining, modeling and correlation by algorithms, are quantitative and qualitativ statistics a major part of our data analysis. Only after the right analysis, the data will transform into information which we can use to build our knowledge out of. For your understanding:

Data exploration: an approach to analyzing data sets to summarize their main characteristics, often with visual methods.

Data Mining: the computational process of discovering patterns in large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems.

Datamining involves 6 common classes of tasks:

Anomaly detection - (Outlier/change/deviation detection) The identification of unusual data records, that might be interesting or data errors that require further investigation.

Association rule learning (Dependency modelling) – Searches for relationships between variables.

Clustering – is the task of discovering groups and structures in the data that are in some way or another "similar", without using known structures in the data.

Classification – is the task of generalizing known structure to apply to new data.

Regression – attempts to find a function which models the data with the least error.

Summarization – providing a more compact representation of the data set, including visualization and report generation.

9. Visualization

The primary goal of our data visualization is to communicate information clearly and efficiently via statistical graphics, tables and information graphics. Numerical data may be encoded using dots, lines, or bars, to visually communicate a quantitative message. Effective visualization helps our users to analyze and reason about data. It makes complex data more accessible, understandable and usable. Users may have particular analytical tasks, such as making comparisons or understanding causality, and the design principle of the graphic (i.e., showing comparisons or showing causality) follows the task.

In these 9 steps ABRAMS World Trade Wiki transforms data into knowledge, we are confident our platform will be of a big help to keep your competitive edge!

Question 6

Are personal data protected?

Accepted Answer

The knowledge portal ABRAMS world trade wiki, together with other world organizations, are united in their goal to strengthen world trade through transparency, making it more efficient and secure.

For the protection of private individuals, we use our algorithms to filter out information with recognizable personal data, e.g. shipments from international expatriation companies. Should you still be able to identify your personal data, you can of course send us a notification, these will be filtered out immediately: privacy@abrams.wiki

Question 7

What about data quality and errors?

Accepted Answer

We have over the years not only processed, optimized and standardized many millions of data sets by the means of very refined algorithms, but we have also continuously invested heavily in the intelligence and labours of our co-workers for the curation and quality control of these data, so that we are extremely confident in the exceptional quality of the data that we are able to present to you.

If you would like to improve the data quality in individual cases (for example curating companies in a verifiable and meaningful way), or content doesn't appear plausible or recognizable to you, we would be pleased if you would send us a written message to: quality@abrams.wiki

Question 8

What should I generally know about the data volume and quality?

Accepted Answer

Every user of the knowledge portal ABRAMS world trade wiki should absolutely be aware of the coverage and limitations of the data for a good qualitative and quantitative assessment of all information:

Coverage

The knowledge portal ABRAMS world trade wiki contains detailed import and export data, differentiated in statistical- and trade data, reported by various authorities and institutions from around 200 countries around the world.

Statistical data: These are monthly and annual statistical data, processed, published and based on the original formats, as from 2010 till current year. In the absence of data we calculate interpolated values. The same applies to very recent or future data, these are calculated as predicted values through extrapolation and therefore seperately visualized, to be interpreted exclusively as approximated values without entitlement representing the real values.

Determination and rectification of significant deviations (outliers) in reported values is done by means of a RANSAC algorithm based on a trend calculated out of values before and after the reported value. These values are also explicitly visualized in our published data.

The statistical data per country are continuously updated, depending on the availability of national data collected by UN Comtrade.

Trade data: data disclosed by e.g. governments and governmental bodies or institutions based on the Freedom of Information Act.

In the case a country grants access to such data, there are primarily three different versions:
A) Only companies with addresses of the country concerned are disclosed, for both import- and export data only the country of the trade partner (foreign company) is published, so not the name and address of the trade partner itself. We call such data "half-open".
B) Both companies with addresses in the business relation (international suppliers and customers) are disclosed, so in an import database of a strong import nation, not only the data (importing companies) of the disclosing country are available but also the data (exporting companies) of e.g. more than 100 countries worldwide, selling to this (target) country. We call such data "fully-open".
C) In some countries transit information is also collected and disclosed. Strictly speaking this is neither an import nor an export of the disclosing country, these are different nations. Depending on the database, these data can again be type A) or B)/semi-open or fully-open.

Trade data are collected as raw information, processed through a wide variety of processes, software and algorithms, and made uniformly available in a standardized database. In total, the database contains information about approximately 75 million shipments and 250 million article descriptions involving 6.5 million companies worldwide. Trade data are continuously updated, depending on the availability of the national data provided by the concerning country.

Limitations

Users of the ABRAMS world trade wiki should have a clear understanding of the limitations of the portal, which is why we are exclusively working with representatives of professional institutions as e.g. authorities, consultants and certified bodies. The following headings set out below should be carefully read before you start working with the tools and data of the knowledge portal:

Important statement: Is the entire world trade demonstrated or represented? Clearly not!

Both statistical- and trade data always represent only a part of the total world trade. It is not a matter of what percentage is available, but rather what knowledge generally can be generated, helping decision-makers in their decision making process. It must therefore be clear that it is purely about international trade relations. Solely national activities of companies are e.g. rarely present. As a result, all generated statistics with percentages etc. are to be understood in a general way and exclusively based on the IDENTIFIED information.

With respect to the statistical data based on UN Comtrade, we expressly refer to their disclaimer, which sets the framework conditions for the interpretation of our further processed data. The disclaimer is available at: https://comtrade.un.org/db/help/uReadMeFirst.aspx

The key elements, as exemplary highlighted below, are just an example and explicitly just an extract of the official disclaimer:
The values of the reported detailed commodity data do not necessarily sum up to the total trade value for a given country dataset. Due to confidentiality, countries may not report some of its detailed trade (for example waepons). This trade will - however - be included at the higher commodity level and in the total trade value. For instance, trade data not reported for a specific 6-digit HS code will be included in the total trade and may be included in the 2-digit HS chapter.

Countries (or areas) do not necessarily report their trade statistics for each and every year. This means that aggregations of data into groups of countries may involve countries with no reported data for a specific year. UN Comtrade does not contain estimates for missing data. Therefore, trade of a country group could be understated due to unavailability of some country data.

Data are made available in several commodity classifications, but not all countries necessarily report in the most recent commodity classification. Again, UN Comtrade does not contain estimates for data of countries which do not report in the most recent classification.

When data are converted from a more recent to an older classification it may occur that some of the converted commodity codes contain more (or less) products than what is implied by the official commodity heading. No adjustments are made for these cases.

Imports reported by one country do not coincide with exports reported by its trading partner. Differences are due to various factors including valuation (imports CIF, exports FOB), differences in inclusions/ exclusions of particular commodities, timing etc. The recommendations for international merchandise trade statistics can be found in the International Merchandise Trade Statistics Compilers Manual . Additional methodological information can be found on the same web page.

Based on the before mentioned limitations of the statistical data, we emphasize additionaly that for trade data, depending on the source and extention, only a part of the international transactions is shown. It can occur that a "fully-open" database contains comprehensive information about maritime shipments but nothing about air- train- or truckfreight because this was not recorded or disclosed.

Furthermore, there is a possiblity that because of aggregation and unification of all databases, individual shipments occur multiple times e.g. a shipment of a Chilean company to a customer in the USA has been recorded in the Chilean export database, but also in the import database of the USA. We try to correlate such shipments as much as possible however, it can lead to statistical imprecisions of analysis which can e.g. better be analyzed or identified in the "FreeSearch" section or in the shipment tables of "Company Transparency".

As with statistical data from countries, although rare, trade data can occasionally be no longer available from individual countries. This might be temporary (technical problems) or for an unpredictable period of time, e.g. in case of war situations, embargo policy or legislative amendments. In such cases, all statistical analysis involved, will have imprecisions.

Question 9

Are there special terms of use for the knowledge portal?

Accepted Answer

Yes, before you are able to use the knowledge portal ABRAMS world trade wiki, you must read and agree to the terms of use. These are shown during your personal registration and are available at any time here.

Frequently Asked Questions