Appendix F  
Description of the Commission’s Survey Methodology

 

Survey Methods

In its letter, the Committee on Finance of the U.S. Senate (Committee) requested that the U.S. International Trade Commission (Commission or USITC) provide a report that describes foreign censorship policies and practices and another report that quantifies their trade and economic effects on U.S. businesses. The Commission’s second report was to use, among other information sources, primary data collected from a survey of U.S. companies. To make the scope of the survey manageable and to ensure our ability to collect responses that are representative and statistically meaningful, the survey aimed to collect information on the impact of these policies and practices on U.S. businesses engaged in China. The Commission developed a questionnaire to collect this information, conducted field cognitive testing of its questionnaire with companies in June and July 2021, and submitted its questionnaire to the U.S. Office of Management and Budget for approval in August 2021. After receiving the approval in October 2021, the Commission sent the questionnaire to 3,787 U.S. companies.

Surveying for this investigation consisted of three major steps. First, the Commission combined information from multiple databases available through Bureau van Dijk (BvD) to identify companies of interest and generate a list of U.S. companies more likely to have recent operations in China. Second, the Commission randomly selected individual companies from that list to survey and sent them questionnaires. The Commission also conducted extensive outreach to the surveyed companies to ensure adequate response rates. Third, the Commission combined the responses from individual questionnaires to produce statistically representative estimates of U.S. companies’ experiences with and perceptions of direct censorship or censorship-enabling acts, policies, and practices (i.e., measures) in China and their effects on U.S. businesses.

Sampling Frame

The first step in determining which companies would receive the survey was generating the sampling frame, which is a list of companies from which the sample was selected. The list is formed with the goal of identifying$—$to the extent possible$—$U.S. companies commercially active in China, which is our target population. The metadata in BvD’s ORBIS databases provided indication of such a connection when U.S. companies active between March 2020 and March 2021 met at least one of the following criteria: (1) had foreign shareholders located in China (at least 10%); (2) had foreign subsidiaries located in China (at least 10% ownership); (3) were involved in foreign direct investment projects in China; (4) were involved in mergers and acquisitions in China; or (5) filed a patent in China.[1] The frame does not contain businesses that may have previously provided or attempted to provide products or services in China but are not currently active in China based on the above criteria. Finding these latter businesses would not be possible without an economy-wide survey which would be more resource and time intensive.

Holding companies and government entities (other than universities) were excluded from the sampling frame unless the holding company was found to have operations in the U.S. and China. Subsidiary companies were excluded when its parent company was U.S. owned and already in the population.

The Commission used stratified random sampling to select companies from the sampling frame. In the stratified sampling process, the frame is first divided into distinct strata (subpopulations), and then companies are independently selected from each stratum. By constructing strata that contain relatively homogeneous (similar) companies, stratified sampling can produce statistical estimates with lower standard errors than simple random sampling, in which all companies on the list have the same probability of being selected. Companies in this investigation were stratified through a combination of industry, size, and whether the business has operations in China or has only filed a patent in China.

The stratification plan was based on three sets of expectations. First, censorship is likely to impact businesses in service industries more than manufacturing, agriculture, or mining, so the population is stratified by industry group (Services only, Manufacturing only, or Services and manufacturing) using primary and secondary North American Classification System (NAICS) codes obtained through BvD.[2] Second, businesses with operations in China are expected to respond differently than businesses that have filed patents only in China. Third, small and medium-sized enterprises (SMEs) are expected to experience censorship-related measures differently than large businesses.

Table F.1 presents the size cut-off criteria used to stratify into SMEs and large businesses by industry group. These same criteria were used for analyses by size in chapter 2.

Table F.1 Size cutoff criteria for size-based stratum by industry group, by number of employees and revenue in billions of dollars

Prefatory text: SME = small and medium-sized enterprise.

Industry group	SME	Large
Services only	<= $1 billion in revenue	> $1 billion in revenue
Manufacturing only	<= 500 employees	> 500 employees
Services and manufacturing	<= $1 billion in revenue	> $1 billion in revenue

Source: Gartner, “Small and Midsize Business (SMB),” accessed February 24, 2022.

Note: The U.S. Small Business Administration uses North American Industry Classification System (NAICS) codes to determine whether to use dollars or number of employees for size cutoffs. In general, manufacturing NAICS codes (31$–$33) use number of employees for size classifications while other NAICS codes use dollars. USITC used these standards to determine the metric for cutoffs for industry groups.

A power analysis was conducted to determine the minimum sample size needed per stratum in order to produce statistically valid results with a 90 percent two-sided confidence interval (CI), based on the size cutoff criteria listed in table F.1.[3] A sample of 3,787 businesses based on our minimum sample size calculation per stratum was selected as shown in table F.2. Large businesses, regardless of their industry or activity in China, were sampled with certainty. SMEs with only patents in China were sampled at the lowest rate since they are less likely to have operations in China and consequently are more likely to face censorship.

Table F.2 Sample selection rates per population stratum, by business size, types of activity, and industry

SME = small and medium-sized enterprise.

Size, Activity, Industry	Population (number of companies)	Sample size (number of companies)	Sampling rate (%)
SME, Operations, Services only	412	389	94.4
SME, Operations, Manufacturing only	148	102	68.9
SME, Operations, Services and manufacturing	265	226	85.3
Large, Operations, Services only	230	230	100.0
Large, Operations, Manufacturing only	240	240	100.0
Large, Operations, Services and manufacturing	211	211	100.0
SME, Patents only, Services only	1,725	942	54.6
SME, Patents only, Manufacturing only	654	377	57.6
SME, Patents only, Services and manufacturing	1,195	580	48.5
Large, Patents only, Services only	137	137	100.0
Large, Patents only, Manufacturing only	260	260	100.0
Large, Patent only, Services and manufacturing	93	93	100.0
Total	5,570	3,787	68.0

Source: USITC calculations.

Response Rates

Based on the Commission’s authority under section 332(g) of the Tariff Act of 1930 (19 U.S.C. 1333(a)), all companies that received a questionnaire were legally required to complete it. The companies included in the sample received an initial mailing notifying them of the forthcoming questionnaire, a letter containing instructions for completing it within 30 days, and two follow-up mailings reminding them to complete the questionnaire. The Commission also conducted extensive outreach via email to all sampled companies to accommodate increased teleworking practices during the COVID-19 pandemic. The latter helped boost the response rates to a level not seen previously at the Commission.

The survey had an overall response rate of 73.1 percent. These responses include companies that were not active in China, no longer in business, still in product development phase, or were otherwise exempt from the survey. Table F.3 presents sample sizes and response rates by the sampling frame’s strata. Of the 3,787 questionnaires mailed to companies in the sampling frame, 2,767 responses were received which included 1,183 companies not active in China, no longer in business, or still in the product development phase. After all adjustments, there were 1,584 businesses active or recently active in China that provided complete and timely responses.

Table F.3 Response rates per population stratum, by business size, types of activity, and industry

SME = small and medium-sized enterprise.

Size, Activity, Industry	Sample size (number of companies)	Responses (number of companies)	Response rate (%)
SME, Operations, Services only	389	254	83.4
SME, Operations, Manufacturing only	102	75	76.2
SME, Operations, Services and manufacturing	227	173	80.4
Large, Operations, Services only	230	181	73.5
Large, Operations, Manufacturing only	240	193	78.7
Large, Operations, Services and manufacturing	211	176	65.3
SME, Patents only, Services only	941	642	88.2
SME, Patents only, Manufacturing only	377	276	69.0
SME, Patents only, Services and manufacturing	580	400	78.5
Large, Patents only, Services only	137	110	73.0
Large, Patents only, Manufacturing only	260	205	80.3
Large, Patent only, Services and manufacturing	93	82	68.2
Total	3,787	2,767	73.1

Source: USITC calculations.

Note: Responses includes businesses active in China, businesses not active in China, and any exemptions.

Weighting and Analysis of Responses

Once the Commission received completed questionnaires, it reviewed them to ensure that respondents had properly reported all data. In cases where data were missing or appeared inconsistent, the Commission contacted the respondents to obtain corrected data.

After all the data were collected and reviewed, the Commission combined the responses from individual companies to produce statistically valid estimates of the effects of censorship-related measures. As noted above, the sampling rate differed by stratum, and so did the response rates. As a result, the Commission weighted the responses of companies in different strata to produce estimates that would represent the entire population.

The weighting methodology for the foreign censorship survey responses consists of a sample selection weight and a nonresponse adjustment factor. The sample selection weight accounts for companies that were not sampled. For each stratum, g, in our frame, the sample selection weights are calculated as follows.

Sample weights = $\frac{Total number of businesses in stratum g}{Total number of sampled businesses in stratum g}$

Strata with the lowest sampling rates received the highest sample selection weights since each sampled company in these strata represented more companies in the population than sampled companies in other strata. Sample weights are then multiplied by a nonresponse adjustment factor to determine final weights. The nonresponse adjustment factor accounts for companies that did not respond to the survey which is computed as follows.

Nonresponse adjustment factor = $\frac{Total number of sampled businesses in stratum g}{Total number of responding businesses in stratum g}$

Responding businesses includes businesses that are active in China, businesses that not active in China, and exemptions.

Final weights = Sample weights × Nonresponse adjustment factor

These final weights will be applied to all responses, including those of exempt companies (table F.4).

Table F.4 Average final weights per population stratum, by business size and industry

SME = small and medium-sized enterprise.

Size and Industry	Valid responses (number of companies)	Average final weights
SME, Services only	328	2.21
SME, Manufacturing only	198	2.27
SME, Services and manufacturing	300	2.40
Large, Services only	209	1.26
Large, Manufacturing only	334	1.26
Large, Services and manufacturing	215	1.18
Total	1,584	1.79

Source: USITC calculations.

Grouping Product and Services Categories

The Commission questionnaire covered products and services that were fairly disaggregated, particularly for media and digital services. Reporting at the level of disaggregation that the data were collected would reduce the amount of information that could be reported because of the obligation to avoid disclosure of confidential business information. Hence, the Commission grouped product and service categories in two broader product groups for analysis. Table F.5 shows the products and services in question 1.3 that were grouped for analysis.[4]

Table F.5 Grouping of questionnaire product and service categories, by product and service grouping used for analysis.

Analysis product/service grouping	Questionnaire product and services
Media and digital services	Communications services
Media and digital services	Film and Television (TV)
Media and digital services	Music development and/or distribution, licensing, radio broadcasting
Media and digital services	Video game development and/or distribution
Media and digital services	Information content development and/or distribution, and educational services
Media and digital services	Search engines
Media and digital services	Social media, platforms for user-generated content (including reviews), and networking platforms
Media and digital services	E-commerce shopping platforms for goods and services
Media and digital services	Individual company’s online store or web application for selling goods or services, including, for example, provision of electronic payment services
Media and digital services	Cloud storage, computing services, and software
All others	All other services
All others	Information, Communications, and Technology (ICT) products (physical goods)
All others	All other manufactured goods
All others	Agricultural products

Source: USITC, Foreign Censorship Survey, 2021, question 1.3.

Grouping Censorship Acts, Policies, and Practices and Censorship-Enabling Acts, Policies, and Practices

The Commission questionnaire covered censorship acts, policies, and practices, and censorship-enabling acts, policies, and practices that were fairly specific. To generalize the issues for simplicity of analysis and avoid confidential data disclosure issues, the Commission grouped each censorship policy or practice into broader groups based on the characteristics of the policies (table F.6 and table F.7).

Table F.6 Grouping of direct censorship categories by acts, policies, and practices

Categories for analysis	Questionnaire acts, policies, and practices
Internet shutdowns	Short-term internet shutdowns (lasting 48 hours or less)
Internet shutdowns	Long-term internet shutdowns (lasting longer than 48 hours)
Blocking or filtering and targeted denial	Blocking or filtering of one or more of your digital products/services based on the content or as reprisal for speech-related activities
Blocking or filtering and targeted denial	Targeted denial of market access of one or more of your digital products/services based on speech related activities
Blocking or filtering and targeted denial	Targeted throttling or slowing of access to your digital products and services
Harms or threats	Harm or threats of criminal or civil harm, exit bans, or other forms of reprisal against your organization’s employees based on speech-related activities
Harms or threats	Harm or threats of criminal or civil harm, government-initiated boycotts, or other forms of reprisal against your organization, brand, or products based on speech-related activities
Restrictions on or requirements to modify content	Restrictions on or requirements to modify the content of any of your organization’s products, services, or public-facing materials on the grounds that it is objectionable for any reason

Source: USITC, Foreign Censorship Survey, 2021, question 2.1a.

Table F.7 Grouping of censorship-enabling categories by acts, policies, and practices

Categories for analysis	Questionnaire acts, policies, and practices
Data localization measures and local presence requirements	Data localization measures requiring in-country storage of data (either a copy of the data or sole location) that have affected the pursuit of business activities related to speech
Data localization measures and local presence requirements	Local presence requirements, which may include a physical location or local employees, that have affected the pursuit of business activities related to speech
Foreign ownership restrictions and licensing restrictions	Foreign ownership restrictions that have affected the pursuit of business activities related to speech
Foreign ownership restrictions and licensing restrictions	Licensing restrictions that have affected the pursuit of business activities related to speech
Requirement to only use state-approved virtual private networks	A requirement to only use state-approved virtual private networks that have affected the pursuit of business activities related to speech
All other	Overly broad, opaque, or unreasonably burdensome intermediary liability rules related to the monitoring and/or take down of user-generated content on your services
All other	Requirements to turn over personal information of customers or users that have affected the pursuit of business activities related to speech
All other	Requirements to publicly apologize for statements made by the business or by employees for speech-related activity in order to continue the pursuit of business activities in China

Source: USITC, Foreign Censorship Survey, 2021, question 2.2b.

Economic Impact of Chinese Censorship-Related Measures

Section 4 of the questionnaire asked respondents to estimate the economic impact of censorship-related measures in China. Businesses noted that quantifying these impacts is difficult for a variety of reasons including being unable to directly relate increased costs or revenue losses to Chinese censorship rather than some other barrier to trade (e.g., market access) and not having the resources to analyze the impact of censorship on its business. Despite these limitations, most of the economic impact estimates in section 4 of the questionnaire came from internal business estimates.[5]

 

 

Bibliography

Gartner. “Small and Midsize Business (SMB).” Gartner Glossary, accessed February 24, 2022. https://www.gartner.com/en/information-technology/glossary/smbs-small-and-midsize-businesses.

U.S. Department of Commerce (USDOC). Bureau of Economic Analysis (BEA). “A Guide to BEA’s Direct Investment Surveys.” Accessed March 10, 2022. https://www.bea.gov/sites/default/files/2018-04/a-guide-to-bea-direct-investment-surveys.pdf