The United States (U.S.) has been one of the greatest advocates for free trade since the end of World War II, and the importance of it to the U.S. economy cannot be underestimated. As many as 40 million jobs in the U.S. are directly supported by international trade as of 2018 (Trade Partnership Worldwide LLC), and as much as half of the entire U.S. manufacturing industry is supported by exports (International Trade Administration). Imports introduce new goods and boost the annual earning power of the average American by over $18,000 annually (Hufbauer and Lu, 2017).

   COVID-19 disrupted the global supply chain. The first effect was a dramatic and rapid reduction in the labor force in affected locations as workers fell ill or were forced to quarantine (Gruszczynski, L. (2020). International trade was impacted further as borders were closed to commerce and travel (Kerr, W. (2020)). This further exacerbated supply chain disruptions within and among countries as goods themselves were prioritized differently by those in quarantine. Most international trade reduced dramatically especially for manufactured goods and automobiles. However, food and agriculture supply lines were relatively unaffected in the short term due to narrow time windows between harvest and consumption, and the fact that even during a pandemic, the population still needs to eat for survival (Kerr, W. (2020)).

   It is still too early to determine if COVID-19 has permanently disrupted the global supply chain or if it has just hastened trends already underway related to the geography of trading partners and the basket of goods traded among them. These long-term trends will need to be discerned over years, isolating the effect of any prolonged economic recession, especially among countries that are not employing economic stimulus measures. The applied tariff rates of countries is one change that may or may not prove transitory or affected due to the emergence of this virus. These rates may have a confounding effect with the disruptions caused by the direct or indirect measures of COVID-19. It is still too early to tell if the pandemic will lead to a more decentralized global economic system, but this is a possible long term scenario (Gruszczynski, L. (2020).

Proposed Questions¶

 ● How has the composition and/or volume of maritime imports and exports from the US changed over time?

 ● Are there any confounding effects impacting the volume of imports and exports of the targeted commodities?

 ● How did COVID-19 impact the volume and composition of international maritime trade?

Data Collection and Processing¶

   The questions presented in the analysis are complex, interconnected, and require a variety of data sources to answer sufficiently. Data must establish historical import and export volumes and composition . There also must be data that demonstrate COVID-19’s effects on the supply chains and workforce. Finally, data must be included that establishes potential confounding effects on trade volumes, including historical rates for applied tariffs, currency exchange, and unemployment.

Maritime Imports & Exports

   The U.S. Customs and Border Protection collects information about each shipment. Descartes Datamyne, as well as other industry data sources such as IHS Markit’s PIERS and Panjiva, compile this information and enhance it with additional data fields. The data was queried from Descartes Datamyne for U.S. imports and exports separately, from January 1, 2010 through December 31, 2020. The geographic scope includes all maritime trade shipments through the United States’ ports, to and from all trading partners. The data was downloaded in 70 queries of 1 million records or less, separately for imports and exports.

   Both the imports and exports sets contain features import arrival, business name and location, product identifying features, port of arrival and departure, source country, and other characteristics relating to the shipment. Datasets will be concatenated into one table. Data consists of all bills of lading (shipment receipts) into or out of a U.S. maritime port from January 1, 2010 through December 31, 2020. Maritime trade is best measured in volumes, such as metric tonnage, TEUs, or total shipments, however the last two are relevant to maritime trade only. Trade in metric tonnage is a major indicator of the economic vitality of the national and global economy.

Preprocessing¶

   The code that was used for preprocessing and EDA can be found here. First the environment is set up with the dependencies, library options, the seed for reproducibility, and setting the location of the project directory.

Preprocess Imports¶

   Since the initial trade data had to queried in batches separately for the imports and exports, this resulted in many separate files generated. The structure of the Data directory resulted in subdirectories for Imports and Exports. Let's set the path to where the Imports data is stored. There are 55 separate .csv files, so these need to be concatenated together to make a single set. Using glob, we use the .csv commonality to read and concatenate all files matching this extension. Then the variables that are extraneous are dropped and the columns are renamed to more relatable terminology without spaces. Since there queries were completed separately, this generated missing values in the Date features so those are removed. This variable was imported as a float so a lambda function was used to remove the extra .0 resulting in a numerical value containing the year month day without hyphens. Since this is the Imports set, let's add a column denoting a characteristic about this data. Since Container_Type_Refrigerated and Container_Type_Dry have many levels, let's create a binary features by using a mask for all the observations >=1 to be recoded as 1. After this processing, the duplicates are dropped and there over 35 million rows and 17 features. Let's now view the first 5 observations. There appear to be differences in the spacing between words different features so string processing definitely need to be utilized.

Preprocess Exports¶

   There are 36 separate files for the initial Exports set. So let's move to that directory and perform the same type of concatenation as was used for the Imports set, drop features that are not needed and rename the columns. Since Short Container Description will be used to create features for this set, let's select the non-missing for this and the non missing from US_Port. The Date variable was imported the same type as in the previous processing, so let's use the same aforementioned approach. Then create the Trade_Direction feature denoting these are exports. Since these are exports, they do not have a foreign company so let's add Not Applicable to Exports for the Foreign_Company variable. Drop any duplicate rows, now totaling over 23 million observations with the same amount of columns as the Imports set.

   First, let's filter the Containerized exports. The text in Short Container Description can be utilized to create a binary feature by using the condtions that if the word *REEF*, *Cold*, or *Deg* exists in the Short Container Description feature, then we can create a new variable Container_Type_Refrigerated and assign the value equal to one. Since these exports are refrigerated, let's assign them as Container_Type_Dry equal to zero. Then, if the text in the Short Container Description feature does not include the aforementioned words, we can create a new dataframe where we assign the value equal to one to Container_Type_Dry and Container_Type_Refrigerated equal to zero. These two filtered dataframes are then concatenated together by row. For the observations which are not Containerized, we can create a new dataframe where we assign both Container_Type_Refrigerated and Container_Type_Dry equal to zero. Then this can be concatenated by row with the Containerized filtered data. Now the features Short Container Description and Containerized can be removed from the Exports set and any duplicates dropped.

Now, the Imports and Exports sets can be concatenated, and the quality of the data in regards to data types, missingness and uniqueness can be examined.

   Most of the data is complete but two variables, US_Company and Foreign_Company have missing values. TCVUSD has the most unique, which makes sense because each trade transaction mostl likely contains a different value of money. The features HS_Description and HS are strings so any small deviation in the wording, spaces or punctuation could result in differences. It's rationale there would be more foreign ports and US ports as well. Since the Imports set is larger than the Exports set, more U.S. companies than foreign makes makes logical sense.

Preprocess Concatenated Imports/Exports¶

   Let's now create a new variable called HS_Class by pulling the first two digits from the HS Description feature to make this new feature with cleaner and more homogenized labels for the HS family classes. Then we can drop HS_description, filter HS_Class for the 23 selected groups to examine for the proposed questions and convert to a string since it is a categorical variable. We can also create a new variable called HS_Mixed as a Boolean value denoting TRUE if there are more than 6 digits in HS and FALSE if there are 6 or less digits in HS. Then HS can be removed from the set.

   Now, let's set the path to the Warehouse_Construction directory where the various sets are stored, read the HS_Groups file and examine the first five observations. We can remove Description and convert HS_Class to type int. This can be used as a key to join the HS_Groups set to the main table. This new feature HS_Group_Name, which groups similar HS_Class items together, will reduce dimensionality.

   Since some of the observations contain missing information for US_Company, let's filter the sets into different datafames to process this variable to obtain the U.S. state where company is located. Using rsplit to split the text where the last comma exists results in 75 unique U.S. company states, so this method is not 100% effective. For the observations that do not contain information for the US_Company, we can allocate Not Provided to this feature and the created US_Company_State feature. The two sets are then concatenated to complete the set. Since the presence of a comma separates the companies which have the same name but exist in different states, we can utilize rsplit again, but now to remove everything after the last comma. This results in aggregrating the same companies together by removing the state information.

   Next, the Foreign_Company feature is organized in a similar pattern as US_Company, but this contains the country which the company is located. Also, the parentheses exist in this string so they need to be removed first. The observations that contain information for this variable can be selected into a set, then all of the parentheses removed, and the last two characters extracted from the string creating Foreign_Company_Country. Then the observations without foreign companies can be assigned Not Provided for Foreign_Company and Foreign_Company_Country. Lastly, the sets can be concatenated by row returning back to a single table.

Extract Names for Clustering using OpenRefine and Binning from Original Data¶

   Data dictionaries stored in .csv files are generated, which map each of the unique values from the raw fact data to standardized values using the clustering capabilities of Google OpenRefine. These dictionaries can then be joined to the raw fact data to replace entire fields with the standardized data for that field. This will reduce the occurrences of erroneous data, reduce the number of factor levels, and provide a set of key values for joining the datasets together after some preprocessing.

   To create one of these data dictionary .csv files, the unique values from one of the columns in the fact set will be extracted into a file named [field_name].csv and stored in the Keys_and_Dictionaries directory. This is then loaded into Google OpenRefine, the field containing the row number will be turned into a numeric value and then faceted. There are memory constraints of OpenRefine, so this controls the number of records fed into the clustering algorithms by selecting a range of the row numeric facets. The column containing the unique field values will be duplicated and renamed [field_name]_Clustered containing the smaller list of clustered values.

   In the current format, there is a large number of unique strings contained within the companies, countries, ports and carriers features. This will result in high dimensionality if these features are utilized in modeling techniques. There is the potential that there are spelling and formatting errors which might be contributing to this.

   In addition, ISO standard country names and abbreviations are essential for joining sets of data because mismatched names can result in the improper alignment and loss of observations. The datapackage is utilized to generate a file containing the country name and a two digit country code (ISO 3166-1).

Perform Clustering using OpenRefine¶

   Note that for this field, there are over 1.2 million unique values for US_Company, so the numeric facet for the row number column is a necessity to avoid memory issues. A range of these numeric row values will be chosen spanning 400,000 records at a time to be sent through the clustering algorithm.

   In this way, the clustering can be done in a few batches on large datasets running through several of the key collision algorithms until each batch does not produce clustering suggestions. US_Country_Clustered Diagram given below is an example of applying the Beider-Morse function to the first 400,000 records in the US_Country_Clustered column.

   The interface clearly shows which values it believes should be clustered together and which value should replace them. In this way fields are combined which probably refer to the same entity, as they vary by ordering of words or abbreviations (ex: Smith, Thomas vs Thomas Smith), by the spacing between letters and numbers (ex: Thomas Smith vs Th omasSmith), and by variations in numeric prefixes or suffixes as seen in the example above. After making the reasonable selections, merge selected & close is chosen before repeating the process with a different function like fingerprint or n-gram fingerprint until no clustering suggestions are left for that batch. Then the next batch of records is selected before repeating the process. After all the batches have been processed in this way, all of the records in the entire column can be fed into the clustering functions now that the number of unique values has been reduced. The effectiveness of this method will vary by both the amount of unique values in a field and by the quality of the data in the field itself. For the US_Company field, the number of unique values went from 1,210,791 down to 880,829, a 27.3% reduction in values.

   At this point the column containing the row numbers is dropped and the dataset can be exported to a .csv file named [Field Name]_Clustered.csv to become one of the data dictionary files. The dictionary files are stored in a directory called Keys_and_Dictionaries so let's navigate there to read the data containing the clustered U.S. company names generated by OpenRefine and perform an inner join using US_Company as the key. This original feature can then be removed from the set and any duplicate observations removed. Next, read the data containing the clustered foreign companies and perform similar processing.

   Now we can finish merging the rest of the sets with the generated clustered names using the original feature as the key starting with the trade carrier, then U.S. port, foreign company country and foreign country. The sets are merged after the foreign_country_clustered set because this requires the standard names as a key. Then Country_continent_region_key is then used to obtain the regions for Foreign_Country_Region and Foreign_Company_Country_Region fields one at a time. Then the sets can be removed to clear memory.

Postprocessing after merging clustered features by OpenRefine¶

   The results from the clustered data added to the set reveal some inconsistencies within some of the features. Therefore, let's use another str.rsplit at the last comma to isolate the US_Port_State from US_Port_Clustered so the state abbreviation can be useful as key to match with the full state names. There are also inconistencies within US_Port_State with spacing/formatting, so let's standardize the state and territory codes. Another change that is needed is to recode groups that are not actual states to demarcate these observations from the rest, so let's use NOT DECLARED. Lastly, some of the information contained within the Foreign_Country_Name_Clustered and Foreign_Company_Country_Clustered features contain verbose and inconsistent names, so let's standardize these to match the names in the tariff set, which will be used later.

   Given the large number of unique U.S. companies, let's set a threshold to retain only the U.S. companies with at least 100 total metric tons over the time period (2010-2020). We can groupby the sum and then filter the observations meeting this threshold. This creates a new feature we can name as Metric_Tons_Totals. Now, the number of U.S. companies is significantly less compared to the initial amount of companies.

Conditional Binning of U.S. and Foreign Company by Metric Tons for Mapping¶

   Let's now define a function that uses the created Metric_Tons_Totals feature to bin metric tonnage into specified groups using the following criteria:

• a. micro = 100-1000
• b. small = 1001-10000
• c. medium = 10001-100000
• d. large = 100001-1000000
• e. huge = 1000000+

   These groups can be specified in a list to create the feature Company_size where np.select assigns the conditions to the respective group. We can also use this function for processing the foreign companies, so let's rename the clustered company feature to Company. Now we can create the bins with the function, drop Metric_Tons_Totals and then rename the feature back to the original name. This processed binned data can then be merged with the main set.

   Similar methods that were used to filter out the U.S. companies who were not involved in trading over 100 metric tons during the 10 year time period were used to process the foreign companies. Conditional binning was performed and then the binned data was joined with the main set.

Outlier Testing of Quantitative Variables¶

   Since Metric_Tons will be utilized as the dependent variable or target when modeling, let's filter the observations within three standard deviations from the mean and use a boxplot to examine the data. This results in Metric_Tons less than 2,000, but the data is quite skewed. We can then filter the set to less than 250 Metric_Tons to retain a large amount of data. However, this still does contain data that is in long right tail.

Now, we can filter the data where Teus and TCVUSD are within three standard deviations and replot the distributions.

   The original Date feature needs be processed into the format comprising of year-month-day. The original string is treated as an int type with the specified format and pandas.to_datetime can be utilized to accomplish this task. A function can then be utilized to extract the year, year-week, year-month all in one block of code.

Merge Trade Data with Other Data Sources¶

Unemployment¶

   The Bureau of Labor Statistics conducts a Current Population Survey of households every month in the United States. The survey is carried out to create datasets encompassing the labor force, employment, unemployment, people not included in the labor force, and other labor force statistics. The data was collected by workers conducting the survey by telephone while the Bureau of Labor Statistics encouraged businesses to submit their data electronically. The dataset shows the raw count of national civilian unemployment by month (in thousands of people) as well as the national civilian unemployment rate also by month. The collection of the survey data was impacted by the COVID-19 pandemic.

   Source: United States, BLS. “Charts Related to the Latest ‘The Employment Situation’ News Release | More Chart Packages.” U.S. Bureau of Labor Statistics on February, 9, 2021.

Macroscopic Details

   The data consists of 3 attributes and 133 records including a date for the national civilian unemployment data for the United States only from January 2010 through December 2020. With the onset of the COVID-19 pandemic and stay at home orders across the United States, the pandemic increased unemployment rates far above where they had been throughout much of the previous decade.

   Now we can navigate to the Warehouse_Construction directory and examine the Unemployment set. The Year-Month feature contains a -, so it can be treated as a string and converted to the proper format. An inner merge can then be completed using DateTime_YearMonth as the key for the main set and Year-Month for the Unemployment set. The unemployment rate can act as a better surrogate variable than the numerical amount without adding additional ways to normalize to represent this potential confounding factor in trade imports/exports so this and the monthly time features can be removed. Lastly, to match the format of the variables without spaces present, we can rename Unemployment Rate Total as Unemployment Rate Total.

Tariffs¶

   The World Trade Organization compiles the annual applied tariff rates that member countries set towards the world, on the harmonized service code level. The dataset includes the number of tariff line items, annual averages, and minimum and maximum bounds. Countries were selected based on the United States’ top trading partners, measured in metric tonnage, during the year 2019. Tariff rates were collected for these countries for the years 2010 through 2020. This dataset also notes if the U.S. had a free trade agreement with these top trading partners for each of the 11 years. This qualitative variable will compliment our average applied tariff rate by HS chapter class.

   Source: The World Trade Organization via the data query tool downloaded on February 8, 2021.

   Source: Office of the United States Trade Representative downloaded on February 11, 2021.

Macroscopic Details

   The original datasource included only the primary countries and 18 attributes. Subsequent additions reduced the working attributes to six, but expanded the scope of countries involved, increasing total records from 4,853 to 11,892. The United States and 62 other countries are included. These are the major trading partners of the U.S., as of 2019, measured in metric tonnage. All European countries are included, regardless of their individual trade numbers with the U.S. Data was collected from the WTO and the Office of the United States Trade Representative to create a new table. Each year includes 23 records for average applied tariff rates for the HS chapter classes included in this project. There are additional attributes to indicate whether a free trade agreement exists between the country and the U.S., and whether that country is a member of the European Union. There is missing annual data if a country was not a member of the World Trade Organization (“WTO”) for any of the time period, if it was a member of the European Union (they are reported as simply the European Union), and if they have not yet submitted their 2019 or 2020 data to the WTO. This was later adjusted by assuming that the applied tariff rates were held constant for these countries if no updates were published.

   Examining the Tariffs set, Country, HS_Family, Tariff_Year can be utilized as the keys to add this information to main set using a left merge to the main table. Then European_Union, Free_Trade_Agreement_with_US can be recoded to True/False and Average_Tariff can be reformatted.

Currency¶

   Currency rates are extracted from here, which collects live rates for currencies, currency codes, financial news, commodities and crypto currencies. The data set contains 8 columns and 2,145 rows. The timeframe is from January 2010 to February 2021. All countries that are included in the tariff table are included in the currency dataset.

   Source: Stock Market Quotes & Financial News retrieved on February 12, 2021.

Macroscopic Details

   The data queried has 8,175 records and 10 attributes using a time frame from January 2010 to Feb 2021 The currency rate is an important variable to address our research questions as it has a great impact on imports and exports. If a currency rate increases, importing and exporting volumes may fall for a country as the cost of goods is too great and may look elsewhere to source. As we build our model, we would like to understand if currencies have any correlation with the other variables.

   After examining the information contained in the Currency set, unnecesary columns can be dropped. The Month_Year feature, which is actually year-month, can be processed using the same approach as what was used for Year-Month in the Unemployment. Month_Year and Country can be used as keys for the left merge to the main set and then features that initially were retained in the initial EDA can be dropped from the set. This was decided because the weekly information can provide more insight and grain than monthly trade. Also Continent level trade was not specific enough, Foreign_Country_Name_Clustered and US_Company_State contained high dimensionality and the keys to create the warehouse were not needed for modeling.

COVID-19 State Mandated Closures¶

   The U.S. government granted the state government to the abiltiy declare if/when there was an official closure of public areas / services with the goal of preventing/hindering the increased likelihood of new COVID-19 cases reaching an exponential distribution with other distributions considered given past events. This would reduce the amount of person-to-person contact since the transmission route of COVID-19 includes being able to be aerosolized via droplets, which are not visible by the human eyes when an individual is considered infectious. This included working-from-home options to maintain daily operations by working remotely.

   Source: Kaiser Family Foundation, initially compiled on April 5, 2020. The data consists of 51 rows and 3 columns.

   Now we can navigate to the Warehouse_Construction directory, and read/examine the KFF_Statewide_Stay_at_Home_Orders set. Then navigate back to Keys_and_Dictionaries directory, join the state_abbreviation_key with the main set so that the KFF_Statewide_Stay_at_Home_Orders can be joined using State as the key.

COVID-19 Cases and Deaths in the United States¶

   The New York Times has released, and is continuing to release, a series of data related to COVID-19 from state and local governments as well as health departments. This information is obtained by journalists employed by the newspaper company across the U.S. who are actively tracking the development of the pandemic. This source compiles information related tof COVID-19 across the county and state level in the United States over time. The data starts with the first reported COVID-19 case in the state of Washington on January 21, 2020. The total number of cases and deaths is reported as either “confirmed” or “probable” COVID-19. The number of cases includes all reported cases, including instances of individuals who have recovered or died. There are geographic exceptions in the states of New York, Missouri, Alaska, California, Nebraska, Illinois, Guam and Puerto Rico. Data was pulled from the New York Times Github on February 5, 2021.

Macroscopic Details

   The data ranges from January 21, 2020 to February 5, 2021 with the dimensions of 6 columns and 1,001,656 rows including time, place, fips (geographic indicator) and the number of COVID-19 cases and deaths attributed to COVID-19.

   Let's now convert the provided date using pandas.to_datetime and utilize timedelta(7, unit='d') to subtract a week. Then the weekly sum of the cases and deaths separately for each U.S. state can be calculated, generating the cases_weekly and deaths_weekly features. From the renamed Date_Weekly_COVID, the Year can be extracted and filtered to 2020 to match the extent of the trade information. Since the US_Port_State feature in the main table contains the abbreviated state name, we can merge the COVID-19 set with the state_abbreviation_key, so this can be used as a key, remove the unnecessary columns and select the non-missing information.

   Then the aggregrated weekly COVID-19 information can then be outer merged with the main table using the the keys defined as the abbreviated state name and DateTime_YearWeek. Then the State_Code feature can be dropped.

   Since COVID-19 became medically known late 2019 and early 2020 for the general public, the prior time periods contain missing data for the weekly cases and deaths given it was flu season and a rapid diagnostic test had not been developed given the introduction of a novel virua.. So let's filter the trade related column, HS_Mixed, which contains a small percentage missing observations, and fill the missing values with 0s.

   However, there are observations that do contain missing values for HS_Mixed, which are mostly likely related to trade not occurring during these weeks at a location. To retain the grain without losing all off these observations, the quantitative variables can be filled with 0s since trade did not occur in the location at the time. This subset can then be concatenated via rows to the main data.

Feature Engineering COVID-19 associated vars¶

   Next, the number of days between when a state mandated closure was announced and was made effective can be calculated by first coverting both to proper datetime format and then substracting Date_Announced from Effective_Date, creating State_Closure_EA_Diff. The original features can then be dropped.

   Next, we can examine the number of U.S. states in the set for a sanity check. Then filter the set into subsets by the weekly times when there were and were not COVID-19 cases. A pivot table with the weekly COVID date as the index can be generated that utilizes the cases_weekly feature for the different states. Then the missing case data can be filled with zeros and the first nonzero date for COVID cases can be leveraged using the pandas DataFrame.ne() method. This can then be sorted with the earliest first occurence to create the feature Time0_StateCase, or the first weekly date of COVID-19 cases in each state.

   This created set can then be left merged using place and time so that the date can be paired with the number of cases per this date. Selecting these features, which decreases the dimensionality, and the year-week can be used to create the temporary Year_M that functions as a key with US_Port_State to merge back with the complete set.

   Then, the cases from each of the weeks where COVID-19 occurred can be subtracted from the the initial amount of cases to generate a percent change feature called cases_pctdelta. For the set that contains missing values for Date_Weekly_COVID, the created features generated in the other set can be labeled with Not Applicable and zeros so the sets can be concatenated by row.

Similar methods can also be utilized for the deaths_weekly feature that were used for cases_weekly as demonstrated above.

   Let's now drop the variables not worth considering during the next steps for variable selection.

Exploratory Data Analysis (EDA)¶

Feature Selection with Random Forest and XGBoost¶

Question: How has the composition and/or volume of maritime imports and exports from the U.S. changed from 2010-2015¶

Random Forest - Feature Importance¶

   To approach this question, let's first subset the observations containing the years 2010 - 2015. Then drop time related variables, including COVID-19 features, and the ones not related to the question. Then remove the rows with the columns having missing values to maximize the completeness of the set.

   To set up the RandomForestRegressor, let's create dummy variables for the categorical variables and allocate the features and the target variable, Metric_Tons. Now train the regressor using the default parameters using the parallel_backend from joblib to maximize the amount of CPU cores and then save the model as a pickle file. From the fitted model, the features can then be sorted by gini importance to identify which are the most important features.

   The most important features are:

• Teus
• TCVUSD
• HS_Group_Name_Finished_Goods
• Average_Tariff
• US_Unemployment_Rate
• Container_LCL/FCL_LCl
• Foreign_Company_Country_Continent_Asia
• HS_Group_Name_Edible_Processing
• HS_Mixed
• US_Port_Coastal_Region_Southeast
• US_Port_Coastal_Region_Northeast

   The least important features are the smallest carrier sizes and the locationas where trade most likely would not be probable of importing from or exporting to from the U.S.

XGBoost - Feature Importance¶

   XGBoost can also be utilized to run on the GPU for faster runtimes using similar methods that were used for the RFRegressor.

   When comparing the feature importances from XGBoost to Random Forest, Teus is the most important while Average_Tariff is second and TCVUSD is third. Fourth is US_Unemployment_Rate, HS_Group_Name_Finished_Goods is fifth, HS_Group_Name_Raw_Input is sixth, foreign_company_size_large is seventh, Container_LCL/FCL_LCl is eighth, Foreign_Company_Country_Continent_Asia is ninth and HS_Group_Name_Edible_Processing is 10th. The ports did not show up till 17th and 18th and they reversed in importance.

Random Forest without Teus & TCVUSD - Feature Importance

   Let's now drop the features, Teus and TCVUSD, when creating X. Using the same approaches, let's examine if the feature importance is conserved for both the default models using Random Forest and XGBoost.