2 Comments
May 3, 2023·edited May 3, 2023

Here's a similar approach to the problem of clustering countries based on their cultural similarity, this time using an NJ tree:

Code -

import numpy as np

import pandas as pd

from Bio import Phylo, AlignIO

from Bio.Phylo.TreeConstruction import DistanceMatrix, DistanceTreeConstructor

df = pd.read_csv('culturaldistance-80-countries-36-dimensions-2005-2014-years-table-combined.csv')

df.fillna(0,inplace=True)

names = df['Name'].copy()

del df['Name']

n = len(df)

M = df.to_numpy()

constructor = DistanceTreeConstructor()

matrix = np.tril(M).tolist()

matrix = [matrix[i][0:i+1] for i in range(0,len(matrix))]

NJTree = constructor.nj(DistanceMatrix(names=list(names.values),matrix=matrix))

Output -

_ Mali2005-2014

_|

| |_ Burkina Faso2005-2014

|

| , Lebanon2005-2014

| ____|

| | |_______ Bahrain2005-2014

|,|

|||, Kuwait2005-2014

||||

|| |___ Algeria2005-2014

||

|| _____ Philippines2005-2014

||,|

||||_ Malaysia2005-2014

|||

||| _ Zambia2005-2014

||,|

|||| __ South Africa2005-2014

|||||

| | |____ India2005-2014

| |

| | _ Guatemala2005-2014

| | ,|

| | ||_ Ecuador2005-2014

| |,|

| ||| ___ Trinidad and Tobago2005-2014

| ||||

| |||| _ Peru2005-2014

| || |,|

| || |||, Mexico2005-2014

| || ||||

| || | |___ Colombia2005-2014

| || |

| | |__ Brazil2005-2014

| |

| | __ Romania2005-2014

| |,|

| |||__ Moldova2005-2014

| ||

| || __ Poland2005-2014

| || |

| | | , Cyprus2005-2014

| | | |

| | | | _____ Serbia and Montenegro2005-2014

| | |,|,|

| |_||||| __ Italy2005-2014

| |||||__|

| |||| | __ United States2005-2014

| |||| |__|

| || | |_ Canada2005-2014

| || |

| || | _ Chile2005-2014

| | |_|

| | |_ Argentina2005-2014

| |

| |, Singapore2005-2014

| ||

| || ____ Taiwan2005-2014

| || __|

| || |____ South Korea2005-2014

| || |

| || | ___ Viet Nam2005-2014

| || |_____|

| | |______ China2005-2014

| |

| |, Kazakhstan2005-2014

| ||

| ||, Ukraine2005-2014

| ||

| || __ Russia2005-2014

| ||

| || Belarus2005-2014

| ||

| | ___ Bulgaria2005-2014

| | |

| |_| ___ Uruguay2005-2014

| ||

| || _ Hungary2005-2014

| |,|

_| |||_____ Estonia2005-2014

| ||

| | ____ Hong Kong2005-2014

| | |

| |_| , Slovenia2005-2014

| | _|

| || |__________ Japan2005-2014

| ||

| | , Spain2005-2014

| | |

| |_| , Great Britain2005-2014

| | __|

| | | |___ France2005-2014

| |_|

| | ___ Finland2005-2014

| ||

| || _ New Zealand2005-2014

| ||

| || Australia2005-2014

| ||

| | _ Germany2005-2014

| ||

| || __ Netherlands2005-2014

| | |

| |_| , Switzerland2005-2014

| | |

| |_| ___ Sweden2005-2014

| | ____|

| |__| | Norway2005-2014

| |

| |____ Andorra2005-2014

|

| _ Nigeria2005-2014

|_|

| | , Zimbabwe2005-2014

| |_|

| |___ Ghana2005-2014

|

| ___ Morocco2005-2014

,|

|| _ Palestine2005-2014

|||

| |__ Tunisia2005-2014

| |

| | , Libya2005-2014

| |_|

| |___ Qatar2005-2014

| |

| | _ Yemen2005-2014

| |__|

| |____ Iraq2005-2014

| |

| | , Jordan2005-2014

| |__|

| |______ Egypt2005-2014

|

| _ Iran2005-2014

| |

|_|__ Turkey2005-2014

| |

| | __ Kyrgyzstan2005-2014

| |_|

| | | ____ Uzbekistan2005-2014

| | |__|

| | |_____ Azerbaijan2005-2014

| |

| | ___ Georgia2005-2014

| |_|

| |_ Armenia2005-2014

|

| ___ Pakistan2005-2014

|_|

| |________ Indonesia2005-2014

|

| _______ Thailand2005-2014

||

| ______ Rwanda2005-2014

||

|______ Ethiopia2005-2014

Not as pretty, or easy to read, but I think it shows the "Eastern" clusters a little more clearly. Or, you know, more clearly if this post interface didn't suck.

Expand full comment
May 2, 2023·edited May 2, 2023

Mike!!

I had just gotten around to checking back at the old (ish) forums, to see if anybody was still posting, now that we are finally more or less in the event horizon of the present crisis, and lo and behold I stumble on this. Good on ya!

I am glad to hear you have written a book. I also see that said book appears to be published in Britain at textbook prices. This is unfortunate. Even if it weren't, would rather have the data you are using in an electronic format (the way the WEIRD stuff towards the bottom is). Do you have links to the data you are using to generate these graphs?

Now, speaking to the substance of this post.:

1. Did anybody ever get around to conclusively identifying the 19th K-Wave, and extending the data in 'Leading Sectors and World Powers' using proper figures for that commodity?

2. I see we are still suggesting that the past decision phase was somehow anomalous, and apparently should have ended after 4 years, in your view, rather than the 30 years it actually took, despite that being a fairly normal length compared to the previous ones. What metric are you using to suggest an anomaly?

3. Now, using the T&M model, are we quite certain that China is the challenger, given that that role tends to be filled by someone who doesn't end up as the next hegemon? I mean, Russia/USSR filled the challenger role during the Cold War, and has consistenly held pride of place there during the present 4T (using the S&H model, which I know you have moved away from), from the war with Georgia in 2008 on. Wouldn't a more normal decision phase involve something more like declining hegemon US vs challenger Russia leads to new hegemon China (or India, or whatever)?

4. Speaking of the (old?) Cold War, have you abandoned the idea of a reset in the hegemonic cycle in the 1990s with the fall of the Soviet Union?

5. Addressing the bit where the post becomes presciptive rather than descriptive, does this look at all likely to you? I will be honest, I found the switch from describing what is going in, to offering a very specific set of policy prescriptions on the part of the Russians, Americans, and Europeans that I hadn't seen floated anywhere else, to be a little jarring.

6. Have you been tracking the drop in the use of the dollar as a reserve currency, particularly its dramatic drop below the 50% mark in the past year? Nothing else is ready for primetime yet, but at this stage nothing is supposed to be.

7. At a first glance with the WEIRD stuff, I wanted to try clustering, to see if we could derive Huntington's civilizations empirically. So, I took the provided dataset from 'https://world.culturalytics.com/', downloaded the entire distance matrix, took the combined dataset (for 2005-2014, rather than broken into two periods), and ran it through the following Python code in my handy-dandy Jupyter notebook:

import pandas as pd

from scipy.cluster import hierarchy

from scipy.cluster.hierarchy import dendrogram

import numpy as np

import matplotlib.pyplot as plt

import scipy

df = pd.read_csv('culturaldistance-80-countries-36-dimensions-2005-2014-years-table-combined.csv')

df.fillna(0,inplace=True)

names = df['Name'].copy()

del df['Name']

n = len(df)

M = df.to_numpy()

import scipy.spatial.distance as ssd

# convert the redundant n*n square matrix form into a condensed nC2 array

y = ssd.squareform(M) # distArray[{n ch

fig, axes = plt.subplots(1, 2, figsize=(18, 13))

dendrogram = hierarchy.dendrogram(scipy.cluster.hierarchy.linkage(y, method='single', metric='euclidean',),color_threshold=.039, orientation='right',labels=names.values)

Which yield me the following:

https://drive.google.com/file/d/1EnsFSyUEXQs4RJGWySd1BuiKTEl49r_J/view?usp=share_link

Now, this was very rough work, I eyeballed the threshold, made a few assumptions with the other hyperparameters, and only used the top level score, and not the full set of dimensions, but this suggests that there is something measurably there to Huntington's groupings.

Expand full comment