Β Details
Assignment 2: assignment_2.zip (https://gatech.instructure.com/courses/485354/files/65889067?wrap=1) (https://gatech.instructure.com/courses/485354/files/65889067/download?download_frd=1)
In the zip file above you will find:
A2.ipynb : a Jupyter notebook (.ipynb) file data/ : a directory called βdataβ with one or more files
To complete this assignment, follow the steps below:
- Extract the contents into a local working directory (which weβll refer to as LWD) on your machine
- Open your Conda CLI and navigate to your LWD
- Open Jupyter from the Conda CLI (a Jupyter interface should open in your default browser)
- Within the Jupyter interface, open the Jupyter notebook (.ipynb) file and follow the instructions provided there
- When you have completed all the work possible/desired:
Β with the exception of cells that include a single function definition that you added as utility function in the appropriate section, delete any cells that you added that were not already part of the notebook (such as testing cells). Extra cells can sometimes trip up the autograder and require manual intervention. The smoother all the notebooks run, the sooner students can receive feedback.
Assignment 2Β Classification of Networks
Do not add any additional cells in this assignment. If you write additional functions or print statements for testing, please remove them before submitting the assignment. Any additional functions must be defined inside the existing code cell, either as nested functions or inline with the current code. All code and comments should be written in between the lines ####IMPLEMENTATION STARTS HERE#### and ####IMPLEMENTATION ENDS HERE####. Please do not remove cell tags (e.g. βexportβ and βtestβ).
Setup
Activate the conda environment you created in Assignment A1. If you used the same environment name as shown in A1, run:
conda activate cs7280_env
If you chose a different name for your environment, replace <your_environment_name> with your actual environment name, like this:
conda activate < your_environment_name >
Imports
import copy
import math
import networkx as nx
import numpy as np
import random
import scipy as sp
Part 0Β [0 pts] Starter empirical network
We will use a starter undirected empirical network to check the various functional tools we will build for exploring networks. The sample network is as follows:
- Tortoise social networkΒ (undirected)
In the function below, load the tortoise social network data into a NetworkX graph. The data is stored at relative path"data/tortoise_lm_1998.txt"Β as an edgelist file. Use NetworkXβsΒ read_edgelistΒ function. Create usingΒ nx.Graph.
def tortoise_social_network() -> nx.Graph():
"""
Return
G networkx Graph tortoise social network
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
G = nx.Graph()
return G
####IMPLEMENTATION ENDS HERE####
Sanity Check
You can print the return graph of the function below and what should be shown is text of the form:
Graph with 40 nodes and 91 edges
print(tortoise_social_network())
Part 1Β [20 pts] Degree Analysis Warmup (Friendship Paradox)
For this part, we will build functions that allow us to analyze the degrees of nodes in relation to the degrees of their neighbors.
1.1Β [6 pts] Node degree, neighbor degree average, neighbor degree max
In the function below, compute and return a dictionary based on undirected parameter graphΒ GΒ as follows:
- The keys of the dictionary are the nodes ofΒ
G, - For each node key, the value is a 3-tupleΒ
(deg, avg_nbr_deg, max_nbr_deg), whereΒdegΒ is the degree of the node,Βavg_nbr_degΒ is the average degree of all the nodeβs neighbors, andΒmax_nbr_degΒ is the maximum degree among all the nodeβs neighbors.
def get_degree_neighbor_degree_dict(G:nx.Graph) -> dict:
"""
Params
G networkx Graph undirected network
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
deg_nbr_deg_dict = {}
####IMPLEMENTATION ENDS HERE####
return deg_nbr_deg_dict
Sanity Check
Running the cell below should yield the following output:
(6, 8.5, 10)
print(get_degree_neighbor_degree_dict(tortoise_social_network())[str(18)])
1.2Β [6 pts] Node vs neighbor degree competition (frequency)
In the function below,
- Compute the ratio of nodes (out of all nodes) in undirected parameter graphΒ
GΒ that average degree of a neighbor is strictly larger than the degree of the node itself (storing this ratio in the variableΒavg_nbr_deg_larger), and - Compute the ratio of nodes (out of all nodes) inΒ
GΒ that the maximum degree of a neighbor is strictly larger than the degree of the node itself (storing this ratio in the variableΒmax_nbr_deg_larger)
def node_to_neighbor_deg_comparison(G:nx.Graph) -> (float, float):
"""
Params
G networkx Graph undirected network
Return
avg_nbr_deg_larger float ratio of nodes for which the average neighbor degree is (strictly) larger than the node degree
max_nbr_deg_larger float ratio of nodes for which the maximum neighbor degree is (strictly) larger than the node degree
"""
####IMPLEMENTATION STARTS HERE####
# These lines are placeholders
avg_nbr_deg_larger = -1.0
max_nbr_deg_larger = -1.0
####IMPLEMENTATION ENDS HERE####
return avg_nbr_deg_larger, max_nbr_deg_larger
1.3Β [4 pts] Node vs neighbor degree competition (average)
In the body of the function below, for the undirected parameter graphΒ G:
- Compute the average degree among all the nodes ofΒ
G, and - Compute the average degree among all the neighbors of nodes ofΒ
G. Note that any given node is likely to be counted as a neighbor multiple times because it is likely the neighbor of multiple other nodes (and, in fact, this multiple counting is why the Friendship Paradox exists).
def avg_deg_and_avg_nbr_deg(G:nx.Graph) -> (float, float):
"""
Params
G networkx Graph undirected network
Return
avg_deg float average degree among all nodes of the parameter network G
avg_nbr_deg float average degree among all neighbors of the parameter network G
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
avg_deg = -1.0
avg_nbr_deg = -1.0
####IMPLEMENTATION ENDS HERE####
return avg_deg, avg_nbr_deg
Sanity Check
Running the cell below should yield the following output:
(4.55, 7.0989010989010985)
print(avg_deg_and_avg_nbr_deg(tortoise_social_network()))
1.4Β [4 pts] Does a network display the friendship paradox?
In the body of the function below, return a boolean ofΒ TrueΒ if the parameter networkΒ GΒ displays the friendship paradox (the average degree of the graphβs nodes is less than the average degree of the neighbors of nodes) andΒ FalseΒ otherwise.
def network_displays_friendship_paradox(G:nx.Graph) -> bool:
"""
Params
G networkx Graph undirected network
Return
disp_friend_pdx boolean True if the parameter network G displays the friendship paradox. False otherwise
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
disp_friend_pdx = None
####IMPLEMENTATION ENDS HERE####
return disp_friend_pdx
Part 2Β [20 pts] Degree Analysis (Power Law Degree Distribution)
A network with a degree distribution that follows a power lawΒ Β forΒ Β is often called aΒ scale-free networkΒ since it will have the property that no matter how many nodes/edges the network has, the degree distribution will look similarly (ie, most nodes will have a degree that is much small than the average degree, while very few yet definitely some nodes will have degree that is significantly higher than the average degree).
There is ongoing discussion in the literature regarding the frequency and domain-preference of scale-free network appearance. Some articles to read if you are interested:
- Broido and Clauset (2018) β Scale-free networks are rare
- Holme (2019) β Rare and everywhere: Perspectives on scale-free networks
- Artico et al (2020) β How rare are power-law networks really?
- Smith et al (2020) β Scarcity of scale-free topology is universal across biochemmical networks
- Liu et al (2021) β Scale free is not rare in international trade networks
2.1Β [2 pts] Degree distribution (natural, unmodified)
In the function below, for the parameter graphΒ G:
- Compute all the unique degrees of the nodes in the parameter graphΒ
GΒ and store that information in the listΒdegs. For example, if two nodes ofΒGΒ have degree 3, the value of 3 would only be recorded once in the list. Make sure this list is sorted in ascending order (degree 1 should occur before degree 2, for instance). - For each unique degree inΒ
degs, find the corresponding count of how many nodes have that specific degree. Store that information in the variableΒcounts.
def degree_distribution(G:nx.Graph) -> (list, list):
"""
Params
G networkx Graph undirected network
Return
degs list a list of integers holding all the (unique) degrees held by at least one node of G
counts list a list of integers holding all the corresponding counts of the degree values in degs
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
degs, counts = [], []
####IMPLEMENTATION ENDS HERE####
return degs, counts
Sanity Check
Running the cell below should yield the following output:
([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13], [7, 9, 7, 2, 1, 2, 2, 1, 5, 2, 1, 1])
print(degree_distribution(tortoise_social_network()))
2.2Β [10 pts] Degree distribution (logarithmically scaled and binned)
In the body of the function below, take the degrees and counts from the previous subpart and do the following:
- Exchange each degreeΒ Β and count valueΒ Β withΒ Β andΒ Β using NumPyβsΒ
logΒ function, - Using theΒ
bin_widthΒ value, create bins of that width. For each bin intervalΒ , average the log-scaled degree valuesΒ Β for values that fall within that bin to generate the valueΒ Β for that bin. For each log-scaled degree valueΒ , average the corresponding log-scaled count valuesΒ Β to the get the corresponding valueΒ Β for that bin. Store the valuesΒ Β andΒ Β at the same index level in theΒdegs_logΒ andΒcounts_logΒ lists.
Note:Β Ensure that binning starts from 1. After binning, make sure to remove any bins that have no data points (i.e. empty bins), even if the output appears correct without this step. Removing empty bins is essential for the correctness of the final result.
def deg_dist_binned_and_log_scale(G:nx.Graph, bin_width:float=0.25) -> (list, list):
"""
Params
G networkx Graph undirected network
bin_width float the horizontal width of the bins
Return
degs_log list a list of values corresponding to the degs variable from the last sub part, but
scaled with natural log
binned into bins of width bin_width
counts_log list a list of values corresponding to the counts variable from the last sub part, but
scaled with natural log
binned into bins of width bin_width
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
degs_log, counts_log = [], []
####IMPLEMENTATION ENDS HERE####
return degs_log, counts_log
Sanity Check
Running the cell below should yield the following output:
([1.729811471360218, 2.4218099077513178], [0.8049699433098928, 0.23104906018664842])
print(deg_dist_binned_and_log_scale(tortoise_social_network(), 1.2))
2.3Β [8 pts] Determine if a network is scale free or not
In the function below:
- perform a statistical hypothesis test for linear correlation using SciPyβsΒ
sp.stats.pearsonΒ function, - Compare the p-value returned from the hypothesis test to the passed significance levelΒ Β stored in theΒ
sig_level_alphaΒ argument, - Check the Pearson correlation coefficient returned from the hypothesis test,
- If the p-value from above is less than the significance level and the Pearson correlation coefficient is negative, then return a boolean ofΒ
True. Otherwise, return a boolean ofΒFalse.
Note: Use the log-scaled and binned degree distribution generated by the deg_dist_binned_and_log_scale function with its default bin width as the input to the Pearson correlation test. Ensure that you use the two-sided version of the test provided by theΒ sp.stats.pearson.
def is_scale_free_network(G:nx.Graph, sig_level_alpha:float) -> bool:
"""
Params
G networkx Graph undirected network
sig_level_alpha float significance level
probability of making a type I error
Return
network_is_scale_free boolean True if network is scale free, False otherwise
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
network_is_scale_free = None
####IMPLEMENTATION ENDS HERE####
return network_is_scale_free
Part 3Β [20 pts] Clustering Analysis
The small-world property of a network is identified by a network having two properties:
- High clustering
- Low average shortest path length
In this part, we build tools for analyzing whether a network has high average clustering compared to a random network.
3.1Β [2 pts] Average clustering of a network
In the function below, compute and return the average clustering value for the parameter networkΒ GΒ by using NetworkXβsΒ average_clusteringΒ function.
def network_average_clustering(G:nx.Graph) -> float:
"""
Params
G networkx Graph undirected network
Return
net_cc float average clustering value for the network G
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
net_cc = -1.0
####IMPLEMENTATION ENDS HERE####
return net_cc
3.2Β [2 pts] Transitivity of a network
In the function below, compute and return the transitivity for the parameter networkΒ GΒ by using NetworkXβsΒ transitivityΒ function.
def network_transitivity(G:nx.Graph) -> float:
"""
Params
G networkx Graph undirected network
Return
net_trans float transitivity of the network
"""
####IMPLEMENTATION STARTS HERE####
# This is a placceholder
net_trans = -1.0
####IMPLEMENTATION ENDS HERE####
return net_trans
3.3Β [6 pts] Distinction between average clustering and transitivity
In the function below, construct and return a graph with:
- NodesΒ , whereΒ Β is theΒ
node_ctΒ value, - An edge betweenΒ Β andΒ ,
- An edge betweenΒ Β and everyΒ Β forΒ ,
- An edge betweenΒ Β and everyΒ Β forΒ ,
def example_network(node_ct:int) -> nx.Graph():
"""
Params
node_ct int number of nodes for the network
Return
G networkx Graph Graph with nodes A,B,C_1,...,C_{n-2}
edge between A and every C_i
edge between B and every C_I
edge between A and B
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
G = nx.Graph()
node_ct = max(node_ct, 2)
####IMPLEMENTATION ENDS HERE####
return G
Run the cell below to observe how the average clustering and the transitivity of the example network changes as the number of nodes grows.
# Run this cell (when the appropriate functions are ready), but do not modify the contents
for i in (10, 100, 1000):
print()
G_example = example_network(i)
print(G_example)
print("network average clustering : ", round(network_average_clustering(G_example),3))
print("network transitivity : ", network_transitivity(G_example))
Use the information from the run cell above to determine what the limits of the average clustering and the transitivity of the example network is as the number of nodes tends toward positive infinity. Put those limits in the function below. Note that all that needs to be changed about the function below is the two float values.
def avg_clust_trans_tend_toward() -> (float, float):
"""
Return
avg_clust_tends_toward float the limit of the average clustering as n -> infty
trans_tends_toward float the limit of the transitivity as n -> infty
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
avg_clust_tends_toward = -3.14
trans_tends_toward = -3.14
####IMPLEMENTATION ENDS HERE####
return avg_clust_tends_toward, trans_tends_toward
3.4Β [10 pts] Statistically high average clustering
In the function below:
- Generate 100 random networks using NetworkXβsΒ
fast_gnp_random_graph. When generating the random networks,ΒnΒ should be the number of nodes ofΒGΒ andΒpΒ should be the proportion of edges (to the total possible) ofΒG. After generating a random graph, remove all self-loops (disregarding the fact that this will generally lessen the proportion of edges that the random graph has when compared toΒG. - For each random graphΒ , compute the average clustering ofΒ Β and store that value.
- Using SciPyβsΒ
sp.stats.ttest_1samp, perform a one-sample t-test with the sample being the list of average clustering values for the random networks and the constant value being the average clustering of the argument networkΒG. Use the appropriate value for theΒalternativeΒ parameter (one of βtwo-sidedβ, βlessβ, or βgreaterβ).
def network_has_stat_high_avg_clustering(G:nx.Graph, sig_level_alpha:float) -> bool:
"""
Params
G networkx Graph undirected network
sig_level_alpha float significance level
ie, probability of making a type I error
Return
stat_high_avg_c boolean statistically high average clustering
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
stat_high_avg_c = None
####IMPLEMENTATION ENDS HERE####
return stat_high_avg_c
Part 4Β [20 pts] Path Length Analysis
The small-world property of a network is identified by a network having two properties:
- High clustering
- Low average shortest path length
In this part, we build tools for analyzing whether a network has low average shortest path length compared to a random network.
4.1Β [4 pts] Average shortest path length of network
In the function below, compute and return the average shortest path length of the parameter graphΒ GΒ using NetworkXβsΒ average_shortest_path_lengthΒ function.
def network_average_shortest_path_length(G:nx.Graph) -> float:
"""
Params
G networkx Graph undirected network
Return
network_aspl float average shortest path length of the network
"""
####IMPLEMENTATION STARTS HERE####
# This is a place holder
network_aspl = -1.0
####IMPLEMENTATION ENDS HERE####
return network_aspl
4.2Β [16 pts] Statistically low average shortest path length
In the function below:
- Generate 100 random networks using NetworkXβsΒ
fast_gnp_random_graph. When generating the random networks,ΒnΒ should be the number of nodes ofΒGΒ andΒpΒ should be the proportion of edges (to the total possible) ofΒG. After generating a random graph, remove all self-loops (disregarding the fact that this will generally lessen the proportion of edges that the random graph has when compared toΒG. - For each random graphΒ , compute the average shortest path length ofΒ Β and store that value. When a random graph is not connected, take the largest connected component of the graph and compute the average shortest path length over it.
- For each average shortest path lengthΒ Β of a random graph, compute the ratioΒ Β betweenΒ Β and the average shortest path lengthΒ Β of the argument networkΒ
G. Take the average among all the differences and divide byΒ Β to get an average proportionality difference. If this is less than the parameter valueΒsig_level_alpha, returnΒTrue. Otherwise, returnΒFalse.
def network_has_stat_low_avg_path_length(G:nx.Graph, sig_level_alpha:float) -> bool:
"""
Params
G networkx Graph undirected network
sig_level_alpha float significance level
ie, probability of making a type I error
Return
stat_low_avg_pl boolean statistically low average path length
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
stat_low_avg_pl = None
####IMPLEMENTATION ENDS HERE####
return stat_low_avg_pl
Part 5Β [20 pts] Network Classification
In this part, you will bring the functional tools youβve built to bear on a set of undirected empirical networks with the goal of correctly characterizing and classifying them.
5.1Β [4 pts] Suite of empirical networks
We consider the undirected empirical networks shown below:
| Network name | Network type | Source URL | Directedness | Relative path |
|---|---|---|---|---|
| Bat food sharing | Mammal social network | Source | Undirected | data/bat_foodsharing.edgelist |
| Chesapeake road | Transportation network | Source | Undirected | data/chesapeake_road_data.txt |
| Contiguous US states | Geographic network | Source | Undirected | data/contiguous_us_data.txt |
| Tubular reactor | Flow network | Source | Undirected | data/tubular_reactor_data.txt |
In the body of the function below, use NetworkXβsΒ read_edgelistΒ function to load each of theΒ undirectedΒ networks in the table above into aΒ nx.GraphΒ object and return the graphs in the order shown in the docstring of the function. Note that several of the graph data files (such as the bat food-sharing file) are equipped with edge weights. We will not use these weights, but one will need to feed theΒ read_edgelistΒ function the proper arguments in order to read the data correctly. See the example on theΒ read_edgelistΒ documentation page withΒ nodetype=int, data=(("weight", float),)).
def empirical_undirected_networks() -> tuple:
"""
Return
G_bat_food_sharing networkx Graph
G_chesapeak_road networkx Graph
G_contiguous_us networkx Graph
G_tubular_reaactor networkx Graph
"""
####IMPLEMENTATION STARTS HERE####
# These are placeholder lines
G_bat_foodsharing = nx.Graph()
G_chesapeak_road = nx.Graph()
G_contiguous_us = nx.Graph()
G_tubular_reactor = nx.Graph()
####IMPLEMENTATION ENDS HERE####
return G_bat_foodsharing, G_chesapeak_road, G_contiguous_us, G_tubular_reactor
Sanity Check
Running the cell below should yield the following output:
Graph with 21 nodes and 72 edges
Graph with 39 nodes and 170 edges
Graph with 49 nodes and 107 edges
Graph with 100 nodes and 248 edges
# Run but do not modify the contents of this cell
for g in empirical_undirected_networks():
print(g)
5.2Β [4 pts] Classify networks as displaying the Friendship Paradox or not
Run the cell below to see what the functional tools youβve built have to say about whether each of the undirected empirical networks displays the Friendship Paradox or not.
# Run this cell, but do no modify the contents
for g in list(empirical_undirected_networks()):
print(network_displays_friendship_paradox(g))
Fill in the correct boolean in the function below to match the results of the run cell above.
def empirical_networks_friendship_paradox_results() -> (bool, bool, bool, bool):
"""
Return
bat_foodsharing_shows_friendship_paradox bool
chesapeak_road_shows_friendship_paradox bool
contiguous_us_shows_friendship_paradox bool
tubular_reactor_shows_friendship_paradox bool
"""
bat_foodsharing_shows_friendship_paradox = None
chesapeak_road_shows_friendship_paradox = None
contiguous_us_shows_friendship_paradox = None
tubular_reactor_shows_friendship_paradox = None
return bat_foodsharing_shows_friendship_paradox, chesapeak_road_shows_friendship_paradox, contiguous_us_shows_friendship_paradox, tubular_reactor_shows_friendship_paradox
5.3Β [4 pts] Classify networks as scale free or not
Run the cell below to see what the functional tools youβve built have to say about whether each of the undirected empirical networks is a scale-free (ie, power law) network or not.
# Run this cell, but do no modify the contents
for g in list(empirical_undirected_networks()):
print(is_scale_free_network(g, 0.05))
Fill in the correct boolean in the function below to match the results of the run cell above.
def empirical_networks_scale_free_results() -> (bool, bool, bool, bool):
"""
Return
bat_foodsharing_is_scale_free bool
chesapeak_road_is_scale_free bool
contiguous_us_is_scale_free bool
tubular_reactor_is_scale_free bool
"""
bat_foodsharing_is_scale_free = None
chesapeak_road_is_scale_free = None
contiguous_us_is_scale_free = None
tubular_reactor_is_scale_free = None
return bat_foodsharing_is_scale_free, chesapeak_road_is_scale_free, contiguous_us_is_scale_free, tubular_reactor_is_scale_free
5.4Β [8 pts] Classify network as small-world or not
In the body of the function below, use yourΒ network_has_stat_low_avg_path_lengthΒ andΒ network_has_stat_high_avg_clusteringΒ functions from the earlier parts of this assignment to assess whether the parameter networkΒ GΒ meets both of the conditions needed to be considered a small-world network. If it does, then returnΒ True. Otherwise, returnΒ False.
def is_small_world_network(G:nx.Graph, sig_level_alpha:float) -> bool:
"""
Params
G networkx Graph undirected network
sig_level_alpha float significance level
ie, probability of making a type I error
Return
network_is_small_world boolean statistically likely to be a small world network
"""
####IMPLEMENTATION STARTS HERE####
# This is a placeholder
network_is_small_world = None
####IMPLEMENTATION ENDS HERE####
return network_is_small_world
Run the cell below to see what the functional tools youβve built have to say about whether each of the undirected empirical networks is a small-world network or not.
# Run this cell, but do no modify the contents
for g in empirical_undirected_networks():
print(is_small_world_network(g, 0.05))
Fill in the correct boolean in the function below to match the results of the run cell above.
def empirical_networks_small_world_results() -> (bool, bool, bool, bool):
"""
Return
bat_foodsharing_is_small_world bool
chesapeak_road_is_small_world bool
contiguous_us_is_small_world bool
tubular_reactor_is_small_world bool
"""
bat_foodsharing_is_small_world = None
chesapeak_road_is_small_world = None
contiguous_us_is_small_world = None
tubular_reactor_is_small_world = None
return bat_foodsharing_is_small_world, chesapeak_road_is_small_world, contiguous_us_is_small_world, tubular_reactor_is_small_world



