CBC news comment and replies interaction network visualization

The CBC (Canadian Broadcasting Corporation) news website articles often have a comments section. It would be interesting to see the interactions between comments and replies, and to understand which person makes the most comments, and frequently used words and phrases.

See the results: https://sitrucp.github.io/cbc_comments/image_grid.html

Another post details a method to retrieve the comments. Comments include a timestamp when it was posted, comment text, and comment user name, and if it is a reply, then name of the comment user being replied to.

This information can be aggregated to get count of posts by comment user name or date/time. It can also be used to learn more about comment user interactions by visualizing the comment and reply user names in a network visualization using the Python NetworkX module. Code used is provided below.

The visualization below illustrates the relationships between 104 comments and replies by comment user for an article “Unvaccinated travellers over the age of 12 barred from planes and trains as of today” (Note comments data fro this visualization were obtained just after the article was posted when it had about 100 comments and replies. Today it has 2000+ comments.)

The red circles are “nodes” which represent the comment users. The node size corresponds to the user’s total number of comments or replies. The lines are “edges” and connect nodes. Edges represent reply from one user to another user. The edges have arrows that indicate the direction eg who replied to who.

The edge line widths represent the number of interactions between two nodes. Interactions are comment replies from one person to another (in either direction). The more interactions, the wider the edge line.

Most of the article comments sections that were analysed had one or more prolific commenters (represented by larger size nodes). In addition, there are comment users that have a greater number of replies (represented by edges).

Examples of visualizations provided below. View complete list of CBC comments visualizations here. Click on the image to view full size as some of them are very big and you will be able to zoom in to get more detailed view.

The first image below is a snapshot of a part of the entire visualizations. Seen at this level it is clear that each story’s comments has its own “fingerprint” of comments and their interactions.

On COVID restrictions, our governments keep firing up the gaslights and shifting the goalposts

In a fiery speech, O’Toole says Canada is ‘drowning in debt and division’ on Trudeau’s watch

Trudeau calls for global carbon tax at COP26 summit

RCMP union says it supports a Mountie’s ‘right’ to refuse vaccination

View more CBC comments visualizations here.

Python code to create the NetworkX charts is provided below and in Github repository.

import networkx as nx
import matplotlib.pyplot as plt
import math

# Drop comments without any replies
df.dropna(subset=['replied_to_user'], how='all', inplace=True)

# Build NetworkX graph
G = nx.Graph()

# Select data to use in graph from dataframe with full data
G = nx.from_pandas_edgelist(df, 'comment_user', 'replied_to_user', 'minutes')

# Create node size variable
d = nx.degree(G)

# create edges, and weights list for edge colors
# weights are minutes from first comment 
edges, weights = zip(*nx.get_edge_attributes(G,'minutes').items())

# create variable to increase graph figure size based on number of nodes to make more readable
factor = math.sqrt(len(G.nodes()) * 0.01)

# Create plot
plt_width = 25 * factor
plt_height = 25 * factor
fig, ax = plt.subplots(figsize=(plt_width, plt_height))
fig.set_facecolor('black')
ax.set_facecolor('black')

# create layout kamada_kawai_layout seemed best!
#pos = nx.spring_layout(G, k=.10, iterations=20)
#pos = nx.spring_layout(G)
pos = nx.kamada_kawai_layout(G)
#pos = nx.fruchterman_reingold_layout(G)

# draw edges
nx.draw_networkx_edges(
    G, 
    pos,
    arrows=True,
    arrowsize=20,
    edgelist=edges,
    edge_color=weights,
    width=1.0,
    edge_cmap=plt.cm.spring,
    node_size=[(d[node]+1) * 200 for node in G.nodes()], # tells edge to go join node on border
)

# draw nodes
nx.draw_networkx_nodes(
    G, 
    pos,
    node_color='red',
    alpha = 0.7,
    edgecolors='white', #color of node border
    node_size=[(d[node]+1) * 200 for node in G.nodes()],
)

# draw labels
nx.draw_networkx_labels(
    G, 
    pos, 
    labels=None, 
    font_size=10, 
    font_color='white', 
    font_family='sans-serif', 
    font_weight='normal', 
    alpha=None, 
    bbox=None, 
    horizontalalignment='center', 
    verticalalignment='center', 
    ax=None, 
    clip_on=False
)

# create variables to use in chart title
min_comment_time = df['comment_time'].min()[:-3]
max_comment_time = df['comment_time'].max()[:-3]
comment_count = len(df)

# create chart title text
title_text = file_url + '\n' + str(comment_count) + ' comments & replies '+ 'from: ' + min_comment_time + ' to: ' + max_comment_time

# add chart title
plt.title(title_text, fontsize=26 * factor, color='white')

#  save the image in the img folder:
plt.savefig(file_path_image + 'network_' + file_name + '.png', format="PNG")