Sessions

Tentative themes for each session

• Thursday August 18th: Introduction meeting File:IntroductionMeeting.pdf
• Thursday September 1st: Session 1 - Introduction to big data. Big-data processing. Spark
• Thursday September 15th: Session 2 - More about Spark. Data sources. Twitter
• Thursday September 29th: Session 3 - Streaming Spark. Big-data architectures. Kafka
• Thursday October 13th: Session 4 - Cloud computing. NREC an Openstack
• Thursday October 27th: Session 5 - Cloud management. Terraform and Ansible. Docker and Kubernetes
• Thursday November 10th: Session 6 - Societal issues. Privacy. GDPR
• Thursday November 24th: Session 7 - Essay presentations
• Thursday December 8th: Session 8 - Project demonstrations

Session 1 - Introduction to big data. Big-data processing. Spark

• Kitchin, chapters 1, 4-5
• Chambers & Zaharia, chapters 1-3, 12, 15
• Slides: File:S01-BigData-published.pdf File:S01-Spark-published.pdf

Supplementary:

• Section 1 in Opdahl, A. L., & Nunavath, V. (2020). Big Data. Big Data in Emergency Management: Exploitation Techniques for Social and Mobile Data, 15-29. Paper
• Spark 3.3.0 Overview and Quick Start (with Python examples)

Session 2 - More about Spark. Data sources. Twitter

• Chambers & Zaharia, chapters 4-9 (chapter 10 on SQL is also very relevant)
• Kitchin, chapter 3
• Slides: File:S02-OrganisationINFO319-published.pdf File:S02-DataSources-published.pdf File:DanielRosnes-Introduction-to-Tweepy-and-Twitter-API-2.0.pdf File:S02-MoreSpark-published.pdf

Guest presentation: Daniel Rosnes on using Twitter data for the news: Introduction to Twitter API v2 and Tweepy

Supplementary:

• Chambers & Zaharia, chapter 10 (perhaps mandatory too)
• Twitter API v2
• Tweepy: Twitter for Python
• Tweepy Documentation

Session 3 - Streaming Spark. Big-data architectures. Kafka

• Chambers & Zaharia, chapters 20-21
• Gallofré, M., Opdahl, A. L., Stoppel, S., Tessem, B., & Veres, C. (2021). The News Angler Project: Exploring the Next Generation of Journalistic Knowledge Platforms. In Proceedings of Norsk IKT-konferanse for forskning og utdanning. Short Paper and poster: File:A1-Poster-NIKT2021.pdf
• Kafka Introduction

We may have to postpone the Guest Talk on architecture and the News Hunter platform to a later session.

Supplementary:

• kafka-python API
• Berven, A., Christensen, O. A., Moldeklev, S., Opdahl, A. L., & Villanger, K. J. (2020). A knowledge-graph platform for newsrooms. Computers in Industry, 123, 103321. Paper
• Opdahl, A. L., & Tessem, B. (2021). Ontologies for finding journalistic angles. Software and Systems Modeling, 20(1), 71-87. Paper

Session 4 - Cloud computing. NREC and Openstack

• NREC and OpenStack, the following sections/pages: Introduction, Project application, Logging in, The dashboard, Create a Linux virtual machine (skip: Windows), Using SSH, Working with Security Groups, Create and manage volumes, Create and manage snapshots (skip: images), Instance console

Guest presentation: Sohail Khan on computer vision and deep networks for image analysis

Comment: There are not so many readings for this session, because it is where we will start running Spark in a cluster, so there will be practical work that takes some time. Computer networks and image analysis is not a mandatory part of the course, but something you may want to use in your projects. Sohail's presentation will include suggestions for further reading.

Session 5 - Cloud management. Terraform and Ansible. Docker and Kubernetes

• TerraForm and NREC part I, part II, and part III
• How Ansible Works and the Ansible Community portal
• Docker Docs: Docker overview and Get started
• Learn Kubernetes basics, modules 1-6

Guest presentation: Marc Gallofré Ocaña on the News Hunter platform and its big-data ready architecture

Comment: I really hope to have the time to introduce Docker and Kubernetes properly in the course. In a pinch, we can use some of sessions 7 and 8 too for this.

Session 6 - Societal issues. Privacy. GDPR

• Kitchin, chapters 12-19 (only a few of them are mandatory, perhaps 12-13 and 17 - TBA)
• What is GDPR, the EU’s new data protection law?

Guest presentation: Laurence Dierickx on aspects of big-data quality

Supplementary:

• Kitchin, the rest of chapters 12-19 (perhaps 14, 18 and 19)
• EU's General Data Protection Regulation (GDPR) - the official legal text