Topics¶
The main topics of this worksheet are:
- Pillow
- numpy
The problems are labeled according to which of these topics they cover. Make sure to get some practice with each topic.
The main references for these topics are:
- Lecture 24
- Lecture 25
- Lecture 26
- Pillow tutorial
- Chapter 2 of VanderPlas
Instructions¶
- Problem 1 is handled differently than the others:
- Tuesday discussion students: Problem 1 will be presented as an example at the start of discussion
- Thursday discussion students: Please complete Problem 1 before discussion and bring your solution
- For the other problems:
- Work on these problems in discussion.
# MCS 275 Worksheet 10 Problem 1a
# Jennifer Vaccaro
# I completed this work in collaboration with the Tuesday discussion
from PIL import Image
# Create a white image
img = Image.new("RGB",(8,8),(255,255,255))
for i in range(8):
for j in range(8):
# light blue
if i%4 == j%4:
img.putpixel((i,j),(0,255,255))
# magenta
if (i+2)%4 == j%4:
img.putpixel((i,j),(255,0,255))
# dark blue
if i%4 == (-j-1)%4:
img.putpixel((i,j),(0,0,255))
# Resize the image without smoothing between pixels
img = img.resize((255,255),Image.NEAREST)
img.save("tuesday.png")
B. Here is a 16x16 numpy matrix of integers (dtype int64):¶
array([[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
[20, -1, 30, -1, 40, -1, 50, -1, 60, -1, 70, -1, 80, -1, 90, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1],
[20, 10, 30, -1, 40, -1, 50, 10, 60, -1, 70, -1, 80, 10, 90, -1],
[-1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1],
[-1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1],
[-1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1],
[20, 10, 30, -1, 40, -1, 50, 10, 60, -1, 70, -1, 80, 10, 90, -1],
[-1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1],
[-1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1, 10, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
[20, -1, 30, -1, 40, -1, 50, -1, 60, -1, 70, -1, 80, -1, 90, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1],
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]])Find a way to produce this matrix and store it in a variable named A.
This is easy to do if you just copy the matrix into your source code. But you can do it with much less code using numpy indexing. (You'll want to examine the matrix carefully to spot all the patterns!) What's the shortest way to do it? I was able to do it in 3 lines, 70 characters total, including newline characters (but not counting the line that imports numpy). Can you do better?
# MCS 275 Worksheet 10 Problem 1b
# Jennifer Vaccaro
# I completed this work in collaboration with the Tuesday discussion
import numpy as np
# The matrix is mostly filled with -1s
A = np.full((16,16),-1)
# Then there are columns of 10s, spaced by 3
A[4:12,1:14:3] = 10
# Finally, there are rows with increasing multiples of 10
A[1::4,::2] = np.arange(20,95,10)
# Print out A to check
print(A)
2. Needle in a haystack, part 1 (Pillow)¶
Download and extract this zip file:
It contains 1024 image files in two directories (haystack1 and haystack2). If you look at them, they all appear to contain black and white "static". These files are used in this problem and the next one.
Let's say that a pixel is bluish if its blue component is larger than its red or green components. So, for example, (62,30,62) is not bluish, but (5,6,7) is bluish.
The files in directory haystack1 are numbered 000 to 511. They are PNG image files, all of the same size. Among all of these image files, there is only a single bluish pixel. Which file is it in, and at what location?
# MCS 275 Worksheet 10 Problem 2
# Jennifer Vaccaro
# I completed this work on my own, in accordance with the syllabus.
from PIL import Image
# directory string
d = "haystacks/haystack1/"
for i in range(512):
# Make a filename with padded 0s on the left of i
iname = "{}img{:03d}.png".format(d,i)
# Open the image file
img = Image.open(iname)
w, h = img.size
# Iterate through the pixels and check whether any is bluish
for x in range(w):
for y in range(h):
r,g,b = img.getpixel((x,y))
if r<b and g<b:
print("Bluish pixel found at ({},{}) in file {}".format(x,y,iname))
# There's only one bluish pixel, so now we are done
exit()
3. Needle in a haystack, part 2 (Pillow)¶
This is a more complex version of the previous problem. Only work on it if you will still have enough time to do the numpy problems below. (If unsure, skip ahead for now and come back to this later.)
The files in directory haystack2 have 36-character filenames (plus an extension ".png"), and have varying sizes. They contain a secret message, which you can decode as follows:
Examine all of the image files in alphabetical order. In each one, look for bluish pixels. There may be no bluish pixels at all, or a single one, or many of them. Whenever a bluish pixel is found, let (x,y) be its position and let r be the remainder of dividing x+y by 36. Take the character in the filename that appears at index r. If that character is a digit, ignore it. Otherwise, add that character to the end of the message.
Hint: It may be helpful to use os.listdir from the os module. It takes a directory name and returns a list of filenames in that directory.
# MCS 275 Worksheet 10 Problem 3
# Jennifer Vaccaro
# I completed this work on my own, in accordance with the syllabus.
import os
from PIL import Image
# Create empty string to store the secret message
message = ""
# Navigate to the correct directory
os.chdir("haystacks")
os.chdir("haystack2")
# Iterate through the pngs in the directory
for f in os.listdir():
img = Image.open(f)
w, h = img.size
# Iterate through the pixels and check whether any is bluish
for x in range(w):
for y in range(h):
r,g,b = img.getpixel((x,y))
# If pixel is bluish, get the associated character.
if r<b and g<b:
remainder = (x+y) % 36
# Only add to the message if the character isnt a digit
if not f[remainder].isdigit():
message += f[remainder]
# Print out the secret message
print("Secret message!")
print(message)
4. Max, Min, Mean and RMS (numpy)¶
Download and unzip this file to obtain a single CSV file:
It has no header row. There are 30 rows of data, each of which has 10,000 fields which are floating-point numbers.
Read the file and put each line into a numpy vector. Then, compute the following four quantities related to the entries in that vector:
- The maximum of the entries
- The minimum of the entries
- The average (arithmetic mean) of the entries
- The square root of the sum of the squares of the entries (sometimes called the RMS or root mean square, or in mathematics, the $\ell^2$ norm)
Write these rowwise summary statistics to an output CSV file that has the following header row:
max,min,mean,RMSwhich is then followed by 30 lines, one for each line in the input file.
Once you have a row of the input file converted to a numpy array, there shouldn't be any need for explicit iteration over its entries.
Hint: Reading a row of the CSV in the usual way will give you a list of strings. You need to convert those to floats before asking numpy to create an array out of them.
# MCS 275 Worksheet 10 Problem 4
# Jennifer Vaccaro
# I completed this work on my own, in accordance with the syllabus.
import csv
import numpy as np
# Set the read and write file names
in_file = "ws9series/series.csv"
summary_file = "summary.csv"
# Open the in_file and create a csv reader object
with open(in_file, "rt", newline="") as fin:
rdr = csv.reader(fin)
# Open the summary_file and create a csv DictWriter object
with open(summary_file, "w", newline="") as fout:
writer = csv.DictWriter(fout,fieldnames=["max","min","mean","RMS"])
# Write the header row, since we set the fieldnames
writer.writeheader()
# Read through the input csv reader
for row in rdr:
# Make row a list of floats
row = [float(val) for val in row]
arr = np.array(row)
d = {}
# Calculate all of the summary statistics
d["max"] = np.max(arr)
d["min"] = np.min(arr)
d["mean"] = np.mean(arr)
d["RMS"] = np.linalg.norm(arr) # Thanks Marcus for finding this function!
# Write them to the summary file
writer.writerow(d)
Another way to find the RMS, not using np.linalg, would be to take the dot product of the array with itself, e.g.
d["RMS"] = np.sqrt( arr.dot(arr) )
5. Impact crater (numpy)¶
Write a function crater(n) that takes a single argument, an integer n, and returns a numpy array of shape (n,n) whose entry at position (i,j) is equal to
$$ \sin \left ( \frac{\pi i}{n-1} \right ) \sin \left ( \frac{\pi j}{n-1} \right ) $$
unless that quantity is larger than 0.7, in which case the entry should instead have the value 0.
Use numpy arrays, and try to avoid loops.
Hint: numpy as the constant $\pi$ built in as np.pi
# MCS 275 Worksheet 10 Problem 5
# Jennifer Vaccaro
# I completed this work on my own, in accordance with the syllabus.
import numpy as np
def sinsin(n,i,j):
"""Calls the product sine function defined on the worksheet,
either on a scalar or on a numpy array
n: the size of the crater
i: the x coordinate(s)
j: the y coordinate(s)"""
return np.sin(np.pi*i/(n-1))*np.sin(np.pi*j/(n-1))
def crater(n):
"""Returns a crater numpy array of size n"""
# Create an n-by-n numpy array of coordinates
x = np.arange(0,n,1)
y = np.arange(0,n,1)
xx,yy = np.meshgrid(x,y)
# Create a new numpy array which calls the function
arr = sinsin(n,xx,yy)
arr[arr>0.7] = 0 #mask any values >0.7 with zeros
return arr
if __name__=="__main__":
# Make the crater numpy array print nicely
np.set_printoptions(precision=3,suppress=True)
print(crater(12))
Also, can you explain the name of this problem?
The array slopes upward at an increasing rate, then cuts down to 0 in the middle. So interpretted as a greyscale image, it resembles a crater from an impact.