A11 – Playing Notes by Image Processing

Would it be just the coolest thing if you'd be able to play a song using just its music sheet and scilab? Well its possible! :D

In this activity, we will be playing notes in scilab using the image processing techniques we have learned so far.

The music sheet that I chose is Jingle Bells (I know..I know... I'm just so excited for Christmas. Aren't you?).

Figure 1: Music sheet of Jingle Bells

Given the music sheet above, we find the location of the notes (notes present: whole, half and quarter note) using the imcorrcoef() function.

These are the templates that I used:

These are the locations of the notes:

half notes:


found in 1st line of music sheet


found in 2nd line of music sheet


found in 3rd line of music sheet


whole notes:


found in 1st line of music sheet


found in 3rd line of music sheet

quarter notes:


found in 1st line of music sheet


found in 2nd line of music sheet


found in 3rd line of music sheet

With the use of correlation, we are able to locate the respective notes in the sub-images of the music sheet. The next thing to do would be to know the location of each note on the staff for us to identify its pitch.

For assigning the pitch of the notes we need the frequencies of each pitch:

C = 261.63 Hz;
D = 293.66 Hz;
E = 329.63 Hz;
G = 392.00 Hz;
F = 349.23 Hz;

and assign the time for which each note will be playing:

quarternote: 0.25 sec
halfnote: 0.5 sec
wholenote: 1 sec

.....


I would like to give my self a grade of 5/10 for this activity since I wasn't able to play the notes but was able to locate the notes in the music sheet. :(

Frequencies of the pitches came from this site:
http://www.phy.mtu.edu/~suits/notefreqs.html

other reference:
186 handouts for activity 11

A12 – Color Image Processing

Have you ever had your photo taken and you somehow appeared bluish or maybe orangish? You may find the photo "artsy" but do you ever wonder why this happens so? Its because digital cameras have settings called white balance. White balancing just refers to the transformations the camera does to make a white object in your image appear white. White balancing depends on the illumination condition under which the image was taken. That's why if you select an inappropriate white balance setting, your camera transforms the colors wrong.

I have taken an image of colorful objects under different white balance settings as shown below:

Figure 1: Image of colored objects taken under different white balance settings

Notice that using fluorescent and incandescent white balance settings makes your image appear bluish (especially using incandescent). From here we may say that using sunny(daylight) or cloudy are more appropriate white balance settings for this particular imaging conditions.

These are the two automatic white balancing algorithms that are available:

1. White patch algorithm: done by dividing the RGB channels of the image with the RGB components of the part of the image which should be white.

2. Gray world algorithm: white patch algorithm then averaging the RGB channels of the image.

These are the results using the two white balancing algorithms:

Figure 2: Results after performing white patch and gray world algorithm

Both algorithms worked with images quite well but I think that the gray world algorithm performed better since the white piece of paper looked more white upon its application.

The next part of the activity would be to apply both the white patch algorithm and the gray world algorithm on an image of objects having the same color.

Given this image:

Figure 3: Image of object having similar color
(sorry if the image is rotated...I couldn't figure out how to rotate this image)

Here are the results:

Figure 4: Result upon application of white patch algorithm

Figure 5: Result upon using gray world algorithm

Through visual inspection, I would say that using white patch algorithm yield better result than gray world algorithm since white patch algorithm captured the background color better (supposed to be green). This is probably because for the gray world algorithm to work optimally, sample colors from red, green and blue should be present. For this particular image however, only green colored objects are present.

I would like to give myself a grade of 10/10 for this activity since I was able to implement both algorithms and presented results asked.

A10 – Binary Operations

First part: Finding the cells of ordinary cells

In this activity we apply what we have learned about morphological operations (see activity 9) for finding the area of cells.

This operation can actually be easily done using opening and closing operations, however, these functions are not present in scilab. Lucky for us, the said functions can also be done using the erode() and dilate() functions.

Remember this:
Open = Erode + Dilate
Close = Dilate + Erode

The image below is an image model of cells:

Figure 1: Image of cells

We divide this image into 9 sub-images...

Figure 2: Collage of sub-images of original image

To separate the cells from the background, we look at its histogram:

Figure 3: Histogram of one sub-image

Since we know that the background have a gray color, using the appropriate values for im2bw() (near white), we would end up with the binarized image below:

Note: For illustration purposes, the thresholded image above is just one sub-image of the original image, however, all the operations in this activity are applied to all the sub-images.

After getting the thresholded image, we may now apply erosion and dilation operations to fill-up the cells and separate cells that are overlapping. The result after applying these operations is the image below:

Figure 4: Sub-image after erosion and dilation

As you can see, some cells are reduced to smaller sizes than the original while other cells are still not separated from each other(which may be interpreted as larger cells in a 2D image). Since our goal is to average the cell sizes(areas), it would be wrong to use these "reduced in size cells" and "large cells" for finding the mean area. To solve this problem, we look at the histogram of areas of all cells:

Figure 5: Histogram of cell areas

Using this histogram, we may assign a specific threshold locating the cells for which we would get the areas from.

To get the area of a cell, we just add the pixels comprising each cell. Repeat this for all cells and find the mean.

Results:
Mean area: 451.69231
Variance: 66.891858

Second part: Isolating cancer cells

In this part of the activity, our goal is to locate the cancer cells in the image below:

Figure 6: Image of cells with cancer

In this model image, the cancer cells are represented by the larger cells. Fortunately, none of them are overlapping with the normal cells making our work a lot easier.

Basically, the same steps are applied but this time several erosion operations are applied to separate the cells from each other.

Here are the result:

The cancer cells are successfully isolated from the normal cells.

I would like to give myself a grade of 10/10 since I was able to obtain the results asked for.

Thanks to Joseph for the help.

A9 – Morphological Operations

Predicted outcomes upon application of erosion and dilation operations:

Before using scilab to simulate the results when we apply the erosion and dilation operations, we were first asked to predict what these results would be. I used Microsoft word to illustrate my predictions by putting gridlines and drawing squares as pixels.

Here are my predictions...

Figure 1: Predictions for erosion

NOTE: The resulting image is comprised ONLY of the yellow pixels. The white pixels are only included to illustrate how much of the original image are removed.

Figure 2: Predictions for dilation

NOTE: The resulting image is the combination of the yellow and white pixels.



Outcomes upon application of erosion and dilation operators using SCILAB:

EROSION:

Figure 3: Erosion using a 2x2 ones strel

Figure 4: Erosion using a 2x1 ones strel

Figure 5: Erosion using a 1x2 ones strel

Figure 6: Erosion using a cross (3x3) strel

Figure 7: Erosion using a diagonal (0 1; 1 0) strel

NOTE: Blank figure means that no pixel remained after the erosion operation.

DILATION:

Figure 8: Dilation using a 2x2 ones strel

Figure 9: Dilation using a 2x1 ones strel

Figure 10: Dilation using a 1x2 ones strel

Figure 11: Dilation using a cross (3x3) strel

Figure 12: Dilation using a diagonal (0 1; 1 0) strel

I'm glad to see that my predictions are correct! :D

Other morphological operators:

1. Thin: performs a thinning operation on binary objects

Figure 13: objects(left), objects after thinning operation(right)

2. Skel: another thinning operation where the output object can be further pruned by thresholding

below are examples of what your object will look like after skel operation:

Figure 14: objects(left), objects after skel operation(right)

CODE:

stacksize(1000000);

o1 = zeros(11,11);
o1(4:8,4:8) = 1
//imshow(o1,[])

o2 = zeros(8,8);
o2(4,3) = 1;
o2(5,3:4) = 1;
o2(6,3:6) = 1;
//imshow(o2,[])


o3 = zeros(14,14);
o3(3:12,3:12) = 1;
o3(5:10,5:10) = 0;
//imshow(o3,[])


o4 = zeros(9,9);
o4(3:7,5) = 1;
o4(5,3:7) = 1;
//imshow(o4,[])

//Structuring elements
a = [1 1; 1 1];
b = [1; 1];
c = [1 1];
d = [0 1 0; 1 1 1; 0 1 0]
e = [0 1; 1 0]

////////////////////////////////////////////////////////////////

//erode

a1 = erode(o1, a, [1,1]);
a2 = erode(o2, a, [1,1]);
a3 = erode(o3, a, [1,1]);
a4 = erode(o4, a, [1,1]);

subplot(1, 4, 1);imshow(a1, []);
subplot(1, 4, 2);imshow(a2, []);
subplot(1, 4, 3);imshow(a3, []);
subplot(1, 4, 4);imshow(a4, []);

b1 = erode(o1, b, [1,1]);
b2 = erode(o2, b, [1,1]);
b3 = erode(o3, b, [1,1]);
b4 = erode(o4, b, [1,1]);

subplot(1, 4, 1);imshow(b1, []);
subplot(1, 4, 2);imshow(b2, []);
subplot(1, 4, 3);imshow(b3, []);
subplot(1, 4, 4);imshow(b4, []);

c1 = erode(o1, c, [1,1]);
c2 = erode(o2, c, [1,1]);
c3 = erode(o3, c, [1,1]);
c4 = erode(o4, c, [1,1]);

subplot(1, 4, 1);imshow(c1, []);
subplot(1, 4, 2);imshow(c2, []);
subplot(1, 4, 3);imshow(c3, []);
subplot(1, 4, 4);imshow(c4, []);

d1 = erode(o1, d, [2,2]);
d2 = erode(o2, d, [2,2]);
d3 = erode(o3, d, [2,2]);
d4 = erode(o4, d, [2,2]);

subplot(1, 4, 1);imshow(d1, []);
subplot(1, 4, 2);imshow(d2, []);
subplot(1, 4, 3);imshow(d3, []);
subplot(1, 4, 4);imshow(d4, []);

e1 = erode(o1, e, [2,1]);
e2 = erode(o2, e, [2,1]);
e3 = erode(o3, e, [2,1]);
e4 = erode(o4, e, [2,1]);

subplot(1, 4, 1);imshow(e1, []);
subplot(1, 4, 2);imshow(e2, []);
subplot(1, 4, 3);imshow(e3, []);
subplot(1, 4, 4);imshow(e4, []);

/////////////////////////////////////////////////////////

//dilate

a1 = dilate(o1, a, [1,1]);
a2 = dilate(o2, a, [1,1]);
a3 = dilate(o3, a, [1,1]);
a4 = dilate(o4, a, [1,1]);

subplot(1, 4, 1);imshow(a1, []);
subplot(1, 4, 2);imshow(a2, []);
subplot(1, 4, 3);imshow(a3, []);
subplot(1, 4, 4);imshow(a4, []);

b1 = dilate(o1, b, [1,1]);
b2 = dilate(o2, b, [1,1]);
b3 = dilate(o3, b, [1,1]);
b4 = dilate(o4, b, [1,1]);

subplot(1, 4, 1);imshow(b1, []);
subplot(1, 4, 2);imshow(b2, []);
subplot(1, 4, 3);imshow(b3, []);
subplot(1, 4, 4);imshow(b4, []);

c1 = dilate(o1, c, [1,1]);
c2 = dilate(o2, c, [1,1]);
c3 = dilate(o3, c, [1,1]);
c4 = dilate(o4, c, [1,1]);

subplot(1, 4, 1);imshow(c1, []);
subplot(1, 4, 2);imshow(c2, []);
subplot(1, 4, 3);imshow(c3, []);
subplot(1, 4, 4);imshow(c4, []);

d1 = dilate(o1, d, [2,2]);
d2 = dilate(o2, d, [2,2]);
d3 = dilate(o3, d, [2,2]);
d4 = dilate(o4, d, [2,2]);

subplot(1, 4, 1);imshow(d1, []);
subplot(1, 4, 2);imshow(d2, []);
subplot(1, 4, 3);imshow(d3, []);
subplot(1, 4, 4);imshow(d4, []);

e1 = dilate(o1, e, [2,1]);
e2 = dilate(o2, e, [2,1]);
e3 = dilate(o3, e, [2,1]);
e4 = dilate(o4, e, [2,1]);

subplot(1, 4, 1);imshow(e1, []);
subplot(1, 4, 2);imshow(e2, []);
subplot(1, 4, 3);imshow(e3, []);
subplot(1, 4, 4);imshow(e4, []);

I would like to give my self a grade of 10/10 for this activity since I was able to achieve the desired results.

Thanks to Joseph and Dane for helping me with this activity!

Thanks to BA for keeping me company while finishing this activity. :P

references:
1. 187 activity 9 handout
2. http://homepages.inf.ed.ac.uk/rbf/HIPR2/skeleton.htm

A8.D - Canvas Weave Modeling and Removal

For this part of the activity we want to enhance this patch of a painting(from the UP Vargas Museum collection) by removing the canvas weave pattern such that only the paint texture is seen.

Given the painting patch...

Figure 1: Patch of painting

Take its FT...

Figure 2: FT

Design a filter that would get rid of those dirac deltas we see in the FT...(remember that the FT of addition of sine patterns are dirac deltas... :D)

Figure 3: Filter

How do we know that we are choosing the right areas to be filtered you might ask...well let's take the FT of the filter to be sure...

Figure 4: FT of filter

I would say that the FT somehow resembles that of the canvas weave pattern so it means that we are on the right track. :D

Multiply the FT of the patch of painting to the filter...

Figure 5: Image of patch of painting multiplied by filter

Resulting image...no more canvas weave pattern! or atleast reduced..hehe.

Figure 6: Resulting image

CODE: same as in act.8.B with filter input being figure 3.

A8.C - Lunar Landing Scanned Pictures : Line removal

Yey! for another exercise that makes use of what we have learned about FTs.

Basically, we want to enhance this image of the moon's surface by removing the lines that are not supposed to be part of the image.

So given this image of the moon's surface...

Figure 1: Moon's surface
from: http://www.lpi.usra.edu/lunar/missions/apollo/apollo_11/images

we take its FT...

Figure 2: FT of moon's surface image

-we know that when we inverse a crossed-shaped FT, this would results to square patterns. Therefore this corresponds to the lines that we want to remove. From this...

...a filter is designed as...

Figure 3: Filter

To select only the areas we want to retain, we multiply the FT with the filter that we designed...

Figure 4: FT multiplied by filter

Taking the inverse of figure 4 results to...

Figure 5: Enhanced image

...which is an enhanced image of the moon's surface(without the lines)...yey!!! :D

CODE: same as activity A8.B with the filter input being image illustrated by figure 3.