~ read.

Master of Science in Engineering Concluded

Finally I have finished my master's degree in Computer Science in Warsaw University of Technology. As promised that I'd write my experience every semester, this post should be the last post of that series (read here for the 1st semester's story and here for the 2nd semester). Apparently people read those posts as I received some messages asking me further about this program. And even though I did my defense almost 3 months ago in April, I hope I am not too late writing this.

As mentioned in the last post, for the last semester I had only 2 courses left: Thesis (Thesis 2 + 3 and Defense), and English Culture.

English Culture

This course is a humanity course discussing American culture and tradition. There are 3 activities of this course: lecture, presentation, and exam. In general, it discusses general knowledge about the USA (the states, amendments, congress, etc), the history (from the story of Columbus and the birth of America to the American colonies and the origins of slavery there - including the civil right movement). The lecturer itself is an American, so I guess he is qualified to run this course.

For the presentation, every student had to give a presentation related to American culture and traditions. I teamed up with a Vietnamese student and thought that it would be interesting to present about the war with Vietnam. But he's not really comfortable with that and in the end we gave a presentation about the effects of Hollywood.

There was also one final exam. I honestly didn't care much about this course that I only studied one night before the exam. But I still managed to get a 4.5 from this course.

Overall, I would say that this course was OK. It's not as inspiring as the Methodology and Ethical Aspect of Research that I took earlier.

Thesis

I spent the last semester of my studies mostly working on my thesis. During summer, after many sessions with my supervisor (mgr inż. Rajmund Kożuszek) discussing the final topic of my thesis (we explored many things from word embedding to image captioning hoping that I could get some inspirations on how to improve Bag of Visual Words), I decided that I would write about transfer learning in convolutional neural network instead. Started with using this approach to solve different problems, followed by assessing its performance and how to improve it, and ended with quantifying network transferability on small dataset.

Quantifying network transferability on small dataset can be beneficial to see if the characteristics of transfer learning also persist on small dataset. Having a small amount of dataset is such a common problem in machine learning. This experiment could answer if training the network (which is also relatively small) using this small dataset could be helpful to learn a general representation that can be used as a pretrained model for other tasks that also have a small amount of dataset.

We also managed to write a paper based on a simple experiment and published it to a conference for an oral session titled "Does Fragile Co-Adaptation Occur in Small Dataset?". Since I had to go back at the end of April to Jakarta to start my new job and the conference itself is in May, I asked my supervisor to present it.

To be honest, writing is not my forte. During the time of writing, I felt that writing needs a lot of patience. I had a lot that I want to write, but writing itself takes time (especially when it's done in LaTeX). And I hate keeping things hanging on my head. If only there's a technology to transfer directly our thought as a nicely written text! When writing, you'll also keep reading the same sentence over and over again (to convey your idea nicely or simply avoid some typos), that it could really bore you. It had taken its toll on me that at the end of February (the deadline for thesis submission) when I submitted the thesis, I told my supervisor that I wanted to be off the radar for a while. And I did.

After a few weeks without thesis, when I came back again reading it, I actually felt happy. I realized that it's written quite nicely. Now that I'm working as AI Engineer, I keep coming back to my thesis (I wrote the state of the art in ConvNet as one full chapter). I have 2 interns that I supervise at work working on visual search and I asked them to read it so that they could learn the knowledge needed fast. So far, there's no complain :)

Defense

One month after the submission, precisely on April 6th, the defense was scheduled. My supervisor told me earlier that he would ask Prof. Stanisław Jankowski as the reviewer as he probably would enjoy my work. I was happy to hear that. Prof. Jankowski taught us neural network on Evolutional Algorithm course. One day before the defense, I got an email notification that he had submitted his review. I was really excited reading his opinion.

The evening before the defense, I swam at my regular pool and walked roaming the city of Warsaw during the night with hope that, maybe if I am tired enough, I would go to sleep effortlessly. I did not. I couldn't sleep at all even for a few minutes. I could not shut off my brain.

At 6 a.m. on the defense day, I walked to the nearest park (Łazienki), again, hoping that I could clear my mind (it certainly worked), but I felt that I wasn't tired. So I thought everything will be fine. The defense was scheduled at 12.30 p.m, and I went to campus at 12 p.m. to meet with my supervisor first. We talked a bit. He asked me if I want to use the computer in the defense room, which I thought I would be more comfortable using mine. I still copied the presentation anyway to a pen drive in case something goes wrong (it did).

At 12.30, I was called by the committee to enter the room. I was surprised to see all familiar faces (well, Prof. Jankowski is expected and my supervisor also told me that Dr. Podraża would also be there as the committee). But I didn't expect that Dr. Paweł would also be there. We worked together on a machine learning project in AI department previously. You would think that it'll calm me down knowing that I know all of them. But I don't know why it suddenly made me nervous. They are experts in machine learning and computer vision. It would be easy for them to spot the smallest mistake on my thesis.

I started the presentation and I could hear my voice a bit rusty. I saw my supervisor with his eyebrows furrowed. I'd never seen him looking so concerned like that before. My brain was jumping here and there, "Did I say something wrong?", "Did I speak clearly, too fast, or maybe too slow?", and so on so forth. A few minutes after, I felt alright and started to enjoy the presentation...... until the presentation went on and off and completely went off. I realized that I brought my old VGA to HDMI cable. I got panicked and asked them to use their laptop instead.

During the presentation, I was actually looking closely if Dr. Paweł is 'entertained' by my thesis. Even though at that time, I already accepted the offer from my current job, previously, I was still considering taking PhD there. I told Dr. Robert Nowak (another lecturer in AI department) that I was interested in taking PhD in the intersection of computer vision and deep learning, and he told me that I should go to Dr. Paweł.

I ended the presentation within the time limit. Prof. Jankowski then asked me a few question (how I preprocessed the images in some experiments). Dr Paweł then asked me to sit in a chair in front of them for the final exam. The first question was about OOP. I repeated the question while trying hard to remember something that I took 9 years ago. The second question was from Prof. Jankowski about multi layer perceptron. I forgot what the exact question was, but I answered it with some stories of linear regression, logistic regression, and how we can see it as a specific case of neural network. I think that was unnecessarily a long answer, but I was actually hoping to engage more discussions with him (lol). One day before leaving Poland, I visited my supervisor and we discussed a bit how the defense went, and he did say that that was an unexpected answer :)

The final exam ended when they gave some gestures to each other and I was asked to leave the room for a moment. I waited for not more than 5 minutes outside and they asked me to come inside. They were all standing up and Dr. Paweł formally declared that I graduated with honor. I shook their hands and asked for a photo.

Left to right: dr inż. Roman Podraza, dr hab. inż Paweł Wawrzyński, me, prof. nzw. dr hab. Stanisław Jankowski, mgr. inz Rajmund Kożuszek wearing Batik. Overall classification: 6/5 (excellent)

So that was it. After the defense I did some administration stuff, went to my favorite Thai resto with my friends (Thai Style - not far from the Main Building), and went to Park Mokotowski. Warsaw was sunny and beautiful that day. And I slept 15 hours that night.

Leaving Poland

After such exposure of machine learning in Politechnika (be it from the courses, project, or my thesis), I decided that I have to shift my career as an ML Engineer/Researcher. I can take the benefit of being good in maths and CS in this role. But, it isn't easy to get such roles. They are mostly dominated by a big company like Google. I actually applied for Google Brain Residency but I failed. After applying for 5 other jobs, there were 2 offers that I got, one as AI Engineer in Jakarta, and the other as a Python Developer in Warsaw. It was quite obvious to me that I decided to leave Warsaw.

Conclusion

Some people would ask me, "Is the program good and worth it?" Well, in my case, I would say yes. 2 weeks before the defense, I started reviewing all the courses and I realized that I got all the fundamentals that I needed to shift my career as an ML Engineer/Researcher. If you are interested in data science or machine learning, I'd say that the program is good for you.

Of course there's room for improvement for the program, like the name itself is quite misleading. I had a Polish classmate interested in Computer Networks, but felt that the program is not for him because it's too strong on data science stuff. In my opinion, I think they should make the name clearer. And with the current demand, I'd suggest to start a data science/machine learning program with some notes:

Remove the basic courses like Database, Computer Network, Operating Systems, etc. I don't think brushing up on those courses is useful for a master's student.

Restructure some courses. The optimization topic in Evolutionary Algorithm is too fast (it was just 3 weeks. You can read it in my first post that the lecturer even gave the presentation like she wasn't breathing at all). Optimization is very crucial in machine learning, I think it deserves to be a separate course. The optimization part in Parallel Numerical Method could also be merged here.

There was a student in this program telling me that he was struggling with the math and asked me how he could survive there. Well, the fundamental mathematics needed are somehow covered by the current courses like Parallel Numerical Methods and Discrete Random Processes. If you understand Discrete Random Processes, you won't be struggling with the math needed in other courses like Pattern Recognition or Image Processing. I'd say they are in the same 'mental space'. But maybe, we only need Probability and Statistics (I had to review it again when studying EDRP) without other stuff such as queue theory. A review of Algebra and Calculus would be nice too. I think Algebra, Calculus, and Probability and Statistics are more useful than the basic courses that I mentioned before. Maybe they can somehow be merged together as one review course (?)

There are also some topics repeated many times in several courses like neural network. We have this topic in Pattern Recognition, Image and Speech Recognition, Evolutionary Algorithm. I really think that there should be one separate Neural Network course so that we can discuss more advanced topics there. Separate deep learning-based NLP and Computer Vision courses would be nice to have similar to Stanford's CS231n (Convolutional Neural Networks for Visual Recognition) and Stanford's CS224n (Natural Language Processing with Deep Learning).

I see as well that they just announced a post-graduate studies in Data Science and Big Data. Looking at the program here, I think they can take some of them to the master's study program (like data visualization and text mining).

Well there is an AI program in the Faculty of Mathematics and Information Science in Politechnika, but if you see here, the courses are pretty much outdated with the current advancement.

I think that's all that I can say. Pretty sure they would attract more students if they restructure the program a bit.

To end this post, I'd like to thank NAWA (Narodowa Agencja Wymiany Akademickiej) that had supported me financially during my studies there. I'm also grateful to have been supervised by Mr. Kożuszek. He inadvertently guided me to the field of deep learning. I hope we can collaborate again in the future.

See you when I see you Poland!

comments powered by Disqus
comments powered by Disqus