Advice (and opinion) for Taking CS229
27 Dec 2021Intro
Recently, I’ve completed Stanford’s CS229: Machine Learning for a non-degree option1. It was a huge time sink and a bit difficult to juggle with a full time job, but I have some opinions and advice on how to effectively learn and pass the class.
Opinion
In my opinion, CS229 is much more organized and beginner friendly than other ML courses I’ve taken (at UChicago). For instance, students get incredibly detailed lecture notes that mirror the actual lectures, a fairly large TA staff, starter code with clear instructions for problem sets, and clear guidelines for the expected outputs of algorithms (which reduces the burden and confusion when trying to implement algorithms from scratch). This makes the firehose of knowledge you need to learn/understand more like a garden hose - difficult, but more manageable.
However, at a whopping $6000 per course, I wouldn’t recommend it over the same source but taught through a MOOC like Coursera (especially with a massive discount) unless you were trying to get a certificate or credits for the class.
Advice
0. Prepare to commit some time
Even if you intended to or not - this course ends up being a huge time commitment. I’ve spent many weekends and after-work hours reading/watching the course material, office hours, and completing the pset.
1. Don’t be scared if you don’t “get” it the first time
I’ve realized that you have a lifetime (or at least the next couple of years) to really master these concepts and ideas. A lot of the “learning” I’ve had in school didn’t really “click” until I ran into these ideas later in work, hobbies, or different courses.
2. Get a comfortable understanding of the fundamentals
The prerequisites for an ML class are (1) Computer Science, (2) Probability Theory, (3) Multivariable Calculus, and (4) Linear Algebra.
I think it’s fine if you don’t have a strong foundation of knowledge for all of these fields (but I would always recommend it). If you aren’t comfortable with coding and linear algebra, you’re might have a really bad time.
3. Use a debugger (or a Jupyter notebook)
A big portion of the time spend in this class will be spent coding on the homework or projects. Coding anything involves discovering bugs and solving those bugs. It is really easy to get into the trap of not learning a debugger or jupyter and use print() debugging2 - but in reality it will save you much more time using these tools.
4. Optimize your numpy code
It’s good practice to leverage numpy functions as they tend to be more performant. It also has the side effect of being easy to understand and looks clean.
5. Don’t pre-optimize your numpy
On the other hand, don’t shoot yourself in the foot by being overzealous with your numpy operations. If you’re just learning how the algorithm works, it’s okay to start with simple looping with numpy arrays. If you don’t have a solid grasp of the algorithm, and the functions you’re using to compose the algorithm debugging is going to become exponentially difficult.
Optimizations come after it works.
6. Visualize your understanding
Learning things get harder the more abstract they become. A really useful learning method I’ve learning is to visualize by drawing my understanding of the topic. This has an added benefit of being very clear to a peer or TA what you’re talking about.
7. Boost your fundamentals with resources!
The University Of Waterloo has this excellent PDF of various matrix operations and their results! Use this if you’re stuck on a proof or derivative.
There’s also this cheatsheet provided (but I never really ended up using it)
8. Start the Homework early
Start the homework early and start the coding implementations just as early. Since the class has been taught so many times and lecture notes have been refined, you don’t need to wait for the lecture to start the homework. You could just read the lecture notes or watch past lectures.
9. Pick a project topic you’re interested in
If you’re going to spend a bunch of time (1) reading background literature, (2) debugging code, (3) conducting a lot of research, you might as well create something that you’re interested in!
Here’s a link to the repo dump of my final project with the paper!