CSEP 552: Distributed Systems

Course Overview

Distributed systems are central to many aspects of how computers are used, from web applications to e-commerce to content distribution. This course will cover abstractions and implementation techniques for the construction of distributed systems, including client-server computing, the web, cloud computing, peer-to-peer systems, and distributed storage systems. Topics will include remote procedure call, preventing and finding errors in distributed programs, maintaining consistency of distributed state, fault tolerance, high availability, and scaling.

Alumni of this course say that it is among the most intellectually challenging courses that they have taken at UW. We believe the best way to learn the material is to implement the ideas presented in the course, and so there is a substantial programming project.

The calendar page provides a detailed topic list for the course, including readings, problem sets, and labs. In addition, we are using Gitlab, Ed, and Gradescope for various parts of the course.

Lecture

Lectures are Tuesdays 6:30pm-9:20pm in CSE2 G10. We will record lectures via Panopto and make them available to all students in the course. We encourage you to attend lecture in person if at all possible.

Course resources

Course Staff

Staff Contact

The best way to contact the staff is to make a post on Ed. If your question is likely to be useful to other students, please consider making it public. (You can make the post "anonymous" to hide your identity from your classmates, but note that course staff can still see your identity on anonymous posts. See anonymous feedback at the very bottom of this page for submitting feedback without revealing your identity to course staff.) If your message is not relevant to other students, make it private. We prefer you send messages via Ed if at all possible, because it allows any staff member to assist you. If you need to contact an individual staff member directly, then use email.

Assignments

There are two kinds of assignments in this class.

  • Labs: These are significant programming projects. Four total, due every two or three weeks.
  • Problem sets: Due in weeks when labs are not due. Gives you practice with lecture material.

See below for more information on each kind of assignment, as well as the sections on grading, the late policy, and academic honesty.

Labs

The core of the course is to build a highly available, scalable, fault tolerant, and transactional key-value store. Key-value stores are widely used in cloud computing. The project is written in Java, derived from a similar one designed for the MIT graduate distributed systems course. A hallmark of our project infrastructure is extensive support for thorough testing and debugging. Each lab has a model-checking-based test suite that you can use to validate your implementation.

Lab 0 and Lab 1 of the project are to be done individually. (Note that Lab 0 does not have a turnin.) For the remaining labs (2-4), students may optionally work with a partner.

The labs are autograded by a model checker, which is a program that exhaustively tests your solution in all cases up to a certain depth bound. This means that you will need to come up with solutions that work in all cases. The labs are self-grading—we give you all of the test cases we run.

The best way to write code that works in all cases is to carefully think through your design before writing any code. Design before code is especially important for distributed systems where the number of possible code paths is exponential in the number of messages, the cost of uncaught errors in production code is enormous, and it is often infeasible to catch every possible error through testing.

We encourage you to design before coding and to document your design for yourself and your partner in a design document. We will hold informal design review sessions in class where you will provide feedback on other students' designs. Of course, you may find from time to time that you need to update the design as you later write code, but you want to try to minimize that.

We strongly recommend you get an early start on the labs. Many students underestimate the difficulty of the assignments, and leave themselves too little time to finish before the deadline. The most common comment about the labs is that students wished they had gotten an earlier start.

Grades

The class will be graded on an additive points system.

  • Lab 1: 160 points (note the lab 1 tests sum to 320, so for this lab points are divided by 2)
  • Lab 2: 330 points
  • Lab 3: 355 points
  • Lab 4: 495 points
  • Problem sets: 350 points total

Total points possible: 1690

There is no midterm or final exam.

The course is not curved. Your grade on a 4.0 scale is computed by the following formula.

\[ \min\left( \left.\left\lfloor\frac{\mathrm{points}}{35}\right\rfloor \middle/ 10 - 0.6 \right.,\ 4.0 \right) \]

In other words, your grade will be computed by the following table:

Points Grade on 4.0 scale
≥1610 4.0
≥1575 3.9
≥1540 3.8
≥1505 3.7
≥1470 3.6
≥1435 3.5
≥1400 3.4
≥1365 3.3
≥1330 3.2
≥1295 3.1
≥1260 3.0
... ...
≥1155 2.7 (grad sat)
... ...

This grading scheme may be somewhat unfamiliar to you. We will discuss it on the first day of lecture. Be sure you understand it, and feel free to ask any questions about it.

An important consequence the additive points-based grading scheme is that there is a sense in which every assignment is optional. If you are unable to complete some assignment, simply make sure to complete the remaining ones—we give you room to miss points and still get a good grade. For example, if you miss 15% of the available points, that is still some form of an A grade.

Since you will receive the same number of points as your (optional) partner for your combined work on labs 2-4, it is essential during partner-matching that you communicate expectations about your grade target. If you find yourself stuck in a situation where your partner wants to do significantly more or less work than you do, please contact the staff.

Most students find that they are able to complete all of the assignments to a high degree of quality, but that the assignments require a decent amount of effort. Students find the class time-consuming but rewarding, and so grades in this class are generally high.

If you are taking the class S/NS, note that per university policy graduate students receive S credit for any grade 2.7 or higher. Thus, a graduate student would need to earn at least 1155 points to receive S credit.

Late Policy

Our late policy is designed to give students maximal flexibility without having to ask us for permission in most cases, while still allowing us to grade assignments and get them back to students in a timely fashion. If you have an extenuating circumstance that causes you not to be able to complete the work on time, especially if due to something outside of your control, please contact the staff. Often, we are able to work something out that is agreeable to everyone involved.

Each kind of assignment has a separate late policy. Be sure you understand the differences.

  • All assignments, regardless of late policy or extensions, must be turned in by the end of the day Thursday, March 14, 2024, unless the instructor has given specific and individual permission.

  • Each lab comes with a 48-hour grace period, during which work is accepted without penalty. We then deduct 1% off your score for that lab for each additional day that it is late. In other words, if you are making progress, even slowly, you should keep working on the lab, but if you are stuck you should go ahead and turn it in. Since turning assignments in late cuts into the available time you have for the next lab, only use this flexibility if necessary. To turn in your lab after the 48-hour grace period, contact the staff.

  • Problem sets have a 48-hour grace period, during which work is accepted without penalty. No credit is granted after the grace period expires.

Note that, unlike some other course policies you might be familiar with, in this class there is no cap on how much total grace time you can use over the quarter. You can use the grace period on every single problem set and lab and still get full credit.

Academic Honesty

Please read CSE's Academic Misconduct Policy.

You are encouraged to discuss all aspects of the course with and ask for help from the instructor, the TAs, and other students. However, do not cross this line:

Do not share code or written text. Do not look at lab or problem set solutions that might be on the Internet. Do not use someone else's code or text in your solutions or responses.

It is ok to share ideas, explain your code at a high level to someone to see if they know why it doesn't work, or to help someone else debug if they've run into a wall.

This policy applies to use of artificial intelligence (AI) as well. You can ask a chat-based AI agent, such as ChatGPT, questions about the material at a high level, or ask it to explain why a certain code behavior might occur, but you cannot ask it to produce a solution for you. In-IDE agents, such as Copilot, are allowed in this class.

Some work in this class can be completed with a partner. The rest is to be done individually. We will clearly mark each piece of work as to whether it is to be done with your partner or individually. Please contact the staff if you are unsure about any part of how this policy applies to an assignment.

  • Lab 1 is individual.
  • Labs 2-4 are optionally with a partner. You must arrange your partner in advance in consultation with the course staff.
  • All problem sets are individual.

Partner work

You may optionally work with a partner on labs 2 through 4

The labs are difficult, and working with a partner can make things easier because you can discuss the details of your design together. Working with a partner is also a serious responsibility: your partner is relying on you to communicate and collaborate effectively. Do not agree to work with a partner if you are not willing to commit to these responsibilities.

A common misconception about working with a partner is that you should "split up the work". This is usually a terrible idea, especially in this class, because the hard part of the lab is the design process. It might take you 10 or more iterations to design your protocol correctly, and all parts of a distributed protocol can depend heavily on all other parts of the protocol, so there is no way to "split up" this design work. Instead, you should plan to pair program (i.e., work together synchronously) during the design process until you are confident that you have a correct design and have written the design document together. After that, you can either continue to pair program your implementation (we recommend this approach) or you can try to split up the implementation work. When in doubt, work together synchronously. When you do work separately, establish a clear communication channel and keep each other posted about where you are stuck. Again, do not commit to working with a partner if you are not interested in pair programming.

To summarize, by entering a partnership, you are agreeing, at minimum, to:

  • Communicate frequently with your partner during the weeks where design documents and labs are due.
  • Collaborate synchronously on the design document and any challenging implementation tasks.
  • Inform your partner in advance if you will be unavailable for communication or collaboration.

We take the partner agreement extremely seriously and will enforce it. If you flagrantly and repeatedly abandon your partner, you can expect to take a zero on the lab regardless of how much work you put into it.

If your partner breaks this agreement, first try to communicate with them about it. If that doesn't work, contact the course staff.

If you do decide to work with a partner, you can either find a partner on your own (and let us know) or you can ask us to match you with someone. We will send more information about the partnering process later in the quarter closer to lab 2.

When looking for a partner, be sure to communicate about grade expectations.

Anonymous feedback

Anonymous feedback can be sent to the instructor or TAs via feedback.cs.washington.edu.