Before learning discrete math, it’s useful to know how much other math you remember. Last week, I discussed how Khan Academy is uniquely useful for evaluating and improving your skills in math topics through Calculus. But eventually you might reach Mastery level in all Khan Academy math skills. Does that mean it’s time to move on? Not quite.

## After Mastery

If you complete enough *mastery challenges*, you will master all the Khan Academy World of Math skills (except maybe one due to a bug). Once you reach this point, mastery challenges work a bit differently. In a normal mastery challenge, you get five or six problems in different skill areas, at a variety of mastery levels. Once you’re at 100% mastery, you get just one problem (or occasionally two), and it’s always at Mastery level. When you submit an answer for that one problem, the challenge is over.

You can always click the Start button again to get another challenge. The system will keep giving you challenges as long as you want. If you get a challenge question wrong, your rating drops back to Level Two. But if you then start another challenge, the system will present you with a problem from that Level Two skill, and if you get it right, you’ll be back up to Mastered in that skill. So it’s not too hard to stay at 100% mastery until Khan Academy releases its next batch of new skills.

How does the mastery challenge system select which skill to use for your single post-Mastery problem? Even though all your skills are now at the same level, the system doesn’t present problems randomly. Skills you have practiced recently are less likely to come up than those you haven’t seen in a while. But that seems to be the only rule the skill selection system uses. Even though you have mastered Khan Academy Calculus, you’ll still get problems in basic arithmetic. As a result, the mastery challenge system isn’t as useful as it once was after you have mastered all the skills. It needs some manual intervention.

## More Mastery Levels

Even before 100% mastery, the mastery challenge system sometimes presents you with problems from skills you have already mastered. That’s because mastery is not a permanent state. You can forget skills if you don’t continue to practice them. But you don’t need to practice every mastered skill at the same frequency. For example, once you master your multiplication tables in childhood, you never need to practice them again. Partly this is because you practice them in other contexts, but it’s also because they’re not that complicated, so you can commit them to long-term memory. But the Khan Academy mastery system still keeps giving you multiplication problems.

A rating of *mastered* for a Khan Academy skill just means you got several problems in that skill correct with no intervening incorrect submissions. It says nothing about other aspects of mastery:

- How long it takes you to solve problems of that type. Khan Academy keeps track of problem-solving duration, and you can see a graph of that data for each skill. But being very slow at solving a problem doesn’t prevent you from eventually mastering it. It just means you have to move through each level individually (Not Started, Practiced, Level One, Level Two, Mastered), rather than jumping multiple levels.
- Whether you can solve a problem from memory or if you have notes that you refer to as you’re solving it. (Khan Academy isn’t watching you through your webcam).
- Whether you made a lucky guess, or if you really understand the concept behind the problem.

The Khan Academy practice and mastery system does try to target these aspects of mastery. For example, problem writers try to write problems that discourage guessing and require an understanding of the skill being tested. But only you know for sure how well you understand a concept, whether you guessed the answer, and whether you used notes.

Despite these limitations, I think Khan Academy World of Math is still useful post-Mastery. It’s an effective way to get unlimited auto-graded practice problems on a topic, and the instructional content is still available to explain concepts you’re not perfectly clear on.

Since the built-in mastery levels aren’t useful once everything is marked as Mastered, my recommendation is to construct your own rating system for post-Mastery skills. Here’s how that process would work. First, select a skill, either manually or by starting a mastery challenge. Solve a problem for that skill. Then pick the description that applies best to your experience solving the problem:

**Trivial**: No real work is required to solve the problem. You read the problem and the answer is obvious to you. It might take half a minute to check your work, but you can do that in your head.**Easy**: You know how to solve the problem from memory, but it might take a few minutes and require pencil and paper. If you need to make a lucky guess to solve the problem, you should rate it Moderate.**Moderate**: You remember practicing problems of this type, but you have forgotten a few details and need to look them up. Solving it might require on the order of 10 minutes.**Hard**: You’re not sure how to get started on the problem.

The goal with these ratings is to evaluate your true level of competence for skills you have previously learned to a Khan Academy level of mastery. To get an accurate rating, you shouldn’t have solved any problems of the same type in the past week or so (ideally in the past few months). The Khan Academy mastery challenge system takes care of that timing automatically. The wait time ensures that you have the skill in long-term memory.

There are 1498 skills, so if you evaluate five unique skills per day, you could finish in 10 months. At that point, you would have a list of skills to work on. My suggestion would be to ignore the skills rated Trivial and Easy. You know those well enough. The goal is to turn Hard skills into Moderate ones by studying topics you’re missing, and Moderate skills into Easy ones by brushing up on details and practicing to gain fluency.

There will naturally be a range of skills in the Easy range. Although by definition you can solve them all without referring to notes, some problems will always take longer to solve than others. The other consideration is how well you want to know a topic area for future work. Although every Khan Academy math skill is part of a basic math education, you may reach the point where it’s better to move on to skills that the platform doesn’t cover.

Next week, I’ll cover specific examples of problems in each skill category.

*I’m writing about discrete math and competitive programming this year. For an introduction, see A Project for 2019. To read the whole series, see my Discrete Math category page.*