Deliberate practice techniques for software developers
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
Using my QuoraClassifier tool, I’ve gotten about 25% of the way through my question list. So I thought it was the right time to revisit the criteria I’m using to assign primary categories.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
I’m writing a tool called QuoraClassifier to speed up the process of organizing the list of Quora questions I’ve collected. Last week, I used an early version of the tool that let me select a primary category for each question with a single keypress. This week, I added a few enhancements to allow for more specific classification.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
Over the past few months, I’ve been classifying a list of Quora questions. Each question gets a primary category, the topic it is most relevant to. Classification helps organize the FAQ and also allows me to find duplicates to merge on Quora.
Until now, I’ve been classifying questions using a spreadsheet containing one question per row and a column for the primary category. But now that I’m familiar with the types of questions people ask, it only takes a few seconds to decide on a category, so the spreadsheet approach is slowing me down. This week, I wrote a simple classification program, QuoraClassifier, to optimize this process.
The goal of QuoraClassifier is to make it as fast as possible to record a single decision: the primary category for a question. Recording the decision shouldn’t take much longer than deciding on the category.
QuoraClassifier is a simple WPF app. The UI language for WPF programs is XAML, which I used in last year’s project.
Like most of the tools I’ve built for the CPFAQ project, QuoraClassifier reads and writes tab-separated value (TSV) files. When it starts up, it reads a list of question data (titles, URLs, categories, etc.), which it keeps in memory. To save the classification results, it writes a TSV file in the same format. Since this is a quick and dirty tool, not a thoroughly tested application, I’m using two separate files to reduce the risk of data loss. Rather than overwriting the original file, I use two files that I can compare to verify that the changes are what I expect. If everything looks good, I can copy the output file onto the input file before the next classification session.
Once the input data loads, QuoraClassifier displays its single screen. The purpose of this UI is to 1) Display information about a question, and 2) Accept a classification choice. For maximum efficiency, the program lets me select a category using a single keystroke, at which point it displays the next question.
As shown in the screenshot at the top of this post, the UI includes the following fields:
To get through the questions in my list, I need to optimize the classification process. There’s a lower limit to how fast I can read a question title and decide how to classify it. For questions related to competitive programming, I’m probably already at the lower limit. So the only way to speed up the process is to optimize the act of recording a classification.
I decided that the fastest way to classify a question is using a single alphabetic key on the keyboard. Clicking with the mouse or tapping the screen is intuitive for many applications, but for speed, it’s hard to beat a keyboard interface. I assigned a unique letter to each category name. Since multiple categories start with the same letter, I couldn’t always use the first letter of the category. But it doesn’t take long to learn the letter associations: For example, P for People and R for PRoblems. The alternative, like pressing P once for People and twice for Problems, would just slow things down.
Here’s the category list:
And a few more keyboard commands:
With these classification commands, I can classify a question in an average of 6.7 seconds. That’s about 30 hours for the ~16,000 questions in my list, or less than a month of calendar time at an hour or two per day.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
When you’re logged in to Quora, you see more information than an anonymous user does. For example, on the all_questions page for a topic, logged-in users see a title for each question along with how many answers it has, when it was last followed or requested, how many followers it has, and various available actions. Anonymous users just see the question titles.
When I started collecting Quora questions for the FAQ, I noticed this discrepancy between the anonymous and logged in experiences. To collect as much information as possible, I often manually saved pages while logged in, and then ran my tools on the saved HTML. But for individual question pages this wasn’t practical since I’m tracking over 15,000 questions. For those, I wrote a program to download pages automatically. And since that program did not log in, some useful information was not available.
It would be ideal to combine the convenience of automation with the extra data provided to logged-in users. This week, I experimented with using the Selenium testing framework to achieve this. It turned out to be a simple process.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
I have now classified over 1000 Quora questions, using 552 canonical titles, and I think that’s a good spot to move on to some other CPFAQ tasks. But first, here’s how the numbers look compared with my previous checkpoint a few weeks ago.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
As I mentioned at the end of last week’s post, it’s hard to write a good canonical question title. Fortunately, there’s a Quora tradition in which answer writers provide high-quality answers to questions that might be less than stellar. I can take advantage of such questions in CPFAQ by reading all the related questions (and associated comments and answers), writing a canonical title that clearly expresses what the question writers want to know, and including links to the original questions.
Here are some categories of bad question/good answer pairs I have observed on Quora.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
Last week I discussed how question merging works for Quora and CPFAQ. Related to question merging is the idea of canonical questions. Although I have written about canonical questions in the past, I haven’t explained why they’re critical for CPFAQ. That’s the topic for this week.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
If a CPFAQ page has a canonical title and contains a list of Quora questions that all relate to the title, why not just merge all the Quora questions into one canonical Quora question? Good question.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
As I mentioned last week, I’m currently creating FAQ pages, and those FAQ pages rely on canonical question titles. This week I’ll discuss some observations about the set of titles I have so far.
I’m working on a project this year to build a competitive programming FAQ. This is one in a series of articles describing the research, writing, and tool creation process. To read the whole series, see my CPFAQ category page.
For at least the next few weeks, I’ll be creating competitive programming FAQ pages for the most frequently-asked competitive programming questions, according to my analysis of Quora content. That set of pages will give me a foundation on which to add more specialized questions over time. This week, I’ll explain the page creation process that I’m currently using.