About the only social media achievement I am proud of in my life is that I am currently in the top ten of answerers on the Prolog tag of Stack Overflow. I’m especially proud of this because it’s something I’ve achieved by passion alone. I am not an especially talented Prolog programmer. Nor am I a particularly active Stack Overflow user. I don’t use Prolog professionally… in fact I tend to follow a “mama’s cooking” philosophy to it which is probably less than it deserves. I even have a longstanding feud with one of the other dudes.

But I continue to enjoy answering questions on the Prolog tag of Stack Overflow. I tried Documentation once for about an hour and decided it was absolutely the stupidest idea I had ever tried out. In fact I rather hope it fails.

Each Tag is an island

If you are a casual Stack Overflow user—in other words, someone who programs but doesn’t answer questions—you may not realize that tags are informally governed by different groups differently. Stack Overflow overall is not against helping people with homework, but the Prolog tag is very much against it. Why? Because most people do not encounter Prolog as a momentary flight of fancy; instead, they get it as coursework towards some degree. A lot of seemingly innocuous questions have “tells” that give this away, such as a genealogy problem, or a subway route problem, or whatever.

Prolog questions are not exactly in abundant supply. With high-volume tags like Postgres or even Haskell, you have to answer quickly. There are a lot of people who care about pruning duplicate questions. You can gain reputation quickly for a single good answer because there are a lot of eyeballs on a decent question. I’m not prepared to call this a virtuous cycle; it’s just how these things work on a higher-profile tag.

In the Prolog tag, there will be a handful of questions each day, there is no time pressure to answer in a minute or less, and a great answer on a genre-defining question might get ten upvotes. One upvote is probably the mean on the Prolog tag. There are so few Prolog experts on the site, it can take multiple days for an obviously bad question to get closed. So it’s a lot more common to answer a duplicated question.

I actually like this aspect of the Prolog tag. It gives me a chance to hone my writing. Some of my very best writing has gone into Stack Overflow. And I try to write to a specific audience: the person asking the question. I try to meet them where they are. I can guess with (I believe) reasonable accuracy what nonsense notions they believe about what Prolog is doing, because I have had those notions, and recently. Prolog is unusual, in that there are not a lot of great precedents for how it works. So there are a lot of wrong mental models of what it is doing out there.

Another interesting thing about Prolog is that there really aren’t any bad books about it. There were, in the 80s, but the books that are still circulating now are excellent. There is no shortage of great documentation about Prolog, both tutorial and detailed.

“Documentation” is useless for Prolog

Let me summarize the situation:

  • Prolog already has ample (but confusing) documentation
  • There are lots more ways to fail to understand Prolog than ways to understand it
  • Clues about your mental model’s deficiencies are often obvious in the way you ask the question (“what does append/3 return?”)
  • The community is too small to use Stack Overflow’s moderation functions effectively, and the incentives don’t add up for us to use them anyway

What does Documentation bring to the table?

  • Another way to document something already amply documented
  • No help for approaching different mental models
  • Moderation-heavy process for editing and managing documents

What problem is Documentation trying to solve? I guess the idea here is to find a way to move questions (especially frequently-duplicated questions) into some kind of wiki. But that’s more stuff for people to read, who are already having trouble reading. Stack Overflow isn’t just question-answer, it’s programmer-to-programmer technical support. Documentation isn’t for anybody. It doesn’t speak to this use case. It isn’t rewarding to work on. For #prolog folks, it’s a step in the wrong direction. And indeed, almost nothing has happened on it since the day of the debut, because it obviously does nothing for us. There are 16 topics.

What about other tags?

Consider Postgres. The tag is pretty high-volume. There are a lot of server admin questions, a lot of beginner SQL query questions, but a lot of people. The same mental model thing happens, to a lesser degree. It should be “documented” ten or 100 to one compared to Prolog. Instead, it has 22 topics instead of 18. Why?

I think it’s because the same comments apply. Documentation is unicast, Q-A is point-to-point. Wrong mental models of SQL are a common problem that is mostly not addressed by reading the documentation.

Another point on Postgres: it has some of the best official documentation of any project anywhere. What’s there to say about Postgres that isn’t already said in the official documentation? Examples?

Why do we like Stack Overflow?

Stack Overflow is great. We use it all the time, even if we hate it, because Google finds answers to our questions there. It’s the universal programming help desk. Stack Overflow solved a legitimate problem: taking a bunch of programmer technical support discussion and restructuring it into a resource.

Ever searched for something and found the answer on a forum like Java Coder Ranch? Where you have to skip to page 19 to try and find the actual answer to the problem, only to find out you went too far and now you have to page backwards slow to avoid all the “Thanks!” shit? Had to stare at the page for a little too long to try and figure out if you’re looking at the date the comment was left or the date the schmuck joined the forum?

Stack Overflow took a thing that was already happening (asking questions and getting answers) and stripped away the bad and useless shit that was holding that information captive. They came up with a more sensible concept of what was going on and developed a totally new interaction for it.

So… what’s the problem Documentation solves? What thing do we all do that Documentation somehow improves? The truth, we both know it, is that Documentation doesn’t fit into any existing workflow. It doesn’t even fit into Stack Overflow’s workflow, and it was built to fit in there.

Documentation also deprives the programmer of the feeling of helping someone. Which feels better, giving a dollar to a guy on the street, or dropping a dollar into a jar that says “for the homeless” on it? Knowing you are helping a specific person is a huge motivator. Documentation deprives you of that source of pleasure. All the complaints about Documentation on Stack Overflow have the same flavor: “it feels like a lot of work.” It is a lot of work, and so is answering Stack Overflow questions! But answering those questions feeds our desire to be charitable to each other, in a way that writing examples for documentation just does not.

So… the reward scheme is messed up, the value-add is unclear, it’s demotivating, it’s labor-intensive, it’s not going to do anything to staunch the flow of bad questions, it doesn’t clearly improve any real existing problems while creating some new ones… what is Documentation good for?

Stack Exchange: Documentation is a fail. Drop it.