A2 postmortem — the order of the questions

Eight of fifteen files answer every case correctly. The other seven all have a green test suite, and that is the interesting part.

A2 is not marked yet. This goes up first because Wednesday’s class is about exactly the thing that went wrong, and reading it beforehand is worth more than reading it after a grade. Nobody’s code is quoted here.

Fifteen files came in. Eight of them answer every case correctly. The other seven get at least one case wrong — and every one of those seven also wrote down an example claiming the wrong answer, so their tests are green and prove nothing.

There is a theme, and it is worth naming before the list.

else if is not a second if. The branches are tried in order, and each one only ever sees the cases the branches above it did not already catch.

Almost everything below is a version of that.

The solution

fun stripe(u :: Number, color :: String, formal :: Boolean) -> Image:
  doc: "a stripe u tall and u * 6 wide; outlined when formal or big enough to show, black from 30 up"
  if u >= 30:
    rectangle(u * 6, u, "outline", "black")
  else if formal or (u >= 10):
    rectangle(u * 6, u, "outline", color)
  else:
    rectangle(u * 6, u, "solid", color)
  end
where:
  stripe(9, "red", false) is rectangle(54, 9, "solid", "red")
  stripe(9, "red", true) is rectangle(54, 9, "outline", "red")
  stripe(10, "red", false) is rectangle(60, 10, "outline", "red")
  stripe(20, "red", false) is rectangle(120, 20, "outline", "red")
  stripe(30, "red", false) is rectangle(180, 30, "outline", "black")
  stripe(45, "red", true) is rectangle(270, 45, "outline", "black")
end

Three branches, and the first question is the one about 30.

That ordering is not a matter of taste, and it is the whole assignment. Ask about 30 first and the second branch can say formal or (u >= 10) and nothing else, because by the time control reaches it the large stripes are already gone. Ask about 30 last and every branch above it has to defend itself against the large stripes by hand — and the branch that forgets to is the bug.

That is not a prediction. It is what the fifteen files did:

• Five files put the 30-question where no branch above it could catch a large stripe. All five are correct, and four of them write the middle branch as formal or (u >= 10) with nothing bolted onto it.
• Three files put it lower and guarded every branch above it with and (u < 30). All three are correct too — the guards work. They are the price of the other ordering, not a mistake.
• Three files put it lower and left a branch above it unguarded. All three are wrong, in the same place: a large stripe that is also formal.

The remaining four files are shape 2 and shape 3 below.

If you are ever writing and (u < 30) into a condition, that is the code telling you a branch is in the wrong place. Move the branch and the guard deletes itself.

Three shapes that go wrong

These are reconstructions, not anyone’s file.

1. The large case is there, and never runs

if (u < 10) and not(formal):
  rectangle(u * 6, u, "solid", color)
else if (u < 30) or formal:      # a big formal stripe stops here
  rectangle(u * 6, u, "outline", color)
else:
  rectangle(u * 6, u, "outline", "black")
end

Read the middle condition on stripe(45, "orange", true). 45 < 30 is false, but formal is true, so or is satisfied and the branch runs. The "black" below it is unreachable for every formal stripe, no matter how big.

The function is not missing a case. It has all three, in the wrong order, and the third one is dead code for half its inputs.

2. formal is asked after the size, so a small formal stripe stays solid

if u < 10:                        # a small formal stripe stops here
  rectangle(u * 6, u, "solid", color)
else if formal or (u >= 30):
  ...

Same mechanism pointed the other way, and it showed up too. stripe(5, "red", true) never reaches the question about formal, because u < 10 caught it first and did not ask.

Both of these have a fix that is not “add a condition”. It is move a branch.

3. The third case did not survive problem 3

fun stripe(u, color, formal):
  if formal or (u >= 10):
    rectangle(u * 6, u, "outline", color)
  else:
    rectangle(u * 6, u, "solid", color)
  end
end

Two branches. The 30-and-above case that problem 2 asked for is simply gone — and in one file it is still there, correct, in the two-argument version further up the page, which was left behind rather than edited.

Note that problem 3 was an edit to a working function, not a fresh one. Adding a parameter should leave the cases you already had standing.

Your examples agreed with your body

This is the finding, and it is sharper than A1’s.

Of the seven files with a wrong answer, seven wrote at least one example stating that wrong answer — wrong in exactly the way the body is wrong. Of the eight correct files, zero did. There is no overlap at all. The examples split the class perfectly, and every one of the fifteen suites passes.

An example is only worth running when it could disagree with the body. Work the answer out from the rules on the assignment page, and it can. Run the body and write down what came out, and it cannot: you have recorded the bug and then confirmed the bug is still there.

A green where: block is not evidence that the code is right. It is evidence that the code agrees with you.

The good news, and it is real: fifteen of fifteen wrote a where: block, and most of you did the arithmetic — two files carried the body’s u * 6 across into every example instead of working the number out, and a third did it for some. On A1 that was five of seventeen. The habit is landing. It is the source of the numbers that has not moved yet.

The two examples that would have caught everything

Six examples is what problem 3 asked for, and fourteen of you wrote six or more. But most of the six were spent one per branch, and a branch is not where these bugs live.

Every bug above is at a place where two rules meet:

stripe(9, "red", true) is rectangle(54, 9, "outline", "red")     # small AND formal
stripe(45, "red", true) is rectangle(270, 45, "outline", "black") # large AND formal

The first one catches shape 2. The second catches shape 1 and shape 3. Neither is a boundary in the ordinary sense — 9 and 45 are nowhere near 10 or 30 — and neither is reachable by testing one rule at a time.

Two things that can each vary do not give you two cases. They give you six. Three sizes times formal and not. You do not have to write all six every time, but you have to have looked at the grid before you decide which ones to skip.

Two small things

u = 10 is still where < and <= disagree. Two files put 10 on the wrong side of it, and both said so in their own examples. The drill in problem 1 — change < to <=, press Run, watch exactly one example go red — is the thing that finds this, and it takes about fifteen seconds.

formal == true was written in two files. formal is already true or false, so formal == true computes the answer you started with. Worse in the negative: not(formal) says what it means, and formal == false makes a reader stop and check.

Four things that cost nothing here

None of these moved anyone’s mark. Each is worth thirty seconds next time.

Three files arrived under a name other than a2-stripes.arr — a space in it, a doubled .arr, a dot where a dash belongs. They were renamed by hand and graded normally. Worth knowing what the name is for: submissions are collected by exact file name, so a file under a different one is not found at all, rather than found and marked down.

Seven files have no doc: line. A2 did not ask for one, so nothing was lost here. Progress Exam 1 does ask: every function needs a contract and a doc: line. Four days is enough to get it into your fingers — write it before the body, not after.

Two files have three conditions and no final else. They cover every number, so they work. Pyret does not know that: if an input ever reaches the end without matching, the program stops with an error instead of giving you a picture. The last branch is the one you are certain about, so make it else.

One where: block called a function with the wrong number of arguments. Every example in it errored rather than ran, which Pyret reports separately from a failure — Ended in Error, not Failed. An errored example has not checked anything. When you read the test summary, read all three numbers.

Where this goes next

A3 does the same thing on lists, and the ordering problem comes back immediately: a function over a list asks the empty-list question first for the same reason stripe asks about 30 first.

The procedure the whole of this is measuring is on the design recipe page. The step it keeps failing at is the same one every time — examples before the body, worked out from the rules and not from the code.