Debug School 4: The Scientific Method

This is part 4 of the Debug School video series. This episode covers using the scientific method to approach debugging a problem. The scientific method is a scientific, explicit, and systematic to approach solving bugs. It will give you a system of dealing with software bugs.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : hi welcome to debug school this time
00:03 : better check yourself the scientific
00:07 : method sometimes when I'm helping a
00:10 : friend debug a problem they would say to
00:13 : me that's impossible it can't possibly
00:16 : happen and then they would show me that
00:19 : it can't possibly happen because they've
00:21 : already verified this and they've
00:23 : already verified that as if they found a
00:27 : smoking gun that proved that they were
00:29 : right and therefore their computer must
00:31 : be wrong now well computers have been
00:36 : accused of being drunk at times a much
00:39 : more probable explanation is that this
00:42 : is simply a case where a strongly held
00:45 : assumption is mistaken we are biased
00:49 : about the code that we write after all
00:52 : they're like our babies we made them
00:55 : when we write any program we hold
00:58 : assumptions about the code and this is
01:00 : natural it's necessary even because
01:04 : unless you make assumptions you cannot
01:06 : plan and programming is an activity that
01:09 : requires a lot of planning however in
01:13 : the context of debugging clearly
01:16 : something is wrong things are not going
01:20 : according to plan if something defies
01:23 : logic that means at least one of your
01:26 : assumptions are wrong
01:28 : your job now is to figure out which one
01:31 : of those it is you need to get
01:33 : suspicious it is kind of like when you
01:36 : know you have a mole in your spy agency
01:39 : and you really need to find out who it
01:41 : is so in that situation what do you do
01:44 : do you trust your gut the mo is there
01:48 : precisely to deceive you and he or she
01:51 : already has a pretty good track record
01:53 : at doing that if you continue to entrust
01:56 : this to your gut you could spend all the
01:59 : time in the world accusing innocent
02:00 : people while the agency slowly
02:03 : disintegrates in book tracking terms you
02:07 : could be going around in circles
02:08 : indefinitely and never resolve anything
02:11 : instead
02:13 : you would be better off by being more
02:15 : scientific via the Wikipedia the
02:19 : scientific method is a body of
02:21 : techniques for investigating phenomena
02:23 : acquiring new knowledge or correcting
02:27 : and integrating previous knowledge so
02:29 : when we encounter a bug we are going to
02:32 : look at our program as if it were a
02:35 : phenomenon in a sort of black box that
02:39 : behaves in a certain way something that
02:41 : we don't quite completely understand yet
02:44 : but we're ready to learn more about
02:47 : through scientific investigation here is
02:51 : the recipe for the scientific method the
02:54 : first step is you ask a question and
02:56 : then you come up with a hypothesis a
02:59 : sort of guess and an answer to the
03:01 : question and then based on that
03:04 : hypothesis come up with a prediction
03:06 : that can be tested with an experiment
03:08 : the fourth step is to actually test that
03:12 : prediction by running an experiment
03:14 : after running the experiment we analyze
03:17 : the results and come up with a theory
03:19 : that explains those results as well as
03:21 : other existing evidence that we have if
03:25 : there isn't enough data to form a theory
03:27 : we go back to step one and ask another
03:30 : question and we repeat the cycle and
03:33 : we'll do it over and over again until we
03:36 : do come up with a satisfying theory that
03:39 : explains the original question now let's
03:43 : put the scientific method to the test
03:45 : I'm going to debug this program and this
03:49 : program strap is to read in a text file
03:51 : containing a paragraph of text then it's
03:54 : going to go through each word in the
03:56 : text count them up and then at the end
03:58 : print a summary of each word that
04:00 : appeared in the paragraph as well as the
04:03 : number of times the word appeared in the
04:05 : paragraph first I'm going to run this
04:08 : program and as you can see when I run it
04:11 : nothing happened you might be wondering
04:14 : why good because that's exactly the
04:17 : first step so like a good scientist I'm
04:21 : going to be taking notes
04:22 : scientists write in journals but I'm
04:25 : just going to use
04:27 : another text file I'm going to call this
04:29 : file debug notes so question one is why
04:35 : is nothing being printed now I need to
04:39 : come up with a hypothesis for that let
04:42 : me review this source code a little bit
04:45 : so this program is pretty well organized
04:48 : and it has four main sections the first
04:51 : section scripts the text and removes the
04:54 : punctuation the second section splits
04:56 : the paragraph into individual words the
04:59 : third section here goes through the
05:01 : words and counts them up and then the
05:04 : final section here loops through the
05:06 : summary and Princeton now so clearly the
05:10 : last section is responsible for printing
05:12 : things out and if nothing is getting
05:15 : printed I can guess that the summary is
05:18 : empty so hypothesis is the summary as
05:23 : empty now I need a prediction based on
05:26 : that hypothesis my prediction is if I
05:28 : print a summary to the council on line
05:31 : 24 then it will print an empty object
05:35 : let's test it out so on line 24 we will
05:39 : consult that log summary let me run that
05:43 : and indeed it is an empty object so we
05:48 : have confirmed the prediction and for
05:52 : the analysis part I'm just going to say
05:54 : for some reason indeed summary is empty
05:59 : but why why is summary empty so now if
06:04 : went through one iteration of the
06:06 : scientific method steps and I'll repeat
06:08 : it again and again until we find the
06:10 : root cause of the problem again let me
06:13 : look at the code the summary variable is
06:16 : declared and set to an empty object on
06:18 : line 12 so presumably the lines between
06:22 : 14 and 21 here must be responsible for
06:26 : populating it as a sanity check
06:29 : is it even looping through the words so
06:32 : my hypothesis is the for loop that on
06:36 : line 14 may not even be iterating the
06:40 : words and if that were the case that
06:42 : would explain why the summary object did
06:45 : not get populated so my prediction is if
06:48 : I print out some dummy text within the
06:50 : for loop on line 15 that dummy tax will
06:54 : not get printed even once now let's test
06:58 : that so on line 15 I will consult that
07:01 : log some dummy tax I'm going to do my
07:03 : favorite Toby was here let's run the
07:07 : program and indeed none of that text got
07:11 : printed which confirms my prediction and
07:14 : for the analysis I'm gonna say for some
07:17 : reason the for loop on line 14 did not
07:21 : iterate any words why isn't it iterating
07:24 : the words well what if the words array
07:28 : on line 9 were empty to begin with so
07:34 : based on that my prediction is if I were
07:38 : to print the words array on line 10 it
07:41 : will be an empty array so let's test
07:43 : that I'm going to console dot log the
07:45 : words array run it and there are
07:50 : actually words in that array curious so
07:53 : this time my prediction was wrong and
07:56 : the experiment disproves my hypothesis
08:00 : so the words array indeed contained
08:03 : words but for some reason the for loop
08:06 : does not iterate through the words the
08:09 : next question naturally would be why
08:11 : doesn't the further iterate through
08:12 : those words let's review the code once
08:16 : again so there is this for loop and it's
08:19 : trying to iterate the words but that's
08:21 : not working my hypothesis is well maybe
08:25 : the stopping condition of the for loop
08:27 : is erroneous and that's preventing the
08:30 : loop from iterating so prediction if I
08:35 : print the stop condition of the for loop
08:37 : before the for loop starts it should
08:41 : print false because that would explain
08:44 : why it never iterated to loop so let's
08:47 : test that prediction
08:49 : there's a little hiccup here because I
08:51 : need the variable I to be initialized in
08:54 : order to even execute this condition I'm
08:56 : going to need to extract this
08:59 : initialization outside of the for loop
09:02 : then I can cancel that lock the
09:05 : expression in the stop condition let me
09:08 : run that and indeed the stop condition
09:12 : is false at this point so that confirms
09:15 : my prediction analysis the stopping
09:19 : condition is indeed erroneous we
09:22 : expected it to be true causing the
09:24 : iteration of the for loop but this is
09:27 : false next question then naturally would
09:30 : be why is the stop condition I less than
09:33 : words dot length incorrect well there
09:37 : are only two values I should be zero
09:40 : because I had just set it to that so my
09:44 : guess is that words dot length is
09:47 : incorrect and does not return the real
09:50 : length of the array prediction if I
09:53 : print out the words dot length on line
09:56 : 16 it does not print the real length of
09:59 : the array so to test that let me print
10:03 : that out let me run that and indeed it
10:07 : did not return the length of the array
10:08 : at all but instead it printed undefined
10:11 : so that confirms my prediction my
10:16 : analysis is words dot length is
10:19 : incorrect okay so why is what's that
10:22 : length incorrect at this point if you
10:24 : are familiar with JavaScript you
10:26 : probably have already figured it out the
10:28 : length property should be lower cased so
10:31 : my prediction is if I had fixed the
10:33 : casing to lower case length then it will
10:36 : correct the mistake and the program will
10:39 : work correctly
10:40 : let me test that so I'm going to fix the
10:43 : casing of words dot length in both
10:46 : places and run it and indeed that works
10:50 : the word summary is now printed as
10:52 : designed so I'm really glad I got to the
10:56 : bottom of this my final analysis and
10:58 : diagnosis is that the mistake that
11:01 : caused
11:02 : program to malfunction was the miss
11:05 : casing of the length property finally
11:09 : now that I've solved this problem I had
11:11 : better go back and remove all the extra
11:13 : comes without log statements that I
11:14 : added now let me just run it one more
11:17 : time to see that the program is working
11:20 : as it was intended and it is
11:25 : one time one of my students Paul came to
11:29 : me for programming help and he said
11:32 : something along the lines of it's not
11:34 : working and I think I know what it is
11:36 : and then he would tell me his hypothesis
11:39 : so then to that I would say okay now go
11:43 : verify it when he hears that he grumbles
11:47 : and then he settles down and makes an
11:49 : effort and eventually comes up with a
11:52 : way to test his theory and this helps
11:55 : him make progress to itself in a problem
11:57 : eventually what I'm wondering now is why
12:01 : did he have to come to me just so that I
12:03 : can tell him to test his hypothesis
12:05 : because clearly he has done steps 1 2 3
12:10 : in the scientific method but he stopped
12:14 : short before step 4
12:16 : why one reason is that he simply hasn't
12:22 : been following this scientific method
12:24 : and that is why I'm putting together
12:27 : this lesson here but there is also
12:29 : another reason and it's a psychological
12:32 : phenomenon called the streetlight effect
12:35 : and here's the story associated with
12:38 : this phenomenon on a pitch-black night
12:42 : there is a guy let's call him Charlie
12:45 : Charlie is crawling on all fours under a
12:48 : streetlamp intently looking for
12:50 : something Charlie might be a little
12:53 : drunk
12:53 : a policeman sees him and comes over to
12:57 : help hey Charlie what are you looking
12:59 : for Charlie says I'm looking for my keys
13:02 : the policeman helps him look and they
13:06 : look all over and all around but they
13:09 : haven't found the keys so the policeman
13:12 : says charlie
13:14 : are you sure you lost your keys here and
13:17 : Charlie says well actually I think I
13:21 : might have lost them at the park the
13:24 : policeman says well if you lost them at
13:27 : the park then why are we looking here
13:30 : for the keys and Charlie says the part
13:33 : is pitch black and I couldn't see a
13:35 : thing over there
13:36 : this is where the light is
13:40 : people often look for the thing that
13:43 : they're looking for by looking only
13:45 : where the looking is easy but this is a
13:48 : dead end if what you're looking for is
13:50 : elsewhere in the previous story Paul was
13:55 : thought when he couldn't readily come up
13:57 : with a way to test the hypothesis that
13:59 : he came up with and this is a keen to
14:03 : Charlie not wanting to search for his
14:04 : keys in the park because it's too dark
14:07 : when Paul came to me all I had to do to
14:11 : get him to test his hypothesis was to
14:13 : make him think harder and the same thing
14:17 : could be applied to Charlie's situation
14:19 : how can I effectively search for keys in
14:22 : the part oh I know I could bring a
14:25 : flashlight
14:26 : the streetlight effect is something that
14:29 : occurs a lot and so we need to be able
14:31 : to overcome it here's how first follow
14:35 : the scientific method make sure you
14:38 : don't skip on the fourth step of testing
14:40 : the hypothesis I have personal beaten
14:44 : situations where I've acted upon
14:46 : untested assumptions and then ended up
14:49 : wasting both my time and my quarters
14:51 : time second know your tools two common
14:56 : reasons for ineffective debugging that
14:58 : I've seen are not knowing your tools
15:01 : well and not knowing that a certain type
15:03 : of tool exists when you know your tools
15:07 : well it's like having a flashlight by
15:09 : your side for nearly every situation
15:12 : third if it's not obvious how to test a
15:16 : hypothesis take a step back
15:19 : brainstorm research and ask other
15:22 : developers doing this will also help you
15:25 : come up with more than just one way to
15:28 : test a hypothesis which is very useful
15:31 : for situations when you want to be extra
15:34 : certain of something and also for
15:37 : situations where certain tests simply do
15:39 : not work or are unreliable that
15:43 : concludes this lesson on the scientific
15:46 : method I'm going to leave you with some
15:48 : exercises to chew on please work on
15:51 : these because it will force you to apply
15:53 : the concepts of
15:54 : the lesson and thus retain it for future
15:57 : use see you soon