Debug School 6: The Bug Sandwich

This is part 6 of the Debug School video series. It will cover the technique called The Bug Sandwich: the second of the debugging power tools. The Bug Sandwich is also known as delta debugging else where. Similarly to Test Case Simplification, once you have your Bug Sandwich trap setup, the bug simply has no where to escape.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : hello and welcome to debug school this
00:03 : time we're talking about a technique
00:05 : that I lovingly called the bug sandwich
00:08 : have you ever meticulously followed a
00:10 : code tutorial step by step but only to
00:13 : get an error when you actually try to
00:15 : run the code have you ever found a
00:18 : broken featuring your app that you know
00:20 : for sure it used to work before have you
00:23 : ever tried to do something you've done
00:25 : many times before but for some
00:27 : mysterious reason it is simply refusing
00:30 : to work this time
00:32 : have you ever been in a situation where
00:35 : your app works for you perfectly in
00:37 : development only to brick when it is
00:40 : deployed to production what do all of
00:43 : these situations have been common well
00:46 : in each of these cases you can identify
00:48 : a version of the code or test case that
00:52 : is working and an alternate version that
00:55 : is not working what can you do about
00:58 : this well you can certainly throw your
01:01 : hands up and give up be like oh the gods
01:04 : are mocking me or you can take advantage
01:06 : of this situation clearly there is
01:10 : something different about the two test
01:12 : cases what is it if you can find the key
01:15 : difference that makes one of them work
01:18 : and the other one break then you can
01:20 : figure out the root cause of the problem
01:22 : the technique that I'm going to show you
01:25 : in this lesson is called the bug
01:26 : sandwich and that is precisely what it
01:29 : does imagine that you are a bug and you
01:33 : have fallen into a trap you've been
01:35 : dropped into the middle of a room with
01:37 : no exits the two walls on opposite sides
01:40 : of the room have spikes coming out of
01:43 : them and both of them are moving and
01:46 : they're coming right towards you slowly
01:48 : you are sandwiched right in the middle
01:52 : between these two walls and you have no
01:55 : way to escape you are trapped your
01:58 : demise is eminent all you can do is to
02:02 : say your prayers while you await the
02:04 : inevitable for the bug in pursuit this
02:08 : is what it feels like when you deploy
02:10 : the bug sandwich upon it
02:13 : from your perspective it can sometimes
02:15 : feel a little sadistic this is how it
02:19 : works
02:20 : to use the book sandwich you need a test
02:23 : and two test cases a passing case and a
02:27 : failing case initially the passing case
02:30 : should pass the test and the failing
02:32 : case should fail the test the test
02:35 : doesn't necessarily have to be an
02:37 : automated test you can certainly do the
02:39 : test manually the most important thing
02:42 : about the tests is that its results are
02:44 : consistently reproducible your objective
02:48 : is to find the smallest difference
02:49 : between the two test cases such that
02:52 : removing that one difference either
02:54 : causes the passing case to fail with the
02:56 : failing case to pass the actual
02:59 : procedure goes like this
03:01 : first you identify the differences
03:03 : between the two test cases then you go
03:06 : through each difference one by one for
03:09 : each difference remove that difference
03:11 : by either modifying the passing case to
03:13 : match the failing case or by modifying
03:16 : the failing case to match it passing
03:18 : case then you run the test again on the
03:21 : test case that you just modified if the
03:24 : test case result flips ie
03:26 : the passing case now fails or the
03:29 : failing case now passes then you have
03:32 : found the failure inducing difference
03:35 : now let me demonstrate I'm making a
03:39 : to-do list application using the koel
03:41 : web application framework currently the
03:44 : app only displays a list of tasks to be
03:47 : done but I have yet to implement a
03:50 : feature of allowing the creation of a
03:52 : new task as you can see both of the
03:55 : existing tasks have to be in hard-coded
03:57 : into my app right now if I try to create
04:00 : a task by typing it into this text box
04:03 : and then hitting Enter it makes a post
04:06 : request to the tasks URL but I don't
04:08 : have a handler setup for that yet so it
04:11 : comes back with an error not found so
04:14 : I'm going to make a handler for it
04:17 : if implemented correctly this forum post
04:20 : handler should receive the input values
04:23 : and get the values posted from the forum
04:26 : via the object this that request that
04:28 : body
04:30 : and it should receive a parameter called
04:33 : new task name because that's what I
04:36 : chose to call the input element in this
04:38 : form so I'm going to print out this that
04:41 : request that body that new task name to
04:44 : see that the value has been correctly
04:46 : passed in to test this I will come back
04:48 : to the task list type hello and hit
04:51 : enter I'm getting internal server error
04:55 : looking at the terminal I can see that I
04:58 : have an error on line 35 type error
05:02 : cannot read property new task name of
05:05 : undefined so evidently this dot request
05:09 : that body is undefined at the moment and
05:12 : that reminds me that with the quorum
05:15 : work in order to receive post parameters
05:17 : I actually have to install a body parser
05:21 : library as a middleware so I'm going to
05:24 : go ahead and look that up on Google
05:27 : really quick
05:29 : are actually several body parsers for
05:31 : koa I'm just going to select the first
05:34 : result okay I like this readme page
05:38 : because the first thing that I see is a
05:40 : simple example of how to use it which is
05:42 : exactly what I need I'm going to use
05:45 : this as a reference and type the
05:47 : relevant bits into my own app first I
05:50 : need to require the module coop body
05:52 : parser then I need to tell the app to
05:55 : use the middleware okay there we are now
05:59 : let me try that again
06:01 : type in hallo again hit enter and I get
06:05 : the same error now I'm a little confused
06:09 : because I have used this module before
06:11 : in another app and it has worked for me
06:14 : before so why isn't it working for me
06:16 : right now as you might have guessed this
06:19 : is the point where I'm going to employ
06:21 : the bug sandwich I'm going to bring up
06:24 : the other app that I made and I'm going
06:26 : to use that as the passing case this app
06:29 : is called holo and it simply asks your
06:32 : name and that it will greet you by
06:35 : saying your name back to you the key
06:38 : thing that we care about and want to
06:40 : test is that it actually successfully
06:43 : takes input from the forum and renders
06:45 : it back so I'm going to type in my name
06:48 : submit the form and it does say my name
06:52 : back to me so it does appear to work
06:56 : to start the procedure I'm going to put
06:59 : these two source files side by side so
07:01 : that I can easily spot any differences
07:02 : between them
07:03 : the first difference I see is the use of
07:06 : you strict on the left side this
07:09 : probably has nothing to do with the
07:11 : cause of the problem but nonetheless I'm
07:13 : going to remove it by adding this line
07:15 : to the right now since I changed the red
07:19 : side which is the holo
07:21 : app I'm going to retest it and it still
07:23 : passes
07:25 : the next difference that I see is this
07:27 : extra set of parentheses over here on
07:30 : the right I'm going to remove them to
07:32 : make both sides the same but there's
07:35 : probably a good reason these set of
07:37 : parentheses are where they are so I'm
07:40 : going to search in both files for the
07:42 : place that the body parser variable is
07:45 : used here I see that on the left side we
07:48 : have the parens down here where the body
07:51 : parser is registered as a middleware but
07:54 : not on the right side so I'm going to
07:57 : match those as well and again this
08:01 : really should make no difference to the
08:03 : outcome but just in case I'm going to
08:06 : test it out and the hello app still
08:08 : works the next difference I'm going to
08:11 : remove is a big one I'm going to change
08:13 : the entire functional behavior of the
08:15 : app on the left side to match that of
08:18 : the right side I'm making this drastic
08:21 : change because right now the thing I
08:22 : care most about is whether or not the
08:25 : form pose is working and not the
08:27 : behavior of the app itself so I'm going
08:30 : to copy the code that implements the
08:32 : holo app into the left to replace the
08:35 : code for the to-do app
08:42 : now let me retest the app on the left
08:44 : side which used to be the to-do app but
08:46 : now has become the hello app I'm going
08:50 : to type Toby and hit enter here and I
08:53 : get an internal server error which is
08:56 : interesting because we now have a
08:59 : nonworking Hollow app on the left and a
09:01 : working hollow app on the right so we're
09:04 : making progress
09:05 : so right now the left side and the right
09:08 : side are identical all the way up to
09:11 : line 25
09:12 : the remaining differences are number 1
09:14 : the ordering of the two lines here and
09:18 : the port numbers I really don't think
09:21 : the port number is likely to be the
09:24 : deciding factor so I'm going to flip
09:26 : these two lines here and retest and it
09:31 : now works so the flip has occurred
09:34 : namely the failing case on the right is
09:37 : now working so we now know that the
09:41 : failure inducing difference is the
09:43 : difference in the order of these two
09:45 : lines and this tells us that the order
09:49 : is important when it comes to
09:51 : registering middleware with the Khoa web
09:53 : framework what I will do now knowing
09:56 : what I know is to revert the left side
09:58 : back to the to-do app that I had
09:59 : originally and then reapplied this fix
10:03 : so that I can have an actual working
10:06 : to-do app so let me test this
10:10 : and I get a not found
10:14 : looking at the terminal however I see
10:16 : that I did get the name of the task
10:18 : printed out in the terminal which means
10:21 : the forum post did actually work we just
10:24 : have to write the code to handle it so
10:27 : the thing left to do is to add a new
10:30 : task to the tasks array and redirect the
10:33 : page back to the tasks page now let me
10:36 : test this again
10:40 : able to add new tasks to the list just
10:43 : the way that I planned so that is the
10:46 : book sandwich at work now I want to talk
10:49 : about the various situations where you
10:51 : can apply it the first situation is if
10:54 : you are so lucky to have another
10:56 : existing app as I did in the demo that
10:59 : is relevant to what you're doing and can
11:02 : serve as the passing case scenario two
11:06 : is when you are following along a code
11:08 : tutorial and you encounter an unexpected
11:12 : error or behavior when you try to run
11:14 : the program most of the time you have
11:17 : access to downloadable or a copy and
11:19 : paste the bowl demo code which should
11:21 : work right out of the box and providing
11:24 : that demo code works and you will
11:26 : definitely want to test this first to
11:28 : make sure that it's the case you can use
11:31 : that as the passing case scenario three
11:34 : is when you find a broken feature in an
11:37 : application and you know for sure that
11:39 : that feature used to work before if the
11:43 : project is under source control you can
11:45 : likely find a previous version of the
11:47 : app where the bug is not existent so if
11:50 : that's the case you can use that
11:52 : previous version as the passing case in
11:54 : fact the version control system get has
11:58 : a feature called
11:59 : bisect which is used precisely for doing
12:01 : this in a separate lesson I'm going to
12:04 : cover using source control for debugging
12:06 : in a little bit more detail scenario
12:09 : four is when the app is working for you
12:12 : in development but is broken in
12:14 : production or another environment there
12:17 : the challenge is not in finding the
12:19 : difference between two source files but
12:22 : rather finding the difference between
12:24 : different environments as usual I'm
12:28 : going to leave you with two exercises to
12:31 : work on in order to help you to remember
12:33 : this technique in a deeper and more
12:35 : memorable way so that when you actually
12:38 : need it in the heat of battle you can
12:40 : summon it up until next time