Fun with AST Part 2

This is the second hour-long lecture on ASTs which I did for Insiten's Algorithm Friyay. We finish building a script that can tell which variables in a JavaScript program are unused by traversing an AST of the program.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : and and i did like post the last session
00:03 : on my website
00:04 : it's this one fun with asts
00:08 : so if you want to re-watch that one you
00:10 : can
00:11 : um it's even have a transcript
00:14 : and everything um
00:18 : but i'll give you a brief recap of what
00:21 : we did last time
00:22 : uh so basically um
00:26 : we i have this very simple
00:29 : javascript program and what we're trying
00:32 : to do
00:33 : is we're trying to find all the unused
00:36 : variables
00:38 : in the programs like this b variable
00:41 : that's unused because we declared it but
00:44 : then we did not
00:45 : reference it later in the program so
00:48 : we're trying to
00:49 : write a program that can read this code
00:52 : and figure out oh b is was unused you
00:56 : don't need it and then tell you about
00:58 : that
01:01 : so the step one is to parse this code
01:03 : and luckily
01:05 : there are off the shelf open source
01:08 : javascript parsers
01:09 : that we can use we use one called s
01:12 : prima
01:15 : and we just said as prima.pars
01:18 : give it the code as a string and how it
01:21 : gives us
01:22 : an ast structure
01:25 : we save that ast structure into a file
01:29 : like this so so this parse program
01:33 : the way you use it is like you say node
01:36 : parse js and then you
01:40 : give it the path to a javascript file
01:42 : that you want it to
01:43 : parse it parses it and it will spit out
01:46 : a dot
01:47 : ast file so we're looking at this
01:50 : dot ast file and last time we sort of
01:53 : read through the ast structure in some
01:56 : detail
01:57 : and what we also did last time was
02:02 : we wrote this traverse function
02:05 : which traverses this ast
02:10 : and we'll run we'll run this code
02:13 : right now show what we have here
02:16 : so you you run this traverse program
02:20 : and you give to it as argument an ast
02:24 : file
02:26 : and then what it does at this point is
02:29 : it traverses the whole tree
02:31 : uh and along the way it prints out
02:34 : the sort of node or at least
02:37 : some information about the node uh
02:41 : as it's traversing through each node of
02:43 : the tree
02:45 : and in an indented fashion uh so
02:48 : that's what this code does
02:52 : there's like 60 lines of this so
02:55 : we're almost there to solving the
02:57 : problem
02:58 : i think we just finished this sort of
03:01 : basic traversal functionality last time
03:03 : so
03:04 : what we're going to do this time is
03:06 : we're going to
03:07 : use this traverse function to
03:10 : to solve our original problem which is
03:14 : i would like to be able to find all of
03:17 : the
03:17 : unused variables in my program
03:21 : um to do how would we do this by the way
03:25 : uh i want you guys to give me some ideas
03:28 : given that we are now able to
03:32 : you know convert this code into an ast
03:35 : that represents that code and now we
03:38 : have a traverse function
03:39 : that can walk every syntax node
03:43 : in the ast what is the next step
03:47 : we we need to do in order to find all
03:49 : the unused variables in this program
04:09 : yeah yeah so i mean
04:12 : yes that's that's usually a good thing
04:14 : to do because you might want to
04:16 : traverse the tree for different reasons
04:19 : right um now i i could
04:23 : i could um could do
04:26 : maybe for our specific problem though
04:30 : like the problem being
04:34 : you you're saying we need like a generic
04:37 : function
04:38 : but i i also want to get specific like
04:41 : for our problem
04:42 : of how to find unused
04:47 : unused variables
04:52 : what are the steps like step one is to
04:55 : what
05:05 : okay yep fine
05:13 : yeah find all the variable declarations
05:15 : right and then
05:16 : step two
05:33 : yeah and i would how would you do that
05:44 : yeah so so what happens like
05:47 : um let me do a split screen here
05:54 : so like basically we want to find the
05:57 : references
05:58 : to the variables that might be declared
06:01 : right like for example this
06:02 : is a reference this a is a reference
06:08 : so this is statement one statement two
06:12 : this is statement three this statement
06:15 : three
06:15 : is like we're declaring c
06:18 : and the initial value of c
06:21 : is this binary expression uh
06:25 : this plus operation in particular and
06:27 : where the left-hand side
06:28 : is this a and the right-hand side is
06:32 : this one so so i think this
06:34 : identifier a is what you're kind of
06:36 : looking for
06:46 : right
07:00 : foreign
07:14 : i yeah uh like put
07:18 : maybe a dictionary would be more
07:20 : convenient and
07:22 : stick them and
07:26 : put them into a dictionary
07:30 : find all identifiers
07:33 : and remove them from the dictionary
07:37 : does that sound right
07:42 : and then
07:56 : uh
08:01 : also like
08:10 : oh oh okay so so there's there's um
08:15 : um actually we don't want to count
08:19 : the declaration as a reference do we
08:25 : because it
08:26 : because if we if we counted this as a
08:29 : reference
08:30 : then we would automatically count it as
08:33 : used even though it hasn't been used
08:39 : oh
08:46 : oh
08:51 : oh oh oh i i
08:54 : i see what you're saying you're saying
08:57 : um if it don't
08:58 : if we count the declarations also as
09:02 : usage uh reference then if it only has
09:06 : one usage then it
09:09 : uh then that's proof
09:12 : that it was only declared but not used
09:16 : but i i think i have a counter example
09:18 : to that because i can
09:20 : i can say
09:24 : that i never declare and i think that
09:27 : still parses
09:28 : let me see
09:32 : yeah that still parses but what ended up
09:35 : happening is i actually have a reference
09:37 : to z
09:38 : but i never declared z the parser
09:42 : doesn't care
09:43 : when you run it there's gonna be a
09:45 : problem
09:47 : uh you know when you run it it's
09:49 : actually a runtime exception
09:51 : it's it in some programming languages
09:54 : they will check that in compile time and
09:56 : not even let you run it but but a
09:58 : javascript javascript doesn't care it
10:00 : just try to
10:01 : run it when it gets to that line it's a
10:04 : runtime error
10:16 : so
10:20 : i think that's a big problem in a long
10:23 : term sense because
10:26 : like i john's strategy is
10:29 : i'm gonna regardless of whether it's a
10:31 : declaration
10:32 : or a reference i'm gonna count i'm gonna
10:36 : count it the same way but i'm saying
10:39 : that doesn't really work
10:41 : due to this
10:45 : right because if if if we use john's
10:48 : strategy that this would count as
10:52 : uh we referenced it once and
10:55 : his strategy is if yeah yeah yeah
10:59 : we we've mistakenly counted this as a
11:02 : declaration when it's not
11:08 : yeah so i think we actually somehow have
11:11 : to distinguish
11:12 : and identifier that's within a
11:15 : declarator
11:16 : versus an identifier that's not in a
11:19 : declarator
11:21 : and that's actually make it more hairy
11:25 : but uh but it's doable
11:34 : yep yeah
11:37 : so yeah that's true um
11:41 : we could either do that or we could um
11:47 : let me think
11:55 : yeah we could either provide a way to
11:58 : find out
11:59 : what your parent is which is
12:02 : actually not too bad we can do that
12:05 : or another way would be we can
12:10 : we can provide a way for you to override
12:14 : the traversal behavior like you can say
12:19 : hey if i if i am at a
12:25 : or customize the traversal behavior
12:28 : um we could say hey if i if i'm a
12:31 : variable decorator
12:33 : i actually don't want to traverse
12:37 : the identifier it through the normal
12:40 : pathway i'm just gonna say
12:43 : okay i'm a variable declarator i'm
12:45 : declaring this thing
12:46 : don't even bother traversing the
12:48 : identifier and that way it doesn't get
12:51 : to be
12:51 : recognized as a variable reference
12:57 : um so either of those ways
13:00 : works um
13:04 : but yeah like it might be
13:07 : it might be easier to like because both
13:10 : of these steps both step one
13:12 : and step two require a full traversal of
13:15 : the tree
13:17 : and we want to customize
13:20 : the behavior slightly differently for
13:23 : for one case
13:24 : versus another and and yeah we could we
13:27 : could make a
13:27 : copy of this traversal function
13:31 : uh we can make a whole copy of this
13:34 : traversal function and then tweak it
13:36 : for case one and then tweak it for case
13:39 : two but
13:40 : you know there's a lot of duplication of
13:43 : code
13:43 : and it would be nice to use the same
13:46 : traversal
13:47 : logic for both cases um
13:52 : which way so which way do you think
13:56 : is better like we just allow the code to
13:59 : know about what your parent is
14:02 : so if if we allow that that would mean
14:05 : like
14:06 : when we're when the visitor is visiting
14:08 : this identifier
14:10 : in addition to the current node you're
14:13 : looking at
14:14 : we also pass in a parent parameter and
14:17 : then you can
14:18 : ask hey is my parent a variable
14:20 : declarator
14:22 : we can make that happen another
14:25 : thing we could do to solve this is
14:29 : we could potentially have the visitor
14:31 : function
14:32 : like in either case we're going to pass
14:35 : in a visitor function
14:36 : or visit function and maybe the visit
14:39 : function can say
14:41 : return false and if the visit function
14:44 : returns false
14:45 : then i disable the further traversal of
14:47 : the node
14:49 : i think either either way would solve
14:51 : this problem which way do you prefer
15:07 : got it okay let's let's go ahead and do
15:10 : that then
15:11 : um oh anyone want to pair with me
15:16 : last last time we did this like mop
15:18 : programming
15:19 : like a mop programming thing does anyone
15:21 : wanna
15:23 : like we couldn't switch back and forth
15:25 : who's driving
15:26 : that kind of thing does anyone want to
15:28 : control my
15:29 : keyboard
15:34 : cool maybe you could like switch like
15:37 : after 20 minutes or something like that
15:39 : but uh yeah
15:41 : go ahead can you um ask to control my
15:45 : screen or something i don't remember
15:48 : oh okay hello
15:52 : okay all right so
15:55 : so the first thing we want to do so i
15:58 : think a good
15:59 : step towards our goal is like currently
16:02 : what we're doing is
16:04 : when we run this traverse file
16:09 : when we run this traverse script it goes
16:12 : through
16:12 : and sort of prints out for each
16:15 : nude it prints out the type we also have
16:19 : a couple of extra print statements
16:22 : over here like here here and here
16:26 : i'm thinking let's disable these
16:31 : for now maybe for good
16:34 : um and what we're gonna do is
16:37 : we're gonna pass in a visit function and
16:40 : like
16:40 : have the the visit function can do this
16:42 : print out
16:43 : if it wants to how about that
16:47 : okay so so danny i'm gonna let you drive
16:50 : so basically we're gonna introduce a
16:52 : visit parameter
16:55 : to this traverse function
17:02 : oh can you type oh okay uh go ahead
17:06 : and just switch it to a normal keyboard
17:08 : okay
17:09 : so now that means we're gonna have to oh
17:12 : yeah we're gonna need to supply
17:14 : visit over here and we'll just steal
17:17 : line 22 and let that be the body
17:22 : of our past in
17:25 : visit function let's say
17:31 : line 22 and like over here over here
17:34 : like we're just gonna have the visit
17:36 : function print out the
17:38 : type oh actually
17:42 : actually bo both 21 and 22.
17:46 : let's do both 21 and 22 because line 21
17:50 : is the indentation so
17:53 : let's keep the indentation
18:00 : oh oh yeah the shortcuts are not in your
18:03 : favor
18:04 : all right
18:13 : line line 12. oh okay i see what you do
18:34 : oh does it oh you you can't paste can
18:37 : you
18:38 : just say just paste and i'll do it um
18:43 : okay i think we will also want to put in
18:45 : the level
18:46 : as a second parameter just
18:49 : just so that yeah that way we can
18:52 : calculate the indentation
18:54 : uh okay i think that works and then on
18:56 : line 12
18:57 : we will plug in visit node oh i'll let
19:01 : you do this first yeah let's call
19:08 : visit
19:10 : uh and then now the thing we always
19:13 : forget
19:14 : is uh for all the recursive calls we
19:17 : also have to pass in
19:19 : this visit function so like
19:22 : line 31 basically any
19:26 : call to traverse we want to double check
19:30 : that we're passing in
19:36 : visit
19:50 : oops
20:32 : that was a lot of code all right uh all
20:34 : right let's try this
20:36 : let's try this see if that worked
20:40 : uh looks good to me i mean it didn't
20:42 : break it didn't give me an error or
20:44 : anything
20:45 : and it still prints this nice
20:48 : coastline looking kind of thing
20:51 : um okay so i think oh one more thing
20:55 : is we said if the visit function returns
20:58 : false then we want to disable
21:02 : the the the recursive
21:06 : traversal under that node so let's make
21:09 : that happen
21:14 : first
21:23 : oh yeah put it inside the print
21:33 : yep okay so given this
21:37 : actually let's let let's run this first
21:41 : that worked let's change this to false
21:49 : oh what
21:52 : need a parenthesis here
21:56 : okay re retry this
22:00 : okay so if it's true it'll traverse if
22:03 : it's false
22:04 : it will not traverse maybe we try if
22:08 : note type is um
22:13 : if it's no type is not a variable
22:16 : declarator
22:22 : then we want to traverse maybe
22:25 : but otherwise we don't want to traverse
22:29 : does that work
22:36 : i think it worked yeah i think it worked
22:40 : for the variable declarators it kind of
22:43 : stopped
22:45 : yeah but before the expression statement
22:49 : uh for the expression
22:52 : the expression statements in this
22:54 : program they are console.logs
22:56 : so and that went all the way through
22:58 : yeah there's coexpression
23:01 : that's the member expression and then we
23:04 : have
23:04 : the literal here these these are the
23:07 : arguments
23:08 : to the console.log so i think that
23:11 : worked nice
23:12 : very cool so what this means now is we
23:15 : have this generic
23:17 : traversal function that we could use for
23:20 : both either case step one
23:23 : or step two um
23:27 : so let's do step one now
23:31 : maybe maybe make a function called
23:36 : fine for the
23:41 : colorations
23:43 : given an ast
23:46 : and then we'll just say like it's gonna
23:48 : return
23:50 : oh should we maybe pass in a dictionary
23:55 : well yeah find variable declaration is
23:58 : vars maybe and then if we find a
24:02 : variable declaration we will
24:04 : say like
24:07 : we'll just enter and enter
24:11 : a entry into this dictionary but we'll
24:13 : call it verdict or something
24:16 : is that cool okay
24:20 : all right i'll let you go ahead and
24:22 : drive if if
24:23 : if you know what you're doing i'll let
24:25 : you do it
24:27 : let me switch the keyboard back to
24:28 : something you know
24:35 : [Music]
24:44 : so
24:56 : i think we want variable declarator
24:59 : actually because if you look at
25:06 : the declaration yeah the declaration
25:10 : has an array of declarators uh
25:16 : wait a minute what just happened did i
25:18 : expand
25:22 : oh oh there we go um
25:26 : so decoration inside the declaration
25:29 : there's an array of
25:30 : declarators so i think you
25:33 : actually want to detect the declarator
25:36 : and then
25:41 : yeah yeah and then and then you get
25:43 : access to the identifier
25:44 : inside the declarator and put that into
25:47 : the dictionary
26:31 : wait no no we don't need
26:42 : oh
26:48 : it won't actually actually actually it
26:51 : won't unless we return
26:53 : false from our visitor function
26:56 : so what we could do is because
27:00 : we just write a custom visitor function
27:03 : just for the task in step one
27:11 : and like instead of writing this
27:14 : if statement logic directly inside here
27:18 : we might write a nested visit function
27:21 : here
27:26 : okay and and then
27:30 : let's change it back to my keyboard
27:32 : maybe okay
27:34 : uh node so
27:38 : we might want to actually put this logic
27:40 : in here
27:41 : and then at the end we just traverse
27:45 : the node with our special visit function
27:49 : and then this just this top level level
27:52 : is
27:52 : zero and if we did this
27:57 : then we don't have to worry about the
28:00 : traversal
28:01 : we can just return true or false
28:04 : depending on we want to
28:05 : we want the traverse function to do the
28:08 : further traversal or not
28:10 : and so that means we actually don't have
28:12 : to worry about
28:13 : the variable declaration because by
28:15 : default
28:16 : it should traverse down to the variable
28:19 : declarator level it should traverse
28:21 : as deep as we need it to by default
28:23 : unless we return false
28:42 : yeah a pepper i would probably just like
28:44 : no
28:45 : probably just like sorry
28:48 : i like we can just do this
28:56 : just uh verdict
29:00 : and that will make a verdict
29:03 : so we'll just oh just oh you you
29:07 : okay yeah it's just communication yeah
29:09 : you didn't see how i was gonna put it
29:11 : together
29:12 : yeah that's what i had in mind
29:19 : okay let me switch back to your keyboard
29:21 : okay
29:22 : so yeah
29:53 : you
30:17 : no no we can assume it must be an
30:20 : identifier
30:47 : okay oh yeah this way we can track how
30:50 : many times
30:51 : it's been referenced
30:54 : maybe we could maybe we can in like uh
30:58 : impose a linter rule that says you can
31:01 : only
31:01 : reference it for a maximum of 10 times
31:23 : [Music]
31:25 : oh make sure you return false for the
31:28 : variable declarator
31:29 : case and
31:32 : that way yeah
31:36 : that way it won't it won't traverse the
31:39 : identifiers
31:40 : inside the variable declarator
31:49 : uh no you actually want to return true
31:53 : for the default case but false only
31:56 : for the variable declarator because
32:00 : then wait wait it's
32:06 : well because there we could have
32:09 : variable declarations
32:12 : inside a variety of things like inside a
32:15 : function
32:16 : inside a for loop inside an if statement
32:25 : although we're kind of getting ahead of
32:27 : ourselves like for this
32:28 : very simple example it's probably not a
32:40 : problem
32:46 : do it wait no when no longer
32:51 : okay so now we're probably ready
32:54 : to test this so maybe after line 14
32:58 : we can print out the verdict
33:04 : and then we can test it
33:13 : okay i'll go ahead and run that again
33:19 : abc and answer that
33:22 : looks reasonable let me let me take a
33:25 : look at the um
33:30 : uh i can't control my cursor
33:33 : not sure why can you can you go to click
33:35 : on ex1.js
33:38 : okay yeah that looks right
33:41 : that looks right to me those are all the
33:43 : variables
33:44 : that were declared nice
33:49 : so um so
33:53 : all right does anyone else want a turn
33:56 : driving
33:58 : anyone
34:04 : okay
34:07 : no actually no because um the thing we
34:10 : want to count
34:12 : is the number of times it was referred
34:15 : to
34:16 : but but by declaring it we don't refer
34:19 : to it
34:21 : actually
34:25 : does that make sense
34:40 : well let's oh yeah he kind of improvised
34:42 : he deviated from our plan
34:45 : but that's okay
34:53 : we could either keep going with the plan
34:55 : or we could change the plan
34:56 : to uh
35:02 : i put put add an entry
35:05 : at an end
35:11 : i guess so okay then i'm going to change
35:14 : the plan
35:15 : the plan is now to find our book
35:18 : decorator and put an entry
35:21 : put put an
35:24 : entry interdictionary
35:28 : we're at the value
35:32 : with value equal to zero
35:35 : and then find all identifiers
35:41 : [Music]
35:42 : um
35:44 : uh outside
35:47 : of variable
35:50 : declarat
35:54 : actually this is there's something
35:56 : nuanced here
35:57 : that we'll need to pay attention to
35:59 : later
36:00 : so i'll put a star
36:04 : asterisk here and
36:07 : add 1 to the value
36:11 : in the corresponding
36:14 : entry in the dictionary does that sound
36:18 : good
36:21 : okay so so basically we're gonna make
36:24 : another
36:24 : function it's very similar to this one
36:31 : but we're going to call this find
36:33 : variable
36:34 : references
36:37 : and we're going to call it right
36:39 : afterwards
36:41 : like we're doing two separate passes
36:45 : on the tree two full traversals oh not
36:48 : full
36:49 : because sometimes we stop the traversal
36:51 : here by returning false
36:53 : but two traversals of the tree and i
36:56 : guess after the second traversal
36:58 : we should uh expect some of the values
37:02 : be non-zero uh the ones that did get
37:05 : referenced
37:07 : uh yeah so if we do it now
37:11 : that probably won't happen yeah we don't
37:14 : know we have
37:15 : zero values now now we're going to
37:17 : implement some code here to actually
37:19 : count
37:19 : up those values okay so how should we do
37:23 : it
37:23 : i'll let john drive here
37:44 : oh yeah let me let me show the ast
37:49 : here yeah so we're going to descend down
37:52 : this
37:52 : tree the function is going to
37:56 : the visitor function is going to visit
37:58 : every node
37:59 : that has a type field
38:03 : if we choose to return false
38:06 : then we will stop the algorithm from
38:08 : descending
38:09 : into its children um
38:16 : so in this case we're actually looking
38:18 : for
38:19 : identifiers we're looking for
38:22 : identifiers
38:23 : but we also want to make sure that it's
38:26 : an identifier that's not
38:27 : directly inside of a variable declarator
38:31 : because those don't count as references
38:35 : so so that's the trick so so i would say
38:37 : the first
38:38 : thing is to check if the node type is
38:41 : a identifier
38:45 : um
38:48 : uh actually we wanna we want
38:52 : one conditional to check if it's an
38:54 : identifier
38:55 : but a separate conditional to check
38:59 : whether it's a variable declarator i
39:01 : would think
39:02 : but we could do one at a time like we
39:04 : could
39:05 : we could do maybe just do identifier for
39:08 : now like to say
39:09 : if it's an identifier we're gonna grab
39:12 : its name and update the dictionary
39:15 : and that will be incorrect but we can
39:18 : still run it
39:19 : and see what it does and see how it's
39:26 : incorrect
39:31 : oh yeah so
39:35 : so that would be something like this uh
39:38 : this
39:38 : or let's go for this this a it's a
39:42 : reference
39:42 : so that would be statement number two oh
39:45 : no
39:46 : statement number three same one number
39:49 : three over here so you
39:50 : see this a this is an identifier but
39:54 : it's part of
39:55 : a binary expression
40:24 : oh you actually don't have to worry
40:26 : about that anymore because the
40:28 : traverse function has taken care of
40:31 : the traversal like we already did that
40:34 : the giant
40:35 : switch statement thing we took care
40:38 : we yeah it knows which path
40:42 : to go descend depending on what type of
40:45 : node it
40:46 : is and at this point like we're at a
40:49 : higher level now when we're writing the
40:51 : visit function
40:52 : and we no longer have the concern of how
40:54 : to descend down
40:55 : for which type of node so you just can
40:57 : assume
40:59 : you will get to visit all the nodes
41:03 : by default
41:44 : oh yeah oh
41:47 : no that's that i'll do it
41:51 : although this doesn't apply because we
41:54 : got the node would already be the
41:57 : identifier
41:58 : this this case so it would actually just
42:00 : be
42:01 : that name
42:08 : okay go ahead and so now with the name
42:12 : yeah we can we can access or
42:15 : change i should say
42:19 : the entry in the dictionary now
42:26 : would it just be plus plus
42:32 : yeah that is a possibility that it
42:35 : for example with the z variable which
42:39 : was never declared
42:41 : then it would not have been defined so i
42:44 : think what we would end up doing is
42:45 : putting a
42:46 : not a number in the dictionary if
42:56 : yeah key in is better because
43:00 : the way you've done it would think a
43:03 : zero
43:04 : value was falsely
43:10 : uh name name and
43:13 : if yep um
43:16 : i guess if it's not in the dictionary do
43:19 : we even care
43:20 : we don't care for for this case for this
43:23 : case we don't
43:24 : we would care if we were solving like
43:27 : find all the referenced but
43:30 : non-existent variables but but
43:33 : in our case we don't care uh okay i
43:36 : think we can run this and see what it
43:38 : does
43:41 : two one two two so
43:45 : that's more or less what we expected i
43:48 : think so
43:49 : two so for a like because this
43:52 : counts at the moment so
43:55 : we have one and two
43:59 : for a b only has one
44:02 : because it's of its declaration we're
44:04 : counting the declaration for now
44:07 : and then c has one and two and
44:10 : answer has one and two so that seems to
44:12 : be right
44:13 : so now we need to change it so that we
44:17 : don't
44:17 : count the declaration so how can we do
44:20 : that
44:36 : uh anyone anyone that would
44:52 : uh oh yeah
44:55 : let let me restate the problem um
44:59 : so we're at step number two here step
45:02 : number two
45:02 : is i want to find all of the identifiers
45:07 : that are outside of variable declarators
45:11 : uh and basically this is more technical
45:14 : speak but
45:15 : essentially i'm trying to find all
45:17 : variable references
45:19 : like of this form this kind i'm
45:21 : referencing c i'm referencing
45:22 : i'm trying to find all variable
45:24 : references
45:26 : but if i just only look for identifiers
45:28 : there's a problem because these are
45:30 : identifiers too
45:32 : and these are this is a declaration not
45:35 : a reference
45:35 : so i don't want to count this a because
45:38 : this a
45:39 : is inside of a declaration i don't want
45:41 : to count this one i want to count this
45:43 : one
45:43 : because that's actually a variable
45:45 : reference so i need
45:46 : somehow to distinguish between an a
45:49 : that's inside a variable declaration
45:53 : versus an a that is just a reference but
45:56 : not
45:56 : inside of a variable well this is still
46:00 : inside of a variable declaration note
46:03 : but it's just on the right hand side of
46:05 : the declaration
46:06 : so i still want to count this one but i
46:08 : don't want to count this one
46:11 : so that's also a neuron
46:22 : yes it's it's slightly it's even more
46:25 : tricky than i thought but but it's still
46:27 : doable so but
46:29 : but the upshot of it is we we should
46:31 : write another
46:32 : if statement here like maybe an elsif
46:35 : here
46:36 : i should write another if statement that
46:38 : says
46:39 : uh if it's a variable declarator
46:43 : then we need to do something special but
46:46 : how should we handle it
46:47 : so so one
46:50 : one thing is if you look at the ast like
46:53 : one
46:54 : thought would be um
46:57 : like if if we're in here for example if
47:01 : we're just let's just look at this
47:03 : simple one
47:04 : first before we worry about this more
47:06 : complex one
47:07 : this simple one it so that's statement
47:10 : number one
47:11 : over here it can say hey if i'm i
47:16 : if i'm a visiting a variable decorator
47:21 : then i do not want the traversal
47:24 : i do not want to visit the
47:27 : identifier note that's inside of myself
47:30 : anymore because i don't want to get that
47:33 : to get
47:33 : counted as a variable reference right so
47:36 : i can
47:37 : return false to disable that
47:40 : right
47:43 : so that's what
47:46 : we can do um so maybe
47:50 : that's all we do for now
47:54 : let's just do that so let's just say
47:56 : yeah else
47:57 : if the node type is a variable
48:00 : declarator
48:01 : then we're going to return false
48:16 : okay let's see what it does
48:21 : okay so we got
48:26 : zero zero
48:29 : zero and one
48:34 : well can anybody explain
48:38 : why we see this the one
48:41 : is actually correct so we got one of
48:43 : them
48:44 : like the answer we do have exactly one
48:48 : reference to the answer but what about
48:51 : the zeros
48:52 : like why is it that a is not
48:57 : getting counted
49:08 : yeah because it's on the right hand we
49:10 : just said don't traverse
49:12 : any child within this statement and so
49:15 : this the right hand side also doesn't
49:17 : get traversed and here
49:19 : the right hand side also doesn't get
49:20 : traversed and therefore this a doesn't
49:23 : get counted and this
49:24 : c doesn't count it the fact that b has
49:27 : zero is actually correct
49:29 : so that's it that's the only one that's
49:31 : correct but the a
49:32 : and c should have a non-zero value
49:36 : so any ideas on how to solve this
49:40 : basically this
49:41 : this returning false might have
49:45 : been too heavy-handed
49:56 : now so yes statement
49:59 : well no uh c yeah this is the third
50:02 : statement this is the one we want to
50:04 : look at
50:04 : that looks like this here uh
50:08 : so this one basically we want we don't
50:11 : want it to traverse the identifier
50:13 : but we still want it to want it to
50:16 : traverse
50:17 : the initial value that that's basically
50:20 : what it comes down to
50:21 : so this return falls
50:24 : um it says disable traversal for
50:28 : all of it but we want to disable
50:30 : traversal for this one
50:31 : but not disable it for this one is what
50:34 : we need
50:38 : um and can
50:41 : can you think of a way to allow that
51:04 : no it it's still not breath first um
51:09 : it's still not breath first breath first
51:12 : would
51:12 : be in order to do
51:16 : breath first it would be like hey i oh i
51:19 : see that i have a
51:20 : declaration that's fine but i'm not
51:24 : going to drill into children yet
51:26 : and oh i see i have another declaration
51:30 : but i'm not going to drill into children
51:31 : yet i go through everything
51:33 : without looking at the children and all
51:36 : the way until the end
51:38 : and then when i'm done with everything
51:40 : then now i go back
51:42 : okay let me look at the first child of
51:44 : that first thing
51:45 : that i did and then now let me look at
51:48 : the second child of the first thing that
51:49 : i did
51:50 : and and that's how breadth first search
51:53 : works and but this is not doing it that
51:55 : way
51:57 : this is still depth first it's just that
52:01 : we're sort of customizing at which
52:04 : points
52:05 : do we stop traversing stop going deeper
52:11 : that's what we're trying to do
52:30 : oops
52:41 : oh oh you're using like add another
52:45 : parameter
52:46 : it would be nice if we could add another
52:47 : parameter to the visit function
52:50 : and say um
52:53 : that would work but then we're gonna
52:56 : need to
52:57 : change the contract between
53:00 : our traverse library and and our
53:03 : and our consumer of the traverse library
53:06 : and
53:09 : and also the problem is for each
53:14 : situation we might need a different set
53:16 : of parameters
53:18 : maybe we like pass in a context
53:22 : object or something and then you can add
53:24 : anything you want to the context object
53:27 : that's possible but
53:33 : i don't know there's there's a way to do
53:36 : it
53:36 : right now without making a change to the
53:39 : traverse
53:40 : protocol can anyone think of
53:43 : a way
53:46 : okay yeah
53:56 : uh-huh
54:03 : yeah yes
54:09 : yes exactly so so this is like um
54:12 : it's very nuanced what you're doing here
54:14 : but basically it's like
54:17 : hey disable the traversal of my children
54:20 : but actually i wanna still traverse that
54:24 : child just not that one so in order to
54:27 : also traverse that child i'm gonna
54:30 : manually gonna initiate
54:32 : another traversal uh just just for that
54:35 : child
54:36 : so so basically what that looks like
54:39 : is um and it and i know
54:43 : it might seem strange that
54:47 : that we're calling the
54:50 : elder traverse function again on the
54:53 : inside
54:54 : but this is fine
55:04 : yeah and
55:07 : and i mean this does show that you know
55:10 : that this
55:10 : returning the false true false flag
55:14 : isn't all powerful like there's
55:16 : limitations what you can do
55:17 : by returning this flag um and and
55:21 : then this is not super pretty i would
55:24 : say like maybe
55:25 : in terms of api design this
55:28 : might not be necessarily the best way to
55:31 : design it but
55:32 : we can there there's a way for us to do
55:34 : what we need
55:35 : still uh using just this very simple api
55:39 : design
55:39 : so i think it still is still good
55:43 : um it is a slightly counter
55:46 : intuitive i would grant that but
55:49 : basically
55:50 : here yeah we're gonna we're gonna we're
55:53 : gonna tell the top level traversal
55:55 : function
55:56 : hey don't drill into these but i'm gonna
55:59 : start
56:00 : a new top level traversal function
56:03 : just to drill into this one and uh that
56:06 : should
56:07 : solve our problem i think yeah
56:10 : let's i think that's correct now so a
56:16 : a was referenced exactly once b
56:19 : was referenced zero times that's the
56:21 : unused variable
56:24 : c was referenced exactly once and the
56:27 : answer was referenced
56:29 : exactly once nice so now all we have to
56:31 : do is i'm going to write like
56:33 : four bar for let
56:37 : name in vardik
56:41 : um
56:47 : yes zero
56:50 : uptrend
56:57 : sr is unused
57:10 : b is unused nice nice i think we did it
57:14 : uh we could do let me have it i changed
57:19 : that c to b and now it's gonna
57:22 : oh i need to re-parse it
57:28 : replace it and now it says c c is the
57:30 : one that's unused
57:33 : yeah so this works very cool
57:37 : very cool we did it um
57:41 : as you can imagine uh well so to
57:44 : i would say like if you followed us this
57:47 : far
57:47 : uh you you basically got a peek
57:51 : inside of how a linter works and you can
57:55 : imagine doing this kind of thing
57:58 : for other things you might want to
58:00 : detect like
58:01 : i want to detect that i'm referencing a
58:04 : variable that doesn't even exist
58:06 : or maybe maybe they're expecting this to
58:08 : be a global variable like they're doing
58:10 : window that blah and then i did oh
58:15 : i you didn't declare window in here are
58:17 : you expecting it to be a global variable
58:19 : and yeah there are well-known global
58:21 : variables
58:22 : maybe in your lender you have a registry
58:26 : of the well-known global variables and
58:28 : you test them against that i don't know
58:30 : you can do a variety of things and do
58:33 : things like
58:33 : hey i want to detect
58:37 : you know you you probably shouldn't have
58:39 : assignments
58:42 : oh while you probably shouldn't have
58:45 : assignments
58:46 : inside a conditional um that that
58:50 : is either a mistake or it might be error
58:53 : prone maybe that's one of the lintel
58:54 : rules that you want to enforce
58:57 : you can write a little script like this
58:59 : to
59:00 : detect those things um
59:04 : and if you want to go
59:09 : next level you can even write code
59:12 : that can automatically fix up the source
59:15 : code
59:16 : if you want to see i haven't covered how
59:18 : to do it yeah and
59:20 : if if there is interest i can show you
59:22 : that as well but you could say hey
59:25 : this variable is unused i'm just going
59:28 : to automatically delete it from the
59:30 : source code
59:30 : or something or if there's there's like
59:32 : some form some
59:34 : syntax form that you can detect that you
59:37 : know how to fix the problem
59:39 : you can just write the code that
59:41 : automatically
59:43 : fixes the source code uh yeah we're at
59:47 : three o'clock now so this is a good time
59:49 : to stop
59:50 : unless anyone has any questions for me
59:53 : or comments