Parsers and ASTs

I give an explanation of parsers and AST to prep you for the next episode of where we show you how to build a parser for a programming language.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : hello there welcome to another drive-by
00:02 : code session so this is gonna be episode
00:07 : two in the making a programming language
00:10 : series so last time we talked about the
00:16 : different programming language
00:17 : architectures there are there's the
00:19 : compiled programming languages which
00:23 : look like this interpreted programming
00:27 : languages transpiled programming
00:30 : languages and bytecode interpreted
00:33 : programming languages I'm not going to
00:36 : rehash the details if you would like to
00:39 : know about the details please watch
00:42 : episode number one in this series this
00:47 : time I'm going to go more in-depth into
00:51 : the program that you're making whether
00:55 : you were making a compiler you are
00:59 : making an interpreted programming
01:00 : language or a transparent languages or
01:04 : Transpower programming language I should
01:05 : say now I'm actually going to do it for
01:13 : this episode and the next I'm going to
01:16 : do a kind of overview of the whole
01:21 : end-to-end process of building a
01:24 : programming language it'll be like a
01:27 : very very basic bare-bones programming
01:31 : language to sort of get the overview of
01:35 : the end to end big picture out of the
01:38 : way and then after that I will go into
01:41 : more detail for for each of the
01:45 : components so for now we're gonna in
01:51 : this episode we're going to talk about
01:52 : the parser so whether you're making a
01:56 : whatever a programming language
01:58 : architecture that you're gonna be using
02:01 : you will end up building a parser and
02:05 : your parser will want to emit a
02:09 : something called the ast which
02:12 : for abstract syntax tree abstract syntax
02:25 : tree and what is an abstract syntax tree
02:30 : I will show you that in a moment but
02:32 : basically it is a true representation of
02:35 : what the code from the input is once you
02:41 : have the code in AST form next step is
02:45 : to put it through another program that
02:48 : you're gonna write and that program will
02:51 : be either a compiler or an interpreter
02:55 : or transpiler
02:57 : or what have you now to give you an idea
03:05 : of what the what an ast is and and like
03:10 : what the parser does I'm gonna use the
03:14 : example of JavaScript okay so this is
03:19 : the esprima
03:21 : JavaScript parser which is available for
03:25 : free on npm for you to download and play
03:28 : with but if you just want to play with
03:29 : it they have created the author of
03:32 : esprima has created this demo web page
03:36 : where you can write any JavaScript you
03:39 : want on the left hand side and get an
03:42 : ast a representation of the code on the
03:46 : right hand side so this line of
03:49 : JavaScript here
03:50 : well let's ignore the comments because
03:53 : the comments are ignored anyway so this
03:56 : line of code is represented by this ast
04:01 : here and well this this ast may seem
04:08 : kind of complicated it is actually not
04:12 : if you dig into it in detail it's just
04:15 : very verbose so what is the same well
04:20 : first of all this javascript object or
04:24 : JSON object
04:25 : is a representation of this line of code
04:29 : so what is this line of code well
04:31 : actually this is a program right this is
04:36 : this code here it actually could have
04:39 : had multiple lines in it but this code
04:44 : here is a program and a program has a
04:47 : body which consists of a number of
04:50 : statements this one only has one
04:52 : statement so the body is represented as
04:55 : an array and in there is one JavaScript
05:01 : object so that object represents this
05:04 : statement here and then what kind of
05:08 : statement is it well it's a variable
05:10 : declaration statement that makes sense
05:12 : this is a variable declaration and then
05:15 : inside a variable declaration statement
05:18 : there are multiple declaration
05:21 : declarations why is that well as you
05:23 : know in JavaScript you can have multiple
05:26 : variable declarations within one VAR
05:30 : keyword in this one though there is only
05:34 : one so there's only one declaration or
05:37 : the this is called a variable
05:40 : declaration in a variable declaration
05:42 : there's only one decorator inside this
05:45 : variable declaration and that
05:49 : declaration has an ID of answer it's ID
05:53 : is an identifier and the name of the
05:55 : identifier is an answer that's the name
05:58 : of the variable and then in it
06:00 : represents the initial value of this
06:02 : variable and the initial value is the
06:05 : result of a binary expression six times
06:09 : seven and that is represented as
06:12 : something that has a type of a binary
06:14 : expression an operator a left-hand side
06:18 : and our right-hand side so this this
06:21 : javascript object here this sort of
06:24 : composite javascript object is a
06:28 : representation of this line of code and
06:31 : this is what an AST is it's is sort of a
06:35 : tree as a tree as code on abstract
06:37 : syntax tree
06:38 : and it is a true representation entry
06:44 : structure of code another way to
06:51 : visualize this is so you have this line
06:55 : of code same line of code the leaves of
06:58 : the the code or you can look at this
07:01 : code as a series of tokens the token six
07:05 : is a literal the multiplication the star
07:10 : is an operator and then 7 is another
07:12 : literal we can group those three things
07:16 : into a binary expression and then for
07:19 : the variable declaration we can it is
07:22 : also a grouping it's a grouping between
07:25 : a var keyword and identifier an
07:27 : assignment up and an expression of some
07:29 : sort for the initial expression this
07:32 : initial expression just happens to be a
07:34 : binary expression so so next we're gonna
07:40 : get into the programming language that
07:43 : white kit made so I went to Y kit and I
07:48 : said hey white kit I need to make a very
07:52 : very small programming language that I
07:54 : can demo in a short amount of time n2n
08:01 : Jim as I said to him give me just enough
08:05 : to implement the Fibonacci sequence so
08:08 : this language is called my Pio and it
08:12 : can add subtract multiply and divide it
08:18 : has variables and it has a while loop
08:20 : and it can print results and that's
08:23 : basically all that this programming
08:25 : language can do and it is a transpired
08:28 : programming language and it will
08:30 : transpire to JavaScript for simplicity
08:33 : sake and this code example here contains
08:37 : all of the features of this programming
08:40 : language in the next episode we will
08:43 : start actually coding this programming
08:46 : language so I hope to see you next time