Live Code: Making a Programming Language from Scratch

In part 3 of this series, I am planning to make a run script to conveniently execute programs in the language, and implement a way to do conditionals, either an if statement or conditionals via higher-order functions.

Transcript

The following transcript was automatically generated by an algorithm.

00:04 : okay we're live hi hi everyone this is
00:07 : my name is Toby and this is the show
00:10 : where I make a programming language from
00:15 : scratch on stream if you are interested
00:19 : in following along the repo that
00:24 : contains the source code produced on
00:27 : this show is published the programming
00:31 : language we're making is called X
00:33 : programming language and that is because
00:36 : I kind of started this project off the
00:38 : cuff and I didn't even bother to come up
00:43 : with a good name for it um yeah so if
00:48 : you look at the commits let's see I'm
00:53 : I'm I'm I do currently this show on a
00:57 : weekly basis on Fridays and you can see
01:02 : that I made the first commits 14 days
01:05 : ago and then the second commit seven
01:08 : days ago and then now it's today
01:11 : so agenda for this session is first I'm
01:15 : gonna make a run script oh by the way
01:19 : the I've created this YouTube playlist
01:23 : which will list all the episodes so if
01:28 : you're interested in watching the
01:30 : progress of how I made this code that's
01:34 : where you go okay so today's agenda is
01:39 : to first make a run script to more
01:42 : conveniently execute the programs
01:45 : written in the X language because
01:48 : currently you actually have to do three
01:51 : separate steps manually first you have
01:54 : to parse it and then you have to
01:56 : generate the JavaScript code and then
01:59 : you have to execute the generated
02:01 : JavaScript code so first thing is we
02:05 : would make this run script to do those
02:06 : three steps anyone go on next
02:10 : I'm planning to implement a way to do
02:13 : conditionals so we can actually do
02:16 : conditionals either I'm not totally sure
02:19 : yet either I will make a if statement
02:22 : construct and implement everything in
02:26 : the pipeline so including the parser and
02:28 : the code generator or I might or I might
02:33 : use a higher order function way of doing
02:38 : if statements or if we have time I might
02:41 : just do both and compare them and
02:43 : possibly along the way we might do make
02:46 : some semi interesting programs that
02:49 : explore what range of abilities we have
02:53 : at this point so okay let's get started
02:59 : so I'm going to start off with making a
03:03 : run script I don't even have the project
03:08 : started and ready to go so let me do
03:10 : that okay come on Adam start for me okay
03:32 : there is slang computer Adam is slow in
03:41 : starting that's alright I'm also gonna
03:46 : check from time to time my streaming
03:49 : conditions currently it looks quite good
03:52 : okay so this is more or less where we
03:55 : left off so we have we have implemented
04:00 : function definitions so that we can even
04:04 : define custom functions at this point
04:06 : which look like this for some reason I
04:10 : decided to not have like a function
04:12 : keyword or def you're here allowed or
04:17 : Python I thought this would look cute I
04:20 : will let you decide whether that is
04:24 : true or not I'll make the font size a
04:25 : little bit bigger for you to see with
04:30 : okay so let's let's make a run script so
04:34 : currently in order to like write a
04:39 : program say this one and execute this
04:45 : program we have to go through a couple
04:48 : of steps three steps first first we have
04:54 : to parse this file so put the parse our
04:59 : new parts and no part plate eggs which
05:06 : will generate this ast file which if you
05:13 : remember it looks like this it's a Java
05:17 : Script data structure in JSON format it
05:20 : represents the syntax and in each note
05:24 : in this abstract syntax tree contains
05:26 : line number information as well which is
05:30 : super important to me because good error
05:33 : reporting always gives you line numbers
05:36 : okay but so parsing was the first that
05:39 : please ignore this warning you should
05:44 : probably try to figure out a way of
05:47 : muting this warning but I'm using the
05:51 : file system that promises API it is
05:54 : experimental but I like it I don't know
05:59 : how to mute this comment this warning
06:02 : maybe I will try to research into it at
06:04 : some point anyway
06:05 : step 1 parsing step 2 is we have to
06:11 : generate a JavaScript code that is the
06:15 : like a JavaScript version of this file
06:21 : supplying it the ast and now that it's
06:25 : created the JavaScript file we can use
06:28 : the node to execute this javascript file
06:30 : and that's how we run the program so
06:32 : there are three steps to the process we
06:35 : would like to make that easier
06:38 : by making it into one step there's
06:41 : actually a couple of ways of doing this
06:42 : and I have done both of them one is is
06:47 : the bash you smash - all right perhaps
06:53 : I'll just show you both of them and they
06:56 : have advantages and disadvantages so I
06:59 : could make a run script which in bash
07:07 : and if your window user Windows user
07:11 : you're probably thinking well that's not
07:13 : gonna work out for me and that it's the
07:16 : main disadvantage of using this approach
07:19 : is just like whatever you would have put
07:21 : in the come in your console you put in
07:25 : the script windows actually have a batch
07:28 : batch file which is similar to this but
07:31 : the syntax is slightly different because
07:33 : instead of using these guys you would
07:38 : actually want to pass in the think they
07:42 : look like this I don't want to pass in
07:45 : the parameter from the command line and
07:47 : in the way the syntax for doing that is
07:51 : gonna be different
07:52 : bash vs. windows I'm actually gonna do
07:56 : it in bash just to show you what that
07:59 : would look like but at the moment I kind
08:02 : of don't remember so I'm gonna say I'm
08:05 : gonna try to figure it out by
08:11 : experimentation so if I echo this
08:14 : parameter let's see what we got here and
08:19 : in bass I actually have to change a lot
08:23 : of Chaumont
08:25 : the run script so that this is saying
08:29 : add add to my user at the ability to run
08:34 : this file that's what that means so now
08:37 : I can run this and no that doesn't work
08:41 : so I got the syntax wrong I mean
08:43 : thinking maybe it's the % no o % science
08:49 : for Windows isn't
08:51 : I changed the wrong thing anyway
08:56 : no okay I totally blanked its dollar one
09:12 : $2.00 nine that works okay
09:20 : so dollar one is the is the name of the
09:25 : of the parameter the first parameter
09:28 : that's passed to my run script okay so I
09:31 : can use that but so I'm gonna do
09:34 : something like run late x-ray so dollar
09:38 : one would be played at X and then dollar
09:44 : one is the thing I want the the parse
09:48 : tree s to parse that's what I want to
09:50 : pass two parts to us and I guess for
09:56 : generate I'll actually do dollar one and
10:00 : then a ste that probably should work but
10:04 : I'm not 100% sure let me attempt this I
10:09 : might just
10:11 : yeah I'm using tempest oh that totally
10:16 : worked so it did the first two steps it
10:19 : wrote play xk st and it wrote play dr.
10:23 : yes and the last step is i need to
10:26 : execute but i can't hard took play dot
10:29 : to ask because it depends on like if i
10:32 : wanted to run a different file well let
10:37 : me do this this way if I ran into run a
10:40 : different file say function definition
10:46 : example definition eggs it's still
10:56 : executing play doctor yes but in fact I
10:59 : want to execute this file whatever was
11:02 : generated so the way to do that is
11:05 : and you need to replace the tick dollar
11:09 : one which was something something that X
11:13 : and in this case it was example function
11:18 : definition Alex you wanna swap the
11:22 : extension to KS and in bash there's a
11:27 : quite easy way to do that but I don't
11:30 : remember it I've done that numerous
11:32 : times but I still don't have it
11:34 : memorized as what happens when you're
11:39 : our developer I feel like so back by oh
11:46 : extension I can probably find a code
11:54 : snippet I can use on the Internet
11:57 : not too difficult Lee yes I believe this
12:01 : so I think you this syntax except you do
12:09 : it with one and i know im gonna replace
12:15 : X with J yes I believe that's what I
12:20 : want I'm gonna echo this out first to
12:23 : test this yes that totally work yeah
12:32 : that works so and I the reason I did it
12:36 : this fast it's because I've done this
12:39 : before but this line is completely
12:41 : confusing but what it's saying is hey do
12:45 : this little substitution thing except
12:48 : this is the variable dollar one or do it
12:52 : do it do this to the doubt the one the
12:56 : first parameter substituting this dot X
13:00 : for this or removing this might mean
13:03 : removing actually removing the dot X
13:06 : from the end of it and then we're gonna
13:08 : append dr. yes to it so I think this run
13:11 : script works let me try it
13:18 : run play X it does all the steps and
13:22 : execute it but I can alter the example
13:28 : definition X you run that one which gets
13:32 : the result for for that program so great
13:36 : we we have it but again because this
13:41 : this is a this is a pathway a way of
13:47 : this that works on Mac and Linux it
13:51 : won't work on Windows so I also want to
13:53 : show you just in case you are on Windows
13:58 : Oh call this run that Sh
14:02 : and perhaps track this into the source
14:04 : code as well so we have a record of dish
14:06 : but I also show you how to do the same
14:08 : thing using new GS so I'm gonna make a
14:12 : run back to yes again I like to use
14:22 : async await scripts these days it was
14:29 : not so okay so this is how I like to do
14:39 : it and in this case basically we want to
14:41 : do the same thing that run that Sh was
14:45 : doing it needs to execute these three
14:50 : statements and the way to do that is by
14:53 : using the the execute function from the
15:03 : child process okay however I don't want
15:09 : to use this one because this one is
15:11 : callback based and I use a promise if
15:14 : I'd version of this and actually the
15:17 : nucleus Docs if you go to the
15:23 : navier-stocks they tell you a pretty
15:28 : quick one liner you can use
15:31 : to promise if I this guy and I'm just
15:36 : gonna go find that and steal it because
15:42 : that's kind of what I did
15:44 : it's this line so I'm just gonna steal
15:47 : this line and now this exec is the
15:53 : promise defied version of child process
15:56 : that exact and apparently Oh actually
16:01 : need to require this util in order for
16:04 : this to work so nobody s has this util
16:09 : library that comes out of the box now
16:12 : which has this promise is Phi function
16:15 : in it traditionally or before they
16:18 : before they added this from finality
16:21 : promise if I was supplied by various
16:26 : various a node.js modules but now it's
16:29 : in the core now which means I didn't
16:33 : have to npm install anything okay so now
16:37 : I'm actually just gonna run these three
16:40 : statements so but we have to get the
16:45 : input file name first if you've been
16:52 : following along you know why we eat the
16:54 : index to here it's the third thing on
16:58 : the command line I'll print this out
17:01 : first just just to check that the code
17:06 : is good at this point okay so let's say
17:15 : I wanted to run plate the odd X then I
17:19 : have played Alex here checks if anyone
17:23 : is in here just in case nope there's no
17:28 : one watching okay
17:38 : all right so let's let's use the exact
17:46 : function to first part part J is this
17:57 : file we need to await it because it's
18:01 : the promise I promise five version o
18:05 : close parentheses here then the second
18:09 : one is note that generate and then the
18:12 : file name plus ast okay and the final
18:27 : one is we need to actually swap out the
18:30 : extension name which so I'm gonna make
18:44 : okay I'm gonna use the past utility pass
18:48 : I think it's just base name which kind
18:55 : of strips the extension if you tow it
18:57 : what the extension is expected to be in
19:00 : our case the extent is expected to be X
19:03 : and then after we strip off the bat X
19:07 : we're gonna append that they ask to it
19:10 : and that should be the JavaScript file
19:12 : name and then we're gonna exec no yes
19:20 : okay let's see if that works
19:25 : however this doesn't print out print out
19:32 : the output though oh wait no this is
19:40 : fine
19:47 : I'm just gonna run this show you what
19:50 : happens no run yes and then the name of
19:56 : the file
19:57 : let's do play Dex oh I misspelled
20:02 : something somewhere okay it said it but
20:08 : I didn't give you any output and the
20:09 : reason is that oh you'd return something
20:16 : the exact function we're trying
20:18 : something which will actually help the
20:20 : output I'm gonna turn out what it is I'm
20:26 : just gonna do the first one so that how
20:33 : it's doing so it actually returns you an
20:37 : object with a standard out property and
20:40 : the standard error property and you can
20:42 : actually yeah choose to print both of
20:48 : these guys helped so to do that I might
20:53 : just omit to print out the standard
20:55 : error and that's maybe the way I can get
20:58 : around having this warning although in
21:04 : the case that thing's actually errored
21:07 : out then I probably do want to print out
21:13 : the standard error so maybe I spoke too
21:16 : soon so I'm gonna print out standard out
21:21 : and then maybe come so that one the
21:26 : standard error and and well I'm gonna
21:34 : need to do this for each of these
21:36 : commands so perhaps I don't do that and
21:38 : instead roll it right a wrapper function
21:43 : that does it so I'll call this my exact
21:46 : command which would do this
21:55 : but with whatever command is passed in
21:59 : and then it will send out the output of
22:03 : the command here so if I had this and I
22:10 : can just await the result of my exact
22:15 : and then swap these calls to use my
22:22 : exact instead and yeah yeah that that
22:29 : looks kind of good I mean there's too
22:30 : many new lines here so and I don't like
22:33 : that and but we can fix that because the
22:36 : way to have console that look not given
22:39 : your line or the way deuk-gu go this is
22:45 : current without new line I just remember
22:51 : what the Google queries Google query
22:56 : needs to be is that process because I
23:02 : don't memorize this but I memorize what
23:05 : Google query I need to do to get this so
23:10 : I'm gonna do process that standard up
23:12 : standard out that right maybe this would
23:18 : be standard error here but I'll do it
23:20 : like this first okay and I have an idea
23:26 : actually I'm gonna say if I get this guy
23:31 : if the standard error justice that I'm
23:35 : gonna ignore it maybe that way I don't
23:40 : have to see this line because that line
23:42 : kind of bothers me
23:52 : so okay let's do it let's do this
23:55 : but I don't want to prevent other errors
23:58 : from coming out though so what I'm gonna
24:00 : do is split this standard error in two
24:02 : lines error line matches if it matches
24:42 : this I'm gonna continue keyword to skip
24:54 : to the next iteration skip the next line
24:56 : basically woohoo that makes me happy
25:03 : so that's cool
25:09 : maybe I will also like before I run this
25:15 : I'll also say maybe I should print out
25:18 : the command that is being executed as
25:22 : well there so they printed out all the
25:31 : commands that it actually executed which
25:33 : is no parse tree as play X and then the
25:37 : output that it gave and then to come in
25:40 : again and then a output it gave and then
25:42 : to come in and then the output that one
25:44 : gave I think some color coding would
25:46 : help so I'm gonna go ahead there's two
25:50 : new modules that do color coding for you
25:53 : one is called chalk and Beyond the other
25:56 : one is called color colors is the one
26:01 : that I traditionally use
26:05 : I've used it a long time without
26:08 : complaints by sea chalk being a more
26:15 : recent favorite in the node.js community
26:20 : because it's the one that's used by the
26:23 : npm command line tool and probably a lot
26:27 : of other things probably it then it
26:31 : looks like it's more actively maintained
26:35 : maybe it has 44 million downloads per
26:40 : week colors I've used this for a long
26:44 : time - and it has 16 million downloads I
26:48 : would recommend both of them and this
26:51 : one this time I think I'll try a chalk
26:54 : because but the way you use it is the
26:58 : way you would use it is exactly same as
27:00 : colors colors has this thing that some
27:02 : people don't like myself included which
27:05 : is that okay fine how accept your
27:08 : cookies which is that it asks you to do
27:11 : this or they're their preferred way of
27:14 : using it is to overwrite not overwrite
27:19 : but add properties to the string that
27:23 : prototype which a lot of people don't
27:25 : like myself include thing
27:28 : so they made a compromise and say well
27:32 : there's a safe way that doesn't extend
27:35 : string prototype and you can require a
27:37 : color / safe and this is how I normally
27:42 : use the colors module chalk evidently
27:46 : just that's their default way it helps
27:50 : to say truck dot the color name to
27:53 : convert the string to the color of
27:56 : string and in fact one of their selling
28:01 : points that is that it doesn't extend
28:03 : string dot prototype so I mean the usage
28:07 : is exactly the same so I'm gonna use
28:11 : trough this time
28:21 : okay there it is and I'm gonna require
28:25 : truck and then we're needed I'll do
28:33 : truck dot the color before writing it to
28:39 : the process that standard up so here
28:42 : I'll say chalk that color oh wait let me
28:50 : do it for the command first I think it
28:53 : come in it's gonna come in maybe like a
28:55 : blue but lighter blue so cyan I'll do
28:58 : chop that cyan for the command and throw
29:05 : the output I might be yellow but I think
29:13 : yellow is good and if it's a warning no
29:16 : warning should be yellow let me change
29:18 : that which would help put be done green
29:31 : let me see what it looks like okay maybe
29:46 : just don't or like I'm not sure about my
29:53 : choice here perhaps I might also
30:00 : potentially wanna color the final output
30:04 : a different color so maybe I actually
30:11 : feel like the command should maybe be a
30:13 : [Music]
30:14 : lighter color let me let me see what
30:18 : other all the different colors I can use
30:27 : all colors basic color support there's
30:30 : actually 255 colour support there's a
30:33 : true color support what that's crazy but
30:39 : the basic colors home I guess okay so
30:42 : these are the basic colors okay there's
30:47 : not a lot of choice in fact there isn't
30:52 : a lot of choice yeah I'm like actually
30:54 : maybe the warning is the either maybe
30:58 : the warning is red or magenta I'm gonna
31:00 : do magenta for the warning
31:02 : although I'm not seeing any warnings
31:03 : unless I comment this line out for the
31:07 : moment so that I can see the why I think
31:13 : I want yellow for the command and then
31:15 : maybe then for the output is actually
31:18 : green okay so the new line went away for
32:10 : some reason after I split this guy I
32:15 : don't really feel like fixing this
32:17 : problem right now so I'm gonna ignore it
32:21 : but in terms of the display of the other
32:25 : code actually want to color the number
32:28 : like the output of the final file
32:31 : different so maybe I'll just
32:35 : [Music]
32:38 : pass in the color to here and then if
32:53 : maybe D fall it to something that's what
33:10 : I feel like the maybe decompilation ones
33:15 : should be dimmer maybe so maybe I just
33:26 : output color I pass in both colors and
33:33 : maybe I just don't configure the warning
33:35 : color so the command color here's the
33:39 : command color and the output color is
33:42 : the output color and then in each case
33:46 : you pass them in so maybe the command
33:50 : color there's a dim version of these
33:55 : things isn't it isn't there gray I
34:01 : thought there was great I like a damn it
34:09 : there's modifiers
34:18 : maybe I just did the command honestly
34:22 : I'm just gonna dim the come here and not
34:25 : pass in the commune color and then
34:28 : output color probably just also it's
34:35 : only used when you pass in help and if
34:40 : you don't give an output color of
34:44 : default to
34:49 : stay close to them even though dim is in
34:54 : that color so that's maybe up style the
35:06 : output style okay so but if you do want
35:11 : the output color in this case probably
35:14 : do yeah yeah let's go with that
35:23 : everything is grayed out it's great
35:25 : except for the actual output of the
35:28 : program yeah let's go with that
35:30 : I guess it that way if we had a syntax
35:33 : error in this program we would get this
35:40 : yeah we have a syntax error interesting
35:50 : oh the error handling is what what's
35:58 : going on I mean actually see what
36:00 : happened yeah oh I have the ast
36:04 : undefined that's unexpected okay let me
36:14 : think do I want to fix this right no I
36:16 : think I do want to fix this right now
36:18 : so basically when there's a syntax error
36:22 : oh I didn't catch this syntax error or
36:29 : did I it did right the AST file for some
36:36 : reason
36:43 : oh it's not true that there isn't a
36:45 : syntax error but but it's that they're
36:47 : the cargo didn't have any results okay
36:51 : I'm gonna say another conditional to say
36:55 : her result is zero it didn't have any
37:02 : result in that case it wasn't a syntax
37:05 : error
37:05 : yeah because I'll explain because if we
37:14 : got another parenthesis we could have
37:17 : gotten a valid input so and the parser
37:21 : is is a streaming parser which means it
37:25 : you can call feed on it and as many
37:29 : number of times as you want but if you
37:31 : get to the end and it still doesn't
37:34 : haven't have a result then that's an
37:38 : error so you need to catch that I'm
37:42 : gonna go ahead and do it so then I would
37:44 : just say unexpected and it would be
38:07 : better if I could point to the last last
38:12 : character file maybe but I'm not gonna
38:15 : do that right now
38:16 : cuz that would require some thinking so
38:20 : some words thinking so I'm just gonna
38:23 : say unexpected in a file and then I'm
38:26 : going to that's it I think
38:38 : unexpected in a file okay maybe I do
38:42 : have to fix this and I'm probably cross
38:46 : this exit with a one
38:52 : to signal an error in the process okay
39:01 : so command failed so we actually have a
39:06 : C so now our run script it actually
39:10 : errors out in the first come in because
39:15 : because the first come in the power
39:18 : script decided to exit with an error
39:20 : okay and when it does that
39:24 : let's see the error I guess this is the
39:37 : error message not sure I think this is
39:43 : fine and maybe maybe just just me is
39:48 : gonna end up printing out the stack
39:50 : trace here I maybe I just printed out
39:55 : the message and I color that would be
40:02 : fine yeah okay so we did note parse JSON
40:10 : then I'll get this read error message
40:15 : that says this command failed well I
40:19 : guess I don't care to see this this
40:22 : warning but the arrow was unexpected in
40:25 : a file this error message could be
40:27 : better but I'll fix that later
40:29 : so now I just wanna leave that beep okay
40:37 : so now I fix this program run it again
40:40 : then it'll do this the expected thing
40:44 : yeah let's let that be it and then now
40:48 : we can do some little semi interesting
40:53 : programs where would be a
40:56 : semi-interesting program
41:02 : I can add one two three four five six
41:09 : seven eight nine yep that worked okay
41:43 : don't have like a semi interesting
41:45 : program oh we can't even calculate tips
41:48 : because we don't have a decimal point
41:51 : but but yeah so we have this run script
41:55 : now we've done that let's let's go move
41:57 : on to our step two which is to
42:04 : okay so there's two ways to do it one
42:08 : way is to implement a if statement
42:11 : construct which in my initial sketch
42:17 : sketch out an if statement should look
42:20 : something like this but I have another
42:23 : idea which is maybe you could use lambda
42:26 : functions and do do a functional way
42:32 : before we implement this if statement
42:34 : maybe we could like go full functional
42:38 : programming style and have a function
42:42 : that takes in the the conditional in the
42:49 : subsequent block as well as the
42:52 : alternate block so it might look
42:54 : something like this so if it is a
42:59 : function call and in greater than four
43:04 : and then we have a lambda function which
43:07 : maybe it looks like an arrow function in
43:10 : JavaScript and then you can say
43:24 : maybe something like this maybe looks
43:27 : like a block oh that's gonna be weird
43:37 : but then this is very lispy now maybe
43:43 : our lambda functions look like flocks in
43:45 : small talk and then a like parameters
43:50 : are passing like this oh and because we
43:53 : don't even have commas
43:56 : it actually looks more like this
43:59 : interesting idea I don't know what I
44:04 : think about this we will need multi-line
44:08 : support like our argument list we'll
44:13 : need to be able to span across multiple
44:15 : lines there's something about this that
44:31 : I do like would it be possible to write
44:42 : it just like this and do it like a lazy
44:44 : evaluation
45:00 : and or we could do maybe like this like
45:09 : this is an anonymous function this looks
45:15 : weird
45:16 : this looks so very weird for maybe if
45:19 : you if you have just have an empty
45:22 : argument list then that is enough that's
45:32 : kind of crazy actually so that you can
45:40 : this is what like that before
45:44 : is this good that looks like a kind of a
45:47 : weird if statement I have to say it
45:50 : looks like a very weird if statement I
45:58 : haven't decided what I want the callback
46:02 : functions like the in line in line
46:04 : function so look like I decided that the
46:08 : function definitions were going to look
46:09 : like this but like if you had an array
46:13 : and you wanted to map over an array like
46:19 : would it look like this maybe that
46:31 : starts causing problems because my my
46:36 : parser is gonna have to understand that
46:40 : a in line or a value an expression can
46:47 : also be a function that looks like this
46:50 : and this guy can have spaces in them
46:53 : whereas we're using spaces to separate
46:56 : the values right now it visually it
47:03 : looks a little jarring that only a space
47:08 : is separating this indistinct which can
47:11 : also have a space
47:14 : so so that's interesting maybe that's
47:27 : why people have commas ago
47:33 : what does Haskell's arrow function look
47:36 : like
47:37 : Oh house closer it's got parentheses
47:41 : around it everything dude Oh God
47:46 : maybe there's a parenthesis around this
47:51 : thing
47:59 : maybe it's the bracket maybe there's a
48:07 : parenthesis inside the brackets that
48:15 : looked weird item I'm I am we're gonna
48:20 : square this guy so perfectly the the
48:25 : Smalltalk syntax where a block looks
48:29 : like that dad looks weird that
48:36 : definitely looks weird
48:52 : we're gonna do it like Ruby where you do
48:58 : even though it's a parameter of this
49:00 : method you don't put it inside the
49:03 : traveler list it's sitting on the
49:06 : outside but it's implied that it's a
49:08 : parameter of the map function we could
49:11 : do it like this this looks less weird
49:16 : but it definitely is much less see like
49:24 : at this point hmm maybe we're gonna call
49:28 : this a block and the block is like a
49:32 : function but if we use this step then
49:35 : that that means a function cannot take
49:39 : two blocks or maybe a tenth maybe it's
49:42 : just you have one block after another so
49:48 : if we if the if statement was that way
49:51 : oh my god then it actually looks like
50:01 : this and it almost looks like a if
50:06 : statement except that it looks super
50:10 : super weird oh my god this is so weird
50:27 : that I almost but so funny that I almost
50:31 : want to do this so let's do this this is
50:36 : funny okay we're gonna say these guys
50:39 : are blocks wait a minute that that's
50:45 : gonna have conflict with the array
50:47 : syntax isn't it because the array syntax
50:50 : also starts and ends with square
50:53 : brackets and then each thing inside the
50:56 : array is it's a value and it was in
51:02 : blocks each like how do you can tell
51:06 : this is a block and not an array because
51:16 : the print function could wear well
51:19 : return a value which we are putting in
51:21 : this array so this could totally be a
51:24 : valid array syntax here so how does it
51:30 : know this is not an array but it's a
51:34 : block unless we change the array syntax
51:36 : or change the block syntax slightly
51:40 : maybe you do it like this like just
51:44 : change the syntax of the block slightly
51:49 : hmm will change the syntax of the array
51:54 : is another thing that I could do we've
51:59 : overloaded this square bracket character
52:02 : in many different situations
52:22 : I use parentheses for these we might you
52:29 : need to use parentheses for other things
52:35 : like what what if this is like three
52:39 : plus three and now you need to use the
52:40 : parentheses to group it how does it know
52:45 : that this is a used as a grouping
52:48 : mechanism versus an array because
52:50 : there's a plus here and then that
52:54 : changes to an array that can be very
52:57 : confusing for a developer I think and
53:00 : also except for the case of Lists
53:03 : parentheses are supposed to be used as
53:05 : groupings traditionally or for function
53:11 : calls so maybe I just even though this
53:19 : is very very interesting maybe I should
53:22 : just punt on this idea all right you
53:30 : could like do a syntax like this or I
53:43 : could say I could say that maybe there's
53:55 : another symbol used before the block
54:03 : that signals that it is a block like
54:06 : that
54:23 : like if this was a flock then
54:26 : instantiate a block would look like this
54:34 : we still have an issue of how does the
54:37 : parser know that this is a block and not
54:41 : an array or a dictionary for that matter
54:48 : maybe you could use the fact that print
54:51 : the print function is not meant to be an
54:55 : expression is meant to be a statement
54:58 : but you have to know that the parser or
55:02 : the compiler will need to have knowledge
55:05 : about what function is a statement
55:07 : versus with function is an expression
55:10 : and I kind of don't like that although
55:22 : that's that is possible we can say the
55:25 : parser knows that the parser would need
55:30 : the symbol table actually we could have
55:39 : functions that return a value and
55:42 : functions that don't return a value and
55:46 : I feel like in most languages that is
55:50 : very subtle that's a very subtle
55:53 : difference and kind of vague and using
55:56 : that semantics to guide the parser it
56:02 : feels kind of vague and and kind of
56:10 : ambiguous so I don't like it because of
56:13 : that
56:48 : I'm going to research a small talk box
56:54 : I'm gonna see how they did it okay so
57:04 : block let's see
57:16 : okay black and small coke looks like
57:19 : this
57:20 : they named the tram dish and then
57:24 : there's a vertical bar and then a dish
57:31 : is a list of the local variables within
57:34 : the block interesting so in small talk
57:39 : it looks like it's actually not two bars
57:54 : but one bar but what if you don't have a
58:17 : function oh this is where they're using
58:29 : this like their conditional is done via
58:32 : blocks so it's true then
58:37 : a block value I think you can my
58:45 : question is can you have blocks that
58:47 : don't have any progress so what do they
58:50 : look like probably the answer is just
58:54 : doesn't have that part because this
58:56 : looks like a book block and it doesn't
58:58 : have this vertical pipe here yeah
59:06 : but in small talk they don't use they
59:11 : don't use this grab bracket for the
59:14 : array notation though so they don't have
59:16 : that ambiguity problem and I could I
59:20 : suppose change that notation
59:31 : to the dreaded curly bracket but I did
59:35 : say I hate curly brackets in a previous
59:38 : episode yes yes I did say that I don't
59:50 : like curly brackets but then again if I
1:00:04 : change both arrays and dictionaries to
1:00:07 : be start and end with curly brackets
1:00:10 : then that frees up this to be used for
1:00:14 : blocks and then I can do blocks like
1:00:16 : this and I can do if statement can
1:00:19 : actually be a function that happens to
1:00:22 : take in two blocks and a block the
1:00:29 : callback function for map is a block
1:00:32 : that looks like this and that's kind of
1:00:41 : interesting and also in small talk they
1:00:50 : have implicit returns so whatever is the
1:00:52 : last statement of the function the the
1:00:59 : function returns the the value of that
1:01:04 : statement evaluates to instead of having
1:01:07 : every kind of statements I don't have my
1:01:09 : return statements where you just write
1:01:11 : this this is an interesting idea indeed
1:01:23 : I'm not sure if it's a good idea but
1:01:28 : let's do it just to it I'm gonna go back
1:01:31 : on my very strongly stated goal of not
1:01:36 : using curly brackets or
1:01:39 : I guess but uh yeah let's try what why
1:01:44 : the heck not so I'm gonna implement
1:01:48 : these guys as blocks okay so a block and
1:01:54 : that could be a value that's what I'm
1:01:56 : gonna say and within that block you can
1:01:59 : and actually a the body of a function is
1:02:03 : a block so I can even reuse that parser
1:02:07 : logic okay let me go ahead and do that
1:02:12 : so a phone call a function definition
1:02:16 : has this chunk which is actually a code
1:02:20 : block okay so I'm gonna extract that out
1:02:23 : and I'll call this a balk is that
1:02:30 : there's an Open bracket some optional
1:02:35 : space a new line a bunch of statements
1:02:39 : another new line and then as optional
1:02:41 : space and then a closing bracket it will
1:02:49 : just did the thing that will represent
1:02:53 : it it could simply be a set of
1:02:58 : statements or alternatively I could have
1:03:00 : in ast node that has type of so I'm
1:03:08 : gonna say receive these data from these
1:03:14 : matches block oh this is totally gonna
1:03:27 : break stuff I should probably have
1:03:32 : written tests but happen so and maybe in
1:03:37 : a future episode we'll write some tests
1:03:39 : so this is 0 1 2 3 so I had index 3
1:03:44 : that's where the statements are
1:03:49 : so that's our code block and I'm gonna
1:03:52 : extract that out of here and instead of
1:03:57 : doing that I'm gonna say just swap in
1:04:01 : the code block construct that was just
1:04:05 : extract it and the body is now at index
1:04:10 : what's the index of this now zero one
1:04:14 : two three four five six and yep
1:04:28 : so after changing the parser we have to
1:04:31 : actually change the structure of our AST
1:04:34 : tree
1:04:34 : let me regenerate the parser so I think
1:04:37 : you should break now so if I run latex
1:04:45 : it may not work no it worked because now
1:05:09 : stuff will brick because body dot map is
1:05:13 : not a function it used to expect body to
1:05:16 : be in array in a generator but that's
1:05:20 : not so that's mine 83 the generator so
1:05:31 : now body is actually an ast node of type
1:05:36 : code block which in turn has a statement
1:05:39 : property which is an array so this
1:05:42 : should fix it and it did so that was
1:05:45 : good I colored this stuff Oh
1:06:00 : no it has the colors perfect
1:06:04 : okay so I've extracted this code block
1:06:09 : construct but now I actually want to say
1:06:15 : missing this um
1:06:20 : one way I could do this is say that but
1:06:24 : I think we said we didn't like this
1:06:26 : syntax yeah I think I said I don't like
1:06:30 : this syntax I don't know maybe I can
1:06:34 : like this syntax what do you think this
1:06:39 : is easier to achieve this syntax is
1:06:42 : easier to achieve and maybe
1:06:44 : syntactically speaking more consistent
1:06:52 : with the rest of the language if we are
1:06:57 : okay but this syntax this might be
1:07:02 : easier to implement actually I'm gonna
1:07:05 : put a space between the two blocks here
1:07:09 : yeah so I think yeah let's just go
1:07:12 : through this syntax and and then see if
1:07:15 : we can make this work first and after we
1:07:18 : get that working we'll see how happy we
1:07:20 : are about it and if we like the syntax
1:07:24 : actually wait maybe we don't even bother
1:07:27 : implementing the actual if statement but
1:07:30 : we'll see I'll see about it maybe a go
1:07:33 : maybe this this is how we write if
1:07:35 : statements down we can do anything we
1:07:38 : want this is our programming language
1:07:39 : right that's the beauty of creating your
1:07:43 : own things imagining their own world
1:07:46 : okay I think the way I can do that is to
1:07:51 : say you know what a code block is is a
1:07:54 : is an expression an expression can be an
1:08:00 : identifier literal a function call an
1:08:03 : expression can also be a code block okay
1:08:08 : and if that were the case
1:08:15 : and we parse the file so we can let me
1:08:20 : try statement example when I say it's
1:08:29 : too big else I'm gonna say okay maybe
1:08:41 : that's how you do if statement and if
1:08:44 : statement is not an actual if statement
1:08:46 : that if statement is a function if it's
1:08:49 : a function that takes an Deacon devalue
1:08:53 : of the conditioner which is a boolean
1:08:55 : and two blocks block number one and
1:08:58 : block number two and I think that
1:09:01 : probably should work I think I need to
1:09:05 : update the code generator to convert the
1:09:08 : code code block into a JavaScript
1:09:11 : function but that's the only thing I
1:09:14 : really need to make this work so okay
1:09:27 : with a problem oh we don't have greater
1:09:30 : than operator implement oh yeah doing
1:09:45 : that will just say greater than as a
1:09:48 : function yes that's what we do it's a
1:09:58 : very functional language let's say that
1:10:00 : oh I don't even need commas of course
1:10:04 : I'm very less lispy it's getting
1:10:08 : increasingly lispy okay so now it
1:10:12 : doesn't know about this left bracket
1:10:13 : token so that's when we gonna this is
1:10:16 : the parser though actually
1:10:21 : wait this is the parser I would have
1:10:27 : thought oh wait does my tokenizer know
1:10:34 : about if that gif keyword no it doesn't
1:10:42 : so that's not the reason I trying to
1:10:55 : Express have I updated the part
1:10:59 : generator the parser oh yeah it got past
1:11:05 : that okay all right now it's just a
1:11:08 : generator code so let me
1:11:11 : the generator code needs to be updated
1:11:13 : to generate convert code blocks
1:11:18 : JavaScript functions that's actually not
1:11:22 : that hard because we have this ast here
1:11:27 : which it says we have a function call to
1:11:33 : the if function and its parameters are
1:11:36 : first parameter is a function call to do
1:11:42 : to the great GT content which is for
1:11:46 : greater than the second parameter is a
1:11:48 : code block which contains these
1:11:52 : statements there's a actually it's
1:11:57 : actually one code block with two
1:12:00 : statements oh not what I expected I was
1:12:08 : expecting two separate code blocks
1:12:11 : oh no no no there are two separate code
1:12:16 : blocks it's just that this this
1:12:18 : statement is kind of big because it the
1:12:21 : statement is a function called print
1:12:22 : that prints the string and then we have
1:12:25 : a separate code block yeah this is
1:12:27 : perfect so this is what I'm gonna do I'm
1:12:30 : gonna go to the generator
1:12:34 : make a this generator is very small
1:12:42 : right now I'm kind of surprised by how
1:12:45 : small it is
1:12:45 : so I'll just handle the case where the
1:12:49 : the note pass to generate is a code
1:12:53 : block and I'll say well if it's a code
1:13:06 : block unless it's a code block within a
1:13:10 : function definition which I think will
1:13:13 : be fine because oh definition is a
1:13:16 : special case where we don't go this
1:13:19 : logic here to generate a code block but
1:13:22 : if we it was a coup block that was just
1:13:24 : dangling it's not part of the function
1:13:28 : definition then we're gonna return the
1:13:31 : JavaScript function but we'll kind of do
1:13:35 : the same kind of deal that this code is
1:13:38 : doing let you see this here all right so
1:13:46 : this one that map's each statement
1:13:50 : through the generating function again
1:13:51 : I'm just gonna steal this one I don't
1:13:54 : need parameters wait do any parameters
1:13:58 : know oh my code block may need to be
1:14:03 : extended to accept parameters actually
1:14:06 : so I do need to think about that one and
1:14:10 : we might actually change how function
1:14:14 : definitions are written maybe
1:14:17 : okay I'll talk about that later okay so
1:14:20 : I'm gonna indent this body I'll just
1:14:26 : steal that code for now even though it's
1:14:29 : a lot of code duplication I'll fix that
1:14:32 : later once I get it working
1:14:34 : I don't need a name and for now with the
1:14:41 : code blocks don't support parameters yet
1:14:44 : but
1:14:46 : we'll have the indented okay let's see
1:14:58 : if that works
1:15:05 : 987 cannot read property statement of
1:15:09 : undefined so no no it's just note that
1:15:14 : statements cuz note is already the block
1:15:18 : so my copying no maybe it's okay okay GG
1:15:24 : is not defined yeah totally true
1:15:30 : reference error oh it's running the file
1:15:37 : that so that's easy at this point
1:15:39 : because it's already generated the
1:15:41 : JavaScript file in that step the only
1:15:45 : problem is that it's this one we already
1:15:52 : generated this file which looks like
1:15:53 : this which is actually well we can't use
1:15:58 : the if the if as a function call in
1:16:02 : JavaScript can we so I think we'll need
1:16:04 : to call this like dollar F maybe yeah so
1:16:10 : that a special case that the generator
1:16:12 : has to handle but another problem is
1:16:15 : that the GG function is not present
1:16:19 : so I'll just also fix that so say I make
1:16:24 : a function that is called GT actually
1:16:49 : executes but it probably didn't do what
1:16:51 : we wanted it to do it's this
1:16:55 : where is the it's is this javascript
1:16:59 : file so it generated this code but this
1:17:01 : is not what we want because if in
1:17:03 : JavaScript actually is a keyword so what
1:17:06 : it actually did was say if that
1:17:10 : expression and when you in JavaScript if
1:17:13 : you have used a comma to separate
1:17:15 : separate expressions it executes all of
1:17:19 : them but the final value of the whole
1:17:21 : thing is whatever is the value of the
1:17:26 : last expression so in this case it's
1:17:28 : sort of equivalent to if that and that's
1:17:31 : a function is a function truthy in
1:17:33 : JavaScript yes but then if doesn't have
1:17:36 : a box so it has no consequence anyway so
1:17:38 : what I need to do is in the generator
1:17:43 : say that if the function we're calling
1:17:46 : is the if function actually wanna
1:17:51 : generate the call to the dollar if
1:17:54 : function and I supply a I supply a
1:17:58 : dollar if function in my run time okay
1:18:02 : so let's try that
1:18:04 : so perhaps so first let me supply a
1:18:08 : dollar if function and it has the
1:18:14 : conditional and then the consequent
1:18:18 : which is a function and the alternate
1:18:21 : which is also function and then it'll
1:18:24 : basically say if condition is true then
1:18:29 : call the consequent else call the
1:18:34 : alternate okay that's our dollar if
1:18:38 : something and now we want to generate it
1:18:42 : sort of map if it's a function call
1:18:45 : there and if the function name happens
1:18:49 : to be if this is the function name maybe
1:19:01 : this is like source function name and
1:19:09 : is is if if that's the case then we
1:19:22 : actually want to use dollar yet but
1:19:24 : otherwise just use the source function
1:19:27 : okay
1:19:28 : I think that should do it yeah it's fine
1:19:34 : so we had this this is how we're gonna
1:19:37 : write if statements okay and ten is too
1:19:44 : big but four four is fine so now we have
1:19:49 : if statements sort of we're done with
1:19:54 : this so now we're just gonna play around
1:19:56 : just a little bit playing around a
1:20:02 : little bit see what programs we can make
1:20:05 : we're gonna okay what can we do to find
1:20:19 : this function main week yeah let's keep
1:20:24 : this I guess if X if X is greater than Y
1:20:31 : then this is hilarious
1:20:41 : then some is bigger be differ some
1:21:16 : oops I put the can't kick the habit of
1:21:22 : writing a comma sum is less than product
1:21:25 : that's totally true that's totally true
1:21:31 : if I remove reduce the size of this
1:21:35 : product then sum is bigger than the
1:21:39 : product now awesome so so this works we
1:21:44 : have sort of a way to to have if
1:21:47 : statements without actually having if
1:21:50 : statements in the language we if if is
1:21:53 : just a function yeah if if it's just a
1:22:00 : function in this language at least at
1:22:04 : the moment and that's kind of an
1:22:07 : interesting result I'm just gonna try
1:22:13 : this and see what happens and if I don't
1:22:16 : like it later I might change it but for
1:22:19 : now this is what it is okay I think
1:22:21 : that's good for this session today
1:22:26 : basically accomplished my objectives
1:22:29 : which was first to make a run script and
1:22:32 : then to create a way of having
1:22:36 : conditionals whether it's an if
1:22:39 : statement outright or it is this
1:22:42 : functional way out doing if statements
1:22:44 : and yeah I'm gonna end the stream here
1:22:49 : thank you for watching if you are
1:22:59 : you

Live Code: Making a Programming Language from Scratch - Part 3

Transcript