LLVM Tutorial #1: Introduction

I am learning LLVM. In this series I am planning to walk through the introductory tutorial called Kaleidoscope at https://llvm.org/docs/tutorial/MyFirstLanguageFrontend/. Let's see how it goes.

Transcript

The following transcript was automatically generated by an algorithm.

00:04 : okay we're live hi there this is Toby
00:07 : and in this video I'm gonna do a
00:11 : walk-through of the LLVM tutorial the in
00:18 : particular the LLVM tutorial that is the
00:23 : second in this list within this page is
00:27 : called kaleidoscope implementing a
00:31 : language with LVM okay so that's what
00:36 : we're doing but before we start let me
00:39 : talk a little bit about LLVM at least
00:43 : explain briefly what is LLVM so the
00:49 : top-level page of LLVM gorg looks like
00:54 : this and the title for the tagline for
00:58 : LLVM is the o LVM compiler
01:02 : infrastructure o LVM is really big in
01:07 : the c and c++ community because it is a
01:14 : state of the art c and c++ and also
01:18 : objective-c compiler with from what I've
01:23 : heard it's quite a bit faster than the
01:27 : other leading C compilers in particular
01:30 : GCC it the the the executable binaries
01:38 : like executable programs that this
01:41 : system compiles in particular the clang
01:43 : system compiles it's quite a bit faster
01:46 : so but LLVM infrastructure is also
01:52 : popular for another reason in particular
01:55 : you can use some of the modules within
01:58 : this infrastructure as a building block
02:01 : for creating your own programming
02:05 : languages in your compiler design you
02:10 : can use some of the building blocks
02:12 : than LLVM to do that and which is
02:16 : largely why I'm interested in it as some
02:21 : of you might know I have a very big
02:25 : interest in programming languages how
02:28 : they work how to build a programming
02:32 : language yourself have created a series
02:37 : video series called how to make a
02:40 : programming language and which I think
02:44 : only is is the tip of the iceberg when
02:49 : it comes to all the knowledge that you
02:53 : would need to learn to learn how to
02:54 : actually make compilers and so for those
03:00 : reasons
03:00 : [Music]
03:03 : so the tutorial that I'm gonna go
03:05 : through is you can find it if you start
03:11 : from their home page and then go to
03:15 : documentation I believe and then go to
03:20 : get things started
03:22 : slash tutorials and then and then the
03:25 : second this item in this table of
03:29 : contents called LLVM tutorial : table of
03:33 : contents this is the tutorial that i'm
03:37 : going to do the toy language so called
03:40 : toy language that I'll be making just
03:45 : follow step-by-step this tutorial
03:47 : although I may deviate a bit if I feel
03:50 : brave at certain points during the
03:52 : tutorial yeah we're gonna be making a
03:57 : kaleidoscope not making clothes become
04:00 : toy programming language called
04:03 : kaleidoscope there's actually a C++
04:07 : version of this tutorial as well as a
04:09 : objective camel version which objective
04:13 : camel is a popular functional
04:16 : programming language
04:17 : I'm going to follow the C++ version of
04:22 : the tutorial which you can access via
04:24 : this first link here yes and this first
04:34 : video will cover going through the
04:36 : chapter one but um actually let's talk a
04:42 : little bit about what LLVM is first so
04:47 : in my first video in this how to make a
04:50 : programming language series I sort of
04:55 : gave an overview of what is the
04:59 : architecture of a programming language
05:01 : when you're building a program language
05:03 : what are the sort of boxes and arrows
05:06 : within the architecture I'm gonna
05:08 : briefly summarize it here basically you
05:13 : start with some code that the user of
05:17 : your programming language has written
05:20 : the first step in your in your machinery
05:25 : that you're gonna make is you're gonna
05:27 : need to write a parser the parser is dis
05:33 : readable actually think I might have
05:37 : flux turned on which means you may not
05:40 : see all of the colors so let me turn
05:43 : that off so this light is quite a bit
05:46 : bright okay you're gonna write a parser
05:51 : that is gonna take this code and then
05:55 : it's gonna spit out something called an
05:58 : est est is the abstract syntax tree and
06:02 : an abstract syntax tree is like a data
06:07 : structure representation is uh it's just
06:09 : it's a nested data structure
06:13 : representation of what the syntax of the
06:16 : code is that your user wrote when you're
06:21 : at this ast stage then the next step is
06:24 : take this VST and convert it to machine
06:34 : and that step is called the generator so
06:40 : est goes in to the next step of the
06:43 : pipeline called generator and the
06:45 : generator courage machine code okay and
06:49 : the mothers machine code well machine
06:52 : code is code that your processor can
06:55 : execute you can understand and execute
06:58 : so your processor looks something like
07:01 : this and usually says in tell your
07:08 : computer that you're running right now
07:10 : probably has it even your smartphone
07:13 : that you're using right now to maybe
07:16 : watch this video has a CPU that looks
07:21 : like this
07:22 : may or may not see intel on it but it
07:25 : probably does this machine code then
07:28 : goes into the cpu which actually
07:32 : understands these machine code and the
07:34 : CPU will execute those instructions
07:36 : making your program actually work know
07:40 : where this LLVM fit into this picture
07:46 : well LLVM fits within this generator OBM
07:55 : can be used as a component within your
07:59 : code the generator now how does that
08:03 : work let's go back
08:12 : okay let's go back here so instead of
08:15 : this picture if you decide to use LLVM
08:21 : as one of the building blocks of your
08:23 : compiler then this is what you're gonna
08:25 : do instead instead of having your
08:27 : generator generate machine code
08:29 : you're still gonna make a generator
08:35 : you're still gonna take the ast and and
08:39 : run it through your generator but this
08:41 : time your generator is going to generate
08:44 : LLVM IR code LLVM doesn't actually stand
08:54 : for anything if it's not actually an
08:58 : acronym so if you go to the front page
09:00 : of org let me show ya if you go to the
09:08 : front page of it says despite its name
09:12 : LLVM has little to do LOV em the name
09:17 : LLVM itself is not an acronym so I can't
09:22 : really tell you what the M stands for
09:25 : male vm is just the name of this project
09:30 : which is a compiler infrastructure
09:34 : toolset right what is IR IR stands for
09:38 : intermediate representation intermediate
09:48 : representation why is it called
09:49 : intermediate representation well it's
09:52 : it's code there is somewhere in your
09:56 : pipeline but it's not the end result
09:59 : that we need in order for your code to
10:03 : actually run on the CPU because before
10:06 : we get there actually need to run this
10:09 : guy through yet another box and then and
10:15 : after going through that box that's when
10:19 : you get machine code
10:23 : then you get the machine code that can
10:26 : go into your CPU that looks that looks
10:29 : kind of like that okay that's why it's
10:34 : an intermediate representation it's not
10:37 : the code that the user wrote it's it's
10:41 : lower level than that but it's not quite
10:43 : so level that is machine code therefore
10:46 : it's a intermediate representation and
10:48 : this box that will take your LLVM ir
10:53 : code and spit out machine code for you
10:57 : that program is provided by the LLVM
11:02 : project it is called LLC C probably
11:07 : means compile PAC compiler so LLC is a
11:12 : compiler that will take any code that is
11:16 : LLVM that's valid LLVM ir code and
11:20 : generate machine code for you why do you
11:24 : want to do this why is this a good idea
11:25 : well first of all if you do this you you
11:31 : don't have to understand how machine
11:34 : code works which is kind of nice because
11:38 : there's a lot of low-level details that
11:41 : actual CPU you know that there's a lot
11:47 : of learning that you would have to do to
11:49 : get intimate
11:50 : ly knowledgeable about a particular CPU
11:53 : another reason is you might want to
11:56 : target your program for more than one
11:59 : kind of CPU there are different CPU
12:02 : brands different CPU architectures
12:05 : there's the Intel the arm the SPARC
12:09 : those are all different kinds of
12:12 : processors and each of them have a
12:14 : different language set they have a
12:18 : different machine code instruction set
12:20 : so if you wanted to target more than one
12:24 : of them normally you would have to
12:26 : figure out how to generate machine code
12:28 : for each of those are
12:30 : textures if you generate LLVM ir the ll
12:37 : compiler actually knows how to spit out
12:40 : machine code of any of these popular
12:44 : architecture popular cpu types so so
12:51 : yeah so you get a lot of value I love a
12:53 : bang for the buck you just spit out one
12:56 : IR code format and then you get access
13:00 : to all of these different architecture
13:03 : platforms another reason you this is it
13:08 : might be a good idea is that LLVM sort
13:14 : of has a big focus on performance and
13:17 : therefore the compiler has a optimizer
13:21 : package with it or there's a optimizer
13:25 : that you can use with it I should say
13:28 : which is able to generate more optimized
13:32 : code which can run faster on any of the
13:36 : CPUs types that it supports so if if you
13:40 : are not extremely knowledgeable about
13:43 : compilers and how to make really super
13:46 : fast code well you don't have to you
13:48 : just outsource that part the job to the
13:52 : LLVM project and the OEM vm project is a
13:56 : tool chain that does that really well so
13:59 : that's nice how ok so one day I want to
14:04 : say is um the tutorial that we're gonna
14:08 : follow it's called my first language
14:13 : front end without LVM tutorial why the
14:17 : word front end well in this is not front
14:22 : end as in a web user interface front end
14:25 : in in in compiler terms the front end
14:29 : means this bit more or less
14:37 : so this part is the front end the bit
14:44 : that's generating the machine code that
14:46 : is the back end and the code this guy is
14:53 : kind of in between like you're gonna
14:56 : write some a code generator that
14:59 : generates some LLVM IR code with which
15:03 : the back end which is the ll compiler
15:07 : will work on so I guess the the reason
15:12 : the tutorial is called my first language
15:15 : front-end with our VM is because you
15:19 : actually didn't have to write the
15:20 : backend you only have to write the front
15:22 : end which is the parser and then sort of
15:26 : yeah there's this part arguably that's
15:29 : also the front end all you have to do is
15:32 : generate this LLVM intermediate
15:35 : representation and then the back end has
15:38 : been taken care of you you don't have to
15:39 : do to learn machine code and all that
15:43 : stuff which is really hard if you
15:46 : haven't done that before an analogy
15:50 : would be like you you're making a web
15:52 : application but instead of writing your
15:56 : own back-end you're gonna use a
15:58 : cloud-based database or already or yes
16:04 : was it platform as a service that has
16:07 : this sort of database built in which you
16:10 : can just access with REST API calls you
16:15 : can build a whole app by only building
16:18 : the front-end so that's kind of what
16:19 : we're doing right now with building a
16:22 : language okay all right just get started
16:26 : with this so I'm gonna be following
16:30 : along this tutorial this tutorial is
16:32 : gonna be in C++ I've been brushing up on
16:35 : my C++ I've done it a long time ago and
16:39 : really honestly I have never been very
16:42 : knowledgeable about C++ more recently
16:45 : I've been brushing up on
16:46 : and learning specific things that were
16:51 : confusing me partially to get ready for
16:55 : this so let's dive into it let's do this
16:59 : tutorial okay so to my first language
17:06 : front end with LLVM tutorial so this
17:11 : tutorial introduces the simple
17:14 : kaleidoscope language building it
17:16 : iteratively over the course of several
17:18 : chapters showing how it is built over
17:21 : time this lets us cover a range of
17:24 : language design and LLVM specific ideas
17:27 : showing and explaining the code for it
17:29 : all along the way and reduces the
17:31 : overwhelming amount of detail up front
17:34 : We strongly encouraged that you work
17:37 : with this code make a copy and hack it
17:39 : up an experiment warning in order to
17:43 : focus on teaching compiler techniques
17:46 : and LLVM specifically this tutorial does
17:49 : not show best practices in software
17:52 : engineering principles for example the
17:54 : code uses global variables pervasively
17:57 : does not use visitors etc but instead
18:01 : keeps things simple and focuses on the
18:03 : topics at hand okay note it so let's get
18:07 : started so the first chapter is gonna be
18:11 : on the lexer so we're gonna build a
18:13 : lexer the program language we're gonna
18:15 : build looks like this it looks very much
18:18 : like python that's nice I like fight
18:21 : Python this is a fibonacci program so
18:25 : cliche I've done that like a million
18:27 : times that's fine in Chapter six we we
18:33 : use the program to display a Mandelbrot
18:36 : set at various levels of magnification
18:39 : this tutorial really knows its audience
18:42 : oh man I'm crazy about the Mandelbrot
18:45 : set okay so let's follow this tutorial
18:48 : and make a lecture so I'm going to oh
18:50 : one thing actually actually I think
18:59 : I'm gonna do is split this series up
19:01 : into shorter chunks so I'm going to I'm
19:05 : not going to actually I think cut it
19:10 : short here and then start the next one
19:13 : so I'll see you on the other side