Python vs C: Side by Side (Plus Macros!)

I rewrote some Python code in C last couple of weeks to improve the performance of the "recreate" program. Here I show a brief side-by-side comparison of the code before and after with Huiqi.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : rate 2 dot pi i i
00:03 : started rewriting the recreate so
00:06 : in python so that's this is recreate
00:10 : 2 that pi so let's just
00:13 : go to um okay i'll go to this new code
00:17 : function
00:21 : so this is the new code function in
00:23 : python
00:25 : and in c i have i more or less did a
00:30 : uh translation from the python
00:34 : one to the c one obviously not a
00:37 : direct translation some of the things i
00:40 : kind of changed
00:45 : but but most of them were basically
00:49 : more or less the same
00:52 : workflow between the two versions
00:55 : so so the difference is
00:59 : um first i had this
01:02 : parsing kind of interleaved in the
01:05 : processing code
01:06 : which was not the case in python in
01:09 : python i first
01:10 : parsed the things and then fed the
01:12 : results
01:13 : that were parsed into this function okay
01:17 : but for c i decided to
01:22 : i decided to have the processing
01:24 : function do the parsing
01:27 : right inside of it for performance
01:30 : reasons
01:32 : um so this is like whereas the python
01:35 : version i have a heap id
01:38 : which i renamed it to address over here
01:41 : i have a file name i have a name this is
01:43 : like saying
01:44 : hey parse the heap id parse the file
01:48 : name
01:49 : and then when when you parse the file
01:51 : name it goes into this
01:53 : file name variable these are output
01:55 : parameters
01:56 : okay like i i passed these things in but
02:00 : internally it actually sets the value
02:03 : over here
02:04 : so these are output um i
02:08 : developed this um pattern this
02:11 : this core thing that's a macro
02:15 : um i developed this pattern for
02:19 : general error handling similar to
02:22 : um exceptions c does not have exceptions
02:27 : i sort of built my sort of poor man's
02:30 : version of
02:31 : exceptions in a way using this
02:34 : technique i can talk about that later
02:37 : okay
02:37 : that might take a while um
02:41 : but uh okay so this is all parsing i'm
02:43 : just oh
02:44 : this pars var string arcs it actually
02:47 : parses a whole list of string
02:51 : arguments
02:54 : and in python i do it like that
02:57 : over here but in c i do it like this
03:01 : so these three lines correspond to these
03:04 : three lines
03:05 : and then and then this is a database
03:09 : call
03:10 : uh well okay actually this whole thing
03:14 : this whole thing is a one database call
03:18 : and we're we're inserting a fun code
03:22 : here we're using the insert fun code
03:24 : statement
03:26 : to to make a database call and so it's
03:29 : equivalent to
03:30 : this block here okay
03:34 : yeah this this is so in in the python
03:37 : you can see the shorthand
03:38 : which is uh well these are all
03:42 : um parameters these question marks are
03:45 : the parameters
03:47 : and we'll substitute these values into
03:50 : them
03:52 : i have to say bind the first one at
03:55 : index one find the second one at index
03:58 : two
03:58 : find the third one at index three and
04:00 : each time i say
04:01 : bind i have to tell it what type that
04:04 : data is
04:07 : interesting so it's more manual
04:11 : and tedious obviously
04:14 : and also i have to write here if i want
04:17 : null to be represented instead of
04:21 : an empty string let's say that has i
04:24 : need to say
04:25 : bind null instead of buying text
04:29 : whereas in python that happens
04:31 : automatically in internally
04:35 : i i could have wrote written
04:39 : like a helper function that'll do this
04:40 : automatically as well
04:43 : so the where does the by now or by
04:46 : test come from are these like functions
04:49 : that
04:49 : you wrote or where um this is
04:53 : um this gets it this ends up being
04:56 : a um a function from the sqlite
04:59 : library oh okay and this
05:02 : sqlite macro is
05:07 : another thing that another macro that
05:09 : just
05:10 : does a little bit of uh error handling
05:14 : what is a macro what is a macro
05:17 : um oh boy i should
05:21 : probably defer that topic but
05:24 : to summarize what a macro is it's like
05:27 : um
05:28 : it just rewrites your code basically um
05:32 : let me show you and this is a big
05:35 : thing in c it's kind of a big thing
05:39 : in lisp too but lisp the list people
05:42 : will tell you
05:44 : our macros are way better than c's
05:46 : macros
05:47 : because these macros are broken um
05:53 : uh let me show you the this
05:57 : where that macro is oh uh i'll just
06:00 : search for
06:00 : sequel light
06:04 : uh define sql like there it is
06:07 : okay so when i call sqlite it's not
06:10 : a function call it's instead it's a
06:13 : macro
06:14 : expansion and what that means is
06:18 : in re this is uh i think a good way to
06:21 : explain macros
06:22 : is it's syntactic sugar yeah
06:25 : okay it's this is syntactic sugar
06:27 : because this is just syntax
06:29 : expansion it'll replace
06:34 : this with this
06:38 : oh okay before yeah it's like a pre-proc
06:42 : it's sort of like uh you know how
06:45 : typescript
06:45 : just rewrites the typescript into
06:47 : javascript or coffeescript just rewrites
06:50 : the coffeescript
06:52 : this all this does is replaces
06:55 : this syntax with this this syntax it
06:58 : expands it
06:59 : so it ends up looking like that
07:03 : but then it'll swap in the dynamic bits
07:06 : and the dynamic bit is the f so it
07:09 : becomes like that
07:12 : and this this pound sign in front of the
07:15 : f
07:15 : means hey you know stringify that thing
07:19 : okay and so forth uh so so
07:23 : me writing this just magically becomes
07:27 : that
07:28 : okay yeah every time so
07:31 : so so there's a little bit of error
07:33 : handling code here
07:34 : and that that's sort of the mechanism
07:36 : i'm using to to build
07:38 : this sort of like uh sort of like an
07:40 : exception handling system
07:43 : okay yeah so so that every time i do
07:46 : that it's actually
07:47 : a huge block of code or a block of like
07:52 : eight or ten lines every single time
07:54 : here
07:55 : okay so so this actually if i hadn't
07:58 : done this
07:59 : this code would be so much longer
08:02 : because every time i do one of these
08:04 : it's like eight lines of code right you
08:06 : can imagine how long this file would end
08:08 : up being
08:10 : and and my the call is a similar deal
08:14 : where
08:15 : i have uh three or four lines after
08:18 : calling this function to do error
08:20 : handling
08:22 : uh so yeah this file would get really
08:24 : really long if i hadn't done this trick
08:26 : so what's the difference between this
08:28 : and like a function
08:30 : between macro and just a function i mean
08:33 : you can have a function that
08:35 : takes in a function in some cases you
08:37 : cannot
08:39 : okay in some cases you cannot for
08:41 : example
08:43 : um for example i want to return
08:46 : i have a return statement well i can
08:50 : have a function
08:51 : but that function is going to return
08:53 : from itself
08:54 : not from where i am in this scope right
08:57 : that's true or i i'm inside of a
09:02 : while loop and i want to put a brick
09:04 : there
09:06 : well a function cannot break
09:10 : inside of the function it doesn't have
09:12 : it's not in a while loop
09:13 : right there's actually many situations
09:17 : where
09:18 : um you actually cannot write it as a
09:21 : function
09:22 : and but the the macro expansion all it's
09:25 : doing is
09:26 : like replacing your syntax with
09:30 : a new piece of syntax with a little bit
09:32 : of replacement
09:34 : right so it keeps the original scope
09:36 : that's the biggest difference yeah yeah
09:38 : yeah
09:39 : okay gotcha so yeah that's an
09:42 : important but it's also like a
09:46 : very two-edged feature using this macro
09:49 : thing
09:50 : because it's difficult to debug oh yeah
09:53 : you can do crazy things in here
09:55 : in the macro and it's somewhere else and
09:58 : it's like you don't know
09:59 : where yeah it breaks yeah it's it's kind
10:03 : of in in some companies it's actually
10:05 : frowned upon
10:06 : although i see it i looked at
10:10 : various source code in open source and
10:13 : they
10:14 : actually do some crazy magic with macro
10:16 : a lot of the
10:18 : like i looked at the v8 javascript
10:20 : engine and they have some crazy hacks
10:22 : that use macro and also there's
10:26 : there's more fancy features in macros
10:29 : too i'm actually using very basic stuff
10:31 : right now uh you can do much crazier
10:34 : stuff with it in
10:35 : in the python source code they use some
10:38 : macro hacks
10:39 : as well um so this is
10:43 : if if you get into like the c code
10:46 : in the c world or uh a lot of open
10:50 : source code is in c
10:51 : or c plus plus you'll see some
10:54 : stuff and it is kind of hard to debug
10:56 : because you can't
10:58 : just it's harder not not that you can't
11:02 : it's harder to track the code like
11:05 : you're like oh where is this function
11:07 : defined
11:09 : sometimes there's a macro that defines a
11:12 : function
11:15 : okay and you can you can have you can
11:17 : write like
11:19 : you know you can write uh define
11:23 : my even handler
11:27 : like that right and that
11:30 : actually generates a function entirely
11:33 : and then because all it is is syntax
11:37 : expansion
11:37 : right so so then you're like oh
11:40 : i see this you're calling this function
11:43 : here but i don't see
11:44 : where it was defined it was defined by
11:46 : this macro that defines functions
11:49 : there's some crazy stuff like that that
11:51 : you can do which is nice in one way
11:53 : but not nice in that it makes it harder
11:56 : for people to debug later on
11:59 : that's true yeah so