Real World Leaky Abstractions 3: Python and Having No Time

Huiqi and I continue our journey into the realm of leaky abstractions. This time we give examples of low code/no code tools, i.e. code generators, Python 2's range function, and then talk about the prospect of making choices when one "has no time".

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : so the second is like any sort of local
00:03 : no code solutions like
00:05 : uh pi qt designer uh that's what i'm
00:08 : thinking
00:09 : like um like because if when you use pi
00:12 : acuity designer
00:13 : they um they give you all these tools so
00:17 : that
00:17 : you can use this to drag and drop and
00:19 : then generate the
00:20 : code which is very cool but
00:23 : at the same time you know it doesn't
00:25 : cover all the cases that the code can do
00:28 : and it's sometimes like generates like
00:31 : weird stuff or like it will generate
00:33 : like a spaghetti code or whatever so
00:35 : that
00:36 : it generates html first
00:39 : and then i downloaded another library to
00:42 : change
00:43 : that to code so this is the code that is
00:46 : generated
00:47 : so if you had this code you will see
00:50 : it's just like oh you have the tree
00:53 : widget three and three with your sticks
00:55 : and
00:55 : three with your five
00:58 : naming the widgets with numbered labels
01:02 : is not something i would do right so if
01:05 : say if
01:06 : we want to write this code exactly
01:09 : ourselves we wouldn't write it this way
01:11 : right
01:13 : but now it's like what it was this code
01:17 : so do you want to clean it up yourself
01:18 : or do nothing with it and
01:20 : say if oh i come here i'm like oh i
01:22 : don't like this naming commission i want
01:24 : to clean it up
01:25 : but now but if i want to add something
01:28 : in my original you know stuff i can't
01:31 : because you already changed this
01:33 : so now the modified version of because
01:36 : it's not two-way
01:37 : like synchronous yeah it can't
01:39 : synchronize back into the
01:41 : xml file and right then you can't
01:44 : change it with the gui anymore yep right
01:47 : exactly
01:48 : so i i feel like well that's just one
01:50 : example of the no code local solution
01:53 : but there are other
01:54 : ones like uh like for example though
01:57 : the one we did at inside like
02:00 : configurable form or something now you
02:02 : use json
02:03 : to define a form like what if we also
02:06 : did one we also did one for
02:08 : um uh converting the sharepoint
02:11 : data model to to the database to a
02:15 : database
02:16 : data model we took like metadata
02:19 : that's coming from the sharepoint
02:21 : database and we generated
02:23 : c-sharp entity classes
02:26 : uh to represent database tables
02:29 : yeah that's like a one-way street you
02:31 : only generate in one direction
02:33 : and also like when this kind of local
02:36 : local solution have
02:37 : issues you don't know how to
02:41 : fix it because you i guess unless you
02:44 : look at
02:45 : look into what's under the hood but even
02:47 : if you look into what's under the hood
02:49 : and trying to fix
02:50 : it maybe it's hard to fix it because
02:52 : maybe the
02:53 : under the whole structure is too
02:54 : complicated because of the abstraction
02:57 : and then if you want to fix maybe you'll
02:58 : break some other stuff or maybe you
03:00 : don't have assets yeah
03:01 : it's like you don't want to fix it
03:02 : because i think because
03:04 : you don't want to make the code
03:07 : generator
03:08 : very com complicated because it's
03:10 : already
03:11 : complicated making a code generator
03:15 : and if you add a lot of different
03:17 : options different edge cases to it
03:19 : it's going to make it more complicated
03:22 : and more difficult to diagnose and
03:24 : you you're in the first place you're
03:26 : doing this to try to simplify your life
03:28 : and now by making it more complicated
03:31 : you're gonna make
03:32 : it harder right so i think that's part
03:34 : of it so so like
03:35 : when things don't work out you're like
03:37 : oh yeah i did not handle that
03:40 : edge case but i also don't really want
03:43 : to add more
03:44 : handling right so and python
03:48 : this is actually a python 2 specific
03:51 : thing
03:52 : they have changed in python 3 and this
03:55 : this issue this leaky abstraction
03:58 : doesn't exist in python 3 anymore
04:00 : but basically the range function like
04:03 : you're right for i in range 10 and
04:05 : that's going to
04:06 : i is going to be 0 all the way up to 9.
04:10 : that's how that works but then
04:13 : they realized that this is not good
04:15 : enough
04:16 : and they added another function called x
04:19 : range
04:21 : and then after a while they actually
04:23 : recommended people
04:24 : use x range instead of range so like
04:27 : to me that's a leaky abstraction right
04:30 : away
04:31 : because this like to write x range is
04:35 : seems more cryptic than to write range
04:38 : so why why are we doing that and why is
04:40 : that the preferred
04:42 : way to do it and if if python's thing
04:45 : is give make things simple or give the
04:49 : illusion that
04:51 : things are simple then why am i writing
04:54 : the
04:54 : more cryptic thing as the preferred
04:57 : method
04:58 : that's weird um and then in order to
05:01 : understand
05:02 : why you want to use x range you're going
05:04 : to have to uncover
05:06 : more of the how things actually work how
05:08 : about the secrets behind how python
05:10 : works
05:11 : right so the the reason is
05:14 : range returns a list
05:17 : of integers therefore
05:21 : if you're gonna use range with a
05:24 : large number say one million you're
05:27 : gonna
05:27 : python has to allocate enough memory
05:30 : to to hold one million integers
05:33 : okay that's gonna be slow because the
05:36 : allocation of a lot of
05:38 : memory will be slow mm-hmm
05:41 : and and then later on like re
05:44 : reclaiming the memory will also be a
05:46 : little bit slow well
05:47 : maybe not that bad but but just
05:49 : allocating a lot of
05:50 : memory is slow so what x range does
05:54 : is x range doesn't actually allocate
05:58 : the memory for all of the numbers but
06:00 : x-range is actually a generator
06:02 : object or iterable object
06:06 : and uh oh it remembers it's its
06:09 : range and its current cursor within the
06:12 : range
06:13 : so so when when the code goes back into
06:16 : the for loop
06:17 : the for loop will actually ask the
06:21 : generator again hey what's the new i now
06:24 : and the generator just has to reply the
06:27 : new i
06:27 : is this so the generator can just use a
06:30 : counter
06:31 : internally to remember what the current
06:34 : i
06:34 : is and then just increase it when the
06:37 : for loop asks it
06:38 : again so so it doesn't need
06:42 : much memory at all it just it's a big o
06:44 : of one
06:45 : memory whereas range range requires
06:48 : big o of n memory gotcha so for uh
06:51 : i i guess i guess i i would think
06:55 : the original range was like that to
06:57 : begin with but i didn't know it's not
06:59 : uh well it is in python 3.
07:02 : oh okay they fixed it okay gotcha
07:06 : yep yeah so i guess it's like
07:09 : for a developer who used python like how
07:12 : does
07:13 : uh the the developer know like what is
07:16 : the stuff that's under the hood i guess
07:18 : you can try to know it
07:20 : but i guess then it uh defeat the
07:22 : purpose of
07:23 : making it easy to understand i don't
07:25 : know
07:26 : it doesn't totally defeat it but yes but
07:29 : then that's
07:30 : um i don't know the answer to this but
07:34 : like how much percentage of the time
07:37 : of staying ignorant is desirable
07:42 : that's sort of what we're asking right
07:44 : like
07:45 : like how much percentage of the time
07:48 : staying in the simple life is desirable
07:51 : because
07:52 : you can never really get to 100 i guess
07:55 : you can get to 100
07:57 : if you just refuse to learn the
07:59 : internals
08:01 : but that's gonna mean you don't have the
08:03 : ability to fix
08:04 : anything that requires you to understand
08:07 : the internals and
08:08 : maybe you just like call a technician
08:10 : that that's what we do
08:12 : when we can't fix our car right right
08:14 : yeah exactly
08:15 : so i'm going to call in somebody who
08:17 : knows what they're doing and
08:19 : they'll do it i'll pay them yeah i guess
08:21 : at the end you know we can't
08:23 : always know the internal of all the
08:25 : tools that we use
08:26 : because we don't have that much time so
08:28 : it's like which one
08:31 : and which internal do you choose to
08:33 : learn
08:34 : so well it's true that we don't have
08:36 : enough time
08:37 : but um that's like a deceitful way
08:41 : to look at uh it's not that you don't
08:45 : have time you only
08:46 : think you don't have time because
08:50 : because you what you choose to do is up
08:53 : to you right
08:54 : and and so like when you say i don't
08:56 : have time basically what you're saying
08:58 : is i don't have time because my time
09:01 : is reserved for doing other things
09:04 : deframing it as
09:05 : i don't have time it's it's basically
09:09 : you've you've pre-allocated
09:12 : your time for certain things
09:17 : and what i'm saying is it's fine
09:21 : that you don't want to do it but you
09:23 : should be more honest about why
09:25 : and if you say i don't have time the
09:27 : real reason why
09:28 : is because you think there are other
09:32 : things that are more
09:33 : worth your time than this that is true
09:36 : yes and and and it's fine if that's the
09:38 : case but if you you see
09:40 : if you say i don't have time that's sort
09:43 : of like a
09:44 : a lie you know because
09:50 : like like i i think you should be forced
09:52 : to confront
09:53 : the fact that like because a lot of
09:55 : times it says like you just don't want
09:56 : to do this thing
09:58 : right it's not it's not that you don't
10:00 : have time a lot of times it's like
10:02 : i just don't feel like doing this well
10:04 : it's like the choice of
10:05 : whether to use a library right i mean
10:08 : most people
10:09 : use a library because they choose to
10:12 : prioritize
10:13 : maybe making the software quickly versus
10:17 : say uh implementing everything yourself
10:21 : just using a library is faster
10:24 : people make this choice probably people
10:28 : will know
10:29 : say if i use this library i have to face
10:32 : this leaky
10:33 : situation at some point
10:37 : you do one more and i'll do one more and
10:40 : that'll be the end
10:41 : okay uh which one
10:45 : maybe well i think all these are like
10:48 : similar like say like just
10:51 : using like a library versus using
10:54 : something that's more
10:56 : uh more primitive or more low-level
11:00 : because when you use a library like say
11:02 : low dash
11:04 : really don't know what's going on and
11:06 : sometimes you do know what's going on
11:08 : but you still do it
11:09 : because the code looks simpler or for
11:12 : whatever reason um
11:14 : the code looks shorter yeah the code
11:17 : looks shorter so
11:19 : conspicuously conspicuously it might be
11:21 : because
11:22 : uh the underscore is a very short
11:24 : character
11:27 : right right so you you do you do it you
11:30 : use the library but the library is
11:32 : basically like a wrapper of what's under
11:34 : the hood and you have to
11:36 : sacrifice maybe the optimization or
11:39 : something
11:40 : um so and also when the library doesn't
11:44 : work
11:44 : you might not be able to uh easily know
11:47 : why it doesn't work
11:49 : yeah so that's the example of using
11:51 : library i mean this
11:52 : x xcs mpj
11:56 : mpj that uh did an
11:59 : episode on leaky abstraction and he
12:01 : talked about axe heals
12:03 : maybe let's start again i want to hear
12:04 : about garbage collection
12:06 : what is our thoughts about garbage
12:08 : collection well
12:09 : when i write this what i was thinking is
12:12 : you know when you talked about in a
12:14 : language like c
12:16 : and c plus plus you have to do this
12:19 : yourself you have to like clean up the
12:22 : memory
12:22 : uh usage yourself instead of saying
12:25 : javascript
12:27 : it's like automatic uh but
12:31 : when you when the garbage collection
12:34 : mechanism is like more automatic you
12:36 : might face some situation
12:38 : of whatever you want to clean up
12:41 : you don't uh it doesn't get to clean up
12:43 : as you expect it
12:45 : and things like that so yeah that's what
12:48 : i'm and also i'm
12:49 : i was thinking about you know rsjs you
12:51 : know why do you have to like unsubscribe
12:54 : whatever uh yourself and how do you know
12:57 : when to unsubscribe
12:58 : so and why design this way
13:02 : basically the times when the
13:06 : abstraction doesn't work for you in
13:08 : garbage collection is
13:10 : when you have a memory leak
13:13 : uh that doesn't happen very often
13:17 : oh i mean okay maybe it happens more
13:19 : often than you think
13:21 : but it doesn't it's not often that we
13:24 : diagnosed the memory leaks um
13:27 : and my theory is is it happens more than
13:30 : we think
13:31 : but we don't notice it when it does
13:33 : happen
13:34 : most of the time because it's not big
13:36 : enough of a problem that we need to fix
13:38 : it
13:39 : but when it is big enough of a problem
13:42 : and we have to fix it it's usually very
13:46 : time consuming
13:49 : and uh and also
13:53 : you you're gonna need to understand what
13:56 : a memory leak is and how it can happen
13:58 : under what circumstances it can happen
14:01 : what do you need to do
14:03 : for the garbage collector to properly do
14:05 : its job you need to understand that
14:07 : at least even if you don't know the
14:11 : internals in the game industry
14:15 : they hit this more because
14:19 : like a performance is very important to
14:21 : them because they
14:22 : their game has to be smooth the
14:24 : animation of the game has to be smooth
14:27 : uh basically they they feel like they
14:29 : have to take total control
14:31 : over this so that they can
14:34 : maybe like if they need to cheat
14:36 : someplace they can
14:37 : right is that why they
14:41 : don't use javascript to develop
14:44 : games there are javascript games
14:49 : it's not all of them we have games
14:52 : we have a lot of games developed in
14:54 : javascript now we have a lot of game
14:57 : minecraft has a java version
15:00 : you know and java obviously has a
15:03 : garbage collector