Law of Leaky Abstractions

Huiqi and I have chat about Joel Spolsky's Law of Leaky Abstractions: https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/. We touch on various topics including TCP/IP, SQL, ORMs, frameworks, and more.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : i think joe sposky is this guy here
00:03 : with the ping pong paddle he uh he
00:06 : coined this term
00:07 : leaky abstractions okay uh when was this
00:11 : i don't oh okay so previous and next so
00:14 : he wrote this in 2002.
00:16 : an abstraction is designed to
00:20 : hide complexity from you he uses tcp
00:24 : protocol it is the it's the protocol
00:28 : that makes the internet work
00:30 : how computers send information to each
00:33 : other over the network
00:36 : and underneath that there's an ip
00:38 : protocol you can read this
00:40 : in more detail later but he's basically
00:42 : saying that tcp
00:43 : is a reliable transfer protocol in that
00:47 : when you send a packet it gives you a
00:50 : pretty good guarantee
00:51 : that the like the data will get there
00:54 : in the correct order okay
00:58 : like you send a stream of bytes over the
01:01 : tcp connection
01:02 : then the computer that's receiving the
01:05 : byte
01:06 : will get the byte in the same order that
01:08 : you sent it
01:09 : okay and that's actually very
01:12 : non-trivial to do
01:14 : but but it works and
01:17 : the weird thing is that tcp depends on a
01:21 : lower level
01:22 : protocol called ip which does not give
01:25 : that guarantee
01:26 : ip is just like hey you send a fixed
01:29 : number of
01:30 : bytes and then maybe it'll get there
01:33 : maybe it won't
01:34 : i don't know that's what ip gives you
01:37 : so tcp is implemented on top of ip
01:41 : that's why you call they call it the
01:43 : magic of tcp how do they do it
01:45 : on top of a a layer
01:49 : ip which is very very brittle
01:52 : very very a failure prime and that's an
01:55 : example of
01:56 : like oh it's actually the low level is
01:58 : actually
01:59 : not very nice and you know in order to
02:02 : make
02:02 : the data go across and in a reliable way
02:07 : it's actually kind of complicated and
02:09 : non-trivial you have to come up with
02:11 : some tricks to make it happen but well
02:14 : i'm gonna
02:15 : put an abstraction over it so don't you
02:17 : don't have to understand all the tricks
02:18 : that we do
02:19 : under the hood and then so so to you it
02:22 : looks like you're just sending a stream
02:23 : of bytes
02:24 : over and it just works um that's an
02:27 : abstraction
02:28 : abstraction takes something complex and
02:31 : gives you
02:31 : a simplified view of our simplified
02:35 : api simplified way of working with that
02:38 : uh thing like interface basically
02:42 : yeah it is like an interface in the
02:44 : design patterns book
02:46 : there's a pattern called the facade a
02:48 : facade is like a
02:50 : fake appearance right it it's like a
02:53 : costume
02:54 : almost like like even though i am really
02:57 : complicated on the inside
02:59 : i put on a costume and i pretend for you
03:02 : that i'm really simple
03:06 : i'm really nice even though i'm in
03:08 : reality i'm not
03:09 : um the job of an abstraction is to cover
03:12 : up all those ugly details
03:14 : okay so when he uses the word leaky what
03:17 : he means
03:18 : is like the the abstraction doesn't
03:21 : work that there's some ugliness that's
03:24 : showing up
03:25 : trying to hide it isn't working anymore
03:27 : costume has these
03:29 : holes or the custom is malfunctioning
03:32 : so you can actually see the ugliness
03:35 : inside
03:36 : and that kind of thing happens you know
03:37 : when something
03:39 : unexpected is happening in the in the
03:42 : case of the ip
03:43 : the tcp thing is like well you can still
03:47 : have an unreliable network
03:49 : if the network not connected in a
03:52 : nice way then things will get really
03:56 : slow or if you're if your network cable
04:00 : was cut then that you will definitely
04:02 : not get the bits over
04:03 : right right there there's cases
04:06 : that the no matter how fancy your
04:10 : mechanism is
04:11 : will still break sql is actually sql
04:15 : sql itself is a very fancy abstraction
04:19 : right for how data is stored
04:22 : and retrieved but then there's also
04:25 : on top of sql there's classes of
04:29 : fancy orms they're trying to provide you
04:32 : an abstraction
04:34 : to write the code without touching sql
04:36 : itself
04:37 : because a variety of reasons but um
04:41 : but i basically what an orm uh orm like
04:45 : in java is hibernate in javascript
04:49 : type orm is pretty uh
04:52 : popular in the javascript community now
04:55 : as an orm sql itself is a very
04:59 : sort of large complicated language
05:02 : and if you try to put a facade over
05:05 : something that's already
05:07 : large and complicated you're trying to
05:09 : wrap the large and complicated thing
05:11 : there's just a lot of opportunities for
05:13 : abstraction leaks
05:15 : when you do something like that so the
05:17 : prop the problem
05:19 : is the user of the
05:22 : abstraction does not understand what's
05:25 : inside of it so when the problem comes
05:29 : they don't really know
05:30 : what exactly is the problem but that's a
05:33 : potential problem yeah
05:35 : that's what he's pointing to if the user
05:37 : is
05:38 : too dependent on the abstraction the
05:41 : user is too dependent on
05:43 : not knowing the ugly details
05:47 : then when they actually need to know
05:49 : what they are
05:50 : he just has to start from zero he has no
05:53 : knowledge of that stuff you can
05:56 : interpret
05:57 : what he's saying as the the concept of
05:59 : abstraction
06:00 : it does not work 100 of the time
06:04 : you would like to be able to pretend
06:07 : that the complicated thing is simple but
06:10 : it
06:10 : only works some percentage of the times
06:12 : sometimes that doesn't work
06:14 : because your your program is just not
06:16 : working as you expect it to
06:18 : and when that happens it's not good
06:19 : enough to pretend it's simple anymore
06:22 : right but what's the solution of this
06:27 : he doesn't really give you a solution
06:29 : but to me
06:30 : uh what it says is you should
06:33 : re-evaluate
06:34 : the value of an abstraction
06:38 : and abstraction can be anything that
06:41 : is trying to represent itself as a
06:44 : dramatic simplification right that's how
06:47 : they usually sell
06:49 : frameworks and libraries to you look
06:52 : using flutter
06:53 : you can build beautiful mobile apps
06:56 : very easily and and they will be very
06:59 : performant and then
07:00 : development is fast and easy and it's
07:03 : it's very flexible and expressive of it
07:06 : all
07:06 : it's all good it's all good and it's
07:08 : very simple and they will never tell you
07:11 : about
07:11 : those times when the abstraction is
07:14 : failing
07:15 : and and then you actually need to
07:16 : understand what
07:18 : happens in the internal right way but
07:20 : isn't this all the
07:22 : like modern programming language i mean
07:25 : they are all
07:26 : abstraction of the bottom layer language
07:29 : but i don't know how to code in the say
07:32 : assembly
07:34 : to like hold such a complicated app so i
07:37 : have to use the scaffolding
07:40 : of the modern programming language
07:43 : and now the framework and stuff um well
07:45 : you think you
07:46 : have to uh but you don't
07:52 : you don't you don't have to use anything
07:54 : you can
07:55 : learn assembly language and code
07:57 : everything in assembly language
07:59 : if you wanted to right it's probably not
08:01 : a very efficient way to do it
08:04 : right there's a trade-off and my point
08:07 : is
08:08 : that it is a trade-off it's not the only
08:10 : option you have
08:13 : because it like in in a lot of modern
08:16 : projects that i've seen
08:18 : obviously it varies from company to
08:20 : company from project to project but what
08:22 : i have seen
08:23 : is we tend to take on a lot of
08:26 : dependencies
08:27 : not not only a lot of dependencies but a
08:29 : lot of
08:30 : large dependencies that that uh
08:33 : they do a lot of non-trivial stuff
08:36 : in order to abstract you from the
08:39 : complicated things
08:40 : but each one of those has a cost
08:43 : associated with it and
08:45 : and don't 100 percent abstract you
08:48 : from what happens internally because all
08:51 : of them
08:52 : that's the law of leaky abstractions all
08:55 : non-trivial abstractions to some degree
08:57 : are leaky
08:58 : each one that you take in whether it's
09:00 : react or
09:01 : express js or nest js
09:04 : or expo or you know uh
09:08 : just whatever framework it is every
09:10 : single one of them
09:12 : to some degree are leaky and the more
09:15 : the more they try to take on like the
09:17 : more ambitious
09:19 : that uh that project is the the more
09:23 : problems it tries to solve the the more
09:26 : complicated their internals are um and
09:29 : they're more likely that there will be
09:31 : some leaky like
09:33 : the harder job they have to sort of keep
09:36 : keep the reality from you
09:37 : right yeah i think it's the case by case
09:41 : choice it's like if you can work around
09:45 : not work around work within the leaky
09:47 : abstraction and get what you need to be
09:49 : done
09:50 : done and that's good but if it goes
09:53 : beyond
09:54 : what you need to be done it's leaky then
09:57 : you can't do it
09:58 : with this abstraction you have to either
10:01 : do something else or you implement it
10:03 : yourself or whatever
10:05 : yeah you have to look at it as a
10:07 : trade-off i think
10:08 : maybe a lot of people are too eager to
10:10 : jump into
10:11 : a cool framework that's like trending
10:15 : yeah it's my feeling yeah i see a lot of
10:18 : this
10:18 : in uh my company too the the folks just
10:21 : love
10:22 : the tools they are like too uh
10:25 : attracted by the tool like how fancy the
10:28 : tool is
10:29 : but they don't really want to know
10:30 : what's inside of it
10:32 : uh-huh yeah and that's that's like
10:36 : exactly what joe is talking about