Reverse Engineering with GDB

In this video I excitedly tell Huiqi about reverse engineering: my latest obsession.

Transcript

The following transcript was automatically generated by an algorithm.

00:00 : i was inspired by a couple of papers
00:02 : that i read
00:03 : repeatable reverse and repeatable
00:08 : reverse engineering for the greater good
00:11 : with panda this this
00:14 : is a cool paper uh yeah if you would
00:17 : like to read this
00:18 : uh i think it would open your eyes to
00:21 : something
00:22 : basically the technique they use is they
00:25 : run
00:26 : things inside of a virtual machine
00:29 : run run code inside of a virtual machine
00:32 : and by a virtual machine i don't mean
00:35 : like java virtual machine
00:36 : i mean i mean it's like a virtualization
00:39 : it's more like
00:40 : vmware vmware okay where
00:44 : it virtualizes the real machine
00:47 : it so what that means is it takes an
00:49 : executable
00:51 : file uh not even just the executable
00:54 : file like the whole
00:56 : the whole operating system itself
00:59 : first of all like if you have linux or
01:01 : something or windows it doesn't matter
01:03 : it takes the whole operating system and
01:06 : runs it
01:07 : inside a sandbox and it simulates
01:11 : the cpu that would be executing that
01:15 : code but but that's simulated in
01:18 : uh software instead of running directly
01:21 : in hardware
01:22 : why do they need to do that i don't i
01:25 : mean
01:26 : in the case of like in general though
01:29 : the reason why people do that well
01:31 : there's multiple reasons but
01:33 : at the very beginning it was just like
01:36 : hey i have a mac
01:37 : and i it would be nice if i could run
01:39 : linux also
01:41 : um so then or i have
01:44 : like a windows that's running on my
01:47 : computer as well because some of the
01:49 : time i might want to do some work in
01:51 : linux
01:53 : at the beginning that's why so people
01:54 : would just install
01:56 : vmware or virtualbox is another popular
01:59 : one
02:00 : and then people can have both operating
02:03 : systems or
02:04 : all three operating systems maybe um but
02:07 : another good reason to do it
02:09 : is um
02:12 : this might be another good reason to do
02:14 : it
02:15 : let's talk docker is a second let's talk
02:18 : about the docker use case
02:19 : but even before docker people were using
02:22 : it
02:23 : to scale out servers like okay
02:26 : or um or like you can just
02:30 : if you have a same setup for
02:34 : for how you want to configure your
02:36 : machine
02:37 : your then then you can sort of snapshot
02:40 : it and very easily spawn
02:42 : new machines with that exact same
02:44 : configuration
02:46 : yeah so i guess people use that to scale
02:50 : out a server farm that has a lot of
02:52 : servers
02:53 : running the same rest endpoint or
02:56 : something like that
02:57 : okay so if they they need to handle more
03:00 : load because they have more users
03:02 : they can just spin up more servers like
03:04 : that
03:05 : um
03:10 : oh another reason is like if you're a
03:13 : hosting company
03:14 : like aws okay then it's
03:18 : if you just instead of letting people
03:21 : share the same server by
03:25 : you know assigning different people
03:27 : different usernames
03:28 : into the same server and making sure
03:31 : they are playing
03:32 : nice with each other nobody is using too
03:35 : much of the computer
03:36 : at any one point um
03:39 : you just give each person a virtual
03:42 : server
03:43 : and then they can they can install
03:45 : whatever software they want
03:47 : without bothering anybody else
03:51 : gotcha so will this multiple virtual
03:54 : server
03:54 : run in the actual sam uh like hardware
03:58 : is that how it works yeah uh it does
04:01 : i don't know the details because i i
04:04 : haven't built any
04:05 : something like this before but i i
04:07 : imagine
04:08 : it's a mix between virtualizing
04:12 : and and running it running this
04:14 : instructions on the actual hardware
04:16 : because if if you simulated everything
04:20 : in software
04:21 : it's going to be much much slower right
04:25 : if you simulate it to cpu so i think
04:28 : to get better performance you probably
04:31 : want to at some point
04:34 : uh throw the code onto the actual
04:37 : cpu or maybe even on the fly
04:41 : translate some of the code in into a
04:44 : form that you want to be able to
04:46 : uh throw it onto the actual cpu but i
04:49 : don't know
04:50 : i haven't built something like this
04:52 : before or i haven't read enough
04:54 : into it but in any case the the
04:58 : people who wrote this paper uh brandon
05:01 : josh and patrick
05:03 : and tim and ryan these people they
05:07 : came up with a way of reverse
05:08 : engineering um
05:10 : like one good use case for reverse
05:13 : engineering
05:14 : is uh um
05:19 : security breaches so so there are
05:22 : hackers
05:23 : that are trying to hack a server and
05:26 : the way they do it very often is
05:30 : they install a program into your server
05:33 : okay maybe somehow they get access to
05:35 : your computer
05:36 : and then they they downloaded
05:40 : a program that they wrote into your
05:42 : computer
05:43 : and they got that program to execute in
05:45 : your computer
05:46 : okay and who knows what that
05:50 : program is going to do okay
05:53 : that's the purpose of reverse
05:55 : engineering is to figure out what that
05:57 : program
05:58 : did okay
06:01 : and so so the way and there's more than
06:05 : one way this is not the only way to do
06:06 : it but the way that these guys
06:08 : did it is they just grab that
06:12 : binary and or they even actually
06:15 : they suggest that if you run a server
06:19 : you should run it in a virtualized
06:21 : environment like this
06:22 : and then that way when
06:25 : there's hackers that are hacking into
06:28 : your computer
06:30 : they're just hacking your virtualized
06:32 : environment which you have more control
06:34 : over
06:35 : what the the extent of the damage that
06:38 : the hacker can do
06:40 : and then and then second you can
06:43 : actually
06:45 : because this system that they built
06:49 : um they basically is able to
06:52 : intercept every instruction that
06:56 : happens inside the virtual environment
07:00 : and be able to do some analysis
07:03 : on what actually happened they can trace
07:07 : um like for example if your hacker's
07:11 : program
07:12 : uh read the red the system
07:16 : password file to try to steal people's
07:18 : passwords
07:21 : or you can actually track that happening
07:23 : because
07:24 : in order to do that they would have to
07:26 : have make a
07:28 : system call to read that
07:31 : password file
07:35 : and question does that is that related
07:38 : to like how rr
07:40 : works like to check the system call and
07:43 : then to figure out
07:44 : yes what the code did something like
07:47 : that
07:48 : yeah it is related and which is why i'm
07:50 : reading the paper like
07:52 : i i did a while ago i did like this
07:55 : traversal of papers and this
07:56 : is one of the papers that was referenced
07:59 : by
08:00 : by that cloud of papers but yes
08:03 : that that's kind of the the way rr works
08:05 : is it it sort of
08:07 : um tracks all the system calls that the
08:10 : executable has has
08:14 : issued and then later on in the replay
08:17 : phase it'll play them back
08:20 : so rr draws draws
08:23 : techniques from this paper basically
08:26 : okay or or or other or the techniques
08:30 : used in this paper
08:31 : maybe i should say um which which is you
08:34 : you can
08:35 : actually replay replay
08:38 : what that attack program did and but
08:41 : as while you're replaying
08:46 : so so like it's kind of hard to catch it
08:49 : in the act
08:50 : because um because in order to do this
08:54 : kind of analysis
08:56 : it takes a lot of time to do so it'll
08:59 : slow down
09:00 : everything by a lot if you try to do it
09:02 : online
09:03 : so what you do is you do like what rr
09:05 : did you you
09:06 : capture enough information so that you
09:09 : can
09:10 : replay everything later
09:13 : and then while while you're replaying
09:15 : everything
09:16 : you do the analysis right at that point
09:19 : but while you're replaying
09:20 : and that's going to be much slower but
09:22 : that's okay you're doing that offline
09:25 : okay yeah so um
09:29 : but anyway uh i'm not talking too much
09:31 : but uh
09:32 : you can read this paper i'm not gonna
09:34 : give it all away it's
09:35 : it's it's pretty amazing stuff um but
09:38 : that's
09:39 : the part of the thing that inspired me
09:41 : to learn a little bit of
09:43 : about reverse engineering and um so i'll
09:46 : show you one of the
09:49 : one of the articles that i found let me
09:52 : let me grab that article uh
09:56 : i think the name was something rick
09:59 : harris maybe
10:00 : i believe yes um
10:05 : [Music]
10:10 : yep yeah this is was a good article
10:13 : um i i the best part about it
10:16 : is it linked to some example
10:20 : challenges for me to hack
10:24 : um so so
10:27 : uh yeah so so
10:31 : apparently like he this person i guess
10:34 : went to the hoberton school
10:38 : which is like a coding bootcamp i
10:40 : believe
10:43 : but but basically i i
10:46 : cloned this repo called don't hate the
10:49 : hacker hate the code
10:51 : and there's three programs that you
10:53 : you're gonna try to hack
10:55 : um so do you hack a program
10:59 : like what does that mean exactly i'll
11:02 : show you
11:03 : okay yeah so this one is like
11:06 : okay okay this is not a real program
11:09 : obviously it's just fabricated for the
11:13 : purpose of the exercise
11:14 : but you can imagine if this was a real
11:17 : program
11:18 : then it's like asking you for a password
11:23 : and you don't know what it is you try to
11:26 : give a random one it says
11:27 : wrong password so how can i figure out
11:30 : what the password is
11:31 : um there's multiple ways
11:35 : but there's multiple tools and if you
11:38 : want to know more
11:39 : you can read this article i he he talks
11:43 : about a few different tools
11:45 : i'm just gonna show one of them because
11:47 : of
11:48 : uh you know with i don't wanna spend too
11:50 : much time we have a limited amount of
11:52 : time
11:54 : i'm going to show you gdb which is
11:58 : the tool that a c developer c
12:01 : c plus plus developer used to debug
12:03 : their program okay
12:05 : but you can use it even when you don't
12:08 : have access to the
12:09 : original source code that's the point
12:12 : because
12:12 : gdb can also be used to debug assembly
12:16 : code
12:17 : okay so i'll use um
12:22 : gdb
12:24 : to do
12:27 : oh i i and i also use rr with this by
12:30 : the way
12:32 : um okay i'm not gonna use rr for now i
12:35 : might
12:36 : well i might rr allows me to go back in
12:41 : time which
12:41 : is very helpful so anyway for now i'm
12:45 : not
12:45 : i'm i might in the next try uh so
12:49 : so if i do gdb um
12:53 : i need to break i want to be able to
12:56 : break
12:56 : on the first line of the program okay um
13:01 : and luckily
13:04 : let me see i did info file that shows me
13:08 : some info
13:09 : about this file uh
13:11 : [Music]
13:13 : wait a minute that didn't really tell me
13:17 : anything useful but oh it told me the
13:19 : entry point
13:20 : it told me the entry point which that's
13:24 : the first line of the program
13:26 : um but that's not
13:29 : like i could set a breakpoint there
13:33 : so you are going to dig into the
13:35 : assembly
13:36 : yeah okay i'm going to dig into the
13:39 : and if i'm here i can say disassemble
13:45 : except there's no frame selected
13:48 : i think i can say
13:53 : i want to print the 10
13:57 : instructions starting at
14:00 : the program calendar register your note
14:03 : oh i need to run the program
14:05 : first that's why okay i forgot to start
14:08 : the program
14:09 : which is done with the run command
14:13 : oh everything has a shorthand so run
14:17 : s r is short for run so now i
14:20 : can disassemble
14:24 : and and this is the assembly code
14:29 : this is the assembly code and these are
14:31 : like these are
14:32 : these numbers on the left here they are
14:36 : the
14:36 : memory location where this
14:40 : these instructions is in memory
14:43 : and you can think of these almost like
14:46 : line numbers
14:47 : so if you want to set a breakpoint if
14:49 : you want to set a breakpoint on this
14:51 : line
14:51 : then you can copy this line number
14:55 : and you can say i want a brick there oh
14:58 : you have to put a star
15:01 : before so i'm going to paste that memory
15:04 : address and that's like
15:05 : i want to break at that line
15:08 : and then now i can continue and it'll
15:11 : it'll pause at that line that i just set
15:13 : a breakpoint into
15:16 : okay basically
15:19 : you basically have the original code but
15:22 : just in assembly
15:24 : format yeah yep it's it's exactly it's
15:28 : like you can compare it to trying to
15:30 : debug some javascript that's minified
15:34 : oh yeah i have to do that recently very
15:37 : often
15:38 : yeah yeah it's kind of
15:41 : like that um but uh
15:44 : in this case i actually know there's a
15:46 : main function so
15:47 : it's a as a shortcut i'm actually gonna
15:50 : set a breakpoint
15:51 : in the main function and then i'm gonna
15:54 : continue and now i'm in the main
15:55 : function
15:57 : and i can disassemble
16:01 : the code in the main function and then
16:03 : this is basically the source code in the
16:05 : main function
16:06 : okay so how do you figure out which line
16:09 : has to do with the password
16:11 : yeah yeah so like you could read this
16:15 : code
16:16 : and in this case um
16:21 : there's some jump statements there's
16:24 : some compare and jump statements
16:26 : and that's um that's usually
16:30 : where they're checking something so
16:34 : um they're calling this checksum
16:37 : function
16:38 : so that's kind of a clue here
16:41 : okay you might be calculating a checksum
16:44 : based on the password so you can drill
16:47 : down into the checksum function
16:49 : to figure out how the checksum function
16:52 : works
16:54 : and then here after they call the
16:56 : checksum function they're doing a
16:57 : comparison
16:59 : here so presumably they're comparing the
17:02 : result of the checksum
17:04 : with something and that something is a
17:06 : hard-coded number
17:07 : because because this is actually this is
17:10 : dollar sign
17:11 : means it's a literal it's a hard-coded
17:14 : value
17:15 : and literal for that value like this is
17:18 : a hex number
17:19 : i don't d d e e that's a hex number
17:23 : okay i'm gonna use my mac calculator
17:29 : so so the if you don't didn't know this
17:32 : is extremely handy if you're doing
17:34 : anything
17:35 : like you're trying to hack something uh
17:38 : so in the mac calculator there's there's
17:41 : the basic
17:42 : which is what you usually see okay then
17:45 : there's the scientific
17:48 : uh but i'm not a scientist i'm a
17:50 : programmer so you
17:51 : choose programmer
17:54 : so so you use the programmer calculator
17:57 : and then it knows all about
17:59 : hexadecimal numbers so this is saying
18:02 : i'm in the hexadecimal mode the base 16
18:06 : so i'm gonna put in d-e-e in here i
18:09 : could paste it or i could just type
18:11 : d-e-e
18:12 : and uh i can click on 10
18:16 : to convert it into decimal mode so i
18:18 : know that
18:20 : oh this literal number here is
18:25 : 3566.
18:27 : i wonder why they're comparing it to
18:29 : that
18:31 : okay what's yeah right how do you know
18:34 : why they're comparing to that um
18:37 : that might be a magic number so so what
18:40 : i'm gonna do is i'm going to
18:41 : pause on this line here i'm going to set
18:43 : a breakpoint here
18:47 : move this a little okay
18:51 : okay so i'm paused on this line and
18:54 : at this point i can actually look at the
18:57 : register
18:58 : the the value it's comparing to
19:02 : um the way i do that is using the x
19:04 : comma x
19:05 : means um examine
19:08 : uh and it and it actually examines
19:12 : a memory location and
19:16 : rbp is a register that holds
19:20 : that contains some memory address okay
19:23 : and here but here we're going to do an
19:26 : offset we're subtracting 8
19:29 : from rbp so i'm going to do the same
19:32 : thing here
19:35 : and so this basically says examine the
19:38 : value of the location at
19:40 : rbp minus eight okay
19:45 : and it's zero for some reason
19:49 : should it be zero
19:52 : okay so does that mean rbp is
19:56 : eight oh but apparently
20:01 : it hits that again oh it hits that again
20:09 : so i i did continue and then it stopped
20:12 : on this on this same line the compare
20:16 : line again
20:17 : okay and then uh but
20:21 : oh it hit it twice actually
20:24 : okay and they gave me the value of 214.
20:30 : um it gave me the value of 214.
20:34 : which is not equal to 3566
20:39 : so maybe if i can get this number
20:43 : to be 3566
20:47 : then i can crack it right okay
20:50 : right but how do you but maybe it has
20:54 : something to do with the password that i
20:57 : give to it because because i put in
20:59 : hello
21:00 : okay i put in hello it gave me 214.
21:05 : oh so you want to put in a different one
21:07 : and test out what
21:09 : number you would yeah um yeah
21:12 : i'll put in monkey this time
21:17 : and then i will i will uh
21:21 : let me let me look at how many
21:22 : breakpoints i have so
21:24 : info break i'm gonna remove the
21:28 : breakpoints that i don't need
21:29 : because i only care about this six zero
21:32 : zero one
21:33 : or because that is the one that's doing
21:36 : the comparison
21:37 : i i'm most interested in that one i'm
21:40 : going to get rid of the other ones so i
21:41 : can say delete
21:43 : one delete to
21:46 : delete three and i'm gonna
21:48 : [Music]
21:50 : i'm gonna go okay i'm pause there and
21:53 : i'm gonna print out
21:56 : the value inside that register by the
21:58 : way this
21:59 : memory location
22:02 : this memory location here that's
22:05 : that's basically a variable in in a c
22:08 : program very likely
22:15 : oh
22:18 : it's it's 293 this time
22:22 : so it's a different number when i put in
22:24 : monkey
22:28 : 0 again so what does that zero mean
22:31 : oh i think zero means when i
22:34 : i accidentally hit enter twice and then
22:37 : when i hit enter the second time
22:39 : it just went continue again
22:42 : i think oh yeah oh no i'm sorry
22:46 : it just did the same previous uh command
22:48 : again
22:49 : oh but i don't i don't know why oh but
22:52 : it did
22:53 : move
22:53 : [Music]
22:58 : oh wait interesting
23:01 : it might i think it might have
23:05 : i think what i did probably actually
23:08 : subtracted eight
23:09 : from the rbp that's probably why
23:13 : what i don't i wasn't
23:17 : i didn't mean to do that so you think
23:19 : you modified this right now
23:23 : i think i modified the register
23:26 : yeah i did i did but
23:29 : okay i i'm gonna in the future i'm not
23:32 : gonna do that
23:34 : i need to i need to find a way to show
23:37 : the number without actually modifying it
23:40 : okay
23:40 : but anyway um but okay so
23:43 : so what now i'm like you just okay
23:46 : so depending on what password i pass in
23:50 : i can actually change the number and
23:53 : we want the number to be
23:56 : the big number like this oh how are you
23:59 : gonna
24:00 : i'm gonna just put a lot of z's in here
24:04 : try to get the number to
24:08 : oh still wrong oh wait i missed it all
24:11 : right and let's try again
24:14 : oh i already broke there so i'm gonna
24:17 : look at
24:24 : and now it's b70 and b70
24:30 : we're getting close okay
24:34 : i think i i think as as soon as we get
24:38 : to the same number as
24:41 : uh d e e then
24:45 : it it's it's gonna work
24:48 : um so let's see how far
24:51 : are we
24:54 : now just put more z's
24:58 : under it
25:04 : c d e we're getting close to d e e right
25:08 : okay okay one more z
25:14 : uh okay what what's d
25:17 : e minus uh
25:20 : said wait d
25:24 : e is three five six six
25:29 : [Music]
25:33 : so all you have to do is get all of them
25:36 : to add up
25:37 : and uh and if we hit that number
25:40 : that then it'll work um actually
25:44 : yeah let's just stop here um
25:47 : we don't have to go through the whole
25:48 : exercise but um
25:51 : i get the point yeah you get the point
25:53 : so once we once we hit that number
25:56 : it it'll unlock the thing and work and
25:59 : um
26:01 : yeah i guess so what and that there's
26:05 : a couple of exercises uh that are a
26:08 : little bit harder
26:08 : than this one okay
26:12 : i guess what what's the point of this
26:14 : the point is
26:17 : the divine is like um like i
26:21 : i kind of was interested in this like
26:24 : like um i learned a little bit about
26:27 : assembly programming
26:29 : because because of the compiler stuff
26:32 : and
26:32 : um and uh yeah just just
26:36 : learning more about how gdp works like
26:40 : the fact that you can
26:42 : look at memory locations um
26:46 : the different ways of you know or the
26:49 : fact that it can disassemble assembly
26:51 : language
26:52 : and be able to walk the assembly
26:55 : language instructions step by step
26:58 : so that you can debug things even if you
27:00 : don't have the source code
27:02 : that's pretty interesting to me and
27:05 : there's there's
27:06 : actually like a lot of people that do
27:08 : this
27:09 : like in particular security researchers
27:12 : do this kind of thing they're trying to
27:14 : figure out how a
27:16 : virus or a worm
27:19 : works by executing the code and stepping
27:23 : through it
27:24 : um there's also people in the video game
27:28 : industry or maybe not like
27:32 : the people who are really into games
27:34 : they like to make modifications to the
27:36 : game
27:37 : okay uh and this is how they do it
27:40 : oh wow yeah they they walk through the
27:43 : game
27:45 : line by line or they look for stuff in
27:48 : like there's there's tools that you can
27:50 : use to look at
27:51 : this binary file that that that is the
27:54 : program
27:55 : look into it and you can identify hey
27:58 : that's an image
28:00 : right in there and then if you know that
28:03 : that part
28:04 : of the file is an image you can swap in
28:06 : your own
28:07 : image and then then you would change the
28:10 : game right
28:11 : yeah just change the numbers and then
28:14 : you can
28:14 : yeah you can you can use the text editor
28:17 : to to change the numbers and then you
28:19 : can
28:19 : so yeah this this stuff is pretty
28:22 : interesting to me
28:23 : um and because it's also very
28:26 : related to um what we're trying to do
28:29 : with the debugger
28:30 : which is um
28:34 : debugging is actually reverse
28:35 : engineering something
28:37 : if you think about it yeah that's true
28:39 : like
28:40 : you don't know how the thing works
28:41 : you're trying to figure it out um
28:43 : so the fact that there's people building
28:46 : tools
28:47 : to make reverse engineering easier
28:51 : that's that's very close to to what a
28:54 : debugger is
28:56 : yeah that's true yeah uh
28:59 : okay uh
29:03 : that that's a good stopping point there
29:05 : i think uh yeah any other thing
29:08 : maybe you can send me the paper so
29:11 : yeah i'll send you this paper okay this
29:14 : one
29:15 : yeah the the panda one yeah it's a it's
29:19 : extremely it's really incredible paper
29:22 : yeah and then uh and uh yeah if you read
29:26 : this
29:26 : there's another one that's even more
29:29 : incredible
29:30 : so maybe we'll we'll we'll save that for
29:32 : later
29:34 : all right i'll talk to you later then
29:36 : all right talk to you later bye