How do we host and run server software? Let's consider the progression we've been on over the
past 15 years or so. We've gone from software and operating systems that we manage running on
hardware that we own and babysit, to virtual machines, to infrastructure as a service on the
cloud, or even platform as a service on the cloud. And then on from there, we've moved to containers,
usually Docker, maybe running on someone else's cloud. After that, maybe we want to put these into
microservices, which are conglomerates of these containers working together managed by something
like Kubernetes. Well, where do we go from there? I can't tell you what the final destination of this
whole progression is going to be, but I believe we have reached a leaf node in this hierarchy with
our topic today. On this episode 118 of Talk Python To Me with Ryan Scott Brown, we're going to explore
serverless computing. It's an interesting paradigm shift, and I hope you enjoy the conversation.
It was recorded on May 24th, 2017.
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem,
and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm at
mkennedy. Keep up with the show and listen to past episodes at talkpython.fm, and follow the show on
Twitter via at Talk Python. This episode is brought to you by Rollbar and us at Talk Python Training.
Be sure to check out what we're offering during our segments. It really helps support the show.
Ryan, welcome to Talk Python.
Hey, Michael. Glad to be on.
Yeah, it's going to be super fun. We're here to talk about running code out on the internet with
no servers, which seems kind of impossible and awesome all at the same time. That's going to
be exciting.
Yeah, I mean, there are still servers. We'll get around to that.
Of course. Of course. Yeah, of course. Awesome. Before we do, though, let's hear your story. How'd you
get into programming in Python?
So I had kind of a pretty vanilla path. I actually started with my interest in networking. So I was
majoring in systems administration and network admin and actually got brought into the programming
world by a Perl class that I took. And then there was an open source club at my college that everyone
was using Python to write programs for the One Laptop Per Child project. And that was my introduction to
Python 2.
Oh, that's really cool. What school is that?
Rochester Institute of Technology.
Oh, yeah. Okay. That's a great school. I know a couple of people that have gone there and it sounds
like a pretty neat program.
Yeah, they have actually an open source software minor now. They didn't when I went, but the program's
kind of slowly evolved and grown over time.
Oh, my gosh. I've never heard of an actual official university program that is focused on open source.
Tell us just a little bit about that. Do you know anything about it?
What does a person who takes that degree do or that minor do?
It's not a full degree. So usually it'll be in software engineering or in computer science.
In my case, I actually TA'd for some of the classes that then were added to the minor. So we had a bunch
of pilot courses first, and then you get a certification from the university that you're a real minor, and then
you can have real students enrolled.
Awesome.
All of the courses focused around either open source software or the idea of open communities
in different ways. Like we had some Wikipedia editors that would come in and talk and be
TAs. We had contributors to the BSD project, one of whom lives in Rochester and would teach
one of our classes on open source software. And for course credit, you could do things like contribute
to existing open source projects.
All right. So that sounds like a really cool program that you were in in the university.
Now, what do you do now?
Yeah. So now I actually work for Red Hat, who obviously makes all of their money on open source
software and support and consulting around that. And the team that I work on is the Ansible
team. So we make a engine and a kind of language that people write in so that they can orchestrate
all of their servers and cloud services and network devices and all that from a simple YAML
language. So that's not a full fledged programming language. So you don't need the kind of experience
you'd need to write things like chef recipes, which are Ruby. And so you'd have to know Ruby
to write chef recipes. So we try and make that as accessible as possible.
Yeah. I've heard really good things about Ansible and that sounds fun. And Ansible is a thing that
recently came to Red Hat, right? Like recently acquired that project?
Yep. About a year ago. Well, a little more than a year ago now, Red Hat acquired Ansible
Works, which was the company that was behind Ansible, the open source project. And so now
Red Hat sells Ansible Tower, which is a web interface that previously Ansible Works sold on top of
Ansible. And then the Ansible core is still open source and just as kind of community focused as
ever.
Nice. Which side did you come from? Were you working on Ansible before or were you at Red Hat
and then it came there? I was using Ansible, but I was working on OpenStack, which is a
series of open source projects also all in Python.
Yeah. That's quite a big Python project actually.
Yeah. It's actually, I think 50 big Python projects.
Yeah, exactly. It's an understatement to call it large, right?
Huge contributors both to obviously the Python community in terms of bringing things back like
PyPI and other things that become things in the regular Python community, as well as to
the core language. There are a lot of core Python contributors that work on OpenStack.
And so split their time between making Python better and then making things on Python better.
Yeah, it's definitely a symbiotic relationship there. That's great.
So let's talk about this idea of serverless programming. I feel like we started out with just
having machines in data centers, maybe co-locating some of our machines in a data center to virtual
machines, to things like Docker and things getting sort of smaller and smaller. And now we've reached,
I'm not sure you can get much smaller than a single functions almost running on the internet,
right?
Yeah. I mean, that's sort of the direction that we're moving. And it's been a direction in programming
for a very long time from when you had a system that just ran one job at a time, like on punch cards. So
you toss in a stack of punch cards and it would churn through them and then churn out results also on
punch cards. You didn't have multitasking at all. So one program was exactly what was running. So you
were, you had the whole CPU to yourself and then you have multitasking operating systems. So you had
multiple people providing code that would run in different processes that would be isolated in a certain
way. And the operating system would handle scheduling those. And you can sort of think of
function as a service, which is things like AWS Lambda and mostly called serverless, quote unquote. You can
think of those as like a multi-user scheduling, but where you have untrusted entities sharing
compute resources. So you're getting smaller and smaller slices.
Right. Oh, so that's really interesting to track that trend over time, right? From just
sort of OSs to, to running these functions on other people's computers. So obviously there
really are servers out there, like you said, but when it's not your responsibility, it's not
your problem to like babysit these servers. So for example, yesterday, I think I logged into
some of my servers with SSH and I had to update like the unattended install software package on Linux.
So even though of course your code runs on servers, when you don't have to think about it or manage it
or balance it, it really makes a difference, right? Yeah, exactly. It's serverless in the same way that
wireless doesn't have any wires involved. There are still wires. They just aren't directly to your device.
Right. Well, and you know, in wireless, you never care about the wires. Somebody has to care about the
wires, but it's opaque to you. Right. And so as far as you're concerned, wireless is wireless.
Same analogy here. As far as you're concerned, you get particular promise from your provider. Like
we will always have such and such version of such and such underlying dependencies like libc and things
like that, that you would statically link to and a such and such version of a Python interpreter
available and such and such version of whatever else you need. And you would write your code such that
you can rely on that such and such version being there, but you don't install that. You don't deal
with the sort of provisioning that you would do to install. Let's say you need a specific Python 3.
So if you only can run on 3.5 or up, you don't have to worry about making sure that's there. You just tell
your provider, give me this runtime, please. Right. And it just works and you have to worry about it. That's great.
So can you compare this to like, say working with like event driven programming versus like hooking up
to cloud events that, you know, something like Erlang? Yeah. And this is kind of the other side of
serverless. The one side is run this code in the cloud for me. And the other side is hook my code
up to these other things. So real world or virtual world, I guess, events that you want your code to
active on. So let's say you're in your podcast world. You have something that you want to happen
every time a sponsor contacts you. You could hook up a way that when the sponsor fills in the contact
form, it invokes a Lambda function that then maybe calls you if it's super important to get back to
them immediately or any number of other things. But the idea is the event source half and then the run
my code somewhere that I don't have to manage half is what kind of makes up serverless.
Right, right. It's easy to focus on here's a function run it, but really the diverse triggers
that trigger those functions are super important. Yeah, exactly. And it maps to a lot of event driven
concepts that we might be familiar with in Erlang where you have processes that are managed by a
supervisor. So in serverless, that would be individual functions that are managed by your cloud provider,
which under the hood deals with scheduling those across diverse hardware and all that stuff.
And then in Erlang, you also have an event routing system where every process has input and output
kind of event hooks. And you can get that similar thing in Lambda with using APIs to send data out.
So you might send data out over like a MailChimp or a Mandrill, or you might send data out into a
database. That would be your kind of output. And then your input can be triggers from things like
DynamoDB. It can be HTTP events, any number of things. I'm not going to list them all because
this would be a long podcast. Yeah, yeah, absolutely. And you know, to bring it back to like, what could I do?
You know, you're saying something happens. Like maybe I've, I'm going to upload my MP3 file that I want to
release for the week to S3. And that needs to be, you know, maybe have like the artwork and the little,
like the description embedded into the MP3 header. And then it needs to be moved over to the content
delivery. I got to like flip it to publish all these different things. Like I could possibly set that up
serverless, you think? Yeah. And individually, you would have something like the S3 event source. So you would tell
a Lambda function that when a new object appears in S3, and then S3 would notify your function when
that happened, it could download from S3 your file. It would put in the, I forget what the name of the
MP3 metadata format is. Yeah. It would dump in whatever author metadata, show notes. I don't know if that
goes in there. Yeah. And then it could save it to another S3 bucket that would either kick off yet
another Lambda and you'd have this chain, or that would be the S3 bucket you're serving public content from
to your CDN. And it doesn't have to be linear either. You can have kind of forking things. So
you could have this Lambda see, Oh, a new shows up here. I'll put the metadata in and I'll also kick off
the job that will add it to the list on the front page. And I will kick off the job that will add it
to the RSS feed. And those can be a bunch of Lambda functions that each have a small focused purpose.
It sounds really amazing to think of how that kind of opens it up. And to me, it feels like almost like
the event sources and the workflow bit is more important than, Hey, run this function.
Yeah. Because I mean, when you think about it, we've been really good at running small bits of
code for a very long time. And a lot of the value comes in when you can hook those small bits of code
up to something that's a useful thing in the real world. Right. Cause I could make a little program
that simulates a vending machine that will, you know, take pretend quarters and things, and I can type in on
the terminal, but until I'm hooking that up to hardware, that's actually giving someone a soda
or whatever, it's much less useful. Right. Right. Absolutely. Absolutely. So you're talking about
S3 and I'm assuming that that probably ties right back into AWS Lambda, which is probably the most
popular one of these, but it's not the only one. Like I think there's Azure functions, for example,
what are, what are some of the implementations and places where we might find this type of
programming model? There's a load of different providers. There's obviously the big three cloud
providers all have their offering GC or Google cloud platform has what they call cloud functions.
Microsoft Azure has what they call Azure functions. AWS has Lambda, which is a kind of a pawn on function,
right. Nothing to do with, Python Lambdas, right? No. And I, yeah, that's been a problem
sometimes cause you'll pull something into Python to automate. So if you want to automate the deployment
of a Lambda function and you pull that into Python and then you name a variable Lambda, you can have
problems if you're not careful. So why is this keyword gone? I don't know if that actually happens.
There we go. It does. You can override keywords in Python. So you can override true, false, Lambda,
all kinds of things that you should. Yeah. These are not good. And recently I can't speak to the
other cloud providers. I just don't track them that closely, but AWS Lambda recently switched to
Python. Well not switched, made available Python three six, which is pretty cool. Yes, that is
absolutely right. And it has been great. Yes. Cause surprisingly until like a month ago,
this was a Python two only option, right? Yeah, exactly. And I mean, if you think about when it came out,
Lambda was out, I believe in towards the end of 2014 when, you know, Python three has been out for
six years at that point. Yeah. That sounds about right. And I would love to be able to go run a
second experiment and release it with Python three only and see what that does for adoption. Yeah. Yeah.
Yeah. That would have been awesome. These would just be really interesting experiments to run,
but for these other providers, there are some smaller providers that only do JavaScript. So,
Auth0 is an example of one that has a function as a service, but it's JavaScript, Node JS only.
Mm-hmm.
If you look at IBM's OpenWhisk, you can actually provide an arbitrary Docker container as the runtime
for your function. And so it manages the scheduling and it will invoke as many as needed to handle events.
So it's still got the function as a service going for it, but you get a lot more control over what
run times you give it. And so you can not only do Python three, but you can do entire Ruby installs.
You can do Apple Swift, any language that you can fit in a Docker container, which is any language.
Right. You could probably do something like Fortran if you really wanted to torture yourself,
right? Like you could do basically if it can accept a request, you can do it, right?
Yeah. I mean, as long as it accepts, I believe the rules for OpenWhisk is it has to accept data on
standard in and provide a result on standard out when the container is invoked as a process.
And so as long as you can do those things, you're good to go. So COBOL in the cloud.
How interesting. Yay. Let's get more COBOL in the cloud. So one of the things that immediately comes
to mind when I think of this and the Docker variant obviously would be one way to basically get an escape
patch errors. Like how much do you have control over dependencies and requirements? So for example, what if I want to
use like NumPy or requests or something that doesn't ship with Python? Yeah, sure. But I want to use that.
Like, can I go to AWS Lambda and go, Oh yeah. And like, here's my requirements.txt make that happen.
Unfortunately, you can't just upload a requirements.txt as much as I would love to. You can with a Google
cloud functions, you can give them your code and a requirements.txt and they'll do it for you.
That's pretty cool. In Lambda, you provide a zip file, which contains, can contain any dependencies
you want, whether that's C object libraries, if you're using something that links directly to C like
NumPy, or it can just be Python files like requests. So you would use pip on either your local machine or a
build server that would build in all of your requests and pack them into a zip file. And I've
had to do some backflips to get SciPy and SciKit learn working in Lambda. Wow. But you've, you've done
it? No, it absolutely works. And the nice thing is that AWS now provides a Docker image that duplicates
the Lambda environment. So you can download that Docker image, build stuff in there. So it's all going to be
built just right. And then dump it to a zip file on your machine and then upload it to Lambda. So you can
have exactly the same build environment locally for Lambda. And then you build all the SciKit learn
and optimize it for that hardware or for that, that environment. And then you can strip out
everything that isn't needed. So you can do things like compact the .o files. You can strip out a bunch
of a TXT and MD and RST documentation files to get your size down because that relates to how fast your
function can run. Yeah. Cause it's got to somehow take that thing apart and work with it. Right.
I mean, it's got to unzip it, but it's also got to download it to whatever machines running your
code. Cause they don't all have it. Right. Right. Absolutely. Does it get, like warmed up as it
runs? Like how many machines might be involved in executing your code? No. And does it, if it's
running a while? Yeah. It depends on your number of invocations. So the way that it works under the hood
is they spin up basically a container that will handle events and each container will handle one event at a
time. So let's say that you have five events per second and they all take half a second.
Okay. Then on average, you're going to have about three hosts running or three containers handling
requests because if they each take half a second, you've got two that are fully utilized and one
that's half utilized. If that makes sense. Yeah. Yeah. That makes sense. And so if you have a bunch
of events that come in in a very spiky kind of way, so you have 500 requests this second and then
nothing for 10 minutes, it's going to spin up on a bunch of machines to handle these events and then
they'll all spin down basically. And eventually they get evicted if they're not used. What's the
payment model for this kind of stuff? How much difference does it make from say like, I'll just
fire up a VM and just run my stuff there relatively. Yeah. That depends a lot on what your utilization
level can be. So if you get, let's say you have a VM and you get one request a day, you're paying for
24 hours of computer to handle 50 milliseconds or a hundred milliseconds or something. Yeah. And whereas
in Lambda you pay by the unit, I believe is the megabyte second of memory. And that also scales your CPU.
So they have tiers of anywhere from 128 megabytes of RAM up to, I believe a gig and a half. And each of those
is build per hundred millisecond slice by the tier. Yeah. Let's suppose I've got a function that doesn't
really use much memory, right? It's pretty basic and it's run five times a day for 10 milliseconds.
Is this like a penny, a dollar, $5? Oh, that's in the hundreds of pennies. The way I had a blog post that
I calculated out the cost to quote unquote more sensible unit, the, Pico dollar per bite second.
Okay. And if you run, I believe it's, you get a million invocations for something like 14 or a million
milliseconds for something like 1.4 cents. Wow. And so there's a lot of, I mean, you get,
it's more expensive if you were to run a 24 hour EC2 instance and then run a 24 hour Lambda function,
which you can't because there's a timeout, but if you ran them one after another for a total of 24 hours.
Like if it was under a super heavy load, that was basically equivalent to a continuous, right?
Yeah. If you were to run 24 Lambda hours, if you will. So if you run, you know, 24 of them all for one hour solid,
then that would be the equivalent of a little more expensive than the equivalent EC2.
But that would be assuming that you can fully utilize your EC2 instance, which is not all that
common for most workloads because you've got kind of extra capacity to handle random spikes and
variations in what users are doing. So if you're going to get a little bit more traffic, you don't
want to spin up a brand new server every time you get just slightly over the threshold. Right.
Yeah. Yeah. That makes a lot of sense. Whereas Lambda can match very precisely,
you know, one-to-one with the events that you've got.
Hey everyone, Michael here. Let me take just a moment and thank one of our sponsors who makes
this show possible. This portion of Talk Python in May has been brought to you by Rollbar.
One of the frustrating things about being a developer is dealing with errors,
relying on users to report errors, digging through log files, trying to debug them,
or a million alerts just flooding your inbox and ruining your day.
With Rollbar's full stack air monitoring, you get the context, insight, and control you need to find
and fix bugs faster. Adding the Rollbar Python SDK is just as easy as pip install Rollbar. You can start
tracking production errors and deployments in eight minutes or less. Rollbar works with all the major
languages and frameworks, including the Python ones like Django, Flask, Pyramid, as well as Ruby,
.NET, Node, iOS, and Android. You can integrate Rollbar into your existing workflow, send error alerts to Slack
or HipChat or automatically create new JIRA issues, pivotal tracker issues, and a lot more. They have a special
offer for Talk Python To Me listeners. Visit talkpython.fm/Rollbar, sign up and get the bootstrap plan free
for 90 days. That's 100,000 errors tracked for free. But you know, just between you and me, I hope you don't encounter
that many errors. Give Rollbar a try today. Just go to talkpython.fm/Rollbar.
There's all these different event sources. I understand how AWS can invoke our function.
If like say an S3 thing changes, it's all like inside AWS, but you could wire up even like an API to this thing,
right? Like if somebody hits this URL with a post with this JSON body, run this function, right?
Yep. Just like you would put a handler in your Pyramid or Django application, you can attach a Lambda function
to a, what's called in AWS an API gateway. In other serverless platforms, they call it, I believe, a HTTP event.
But it's all the same. Basically, they take, they have something that's running all the time that's waiting for HTTP requests.
And then when someone hits it, it will invoke your Lambda function, get the output, and then send it back.
And you only pay when people are actually using it. So you can run HTTP API that's available that costs you pretty much
nothing in base cost. It's just for requests.
Yeah, that's pretty awesome. And can you do things like map, like SSL certificates and custom domains and stuff to those URLs?
Do you know?
Yep. Yep. You can do custom SSL certificates. You can do custom domain names.
And in your function, you can figure out what domain name it was sent to. So you can do special things in your template, for example.
Let's say you have a short domain and a long domain, but they both show the same site.
If your Lambda function is rendering things or rendering links, it can give the correct URL back.
Yeah, I see.
By looking in the event and seeing all the information about where it was sent and all that stuff.
Okay. That sounds great.
And it has access to all the data sent to it, right?
Like the URL, the query string, the post body, the headers. Is that possible?
Yeah, absolutely. But there's kind of two ways you can do it in Amazon's API gateway.
You can use what's called the Lambda proxy integration, which gives you this giant JSON event that has everything in it.
So it'll have the forwarded by, the X forwarded for, all any other HTTP headers like auth and the query string and the path and everything.
Or you can use subset language that AWS calls a velocity template language.
And you can select very specific little bits.
You shape the event that you get in your function to only the stuff that you need.
And this means that your function runs faster because it's decoding less data.
And there's less kind of a security attack surface area because you're only letting through one or two little things.
And then that's happening before it gets to your code at all.
Right. You don't want to.
There were those mass assignment injection attacks, for example.
Yeah. Or you could just have someone who sends a massive regex or something else that's hard, particularly expensive to decode.
And make your Lambda function slower and, you know, cost you more money in the end, right?
Right. Yeah. It's interesting to think that the more we use these cloud resources on a consumption-based model,
how distributed denial of service has a direct monetary component.
Yeah. I mean, it's more like a banking denial of service.
Yes, exactly.
So if I have, like, my VM running at, say, DigitalOcean or something, and somebody decides to attack it and pound on it,
well, it may degrad or even kill my service.
But I'm still going to pay the $10 a month or whatever I pay, you know what I mean?
Whereas this, it could vary, right?
It can vary, but there are limits.
AWS puts in place two kinds of limits.
They call them, the first ones they call safety limits, which are relatively low just so that you don't outbill,
you don't bill yourself out, right?
Right.
And then they have what are called soft limits, which are limits that they say,
okay, most of our users never hit this limit, but the ones that do can just call us and we'll raise it right up for you.
And then they have hard limits of services that there's some technical limitation where they just can't go above, you know, 40 gig Ethernet, for example.
Sure.
Okay.
That makes sense.
And, of course, in AWS, I'm pretty sure the others have this as well, but you have, like, billing alerts.
Yep.
Right.
You have billing alerts.
You can also monitor specific things about Lambda.
And that's another thing that's really nice in serverless is the provider needs to monitor really well to bill you correctly.
And so you also happen to get really good monitoring because they need good monitoring to bill you.
Right.
And they just surface that for you.
So that lines up real nice.
Oh, yeah.
That's cool.
So maybe that's a good place to look, maybe compare and contrast with traditional web frameworks like Pyramid, Flask, Django compared to, like, this programming model.
I mean, obviously, the way you set up the server is different.
Like, you don't deal with N2NX and whatnot.
But, you know, sort of the paradigms, what do you think?
There's a lot of things that you don't get.
So, for example, you can't just have some super long-running API request because, for example, Lambda has a five-minute timeout maximum.
But usually you'll set that lower so that you don't bill yourself out or so you don't cost yourself too much money.
Because, you know, if your average web request doesn't terminate in five seconds, the user's gone anyways.
So you want to just stop that.
And you also get a lot more control over what's shared and what's not.
So in Django, a Flask, or a Pyramid, you have sort of a shared state that's internal to the server that isn't persisted out to a database, for example.
Right.
Maybe some in-memory static caching and stuff you pre-computed at start and you just can reference that, right?
Yep.
So you can build up pretty expensive caches locally that in Lambda don't make sense to do quite so much.
And so what you would use for that is some other really fast storage system like DynamoDB or a Redis or even Elasticsearch.
Sure.
All of these things give you a really low latency way to just get data back.
And then that would be accessed by your Lambda functions.
And then you would be able to get stuff really quick.
And so that's different in that you don't have the persistence.
And then the other thing is that you don't have the same limitations on language boundary.
And I know this is a Python podcast.
So we'll flip around this example to imagine that if you're writing something in Express, which is a Node.js framework, and then you want to write a function Python because you like Python more.
You're kind of SOL unless you make a new microservice.
Right.
Whereas in Lambda, you can go as granular as you want with languages.
So I can say, oh, well, the user create endpoint is in Python 3.6, but our profile image generator only runs in 2.7 right now because I haven't gotten around to it.
And so you can actually make a more granular migration between languages because you're doing one feature at a time.
I see.
You upgrade a function at a time and execute a function more or less in its own isolated environment, right?
Yep, exactly.
And then if you compare that to a Django project or something, then you've got one Python interpreter for your whole app.
So everything either has to be Python 3 or nothing can use Python 3.
Right.
It's all or nothing.
Yeah.
Okay.
That's pretty interesting.
So the granularity is really cool.
And then you also get the ability to make your dependencies separate.
So if you have certain dependencies that you only want to run against very trusted data, you can make those only in the functions that are invokable by very trusted things.
So you get a lot more security firewalling, not literal firewalls, but a lot more compartmentalization between functions.
And you can even do AWS resource permission distinctions between functions.
So you can do things like, say, this function's allowed to write to S3, but it's the only one.
Everyone else is denied.
Right.
Okay.
That's actually pretty awesome.
Because, again, in like a traditional whiskey app, you have to put those walls up yourself.
And it can be tricky because it's still the same memory in the end anyway, right?
Yeah.
I mean, regardless of how tricky it is, it's just, it's easy to make a mistake.
Sure.
Or to accidentally add an endpoint that probably shouldn't get right to S3 and then so on and so forth.
Sounds like serverless code might be a little bit more safe by default.
It's as safe as you make it.
Yeah.
You can just say, oh, I'm just going to give all my functions full admin and they're just going to execute arbitrary Python that comes in off the internet.
You can just eval every request.
Here, give me that pickled object.
I'll work on that.
No problem.
Yeah.
I also take arbitrary pickled objects and just, yeah, let's go.
That'll be fun.
Let's try that.
But you can do a lot.
There's a couple of really good talks.
One is gone in 60 milliseconds as an example of how even in a serverless context, if you over permission your functions, attackers can still get things that they shouldn't from your Lambda functions.
Of course.
Yeah.
It just sounds like it might be a little easier to exercise some least privilege.
Yeah, definitely.
Type of stuff here.
Okay.
So talking about the dependencies and persistence and caching and things like that, it sounds to me like to really, if we're going to have kind of complicated programs that are
running in this serverless architecture, you kind of need to go a little more all in on the cloud providers.
Like, let's just stick to AWS because we've been talking about Lambda, but, right, this applies generally.
So AWS has DynamoDB and some kind of caching.
I'm guessing Redis.
I haven't played with their Redis option.
Yeah, they offer Redis or Memcached as a service.
Yeah, exactly.
So there's that.
There's RDS.
Storage would go to S3 instead of the file system.
Right.
So do you feel like to be effective with this stuff, you kind of have to go a little more into the various APIs?
Whereas I could use EC2 and, like, basically forget I'm on AWS.
Yeah, yeah, you could.
The downside of doing EC2 and you forget that you're on AWS is that you've, you know, forgotten Amazon's, you know, zillion man years or a zillion developer years that have gone into creating all these higher level services that are basically commodities.
So things like S3, oh, store this blob for me and then let me get it later.
When you're running just on EC2 and you're storing it to disk, you need a backup strategy.
You need to make sure that if that server goes down, it's still available.
So you have to do the replication.
And S3 is just one example, but you have all these services that make your life easier.
And so you do have a trade going on.
So you can choose to use as few provider-specific services as possible, but then you don't get the benefits of using those.
So the example that I like to use is there's an online training company called A Cloud Guru that they built their prototype over about a week on Lambda and using Firebase.
Okay.
And the downside is that they would be locked into that forever.
The upside is that if they couldn't have done that, they couldn't have started.
And so they wouldn't even exist, right?
Right.
And so every time you get up, you're making a trade between going and killing your own hand-raised, going out and growing your own food versus getting it from someone else where you're locked into that provider somewhat.
So you're making a trade-off between what you're able to do in a short amount of time versus how easy it would be to switch to another provider, basically.
Right.
Of course.
And it doesn't necessarily mean you have to stick with serverless, right?
Like you could go and use RDS and Redis and S3 and then switch to EC2 and still use those, right?
You're just kind of just stuck to AWS at that point, but not to Lambda.
Yeah, because a lot of the services that you'll use alongside Lambda, you would use from a traditional application.
So you'll see people that write web applications and then some of the functionality is in Lambda because they didn't want to deal with something like a resource overrun on that particular item.
Or they wanted a special event source that wasn't an HTTP type event.
Or they just liked that context better because it's a language they don't normally work with.
There's all kinds of reasons that you would have kind of a hybrid.
Right.
Of course.
I have a lot of faith in these cloud providers.
Like they very rarely go down.
And, you know, when they do, it's usually really, really short.
But what if you wanted to have some flexibility to say move?
Like maybe could you speak to lock in a little bit?
Everywhere you're going to go, you're going to have some lock in, whether that's just the time it would take to move your data or the time it would take to write your app to make changes.
An example of not very lock in lock in would be something like file systems on Linux because you can switch between X4 and ZFS.
And that's, as far as your code is concerned, the same because they both provide the same interface.
Whereas when you migrate from, like you've talked about on previous podcasts, from Postgres to MongoDB, you've got to do a change to your code to deal with the different modeling that MongoDB does of your data and queries versus what Postgres does.
Right, exactly.
Yeah, that took a couple days of work and a few bugs you had to hunt down, right?
Yeah, that was different, of course.
Yeah, but you made it.
So, and serverless is the same way.
You're going to be embedded in whatever cloud provider you're in because in most contexts, you kind of want to be embedded like that because then you take advantage of their work so that you don't have to.
Right.
But if you're worried about transitioning cloud providers, there are a few things that you can do, like never handling an event directly.
Is you always transcode it into a sensible format.
So instead of relying on the Lambda proxy event format, you have something that transcodes that into just the stuff that you need in a format that makes sense to you.
So that way, at least, you just have to rewrite these shims and then your internal code makes sense.
Sure, that makes a lot of sense.
Or your internal code still works.
Right, right.
When you rewrite the shim and to handle the, say, the Google format of the event or the Amazon format of the event.
Yeah, I guess, you know, probably some proper architecture makes a lot of sense here.
Like, for example, your primary code could directly write to S3 or it could call some other code that says save this file to my thing, wherever that is.
And then that could be implemented to do it to S3, it could be implemented to Azure Blob Storage, whatever, right?
Azure actually has a really cool implementation of this that I like a lot that I hope that more providers will copy.
You make a trigger, they call it a trigger, an event source.
And you can have Azure Blob Store be the trigger, or you can have something like Dropbox be the trigger.
And what it'll do is it will pull down that file, put it in a temporary directory where your function is running, and then invoke your function and tell it about the local file path.
Oh, nice.
So you don't actually deal with, like, the Dropbox or Azure Blob Storage APIs to get files.
It puts them in a local directory for you, and then you use your language's regular file support.
So events like that would be amazing for more providers to implement.
And I think they've done a great job there.
Yeah, yeah, that actually sounds really quite interesting.
Another thing that I guess I see is a bit of a challenge for serverless is what if I want to work on this locally, right?
Like, I want to just fire up my local CPython and run this code and see what happens, you know?
Like, maybe I'm on a plane, and I want to, you know, do a little bit of work before I get to this conference for something I'm doing or some customer demo or something, right?
And it involves this app.
Like, what's the story on local or offline or any of these things?
You've got some options.
For local, there's the kind of what I would call the first-degree local, which is you can develop locally and deploy to a dev environment that just takes your code directly.
So that's kind of first-degree local, but the code's actually not running on your machine at all.
If you want more than that so that you can have a faster feedback loop, because deploys do take time.
Not a lot of time, but time.
You can use something like Atlassian has a thing called local stack that will fake out API Gateway, Kinesis, Dynamo, S3, and a bunch of other services on your local machine.
So they'll be on local ports.
And then you can run code that would use AWS services and point them at your local, local stack.
Nice.
Yeah, maybe just change, like have a different, like a dev local config that has different endpoints.
It would be a config change that would point your code at those local services.
So you can then develop locally, like on a plane.
So once you have local stack downloaded, you would be able to do that.
Other thing is plugin for the serverless framework, which I'm a contributor to the serverless framework, but not to this plugin.
That it runs Docker locally and will use that container image that I talked about and simulate different event invocations to your function.
And so you can either use that with, they have a built-in thing for the DynamoDB developer mode, which is a little jar that you run that puts up a fake DynamoDB.
Or you could use that in conjunction with local stack to get the runtime and the service simulation together.
Yeah, I see.
Yeah, I wonder how much Atlassian came up with local stack to like solve their own local cloud testing problems and just open source it.
Do you know?
I don't actually know the people that made local stack, but it definitely seems like the kind of thing that someone Atlassian size would say.
We have so and so many developers and it would save us so and so much time to not deploy every time they want to test something.
So let's take that time that we're going to save and invest in this tool that's going to, you know, make our developers a lot happier down the line.
Yeah, that actually made me think of continuous integration and testing and verification and stuff like that.
Like, how would I basically verify my stuff in a CI style?
Yeah, so there's some things that are hard, like simulating actual chains of lambdas.
So let's say you have that podcast lambda that we used as an example earlier where it takes an MP3 file, it does stuff, it puts it back in S3.
And then the next function picks up the new file and does something else and so on.
It's hard to simulate that chaining, but because each function is pretty focused, using a regular testing framework to test, given this input, what happens?
And there are several libraries that will pretend to be AWS for you in Python unit tests.
I really like placebo, which you can run it and it will record your interactions with AWS, the interactions of your code, and save them all in order.
And then you can rerun it in playback mode.
And so it will insert itself before your calls go to AWS and just send back the recorded response.
So you can make sure that things are called a certain number of times and that your code handles these responses from AWS the right way.
Okay, that sounds cool.
You'd use unit level testing for that and then you'd have an integration level test where you deploy to a staging environment and then you have scripts that exercise kind of the full life cycle with all of those AWS services that you can't simulate locally or want to make sure that your local simulation isn't different from the current AWS behavior.
Right.
It sounds like mocking might be an important part as well for certain parts.
Actually, with placebo, you don't have to do the mocking because it injects itself into the AWS client library and sort of does that mocking under the hood for you, which is pretty cool.
I see.
Yeah.
It's involved.
You just don't have to write it.
Yeah, it's involved, but you don't write it.
Nice.
I guess that's mockless now.
Yeah, it's mockless.
Mockless unit testing.
This portion of Talk Python To Me is brought to you by us.
As many of you know, I have a growing set of courses to help you go from Python beginner to novice to Python expert.
And there are many more courses in the works.
So please consider Talk Python training for you and your team's training needs.
If you're just getting started, I've built a course to teach you Python the way professional developers learn by building applications.
Check out my Python jumpstart by building 10 apps at talkpython.fm/course.
Are you looking to start adding services to your app?
Try my brand new consuming HTTP services in Python.
You'll learn to work with RESTful HTTP services as well as SOAP, JSON, and XML data formats.
Do you want to launch an online business?
Well, Matt McKay and I built an entrepreneur's playbook with Python for Entrepreneurs.
This 16-hour course will teach you everything you need to launch your web-based business with Python.
And finally, there's a couple of new course announcements coming really soon.
So if you don't already have an account, be sure to create one at training.talkpython.fm to get notified.
And for all of you who have bought my courses, thank you so much.
It really, really helps support the show.
We talked about developing locally, but there's also some other tools that just help with things like deployment
and something called Zappa, which basically as soon as AWS Lambda switched to Python 3,
Zappa's like, hey, we're Python 3.
Because it's like running more or less on top of Lambda.
And there's some others as well.
Do you want to talk about those?
Yeah.
So Zappa is a project that will take your WSGI-ish app.
So it'll take things like API star, now that it supports Python 3, Flask, Django, Pyramid,
and wrap them up in its own kind of fake WSGI that takes the Lambda API gateway events
and will put them into the request object for that web framework and then give it to your function that would work as a Django app,
but is now inside of Lambda, but it doesn't know.
Right.
So you write code as if it were Django or something.
So you pretend that it's Django and then Zappa handles packaging up each endpoint and associating it with API gateway
and then uploading that code to Lambda and hooking it up.
And then you get your Django-ish endpoint.
Or, well, it is a Django endpoint, but now running without Django actually serving the connection.
Interesting.
Yeah, because you basically, when you start these apps normally, you say, here's my WSGI app.
And that's the implementation of WSGI is like, you've received a request, right?
So they just have to adapt to, hey, you've received a request.
It just happened to have come not from a WSGI server, but from somewhere else, right?
Yeah, that's exactly right.
Because you're getting the same sort of data about the request in a different format from API gateway.
And so what Zappa does is it makes that format match the format that WSGI expects and then gives it off to your code.
Okay, cool.
And that'll do deployment as well.
So you say, Zappa deploy, and it'll deploy it up for you.
And it will also let you do some local testing and things of that nature.
Yeah, I wonder if that actually makes it more locally testable because you could just run it as a Django app or...
I think you're still really bound to those services.
So I don't know how much that really helps you.
Yeah, I guess you're right.
Because it's really that you're trying to get to that S3 bucket.
You're trying to get to that RDS instance and so on, right?
Yeah.
And for example, you might have a thing that you're calling that's only available inside of your VPC.
which is a virtual private cloud, which is basically a private little subnet.
Yeah.
And if you're testing locally, you're just not going to have it.
Yeah.
But Zappa is really cool.
Chalice is another one that's made by Amazon as sort of a labs project.
I think it's still point something.
So zero point something.
But it looks a lot like writing Flask, but it uses those decorators that you put on to auto-discover how it should connect your code to API Gateway.
And it has a thing that will try and auto-discover what IAM permissions you need, which is, in my experience, a little hit or miss.
Okay.
And then there's also something called Gordon.
Yeah.
Gordon is a Python frame.
It's both written in Python and you can deploy Python with it, but it's just for deployment.
So you'd write your code like Gordon doesn't exist and then you use Gordon to deploy your code.
Okay.
So things like Gordon, Ansible, the serverless framework you use to specify all these resources around your function, but you write your function sort of independent of them.
So they don't want kind of anything to do with your internal code.
They just want to deploy it.
Interesting.
Okay.
So let's talk about where people are using this.
What are some popular deployments?
We already talked about a cloud guru, which is that online training place for cloud stuff.
Yep.
And then you also probably know of iRobot.
They make the Roomba and they have a new series of Roomba.
I think it's the nine something.
Right.
And that has an associated app and it will map your home over time as it sort of figures out where things are.
And that's actually all backed with Lambda and API Gateway.
Oh, that's crazy.
So is it like you hook it on your Wi-Fi?
Yeah.
It has network.
It just streams its location and info.
I don't think it streams the location.
You'd have to talk to them about that.
They're also huge Python users, so they might be a good guest.
Oh, yeah.
That sounds fun, actually.
The Roomba maps kind of locations in your home and eventually it'll figure out like, well, this is always here.
And this chair is only sometimes here, so it must be mobile.
And so it can figure out the most efficient way to vacuum your house.
But you can also do things like kick it off remotely.
So if you're at work and you're like, oh, no, I have guests that are going to be in for dinner and I'm at work, run the Roomba so I don't look like a slob.
And all of that's going through the iRobot API, both on the Roomba side, because they need to be able to tell the Roomba to go.
And then on your app side, because your app needs to tell them, I want this to happen.
And all of that's going through API Gateway with Lambda.
Yeah.
It seems like these IoT things would be a really good fit for serverless.
They just need to talk back.
Yeah.
Because you don't know what the usage profile is going to be.
And you're really cost sensitive because people pay you however many dollars for the device and then they expect it to just keep working.
Yeah, that's cool.
Another one is Nordstrom, right?
They've been speakers at serverless conf several times and they use Lambda API Gateway and Kinesis, which is an event streaming service.
So you can put as many events as you want on Kinesis and it'll sort of keep them in order to a certain extent and invoke Lambdas for batches of them.
And they have a project called Hello Retail that I'll make sure gets in the show notes.
But is a really nice.
I want to see how an architecture of a real thing works.
So for them, it's a simple retail platform that's an example of all these services kind of together in one place.
Okay, great.
So I guess, you know, all this sounds really good.
There's a few cases where serverless makes things a little harder, but a lot of places where it makes it much easier.
When would you say that we should maybe, so we talked about a lot of places to use it.
When would you say you maybe should not use it?
Like if I'm trying to do this thing, like it's probably not a good use case for this model.
What is that?
There's a few.
There's things that you have right now, things that have a low latency sensitivity.
So if you absolutely positively need a response in X milliseconds.
Right.
So like high frequency trading might not make sense.
And this is like single digit milliseconds.
Yeah.
So you're not going to high frequency trade on Lambda.
But even if you have something like an ad marketplace, Lambda is probably not the best because usually your responses are sitting around.
I believe it's 150 milliseconds to go through API gateway and back and then however long your Lambda function wants to run.
So if you need 50 millisecond latency, then I think you're going to have problems.
I see.
So there's kind of an assumed like 150 millisecond latency just in the whole system.
Yeah.
Just because you're invoking a new container and it's starting and it's loading your code into memory and doing all this stuff.
And subsequent invocations are faster because the code's already in memory and it's just waiting on more events.
But you have kind of a base time that you're going to be spending just communicating over the network between API gateway and Lambda.
Whereas if you're going from client directly to your Django app, there's no like API gateway in between.
And so you are adding a hop even though it's a hop inside AWS.
So it's pretty quick.
The other thing is for WebSockets.
There's not a great Lambda WebSocket story yet, but you can go over to a place like Firebase and they have awesome WebSocket support and they hook into Google Cloud Functions.
So it depends on your provider too.
I see.
Okay.
Well, that sounds like good advice.
Firebase definitely worth checking out.
Yeah.
And they have really good Google Cloud Functions integrations now that were, I think some of them were just announced at Google Next a month or two ago.
Yeah.
Excellent.
So this is probably a pretty good place to wrap things up.
One thing I did want to give you a chance to talk about is you just wrote some video courses on serverless programming, right?
Yeah, that's right.
I have two, one from a little while ago that's just on AWS Lambda and function as a service and how to write code for kind of this new architecture and runtime and all that stuff.
And the most recent one I have is using the serverless framework, which is one of these deployment options, one of these tools to deploy GraphQL APIs, which is a query language.
It's developed at Facebook.
I think you've had a show on it unless it was a different Python podcast.
Yeah, I think it was on a podcast in it actually, but that's, yeah.
So people can check that out.
That sounds interesting.
Yeah.
So that's the serverless framework to deploy a GraphQL API that you would consume from a web front end, like a single page app, maybe using React, or you could use it from a mobile app.
Anything that can speak GraphQL.
And the idea is that you can then write your own backend with a lot less experience than you needed before because now you're not managing like a load balancer and auto scaling groups and all that stuff.
You just deploy your function that talks to your data store that then gives you access to all your data from the client.
That sounds really cool.
So those are both at a Cloud Guru and we can link to that in the show notes so people can check that out.
All right, Ryan, this is really interesting stuff.
Let's close it out with the two questions.
If you're going to write some Python code, what editor do you open up?
That would be Vim, of course.
All right, cool.
And most notable but not really super popular PyPI package you want to draw people's attention to?
I don't know if it's super popular, but I really love StructLog, which is a library for structured logging.
So you import it and you put it into your standard library logging configuration and it intercepts stuff and it can take keyword arguments.
So instead of logging out a formatted line with a bunch of percent S or the curly braces, you log out the name of your event.
So that can just be a short message and then as many key values as you want as keyword args.
And then it'll log them as either JSON or a prettified kind of CLI thing.
And so you can inject a lot more data into your logs and then you can get a lot more out of them if you're parsing them through like the elastic stack for log parsing or using CloudWatch or anything like that.
So in Lambda, you can also use this and it will get parsed as JSON by CloudWatch, which is pretty cool.
Yeah, yeah.
That sounds like a really cool addition.
All right.
Final call to action.
How do people get started with this stuff?
I have a blog about this that has some Python material as well as Node.js.
So whatever people are into at serverlesscode.com, they can just kind of go through there.
I have tutorials, projects, interviews, kind of a mix or hit up those video courses if you're the kind of person that learns from video.
All right.
Very cool.
Thanks for sharing all this serverless stuff with us, Ryan.
It was very interesting.
Yeah.
Thanks, Mike.
My pleasure.
Yep.
You bet.
Bye.
This has been another episode of Talk Python To Me.
Today's guest was Ryan Scott Brown, and this episode has been sponsored by Rollbar and Talk Python Training.
Rollbar takes the pain out of errors.
They give you the context and insight you need to quickly locate and fix errors that might have gone unnoticed until your users complain, of course.
As Talk Python To Me listeners, track a ridiculous number of errors for free at rollbar.com slash Talk Python To Me.
Are you or your colleagues trying to learn Python?
Well, be sure to visit training.talkpython.fm.
We now have year-long course bundles and a couple of new classes released just this week.
Have a look around.
I'm sure you'll find a class you'll enjoy.
Be sure to subscribe to the show.
Open your favorite podcatcher and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, Google Play feed at /play, and direct RSS feed at /rss on talkpython.fm.
Our theme music is Developers, Developers, Developers by Corey Smith, who goes by Smix.
Corey just recently started selling his tracks on iTunes, so I recommend you check it out at talkpython.fm/music.
You can browse his tracks he has for sale on iTunes and listen to the full-length version of the theme song.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Smix, let's get out of here.
I'll see you next time.
Bye.
you