28 March 2018

Compiling Python into an AWS Step Function

Amazon provide a very large collection of services for running your computations, websites, etc., on their infrastructure. One of the approaches these services enable is 'serverless computing', which, as they put it, lets you 'build and run applications and services without thinking about servers'.

Two of the components of this are 'Lambda' and 'Step Functions'. Very briefly: Lambda lets you upload Python (or Go, or...) code and leave Amazon work out where to run it. Step Functions allow you to coordinate multiple interlinked stages of a computation, which can include Lambda invocations.

To specify a Step Function, you describe a state machine in JSON. This struck me as cumbersome compared to the normal way we describe how we want multiple components of a computation to work together: normally, we write a Python (or Go, or...) program. To explore this idea, I put together a 'plausibility argument of concept' for the idea that Python could be compiled into a Step Function specification:

For example, the top-level function summarise() (22 non-blank lines) gets turned into 196 lines of JSON.

There would be plenty still to do to turn this into something production-ready, but I think the idea has promise.

Comments are closed.