Technical Q: how do you run long running scripts on a server quickly?

Hi hackers,

I often have scripts that run for hours on end scraping a resource. It generally produces a CSV/JSON that I would use in various forms.

One way to do this is to provision a new EC instance, create a git for my script and let it run. Download the resource after and remove the instance.

This seems like quite some overhead though, and I was wondering if there isn’t some software that just takes a script, runs it remotely and returns me the output, or any easy wrapper to start and shut down ec2 instances for single use.

How are you guys tackling situations like this?